On Computational Law: Why the History of Computing Could Be the Future of Law
1:56PM Jul 28, 2020
Hackers on planet Earth is back on the airwaves without a tremendous response to the conference this year. Thank you for tuning in for spending time with us for being part of it all. This next talk is very exciting. It's sort of a intersection of law. History of computing and looking towards the future of law, and it's called on computational law, why the history of computing could be the future of law is going to tell some of the story of what's happened in Singapore and extrapolate what that can mean for the future there and elsewhere. I'd like to welcome Mang when one was principal investigator of the research program for computational law at Singapore management University. He's a co founder of legal East calm, and you might remember this name as a previous co founder of Peel box.com, as well as author of RFC, 4408, which is the ever famous SPF anti spam standard that we all rely on so much even if we don't always understand it. So let's go ahead and hear what the what the talk has to say and we will be monitoring the live chat and we'll return for some questions and answers.
Hello hope, 2020, it's 10 o'clock let's begin. My name is Ming, you can say hi to me on Twitter as Ming Wang. I work at a research program at Singapore management University, which recently received a pretty chunky grant to write open source software, a little known fact Singapore is the only country in the world, run by a mathematician and computer scientist, so it kind of makes sense that this could happen here. When I say computational law. People hear robot lawyers. And there are a lot of sensational headlines out there. It's the usual AI is coming for your jobs kind of thing. And this month. There is a lot of hype about things like GPT three. Somebody just did. Plain English in legalese out, and people are like, Hey, we're not gonna need lawyers anymore we have AI. I don't know if I agree with that. But if you talk to an actual lawyer, they will say, oh, Robo lawyers. That's ridiculous. You can substitute for human judgment, every client is different every situation is different. Every case is different. You can't replace me with software. So, okay, fine. I'm convinced. I'll go to a human lawyer. But tell me, why should I go to you. Ah, easy that's an easy question. Easy experience, I've done 1000 cases, just like yours. But seriously, some of my best friends are lawyers my life partner is a lawyer. If I seem critical of the legal profession. Please believe me, things are no better in the world of software. So, this talk is going to look at how software and law, both suck in surprisingly similar ways, and what we can learn from that. Well you don't think software sucks. are you even at the right conference. Go check out this talk, by Gary Bernhardt, and oh by the way I'm going to share all these links in a readme on GitHub. And I will put that link on Twitter, so you can find it. So, what happens when lawyers screw up there are lawsuits and millions of dollars in penalties and maybe people go to jail who shouldn't and people don't go to jail who should. What happens when programmers screw up spacecraft fall out of the sky, patients receiving radiation therapy die of overdoses, the power grid goes down because of a race condition, list goes on.
So, it's really not like we have the moral high ground here.
which makes me think as a programmer. There's got to be a better way. So,
And speaking of signing corporate law is all about the signing
directors resolutions shareholders resolutions shareholder agreements deeds contracts minutes notices, and it all has to happen in sequence and in order. You know what that's called. It's called a makefile. It's a build process it's a dependency graph, you can model all that as a DAG and just automate it. So that's another thing we can do in computational law. In fact, we can compute the dependency graph, as the solution to a constraint planning problem. It's basically a logic problem. Right. The classic. The classic logic problem is the knights and knaves problem. One of the languages we use is Prolog which is perfect for solving logic problems. And what else can you do with graphs, you can visualize the future. Here is a contract represented as a finite state machine. The cool thing about finite state machines, is that they are amenable to formal verification. So let me give a real quick introduction to follow verification. Here's a tweet that this guy called Ken Adams put out a while ago. He had a question about ongoing versus transit. And I saw this and I thought, dude, you're asking linguists when you should be asking computer scientists, because we have the math. To do this, by the way, this isn't just some random person on the internet,
Ken Adams, is the
highly acclaimed author of an authoritative text on contract drafting. So I was a bit pleased to be able to help give him the answer.
So what is the answer.
If you look at full verification, you will see it mentions LTL and CTL linear temporal logic and computation tree logic. I'm not going to go into all the detail about that I will just show you how to use LTL CTL. So you can think about the future you can think about possible futures as a graph of forking paths. And today, we're at the top. And tomorrow we move down one layer. And we want to talk about every possible tomorrow. We say x because that's the next state extra next, and a, which is all of the next states. And so, All Tomorrow's Parties would be a x parties. If you want to say not just tomorrow but forever. We change the X. Next, into a jif or I guess global. So ga means all states in all features. g means there is some future in which some predicate is true forever, or the opposite of that. a f means some predicate will always come true, at some point in every future. So I'm going to skip some of these, but the idea is that CTL and LTL give us a precise and expressive language for talking about system states over time. And this language is what gives us the tools to answer the original question. A an ongoing obligation in every future, you need to keep the information confidential forever right so that's AG, and a transient obligation, you need to pay the purchase price at some point in every future, but you only have to pay it once. So that's a F. And there's a, there's other ways of describing this. There's a standard called legal villanelle, that just got published in April this year. And the language that they use is. There's an achievement obligation, versus a maintenance obligation so instead of transit they have achievement, instead of ongoing of maintenance but it's the same idea and underlying those terms in English, are the mathematics the ag the AF. So, you remember how I said lawyers program in pseudocode computer scientists have formal languages precisely, so we don't have to use pseudocode, we know that transience and achievement mean the same thing in a formal way.
And that's why can Adam said
they're dealing with a mess. And we can help. I know it sounds arrogant, because I mean look at all the bugs right but we can use the same tools to solve software bugs and legal bugs. We use what's called a sat solver, right, to solve these things. So what's a sat solver. Earlier this week, at hope we heard about fuzzing where you basically set up a fast mutating demon outside your front door and challenge them to get in. And with a sat solver, you open the door. Come in. Come in. traverse the internals and try to find bugs from the inside. So have you heard of z three or SMT lib. Or have you heard of TLA plus or alloy. So these are, these are all sort of model checking tools they're, they're written in languages so high level. They aren't even programming languages anymore. They are specification languages, super high level, right now give you an example. You know what a transitive closure is right it's basically all the children and all the children's children and the children's children's children so basically all the descendants, you know that's a transitive closure, all your all your ancestors is another transitive closure. So let's implement a transitive closure in big language. Let's pick closure. Just a little joke. So this is how you do a transitive closure in closure. Okay. And this is how you do transitive closure in alloy, you add a carrot. And that's, that's it. What does this have to do with law. Well, you know, we live in a world where cars can drive themselves computers can play chess computers complete go. why can't computers play contracts or laws right, another word for a loophole in a contract it's a it's a bug. And the whole point of formal methods is to find bugs before they find us in 1994. Someone discovered a flaw in one of Intel's floating point units, and by the time the flaw was discovered Intel had already shipped a ton of processors into the fields, and their recall cost Intel half a billion dollars. And after that they were like okay lads, how do we not do that again. And the answer was, follow verification and 20 years later, Amazon Web Services now uses these methods. So that's a success story right we do learn from our mistakes. Anyway, Now that you've learned temporal logic, let's see it in action, suppose instead of working at AWS, or Intel or NASA, you're at a law firm, working on a multimillion dollar deal. There's probably a lot at stake. Formal methods could help test your code and find bugs at compile time because handling exceptions at runtime. That's called litigation. So let's look at some research from a few years ago that applied these ideas to finding loopholes in contracts. This is some work done by Harter Schneider and some of his collaborators. In this case, the contract they chose was a service level agreement. It's the kind of thing that an ISP or a telco deals with. And they formalized it into the kinds of operators that we just talked about LTL CTL here they look like boxes and diamonds because real computer science professors use symbols right. And after turning this whole contract into code line by line. They fed it to the model checker. Because it's now machine readable. And they said, Can you find any bugs in this. And the model checker said yes. in fact, if this sequence of actions happens, then you lose. And that is an example of a computer playing chess with a contract it's the sort of thing which usually high priced lawyer would figure out by I don't know standing in the shower and letting their mind wander, but we have temporal logics and we have model checkers so we can do better. Now let me show you one last thing. Another little benefit of formalization. It turns out there's a whole genre of lawsuits over commas. Oxford commas, you know, seriously. In 2006, there was a lawsuit in Canada. And this latest one needed a judge to come and try to make sense of a sentence that wasn't even grammatical.
I don't know how that slipped through the drafting but it just like it didn't make sense. And this poor judge had to go figure it out. So, that question, cost, a few million dollars to answer. So in computer science we have these things called parentheses, and we call them nested lists, and we can say things like this. And this is all pretty obvious stuff to any programmer, right, but it's like mind blowing to lawyers, I mean, I really feel sorry for my friends a lot sometimes because if you look at the software world, we take this whole stack of tools and techniques for granted. And it's taken us, you know like 50 years to, to build this stack. But we've come a long way since the days of Fortran and assembly and COBOL and the lawyers, what do they use their most advanced technology is track changes. I mean, they're still programming in a dead language, 2000 years old and I don't mean COBOL I mean Latin. But seriously, if we can fill in some of these holes on the right. There's probably a company or several companies the size of Adobe or Autodesk waiting to be built. And we are going to try to build it here. And that's part of the reason why the government here, chose to fund our research, Singapore is an international center for finance and for legal services we do international arbitration mediation dispute resolution there's a whole generation of lawyers here. We're very proud of their work. And we can see that innovation is coming to law. And if we don't step up. Silicon Valley will be like hey disruptive innovation, we're going to take you out. We're going to totally de laminate the legal industry from the legal profession. Turn your whole world upside down.
Anyway, that's my time. Thank you very much.
There's a lot of excitement in the live matrix chat, I think we all enjoyed hearing the approach that you took and there's a couple different things I'd like to pursue I think the first one there was what's the what's the status what's next in terms of their mailing list and GitHub all the regular ways that people want to, you know,
certainly see what's going on.
Yeah, there, there is GitHub account now we only got the funding earlier this year. And so, since March we've been doing recruitment and the virus made that very challenging. So we've. We're just getting off the ground. Now this grant runs for three to five years. So we have some runway. And there is a GitHub account that's just been set up, we have shipped. Our first working code. And there's also a mailing list, and I've posted those links on the chat so you can go take a look at that.
You know you for programmers like like me and obviously like you. This was such a joy to see the smorgasbord of languages and their different features and capabilities. And so I think I have two questions. One is, it looked like these were your bounded like working on procedural languages I wonder if you look at some of the non procedural languages especially list.
Yeah, you know, the whole paradigms discussion right like there's people talking about the oops the procedurals the functionals the logic. So the thing about law is that lawyers programming pseudocode they program in English, and that's the most multi paradigm language you can imagine because anything you can imagine you can say, it's not necessarily true to anything you can say you can evaluate to a truth statement. And so, we are faced with the challenge of taking these sort of creative writing outputs that come from lawyers and turning them into ultimately Boolean logic and that's not easy. Right. And so we were using a range of languages we're using Haskell to develop the DSL. We're taking inspiration from object languages, where that makes sense like if you're describing shareholders and a class of shares then obviously right classes and objects make sense. But there are situations where you use logic, sometimes even recursion comes up, because legal language will very often say things like, The value of this thing would be the value of this other thing, assuming that this other thing we're not what it actually is right and so you have to go do this sort of hypothetical counterfactual recursive evaluation. So, all the ideas that we see in programming do show up in law. One thing I'm curious
about, and I don't, I don't know if you mentioned is what what usually we call them named entities, so proper nouns names citations to cases which could have some pretty weird punctuation, as well as, you know, needing to identify whether it's a proper name or something. So I don't know if I apologize if I missed it, but I'm wondering how you handle that type of thing and if you end up with your lexicon or mapping.
If you're talking about like defined terms right like at the start of a contract or at the start of legislation there's always a section that says define terms and then you sort of capitalize them, like the low hanging fruit in computational law is just to like go through the document and find any defined terms that you didn't actually define. That's the equivalent of a null pointer. Right. And, and that's the easiest thing you can possibly do is like your symbol table, you've got a bunch of variables. You've got a bunch of function definitions, this is what they mean. And if you try to call a function that hasn't been defined. Then we throw that error at compile time because you wouldn't believe the number of contracts that I've read, where people use sort of undefined defined terms sort of superstitiously, and you're like okay you can't actually do that. But you can in law, people write these things, even if they don't sort of parse validation
and and they sign them, and then it's the judge has to figure it out.
You showed a lot of different examples and some of them I think were really it could make it clear how difficult and how much variety there is, but I would imagine you have an actual test suite for the code and maybe you can tell us something about the test suite,
you know, so the the idea of bringing test suites to law is really interesting right because you can imagine, have, like, a red and a blue, come and they look and they say let's write some legislation, and the red guys like well you know the people that I care about are like a certain demographic, and then the blue guys like well, people that I care about her very different demographic and those are my constituents, and the law itself is like hundreds and hundreds of pages long. And I don't think the politicians actually read the law. Right, the lobbyists write the law. And so, what they should do what they could do is they could say you know I've got 100 test cases here. One of these test cases is a sort of Vietnam vet missing a leg, and another test case is the single mom with three kids and another test case is, you know, a schoolteacher living on a certain salary. And they could say you know what, as long as the proposed legislation passes all the tests. Then we'll find it right we don't need to know what it says as long as it's okay. And so that's sort of the test driven development approach, brought to legislation
and have a crazy idea. Yeah, one in a test suite. Typically what you would do if you were training you mentioned self driving cars for example, you throw anomalies at them so and I know you're early in the software development but picking out throw a recipe throw some poetry in Shakespeare, or something, and see if it, you know, if it pukes appropriately I guess it's probably with Yeah,
you can you can basically fuzz the legislation right you can you can do penetration testing on contracts and on laws and you could say, here's your bug here's a situation never thought about. And so this is a this is a story that comes out of Singapore actually so Singapore for a while, had a policy of, like, if you have two children, that's good enough, sort of like we want to control the population. Don't, don't go for three. Right. and so they would say the government said, if you have two kids, we'll give you incentives and if you have three kids. We'll take them away. And what happened was this happened to a friend of mine. There was the first child, right and they say okay well, we can go for another one. And so they got pregnant, and it turned out. The second pregnancy was twins. And so they went straight from one child to three without ever having the opportunity to stop it. Right. And that's the sort of thing that, like if you had a programmer in the room right if you had a model checker in the room, you would have found that bug.
Right, that's not a, not something unpredictable. Obviously, there's a detailed question unless someone was wondering whether you look at logline logline logline as one liners.
Yeah, there's a number of these sort of constructed languages local and they're not at the top of our list right now. We are just trying to get the basic language working. And once we have sort of an abstract grammar that describes events actions, obligations prohibitions various kinds of deontic and temporal and epistemic modals, those are the technical terms. You can then have whatever concrete grammar you want. And so, you know, that's something that we'll definitely look into but maybe in year three or four of the research program.
Another detail question is just not licensed Are you looking at a GPL or some sort of a free software license for this.
Yeah. We actually had to fight for this. But all of the software that we develop at the research program will be open source. So I don't know if it's going to be GPL I think right now we have MIT license. But we do want to get this out there and obviously, any language or any standard that's closed isn't going to get very much traction. And importantly, building the, sort of, if we are moving laws on to a software footing in an open society, you have to have open laws and in our case, that means open source and open standards.
Yeah, a guy named Carl melamine two discovered that laws are not always open anybody's thought of that suit for free pretty thoroughly by the German court. And of course, of course in the US, we have the next law Westlaw Lexis and Westlaw issues where they try to copyright law, but luckily I've done one good boy stick. Yeah, so I totally agree with that. So in the US, of course you have the United States Code, or provision in Singapore there's something similar in other words, you start with the law books or you start with. You start with the legislation and that sort of thing.
Yeah, the legislation so this is definitely, you know this this whole idea of computational law belongs to the school of legal formalism as opposed to legal realism. And there is the idea of like the civil law system and the common law system right and the common law system is a little hairier and it's more unruly than the civil law system, where, if you look at the German civil code for example, you will see what reads, almost exactly like an owl ontology where they start by defining this is an entity. This is a thing. This is a person, this is an animal, sort of goes from there and, like, Wow, that's really structured right you will not find that level of rigor in the common law system where you basically got like centuries worth of judges saying you know that doesn't feel right to me. I'm gonna go make up a reason why
stream basically just, yeah.
Fascinating so we had to try to redo this question. I'm not sure I get the gist as soon as I mentioned, there would also be room to compare computationally generated contracts to pre existing laws and case law to ensure the contract didn't accidentally repeat existing law. Oh yeah, I think that's where you get it okay
yeah so that's that's a basically what we would consider a consistency check, right, like you could imagine, even without talking about laws just talk about contracts, suppose you're a corporation, right. And you have in sort of your short life, you have signed 20 different contracts with vendors or customers or employees or whatever. And you've got these 20 contracts that govern your behavior as a corporation. And now you've got the 21st, and you're wondering, is there anything in this new contract that sort of contravenes something in my existing contracts. And to answer that question, right, you have to put go back and read every single one and then read the new one, and this is like a classic computational complexity Big O kind of question. Right. And the way to answer that question is not to hire a lawyer and say, you know, every time I have a new contract we'll think about all the previous ones. It's to turn all of this stuff into code, and then run the SAT folder on it. Right. And if we can do that. We really save a lot of time and a lot of money
yeah that's that's, I can see that right away. I'm somewhat of a related question to what you're talking about with contracts. And again, I'm sure Singapore has something similar in the US are sort of a cottage industry is a company called legalzoom.com, for example, you know basically templating, a lot of common contracts so you know wills and rental agreements and stuff like that. And yet it seems that, that there's often a need to go beyond that and that's a big step right so going from $39 for Legal Zoom contract to hiring a law firm to write your custom contract is is you know, a big step even for a small business. So do you think that you might reach into that, I guess is there opportunity for the SEC Yeah,
there is so,
so if you're not aware of it, there's a fantastic startup called do not pay. Play. Joshua Browder he is an undergrad at Stanford, he's actually from the UK originally. And he wrote a parking ticket app that basically gets you out of parking tickets, because there's all kinds of technical things that you can do right. And so they went off, do not pay law is the name, and they went off and they picked up a bunch of funding from Andreessen Horowitz, and they are doing really really clever things that directly satisfy what Clayton Christensen would call disruptive innovation, right now, you hear the term disruptive innovation all the time people talk about it but it actually means something and if you go look it up at Wikipedia you'll see there's a Harvard professor Clayton Christensen who talks about this. And he says, Go find the people who are over served by the current vendors in the market. What that means is if you are a big corporation, doing like an MBA, or a you're a rich person doing a lawsuit, then a law firm will charge you $20,000 just to have coffee. And you're well served by that at the high end of the market but for most people, you're over served you can't afford the fancy law firm, the white shoes and all that coffee. And so, what do not pay has done, they have said, you know, we're going to do things that the average law firm wouldn't even recognize as a legal service, but which the average, the average man on the street would really appreciate, like for example you know there's rules where if you buy a plane ticket and then the price goes down you're entitled to the difference right, or if you're if there's like a three hour delay or a six hour delay, you get certain entitlements. Now most people just let it go because you're like I don't want to have to fight with Delta right about this. But what what Josh has done he said you know that whole system of rules, I can put that into software. And if you just sort of hooked me up to your email account. I will automatically monitor what flights you're on, I will figure out when the prices drop I will figure out when they're delayed and I will do everything that you're sort of pet lawyer ought to do. And I will get that money back for you. So it's it's absolutely genius. I think he's great. Another thing he's done is he's basically said you know just forward your spam to me. And if enough people send me spam. I will figure out who the spammers are and I will automate the person, I'll automate the process of launching a class action lawsuit, and ultimately I'll recover you a few hundred bucks.
Right. Yeah, so we can forward our scam to do not pay law, calm, I guess,
stuff like that.
Yeah, I'm Catholic I know I wasn't aware that it's amazing and I like the Clayton Christian constructive innovation concept of what sounds like a good idea for later after the talk. Um, another question that came in, is about end user licensing agreements so we know how famously long, you know you get these licensing agreements from companies like Apple, and so is there opportunity there to let's say parse and understand or summarize, or is there also opportunity to really track the changes in these things over time and look for, let's say problems.
Yeah, the thing about these end user license agreements is that, you know, a whole bunch of. So a lot of what's inside these agreements actually is unenforceable, and it's it's there sort of out of tradition, because the lawyers who write these things are like well you know, we have to do this because these things ultimately get sort of measured by the client, not by their content, but by sort of by the number of pages they're printed on. Because if you, if as a lawyer, I'd have, if I deliver like a two page thing to my client. They'd be like you didn't do any work on this. And if I do this for a 200 page thing they'll be like oh that's very impressive I'm going to pay you your fee. So, so these things have a tendency to grow. So purely economic reasons. And so, I think if we are successful, if we are successful, then things like eulas and things like MBAs and employment agreements and lease agreements, those things will. They'll be sort of rewritten the way, like early proprietary OSS were rewritten by free unixes by BSD and Linux. And if you know the next time you do a contract, you go download it from GitHub, or some sort of contract store where you can say, look you know this this contract has four stars that contract has five stars. People give feedback on these things people give pull requests, you know, those that future. In that future contracts will represent both sides much more fairly because people have the opportunity to go edit the template in a way that they don't today.
Yeah, this is a democratization process and leveling the playing field for those that can't afford the $20,000 boy right an expensive copy. One of the questions I think relates back to your some of your earlier remarks, the question is it possible to reverse compile existing documents.
Yeah, we, that's a challenge. You'd be surprised how many ambiguities show up in legal text. And I can share a couple of links about specific cases where people have said, there's this guy called Howard Darmstadt, and he spent his entire life as a commercial corporate lawyer. And when he sort of retired. He went into teaching, and in his law class he said okay students. Here's a contract that I've spent the last 30 years working on. I'm very familiar with this template. I'm fairly sure that it's perfect. And so he gave it to the students and his students tore it apart, and they found all these bugs and he was like, Oh my god, you know my entire career I've been using this paper that really has never gone to court so you never even know right it's like imagine, as a programmer, you write a program and it never runs. How would you know, it actually has any bugs right. And yet that's the situation with law you write a contract and maybe it's 10 years before it ever goes in front of a judge. And so if you had, you know, if you're repple compiled was a 10 year cycle, you know, you'd probably be programming Latin to.
Yeah, technology changes during the time as well. Yeah, so many so many so much evolution. Well this is fascinating. I think there's a lot of a lot of interest in the potential here. You know you mentioned a couple times that this is a project for Singapore, and you've also talked a little bit about US law, your common law and civil law. You're being Germany as an example. And I was just wondering how much diversity of law, there is, you know, could you characterize how much diverse either law. Well, in our class and even different jurisdictions within a country. Yeah, so we we see
law as a problem in applied computer science. We think it's an accident of history that law has been around for 2000 years and software has only been around for 50. And so we'd like to ask ourselves the question, what would be different, you know what if, computer science, had been around for 2000 years and we were just inventing law today. How would that be different. And so we do want to build a language, and a standard and a platform that pretty much anybody use, you know, you don't have to be Singaporean or American. If you want to transact with somebody in a foreign country you don't speak their language. No, you should be able to write a contract that both of you understand because it was compiled from the same photonic source. And so that really is our vision. You know I'm an internet infrastructure guy. And I see law as basically a problem in infrastructure. And so if we can, if we're successful I think that will be, you know, there'll be like, maybe some standard some port number some convention some library. You know the whole thing will just turn this after
another RPF excursion. That's right, that's right. So, yeah,
I think we've we've covered most of the questions that came into the through the live chat and we're getting towards the end of our time. Is there anything else you'd like to say to sum up for you mentioned actually some really fascinating links to other projects other activities I think anything else you'd like to share with you. Welcome.
Yeah, we've got
we've got a couple of resources that people should know about the Singapore management University Center for computational law SMU cc law has a GitHub repo and we will post that you can check out my Twitter account, I'll share some of the links. Subsequently, so it's twitter.com slash linglong. And there's also a startup that I've been running with my partner for the last five years that's legally calm, and there are a ton of resources and readings and sort of background. It's amazing that this idea actually goes back 50 years actually goes back a lot longer than that, if you look at the 17th century, lived it's had the idea that it should be possible to calculate and reason and have some sort of mathematical notation, by which one could encode and answer all of the problems of human society. And so that was likenesses vision. And ever since then I think people have been trying to realize it. When he was young he thought this is only gonna take five years, then when he got older, he was like, it's a much bigger project than I thought
that was 300 years ago.
Fascinating say again so a lot of a lot of things to follow up on so I think I think we'll be watching this project, a lot of interest, as I said, we've been talking with mangalam Wong, who is principal investigator and the research program for the organization, especially the competition law as Singapore management University. As long history has wrangled successfully some RFCs, and even developed a standard for anti spam email so so I think I reiterate that because I think your credentials show that there's a likelihood of success here is, thankfully, a hard problem, maybe the biggest, maybe you would say this is the biggest hardest problem you've tackled but certainly it's non trivial in every way.
So we're looking forward to what's next, and thanks again for being with us and hackers on planet Earth.
All right. Goodbye everybody. Thanks for watching.