Stronger, Faster, Safer? Anthropic Co-founder Dario Amodei on Letting AI Find Its Best Self
4:31PM Sep 20, 2023
Speakers:
Devin Coldewey
Connie Loizos
Henry Pickavet
Dario Amodei
Keywords:
models
scaling
ai
years
investors
companies
build
transformer
anthropic
neural nets
stage
llm
long
limits
claude
startup
tend
catastrophic
happen
give
Have your attention. Please welcome TechCrunch General Manager and editor in chief Connie Loizos.
Hi, good morning all thank you for coming to day two of disrupts. It's great to see a full house here. So much in store for you again today. I'm really excited to say the Dario mo de of one of the buzziest AI companies is coming on right after I get out of here. It's also day two of the battlefield competition. It is getting real. The companies that we have are just fantastic. So make sure that you stay around and watch that. Also, I'm really really excited to watch two interviews later today with JB Straubel of redwood materials. And Kyle Vogt of crews whose company is the talk of San Francisco. So a lot going on. We also have a SAS stage today we've got an AI stage, you're not gonna want to miss anything. To get things kicked off. I'm going to hand things off to our US Managing Editor Darrell Etherington. Thanks, guys.
Have your attention please. Welcome TechCrunch Managing Editor Darrell Etherington. Morning, everybody. All right. Everyone's looking bright and bushy. So let's see. We got tons of stuff today. We got lots and lots of stages. It's so many stages. I can't remember them. So I am going to read them. We have the builder stage where you get practical advice about how to build your startup, we have the FinTech stage. You know what that's about space sustainability, AI software as a service, hardware and security. Today specifically, we have the builder stage, so more great content over there. We have the AI stage and the software as a service stage. Also lots of opportunity to network. Go out there, find your co founders, find your investors, build companies, check out the breakouts and roundtables. Those are on the second floor. Check out the deal flow Cafe, which is over there. You can get great coffee in there, and also deal flow I guess. That's for investor badges and founder badges only and then you can also stop by the red wood materials booth and recycle your old electronics. If you have any of those around. Maybe you bought a new iPhone, you're gonna have tomorrow you can just throw out your old iPhone. Today's no phone, it's fine. It's fine, meet people. But first up we have a very exciting session with Dario Amadei on AI. He is the founder and CEO of anthropic, which has built Claude, one of the most amazing generative AI chatbots out there being used by all kinds of companies and he'll talk about that a bit. And he's going to be joined by Devin Coldewey, who will be your moderator for this session. So enjoy
Dario, thanks for joining us here today.
Thanks for having me.
I know you have to catch a flight so we'll get right to it. But we're gonna start at the sort of cosmic scale here isn't new, we're talking about like 100 year level history. Going back to the 50s and 60s and probably ahead into the 2030s 40s 50s. maybe beyond. Can you kind of just locate us really quick on this in the grand scheme on this the century long story of AI
Yeah, so I mean, you know, the I wasn't around all the way back since the 50s 60s.
Okay, but you know, where I think I can say where I think I have had a front seat to to a lot of what's happened in the last 10 years. And in the last 10 years, you know, we've discovered that neural nets actually work. I mean, I remember going to an undergrad class at Stanford back in 2006. And, you know, neural nets were on the syllabus, and you know, they, they the instructor just said, oh, yeah, we'll just skip that. Yeah. Yeah, right. You're on neural net. But but, you know, the last 10 years, there's been this remarkable increase in the scale that we've used to train neural nets and we keep scaling them up, and they keep working better and better. And you know, that that, you know, I was kind of involved back when I was at open AI at studying the scaling and had been able to predict over and over again, doing, you know, GPT to GPT three, and then the further scaling that we've done it anthropic to predict that this scaling will continue to work and the models we continue to get better and better. And, you know, that's the basis of my feeling that you know, what we're going to see in the next 234 years is, you know, what we see today is going to pale in comparison to that, because I've seen that so many times, I've seen that inductive process so many times,
right? Well predict me this are we going to have a quadrillion parameter model next year,
quadrillion parameter model, the Yeah, the scaling laws don't really suggest that and that would be you know, a lot more than the number of synapses in the human brain. So I don't think it'll get that big but you know, the models will certainly will certainly get bigger if you want to get technical about it, they tend to go with the square of the amount of compute, you increase the amount of compute by n squared and you get a factor of N for data and a factor of N for for
size of the model. Gotcha. So no quadrillion, but maybe 100 trillion more than we have now. Okay, more than we have now. Pretty here first. So, today's language models, as you mentioned, they sort of really took off in the last few years and a huge part of that is because of transformer attention method or technique that was invented basically or in 2017. And we've just been scaling and scaling. As you said, since then, when will we reach the limits of the transformer based approach?
Yeah, I mean, you can you can never quite be sure, right. I mean, you know, it's entirely empirical. I'll need you to be sure you've scaled up the last factor of 10. And you know, what happens when you go to the next factor of 10? You don't know for sure. But you know, I've seen I've seen the last seven zeros be added on to the you know, the number of flops you train on. And so my guess is that it's going to keep happening with quite high likelihood. Now, it could, several things could happen. It could stop at some point. Something that could block us is the quantity of data for various reasons. I don't think one of the data is going to block us but you know, there's always some chance that it will do so we could invent something that works better with more efficient than the transformer. If that happens, I think that'll just, you know, that'll just make the curve steeper and make it go on for longer. A lot of the scaling trends we've seen with previous models for the transformer like the LSDM or the RNN, they scale as well, they have the same curve that, you know, they're just they're just offset. They're not as efficient. And so, you know, we could discover something that's even more efficient. But even if we don't, I think I was going to kind of go on for at least another four years or so.
I saw an interesting talk at CVPR by using toy University of Washington. She thought that we should identify the limits first, if you can, and then work inside those or around those. Do you think that that's possible? I mean, as an example, she gave the LLM can't really multiply large numbers very well. I don't know if that's important for most people when they're talking. I don't ask, like, Hey, do this long division for me or whatever. But do you think that we should be trying to identify those sort of fundamental limits or hospitals? Yeah, so I'm not
sure there are. So and so there are I'm not I'm not sure how I am not sure the different ways to measure them. So to take your example, in particular, on one hand, as these models have scaled up, they have gotten not all the time, but 50% of the time are multiplying three digit numbers or selling prompted the right way. And we've seen Yeah, I mean, more than I can do my head for sure. Right. And we're not giving Claude any paper. And there have been papers that show that when you find a model on the problem multiplying, they can multiply here in large numbers. Again, up to the limits of what what they're able to hold in memory, which is the limit of any computational system. So I again, I think it was one years of scaling up to be very skeptical, but also skeptical of the claim that an LLM can't or you know, paper wasn't prompted or finding food and the way that wouldn't be able to eat. I tend to be very skeptical about claims about hard limits. That's not a claim that anything now or anything I'm just skeptical as
skeptics. Skeptics, yes. So you run a giant AI research outfit with hundreds of millions of dollars and the global we won't say agenda but conditions. I can't be the only one wondering. Just fundamentally, what do you do?
Yeah, I mean, I would describe it as things that are very normal and very writing. It's sort of it's a bit of a bit of an unusual experience. Right. So you know, it's kind of, again, fairly normal things that any startup encounters like how do I get latency down on our models? Capacity, you know, trying to recruit this product. These prospective customers and, you know, tell them, tell them about the great thing. And I think the other half of it can be very useful technologies. So, you know, it was just a couple months ago in Congress economy
and, you know, two or three years from now. There'll be more language modeling bioweapons. Folks in the government, a lot of a lot of
ai, ai one dimensional balancing, like that's something that kind of transcends ordinary ordinary question for so there's something real about Serena long back, back back then. I never have I never really seen him.
And those are those times. I've got your testimony, as well. What would be your sort of short takeaways.
What are you hoping
that Congress took away from that?
Yeah, yeah. I think yeah. I know I've been I've been in Congress and in other setting and other setting is important. Right. So various reasons. I've had no view that in two years years, you know, as we scaled and modifying autumn Eiling there were
pretty wide cracks. That almost and that was that's, that's writing. Of that was cited excited and excited. But skill exists in male area area.
models aren't Nodle gold and right away. Some regulators regulate you know, they're there are many issues heavy perspectives, AI to AI tend to be catastrophic, catastrophic. Think about the worst. Mitigating technology if something really bad and you want to catch on cap and you know there might even
be upside the upside. We need to make sure and make sure that we address we get dressed, so that that has that is always that it's always been it's always been it's a little like a little cheesy, capable, wonderful thing. Wonderful thing really useful. really got to be kidding
I don't know that.
Someone said like, it feels like you're bringing in bringing in you know $2 billion, billion dollars.
Our prevented writers from that
kind of always not mentioning right. Now in your knowledge, you're knowledgeable about the technology. And so we try trying to give advice and give advice that kind of stuff it risks out doing without humanity. Again, no one can pretend government should make their own make their own decisions. They shouldn't. They shouldn't. They shouldn't play hard. I'm using starting you know, I wouldn't I wouldn't say but I focus on catastrophic risks and large models. We're not we're not you know, I can't write you know, we're not we're not source restricting institution innovators going to be very slow number
one market dominance, I think that just just protects your sign Texas.
I'm talking to other people talking about how AI is really
convenient
Amazon, Amazon, somebody do you think those are open enough?
Yeah. Yeah, I mean, I think tend to make content available and open to other things. Seen a lot of neural net a lot of work. We do a lot of work on sculpting, validating developments of neural nets, as we know,
even when it doesn't help and allows them in some sense we do we do this. So much that that enables you to defuse the
target right kind of safe system to become truly okay. I
mean, I may not like my hospital right now. biggest rival
towns they've been a lot of Microsoft. Honey, how do you how do you how do you how do you like how do you become the new like, like animal I don't know if
you know, let me let me paint a picture of the additional five years you drove all the research up to three more seconds more scaling was which are which? Which are driving Thank you. When we when we you know what I think I'm some of the most talented people with us. But you know, now we've been around for a little over two and a half. Open AI has been around seven eight years have been around even longer. In that time.
I believe that it's all relative is much smaller. And yet, we've managed to build better than GV or some fans are very wise agents. They know what we can do in our area, some very small amount.
What I always what I always tell investors, we have customers we have a lever really very radically been able to do more with less. And I think relatively soon we're going to be in a position where the playing field is more equal and we're able to learn more. You know, I think I do very well in here to any other player at that time. One thing I would say is partially because it has been on enterprise uptime and I think and I think everyone that area our performance has already been very strong. And I think is getting stronger until
they actually get it what do they actually hear like, oh, this today I hear about it again. are we testing it here? And there and yeah, exactly are they doing
so we found with particular our sweet spot so far has often been kind of knowledge work and professional service. We've seen a lot of use cases in legal cases. One example of that is LexisNexis.
precedent. Yeah, yeah. Or you know,
nating, but you know, to help you correctly, content about public contract, finance and financial services, a lot of those are deployed internally. That is often how the finance finance industry works. Things like counting, you know, we've powered some podium assistance like source graphs, system, productivity, things like notion and slabs. that'll that'll give you a flavor of the reliability of the model 100 contacts, 75,000 words, candlelight, analyzing documents, again, this kind of information than the affiliation knowledge in these cases, people generally choose us. I don't know we're doing very well in this area. Like, you know, it's too early to say but it's quite possible that you know, quad to all it's relatively expensive loss may or may even pay for itself depends when we want three and start using stop using quad two, not out of the question, right, because
three is the the quadrillion parameter model.
Three is some number of parameters. It's more than a lot to even some data that's more than one, two. And some algorithms that are better than I can't see anymore.
Because of the data does entropic use any site hover library Genesis, so
we don't we don't use those data sets. For reference, those are some sets against which I chose a court injunction telling telling people not not to use them. So so we don't,
yes, and officers today, but it's kind of an open question with others. Speaking, not necessarily about this, but you talked about open AI. You spent a lot of time there. What do you think about saying,
What do I think about Sam Walton? I mean, I don't I don't know. I don't know what to say to that question. Well, you know, what, there's several there's several players in the space. All of these players are, you know, I think I think are at least, or at least trying to do the right thing and they have different views about how they're going to combat right. You know, anthropic view has always been, you know, I think I said it on a recent podcast, personalities. You know, I think people are too focused on personalities. I tend not to think in terms of personalities. I tend to think in terms of incentives. And what in profit has always tried to do is create a Race to the Top, do something and then force your do something good. And then that forces your competitors to do the same thing. One thing I'll say one thing I've learned, not just from this, but for many things, you know, it can be pretty ineffective to you know, argue with your boss or argue with someone and say your company shouldn't do accidentally like, especially if your boss is saying, Well, I'm much more effective thing to do is I'm starting the company. We're gonna do X, we'll see how it works. Yeah. And if if if x is working, and people are like, Oh, these are the safe guys, they're doing X, then pretty soon everyone else is going to be doing x as well. And we found that with interpretability. We found that with work on red teaming, we're starting to find that with security. Just yesterday, we released a responsible scaling plan, which is our plan for how to you know how to scale safely. Obviously, we want to scale quickly to be competitive, but we want to do it in a way that, you know, preserves, you know, the model being safe against these catastrophic risks. And so it's a system that lays out different danger levels. For models, you know, models if they pass a hypothetical future model passes a certain test, it becomes a catastrophic, you know, we declared a catastrophic risk. There are stronger security protections you have to put in place, they're stronger red teaming, you can start to get the national security community involved. And you know, we're the first company to have to have done this. We just released it yesterday. But I think that over time, we're going to see that other folks will need to have will need to have an answer to this more and more, it'll be asked, well, you know, this, this makes sense. Why aren't you doing it? And you know, often this takes years to play out. Sometimes it happens immediately. But but you know, I think about creating the right incentives for the ecosystem. And if you create the right incentives for the ecosystem, you know, it doesn't matter the personalities that are running specific companies, the right things happen because the right things are incentivized to happen and that is our hope. Right?
No comments on Sam Altman. It actually I really liked I really liked your perspective actually. That leaving and doing your own thing is at the very hard of everything with TechCrunch and are in the companies that we love to cover. Do like that's the philosophy at the heart of startup is like, well, I don't want to ask my boss at Google or whatever. If I can do this and maybe get it done. It's like I can do it right now. I will leave and and raise hundreds of millions of dollars. So that's that's a difficult part though. That last part, your investors have they pushed for any timelines for returns? Or they know this is kind of a long play. Yeah,
I mean, I think everyone is aware that language models are costing more and more but also generating more and more revenue so ever every everyone kind of understands that, you know, there's a lot of potential for the field for the field in the future. And you know, that that can make fundraising easier, but at the same time, like I said before, like, you know, the game without giving the exact numbers which I can't, the plausible revenues that you can imagine getting from the current generation of Claude, you know, minus the inference costs and the training costs, like those things aren't wildly out of line, right? But the math isn't, you know, like a factor of 100 off or maybe even a fact you know, or even a factor of 10 off
go on. Factor. Exactly. Sorry.
So, I you know, right, you can you can kind of see you can kind of see the headline on these things, right, it potentially, it potentially makes sense as a financial proposition. Even more so if you consider where these models are going in the future, which I think the smart investors are aware of how things are scaling and where they're going. And not just looking at the commercial dissemination of the current models where I think the potential is already very high. But the real potential, you know, instead of, you know, oh, I'm hiring a model to you know, interpret this one course case or something while I'm hiring my model to you know, to act as a co lawyer for my for my wire. That's, that's going to be a bit much bigger, much bigger pie than we then we see today.
And you I feel like you've raised several large sort of trenches where these meaningfully different sort of groups of investors or people with different meaningfully different expectations,
I think I think early on when you know, we basically just had the team we tended to raise more from you know, from from individuals from folks who were particularly passionate about AI safety. And you know, we've always been very sure to like kind of hold true to the the you know, the expectations and the hopes of those folks, as time has gone on. And, you know, we built clusters, we built models, again, there's, there's been, you know, there's been a more practical prospect of a return. So we've had other investors and we've had other investors in the mix at the same time, and I think it's actually been a useful process of getting everyone on the same page of how can we accomplish all of these, all of these kinds of different goals at once. These very disparate goals don't necessarily need to be contradictory, right? There's some tension between them but but we can, we can make them all work if we force ourselves to go through the exercise and confront the potential tensions between them.
Right. And speaking of tension is where did the sort of sandbank been freed to fit into this whole ecosystem of investors? I think we're all kind of curious how that played out.
Yeah, so you know, you know, of course, when you know, Sam invested in our series, series B, we were, of course unaware of unaware
I'm not I do not mean to allege crime. Girl curious, like, I mean, the guy was everywhere. His money was everywhere. And
yeah, I mean, he represented himself as someone who was who you know, who cared a lot about who cared a lot about AI safety and made made big bets. Obviously he was different. Than what he represented himself. As, you know, those those those shares were, were non voting shares. And, you know, now now they're being sold by the bankruptcy estate. So that's pretty much all there is to say,
you ever do ever meet him?
Did I ever meet him? Yeah, I met him a few times.
interesting guy. What do you think about Sam banquet free? No, it's okay. Don't worry about we only have we only have just a minute left. I know you have to run to a thing but let me just get a quick on a scale of one to 10 How dumb is the discourse around AGI Artificial General Intelligence for those who are not familiar with the thing I know you have your opinions on this? Yeah,
yeah. So I don't like the term that I mean, there was a point like 10 years ago where I think actually that term was more useful 10 years ago than it is now even though we're kind of closer to some of the things that it's alluding to. So 10 years ago was like, you know, you could point it this this point in the distance, it seems like one thing, but but now I think we got to this point where, you know, what we thought was a point is actually very complicated, right? So there's different concepts. One is, you can imagine, okay, where do we get a model where it gets to the point where you know, you can hire it as a professional you know, copilot as a lawyer or financial adviser. It has the skills of most educated humans. I don't think we're very far away from that at all. That could be two or three years, then there's all this stuff about you know, the model is going to get to you know, fill the earth with nanobots or you know, build a Dyson sphere around the sun or something like that. Man, you know, I wouldn't say that's never gonna happen. Science fiction is about the future. And sometimes science fiction things happen happen eventually. But that's just a very, very different, not necessarily the same concept. So I think people are mushing together, things that are very practical with things that are that are very speculative. Not that we might not get from one to the other and, you know, relatively relatively short period of time, but I think we're on much more solid ground talking about when our model is going to be able to do most things that humans can do. versus kind of, you know, the societal implications of that or where that might lead us technologically. And kind of both are both are called AGI. And I think it's kind of not, yeah, not not a very useful discourse. So and, you know, eight or nine on a scale
we'll go with we'll go with 8.5. All right. And that's all the time we've got. Thank you so much for joining us here today for having me. Yeah.