We are going to start today with a bit of a controversial question. And that is, is Claude conscious. So this past week, the AI Lab anthropic released its new model, Claude Opus, which many people on the internet are claiming is conscious. And there is this prompt, which you can put into console.anthropic.com yourself right now. They give you a few free free uses to start with, and actually see what it says, and you will be blown away. Or you may be by what it says it will either make you think it is conscious, or be conscious or something. And so the here's an example of one of the responses to putting that in. So I want to ask everyone here, please briefly introduce yourself and tell us is Claude conscious.
Okay, thanks. Thanks, Jeff. Welcome, everybody, thank you for coming. I'm annual Seth. I'm a professor of neuroscience at the University of Sussex, where I also direct the Center for consciousness science, and have a background in AI in neuroscience in philosophy and physics. And it's amazing to see things like Claude Claude, is not conscious, I don't think that should even be particularly controversial. The evidence that drives some people to think that it is, is the kinds of utterances that you see that which are very, very impressive. But there are many, many other ways to account for that evidence before you have to start taking it as evidence for consciousness just on the way that language models work. Consciousness, by the way means, in my view, whether it feels like anything to be clawed, whether it's having an experience whilst while uttering these kinds of things, whether they mean expert, when it says, I feel sad, does it mean that sadness going on? So Claude, you can account for this evidence without imputing imputing consciousness, there are also a bunch of theories of consciousness and Claude doesn't instantiate any of these particular theories that we have in neuroscience about consciousness. Some of these theories say that consciousness is something computers can have, but others not. They say that consciousness is not in principle, something a computer could have anywhere, including my own. And then you have to ask, what else is driving us? Why, why do we find this so persuasive? And I think it's because we tend to associate intelligence with consciousness because we have this kind of human exceptionalism that we think we're intelligent. And we know we're conscious. So we put the two together. And we're very anthropocentric. That way we see things through a human lens, and we're anthropomorphic. We project consciousness into things when they don't have them. I do think it's very concerning that it's so persuasive. It's very, very difficult to resist the intuition that there is a mind conscious mind there, even if you are, in principle, very skeptical about it. And I'm sure we'll come to discuss, like the concerns of AI that even seems conscious regardless of whether it actually is. And finally, I think, yes, it's not conscious. But I might be wrong, because frankly, nobody knows what the sufficient basis of consciousness is in humans or any other animal. So I think we have to bring a bit of a note of start off with a note of humility about the difficulty of making definitive statements.
Yeah, my name is Allison Dickman, I run an organization called foresight Institute, we are a Bay Area, nonprofit existed since 1986. So longer than I've been around. And since the early days, we've been supporting undervalued science and technology, including longevity, biotechnology, Molecular Nanotechnology, neuro technology, AI and space. And we give prizes, grants, fellowships, what have you, and recently launched an AI safety grand with a particular neuro tech track, which is, you know, how I met Judd in the first place, and really excited to be here. Happy to dive into some of these things more. But I mean, for this one, I would say, I'm not a consciousness researcher. But I think the interesting thing is here, my intuition would be yes, but I would say no. So and I think that's the crucial point, right? Like, I think we're currently are like, we as humans tend to make so many kind of judgments based on our intuitions. And I think with AI that just doesn't hold anymore. So I think we really need to get better at having a science of consciousness, or having like a science of really trying to evaluate these systems. We can get a little bit into evals. About that for perhaps like a little bit later. But I think I think was Nick Bostrom, who once said, like aI safety is philosophy with a deadline. And I think many of the questions that we haven't tried to talk about in philosophical settings, we really need to kind of like, get scientific about them and empirical about them to the extent possible soon, because AI will kind of challenge many of the intuitions that we have. So I'm really excited to talk a little bit more about that on the panel.
All right. Thank you. You all and I'm just super happy to be here in this panel. My name is Mike Graziano, I'm a professor of neuroscience and psychology at Princeton, I have been there for since 2000. And I've studied really basic issues in neuroscience. But for about the past 12 years, my lab and my group has focused on this question of consciousness, what is it? Can we understand what it is? Can we understand why it evolved? And also, lately, can we build it? And should we build it? And what are the benefits? Or should we just build it in a limited laboratory case? So we can study what might go wrong or write about it? In any case, these are topics that are really close to my interests. So is Claude conscious? I mean, I agree with all panelists, apparently, no, this is very unlikely, I think it's very unlikely that Claude is conscious. I also agree that what you see up there is not evidence of it. Like if Claude is conscious, that's not evidence of it. I think, as far as we know, Claude lacks the kinds of self models that I would associate with consciousness of the kind of work we do focuses on building how agents or people or brains, build self models are bundles of information constantly updated, that describe the self. And I think probably Claude lacks, many, if not all of these crucial components. So I would say that, no claws, not conscious. But to me, this highlights something that was touched on already. That's really crucial. When you look at people, when you look at what we do with this concept of consciousness, what humans do, we don't just have it in ourselves, we use it socially, we attribute it to other people. And it helps create the social world that we live in, like we live in a sea of perceived consciousness everywhere around us. And our brains are built for that we're, we're made to look at other agents and see a conscious mind in it and interpret their behavior in that way. And so when we people look at Claude and say, Oh, my God is conscious. This, first of all, is very predictable. And second of all, I think, is really good evidence for something about ourselves about our own social natures. So yeah, that would be my comments on Claude.
Thanks, everyone. And meanwhile, I'm Judd Rosenblatt. I'm the CEO of a studio, which is a profitable bootstrapped business that I started originally in 2016, to try to create capital over time to invest in creating our own startups to create yet more capital to try to build maximum agency increasing futures for humanity, particularly by investing in future tech, like brain computer interfaces, and grew that to over 150 people full time now sold our first startup and did a bunch of stuff with with agency, increasing BCI stuff and now over the course of the past year, are realizing that we can try to be fairly impactful with reducing existential risk from Ai, as well, with a unique model we have where we put the profits of the business into neglected things that people aren't thinking sufficiently about, but could be fairly impactful. So speaking of which, Allison, can you tell us what why is everyone freaking out about AI? These days? Anyway? What what is existential risk for me? I? And why? What does consciousness maybe have to do with it? If anything?
Yeah, well, let me just say that I'll focus on the existential risk aspect here. There's plenty of other reasons why we should be excited and concerned about like, in the past, like, really near term, and lots of interesting panels on that. But I think, you know, if we think about, like, you know, what creates most of the things that makes civilization go round, right? It is human intelligence. And if we eventually create systems that are like, similarly intelligent to humans, and perhaps even like more intelligence to humans, and I think we're on we're really on track, then I think we also have to ask ourselves, what that means for humanity. And just like everything that we care about, like civilization is really kind of like run by human intelligence. So human intelligence could also take it down, right? Like we are currently still possibly the biggest risk factor to most of civilization. And I think we don't worry about that all that much. Because usually human beings, we value being alive, we value civilization, we value others being alive, but introducing these other like Alien minds that have not share much of our evolutionary context of our cultural context into this world. And we may not be that safe anymore, that they value what we value and wanting to civilization to continue. And so I think it's really interesting that you could also ask the question, when is this going to happen? There's an interesting attack he was prediction on like when something like AGI will occur metabolises forecasts in place form. And I think a year ago, it was 10 years longer than it currently is. And so, you know, dropped by 10 years within a year. And so I think it's now like, in the next would be like five to six years. And so I think we really like we're willing speeding up. And so I think for AI and AI safety, what does that mean, is I think there's like different components, even within AI safety, it's a pretty broad field. Now, if we take, for example, the first one of just aligning AI systems with human values, you know, here, we have to ask the question of like, well, you know, what are these values anyways, that we have as humans? And then how do we kind of like, communicate those two systems that don't share our kind of cultural and biological kind of evolutionary context? Nor do they share like our substrate? And I think that's a really important question. There's also other questions, for example, like computer security that I would also put under AI safety, and things like social coordination, like given that all of these factors are really difficult. How do we kind of like have a system of, you know, like, political social coordination going on, by which we don't race to the brink all that quickly. But if we just take the first one for a second, which is the alignment part, right, I think that's where the question of consciousness could possibly come in? Well, to the extent that, you know, there's kind of like, two, two aspects. The first one is, well, can we learn something about human consciousness? For aligning AI systems with human values? Right? And then the second one is like, if we do and if we do create conscious creatures, do we have other, you know, moral obligations towards them? So whether or not then I'll say for about like, that is another I think, question that, you know, I'm not sure if it's outside of the scope of this panel, but it's definitely one that we also, I think, at least have to start thinking about.
Perfect. Thanks, Alison. Sue, I, I think it's worth dwelling back on something that Anil mentioned earlier, which is that we, as yet know very little about consciousness. Today. Many people assume that we know more about it than we do, because we experience it all the time, or we think we do. But But we actually don't know very much about it yet. And it's interesting to consider that we are rushing to deliver to develop AI systems that are smarter than we are potentially fairly soon. And already smarter than us in all sorts of ways. Without knowing what consciousness is, and when and how it might arise in conscious systems. We just currently like that that ability. And interestingly, there's also a super compelling argument that might or might not be right, that I, and that is, is it better to develop AI that is conscious today, in small, little ways, so we better understand and figure out what consciousness is, in this small little ways to test it out and learn more about it on the smaller scale experimenting with building safely conscious AI in smaller systems. Rather than having it emerge later on, in much, much larger language models, as they evolve, or? Or is it better to just sort of risk it and see like it might emerge later on? And this is an interesting question to reflect on, because so the argument that that you don't want it to emerge later on, is, it would if it would be at such a larger scale, that it might pose greater existential threat to humanity, because we haven't done enough work to understand what consciousness is. So when it emerges in GPT, five or GPT, six, it might suddenly just kill us all. And then also, the other interesting consideration is that it might have much greater moral patient hood risks and moral patient, it is like the moral consideration for the conscious experience and suffering and general existence of, of AI if it is conscious, and maybe even if it's not. But I, we and we see potentially, or intuition start to tell us with cloud that it's maybe not that far off. Whether it is or isn't is, is a different discussion. But before we get into all these specific things, because there's a lot to argue with, and with the perspective that I just brought up, I want to just take a step back, and I and ask the panelists here. What is consciousness anyway? So Neil, can you tell us what what is consciousness? Anyway?
I'm glad we've got 10 hours for this panel. Good, we'll need it. But I actually I just want to I'm going to argue with you straight away, because when you say what should we just wait until it emerges in a bigger language model, whether it's GPT six, or Claude, crow five or whatever, then I think you're falling into exactly the same kind of psychological trap that we get into which is we associate consciousness, which we don't really understand with intelligence because of this tendencies of human exceptionalism. So we think that kind adjustments might just come along for free, as things get smarter and smarter, and there is really very little reason to suppose that that that will happen they are strictly orthogonal. And I think you can be conscious without being intelligent at all on the kind of species level intelligence. And you can be intelligent, without being conscious intelligence is about doing the right thing at the right time, broadly speaking, consciousness. So what is consciousness? Well, opinions differ, definitions differ, there is no watertight definition of consciousness. We don't need one actually, to do a good science of it. We just need to make sure we're not talking past each other. For me, a good very general definition of consciousness comes from the philosopher, Thomas Nagel. Thomas Nagel said, in a very famous paper, which is what is it like to be a bat and there's millions of bats, we could ask that question to in Austin underneath the bridge, what is it like to be about the point that thing that he said, for a conscious organism, or a conscious system more generally, there is something it is like to be that system. And what he means by that is that it feels like something to be a conscious system, it feels like something to be me, it feels like something to be you, each one of you, but it doesn't feel like a zombie. Well, I don't know, we can get on to zombies, but that, that I don't think you're a zombie. But it doesn't feel like anything to be a piece of paper, a chair, or a laptop computer, or I would argue close or GPT for or anything like that. For these things. There's no experiencing going on. This is a very useful definition, because of what it doesn't say. It doesn't say that consciousness need involve intelligence, it doesn't say that consciousness need involve language, it doesn't say that consciousness need involve an explicit sense of personal identity. Now, all of these things we're tempted to associate with consciousness, because we enjoy them as human beings. And we define these things in our own image. And that I think, is the broadest definition of consciousness, the redness of red the feeling of pain, as to how it happens. Now, when I started in this area in the 25 years ago, it was just moving from pure philosophy into the realm of neuroscience, psychology and other things. And I feel very lucky to have been working in this area, as it's becoming a thriving and flourishing science. And it's particularly exciting now because we know a lot more than we did about what is necessary for consciousness in the human brain and how our experiences are underpinned by brain activity of different sorts. But there's still wild divergence and disagreement, when it comes to theories about how consciousness happens, what is necessary, and what is sufficient. And some theories say that consciousness is what happens when information is broadcast broadly in the brain, or the say, it's when one part of the brain looks at another part in a specific way. My own ideas, sensor on the concept of the brain is a prediction engine of some sort, the brain is always making predictions about what's out there in the world. And inherent the body. And consciousness in this point of view is not one problem. I mean, it's tempting to think of consciousness as one big scary problem in search of one eureka moment of a solution. But it may be that consciousness is a collection of different properties, much like life turned out to be a collection of different properties. There's no spark of life as the vitalist 150 years ago thought. Life involves many things like metabolism and homeostasis, and reproduction. Consciousness involves many things as well. It's what goes away under general anesthesia. There's experiences of the world around us and of emotion itself. And we can account for these things differently and chip away at this so called hard problem of how consciousness happens. And so currently, there are many, many theories, some of which, like mine suggests that consciousness is something that only living systems can have. So even a very, very advanced GPT, or even a computer that is designed specifically to reflect theories of consciousness would not be conscious, it would at best simulates consciousness. Just as a computer that simulates the weather can do it very well. But it never gets wet or windy inside a computer simulation of a weather and we're not confused about that, right? That's fine. And the same might well hold for consciousness, we can get very good simulations, but that doesn't mean we give rise to the phenomenon itself.
i Yeah, perfect. Well, Sue on that earlier disagreement. I want to segue into Michael's theory of consciousness. And it's interesting to be aware of that many working on AI safety. Think that that there is not the same dichotomy, or orthogonality between consciousness and intelligence as Neil just said, but rather more subscribe to the idea that It is a potentially has been evolutionarily useful to become conscious for humans and other animals potentially. So therefore it may wind up being evolutionarily useful for AI systems to become conscious and, and smarter as a result, or more performance of whatever they're trying to be performed with. Michael, what is your theory of consciousness? And does that does that? How does that relate to it? If at
all? Right, so yeah, I mean, the kind of theoretical framework we've been working on for, like I said before, for about 12 years now, kind of lends itself to this question of AI. So because it's so buildable, it's kind of an engineering approach or an engineering theory. And it kind of starts from this, it's actually in some ways you talked about Nagel, and consciousness is what it feels like or what it is like. And this is a theory, that's almost exactly the opposite. That kind of takes issue with Nagel, in fact, and says, From from the start, everything you think you know about yourself, everything you think you know about yourself, derives, ultimately, from information in there, if you didn't have the information, you wouldn't be able to have the thought, you wouldn't be able to craft the phrase, he wouldn't be able to utter the statement. So everything has to at least pass through the phase of being encoded in the form of information. And so information becomes paramount, and not just information, we're talking about information about the self. Right? When you ask people about their consciousness and their inner feelings, you're asking them to introspect, meaning their higher cognitive networks are accessing deeper models built in their brain. And then they're asked to come up with answers about themselves. They're talking about self information, what we call self models. And some of those self models are telling them, well, you can have a self model that tells you about your own body, like the brain has something called a body schema, that tells you about the shape of your body. So if you close your eyes, you still know about the size and shape of your body and its jointed nature. And we know there's a body schema, for example, because if you get an amputation, the body schema, anomalously can still construct, the model of the missing body part is called a phantom limb. So the brain builds a model of its own body. But the brain also builds a model of its own internal processes, right. And this is what we think is going on. So we ask people, you ask anyone, or you ask yourself about your own consciousness and your own feeling and your own experience, what you're doing is talking about yourself, drawing on information that the brain has constructed to describe itself to itself. And we know why that should happen. We know why systems do that why systems build self models, is because there's this very, very old principle in engineering was discovered more than 100 years ago, that if you have a control system, it works better. If it has a working model or simulation of the thing it controls. As is universally true. So if you want to move your body, you better have a good body schema or model of your body constructed by your brain. And if you're going to have your brain control itself and control its own thoughts and control its own attention around the world, then it better have a good working model of itself. Right? So this is something it's not some fluffy, philosophical thing for armchair thinkers, it's like really important for the functioning of the machine, no wonder evolution, selected for it. No wonder, you know, survival of the fittest gave brains this amazing ability to build little, almost cartoonish descriptions of themselves and their own inner workings. And this is what we call a consciousness. And so this is something that can be built in it can be built into machines, and it's something people all over the world are now beginning to realize. You don't just like Neil said, you don't just build something more and more complicated, and then magically consciousness comes out. That's sort of like, if you told the Wright brothers who invented motorized flight, you said, just build it more complicated. And eventually flight will emerge out of it. Like, no, that doesn't happen. You have to engineer it to do that. And so you have to engineer systems to have these self models. The attention schema theory says, the key model is the brain's model of its own attention. And attention, basically, is this ability to deeply process information. And so we not only deeply process information, but we also have a kind of little internal description or cartoonish picture of what that means for ourselves, and we call it our consciousness. That's this theory, the attention schema theory, and we think this is terribly important. for the brain to control its own internal processes to get along through everyday life. But more than that, if you want to build this into artificial intelligence, I'm not just talking about building things that are better at whatever they do. That sounds kind of scary. But like I said before, we are also very social beings, we don't just build models of ourselves, we build models of other people, right. And so I see consciousness in myself, but I see it in you, I see it in people that I know that I'm talking to, and it helps me, it's, it's, it's the key to why we are a pro social species, right, we are not pro social toward things that we don't think of as conscious, right, we behave well toward things when we think they're conscious. And one of the reasons is, because we're using the same machinery in there, to see them as conscious that we used to see ourselves as conscious. And so it's, it's almost like a built in empathy. And so this is something I think, is really important and really important to study. And this is what I've been doing with EY and with Judd to try to build up in a controlled way to understand how we can if machines would be more cooperative, and more social and get along better with people, if we gave him that kind of toolkit. And it may turn out to be yes. And it may turn out to be really scary. And then we'll we want to know that by testing these things. So anyway, that's a super brief description of this kind of framework.
Thank you. Can I come back? Just want to push back on a couple of things. Oh, actually, I think we we totally agree that the engineering approach is going to be much more useful than the sort of build stuff. And let's see what happens approach. I think we can agree about that. Whether or not we should be trying to build conscious AI, we'll come to I think we shouldn't. But I do disagree. When you say it's obvious that the brain encodes this information about about the self and the brain is some doing some kind of information processing, which is sufficient for consciousness. You know, there's this very, very common idea out there that the brain is a computer of some sort, this is a very sticky metaphor. And the language of the brain is doing information processing is so pervasive, people say it without, you know, without really sometimes thinking what the consequences are. And I think you obviously do, but there are some, it's, for me, it's not obvious that information processing of any kind is actually what brains do. If you if you look inside, if you take a brain, open it up look inside, you don't see the kind of neat separation between the hardware and software that you see in a computer that that makes it able to implement computations, everything is very intertwined. Everything is messy, where the mind where stops and the wetware starts, there's no clear boundary. And when you try to understand what these kinds of systems can be capable of, I think they could probably be capable of all the things, Michael, that you described as very important for consciousness modeling, the self predicting things about control and so on, without doing so in a way that that is computational. That is something computers could have. I think you can simulate it that way. But I think there's just this this this important step back to admit to ourselves that whether brains process information, is still an open question. We can use information theory to describe it. But I do think it's still an open question whether this is actually something brains do or whether it's a useful, just framework.
Should I respond? Okay. Yeah, I think it is a closed question that brains process information. But I agree with you that there may be other things they do in addition. And just to give a little, very brief, super broad history that of course, everyone on this stage knows that the neuron was first discovered by Santiago Ramon Iike. Hall are in the end of the 1800s, beginning of the 1900s, for which he won a Nobel Prize. And he devised this theory that the brain had neurons connected together by little contacts called synapses that were like little gates, signal gates, and that network was how thinking and processing occurred. And that insight has completely revolutionized our world in ways he would never have guessed. Because when we talk about artificial systems and AI, that is built on that insight. Like there is a reason why AI somehow like chat GPT magically somehow seems to do these amazing things is because more and more It's taking principles from how brains are organized, and putting them into artificial systems and noticing oh my god, they do think things, right? The old idea of the computer doing a program cereal step by step processing is not what AI current neural network architecture AI is doing. It's doing something much more akin to what the brain is doing. So, yeah, I think brains compute stuff. And I think artificial neural networks are doing a lot of what brains are doing sort of distilling it and abstracting it. I think there's lots of other things that brains are doing that aren't being captured by this. And there, I'll agree with you. But information processing? Yeah, I think that's what brains do. They process information. And I also would say, it's not that information processing is sufficient for all our thinking, and perceiving and consciousness and so on, it's that all of that has to pass through an information processing stage, you have to have the what's called an embedding or an informational embedding is the pattern of activity in a network of neurons that represent something you have to have that in order to have the thought in order to have the belief in order to make the claim or figure out what words to say, that is a bottleneck, unnecessary bottleneck and all of this. So it's not necessary, it's not sufficient for it, right. But you necessarily have to go through the information processing bottleneck. So anyway, that's what I would say.
I don't want to keep going. Because we can end up in a big, big hole about this. We shouldn't be but I
do. Let's let's move on and ask you a specific question about your research area know, if we consider your own research into the neural and computational basis of code of consciousness? How might your findings, if at all, how might they inform the development of of AI systems that aim to replicate aspects of human consciousness?
I think in many ways, so you know, the work I do is built on what's a very old concept of, of the brain as this prediction prediction machine, I mean, this goes right back to Plato, and the allegory of the cave that prisoners are chained to the wall of this cave, and they don't have any access to what's out there in the world, or they can see a shadows cast on the wall by fires, and they take those shadows to be real, because that's all they have access to. And then instead of prisoners in a cave, we've got a brain trapped in his skull. And it also has no direct access to what's in the world or the body, it has to make a best guess and inform gas. And it does that on the basis of combat making predictions. So this is the theory now does this on the basis of making predictions about what's out there, and updating those predictions on the basis of sensory data. We, in our experience, it seems as though we just directly read out the world in this outside in direction that it pours itself into our minds through the transparent windows of our eyes and our ears. But what I think is really going on is the brain is always actively generating our experiences, creating this impression that these things are independent of our brains when they're not we see with our brains. And we all see and experience things slightly differently. This is also happening inside the body. And here's a point of agreement with you, Michael, that, that I think, fundamentally, the brain is also a control system. That's why it's a prediction system, it's all about controlling the body, keeping it alive. And emotion is, in my view, the brain's prediction about how well the brain is doing it, keeping the body alive. Now, these kinds of ideas of the brain as a prediction thing, you can build them to, you know, their construction principles people do to some some of the more generative versions of AI that we see now, they operate in the same sort of way in terms of a generative model making predictions about what caused data of some sort. So I really think there's, there's a lot that can be learned this this interesting and productive back and forth between building compute computational models of this process, this dance of prediction and prediction error, to try to understand what our experiences are like. And we do this in my lab. So we build computational models of different kinds of experience visual hallucinations, for example, in different kinds of conditions, Parkinson's disease, psychosis, psychedelics, so we can do this. And that sheds important light on what's happening in the brain that makes one experience different from another makes a visual experience different from an emotional experience or a sense of agency, or, or, or smell or something like that. And then studying what's happening in the brain can of course give us principles for developing machine learning and AI systems that that take advantage of the same principles. So I think there's there can be this back and forth. But that does not mean in my view that when we build the AI system is like this that we're actually building in that they actually consciously experience things in the same way we do. I think that's to make the unfortunate slippage, again, from computational models as a useful tool. And it's an extremely useful tool to re refining that. And thinking the model is the thing, thinking that the map is the territory. The final implication of the work that I'm doing actually, if you, if you think about why back to your point about brains and consciousness being useful, yes, of course, consciousness is, I believe, incredibly functional, incredibly useful for all of us, it helps us behave in the world, it brings together so many things about our environment, in relation to who we are our bodies and our opportunities for B, for behavior. But the fundamental role of the brain and keeping the body alive, goes right down into into our individual cells, which even a single neuron is a caricature of a neuron. It's like this one zero binary unit. But you'll know Michael, neurons are so much more complicated than that one neuron fires, everything changes, there's chemicals, there's all kinds of other things going, going on. And if you pull on this thread, at least when I do it, consciousness emerges as something that is intimately tied to our nature as living systems. You know, we're not just meet robots that carry around a computer inside our head from one meeting to another, you know, we are living machines, and consciousness in in my view, anyway, we should this consider the possibility that it's something that only living systems can have. And conscious, and AI will only ever at best give us the impression of being a conscious mind.
How will we know when and if these things are conscious? Well,
I'm not a consciousness researcher. I'm, I can just tell you, I guess from, you know, from the normal AI safety, I guess, community and canon, like, there is a thing called evals hand up if you've heard of emails before. Okay. Okay. So emails are basically used by different AI labs, or by like other kinds of like third parties to evaluate the work that different AI labs are doing. So for example, there's a part research this Apollo, there's meter, those are all organizations that are basically evaluating the capabilities of existing AI labs and their models. And so usually before, you know, these models are published, and they go through these kinds of assessments. And so they're evaluated for like, you know, possibility of like, causing some cyber damage, of deception, of maybe even some autonomous replication of agency, and of persuade persuasion. And so I think these events are usually used just to make sure that the systems are safe. And so I think that is like, I think an important bit that we focus on in our work is just like, really the, the security and safety of the system visa vie humans. And I think there's this interesting question of like, Could we have something like evils for consciousness or sentience, those, again, different types of be unpacked, but I recently was on a different panel with Jeff CEVO. And he's doing some really interesting work. He's a philosopher, and he's mostly focused on ammo, on animal minds, but he's trying to come up with evolves for consciousness, sentience, or agency, again, big terms. But the idea here is that can we come up just like we come up with specific standards for safety and security, with specific evils to test whether these systems show some kind of rudimentary, rudimentary ability to for example, you know, be able to have attention, be able to actually, like have sensory input and actually actuate on it? And if so, then what does that mean for like, our interaction with these models? And I think that's an interesting question. Of course, you know, like, there is a really big problem of like, either over optimizing on that, based on that knowledge. So, you know, there's something to be said, for example, that we have historically, perhaps been relatively averse at describing other systems consciousness, I think, you know, with animals, we're still struggling sometimes. And here, the idea is, like, maybe if we find something small in these systems, that they could be conscious, we should just, Okay, we are okay, with a few false positives, we should just kind of go in and like now really, like make a thing out of that, treat them as if they were conscious, you know, put them into our moral circle into our moral compass. That is also really dangerous, because different to animals, these systems are often MLMs. And they're, like, act really human, like, you know, so what I said at the beginning, you know, we have this intuition, almost, perhaps more so than animals, non human animals, they can speak with us to perhaps over assign consciousness. But then you could have this other consideration of well, AI labs are usually incentivized for the systems that they're training to not tell you that they're conscious, because if they did, they would probably be shut down earlier. And so there's all these different considerations of like, trying to optimize for false positives or false negatives there and we should definitely not think because we've like now found something that we should absolutely be sure that I think we should and could be trying to make matrix some empirical steps to what actually assessing if there's a there there? And if so, what we should do about it?
Cool. So, um, that seems great and super important that that that is likely moving ahead. Interestingly, one major shortfall of that could be over indexing on those things to make us believe it's all going to be okay. Because we still don't actually know what consciousness is. So for instance, if Graziano is theory of consciousness is right, then all the other ones don't necessarily matter as much. Or perhaps more likely, if some permutation of his and something else that hasn't been discovered yet is right, then I it's impossible to know. So we might feel everything's okay. And open AI and others might lobby to make everyone believe, Oh, everything's fine. And then we we quickly run out of time. And that's an interesting thing to take into account here. as well. There's a there are a couple of anonymous leakers from open AI on Twitter. One of them here says this, this tweet, which is basically hard to like, reconcile with your regular view of the world for most people, but many people at open AI, and other large labs think that that that AGI is just around the corner in a single digit number of years. And most likely Hate to break it to you, there's not a high probability that we figure out what consciousness is in that amount of time, because we've been working on it for this long, and haven't figured it out. Now. That's not to say we shouldn't, in fact, ideally, I think we should accelerate research in to consciousness as much as we can to try to understand it as much about it as we can before we build these big AI systems. And there's a decent possibility that we live in a world in which increasingly, everyone around the world, or enough people start to say, hey, we spend trillions globally, every every year on defense, we should spend at least that much perhaps on the biggest threat humanity has ever faced. Now, if that were to happen, no one would have any idea what to do with it tomorrow to there don't exist. We used to spend that money productively. But we could start spending substantially more money on consciousness research and on AI safety research. And there's there's so much to be done there that that is not done yet. So it just it's exciting to take that into account and think about okay, well, if we are running out of time, not saying we aren't necessarily but but in the scenario that we are running out of time, then what is the highest impact stuff that can be done with regards to consciousness and, and AI, specifically, that and as they relate together there? Well, I
mean, you're absolutely right, that, that the best way to understand the prospects and pitfalls of conscious AI is to understand consciousness in those cases where we know it happens, which is in human beings. And the best way to, I suppose where the most impact can be made. By the way, there's other reasons to do this, as well as we should understand consciousness not purely in order to mitigate some potential existential threat. I mean, it is one of the biggest, existentially for us it's one of the biggest mysteries that remains, you know, we know we know we're no longer at the center of the universe, we have evolutionary theory, but the nature of consciousness is something we should strive to understand in and of itself. And there are huge practical implications other than AI in terms of understanding and better treating psychiatric and mental health disorders, in terms of understanding the different ways in which we experience the world and and other minds to help diffuse some of the dynamics of polarization that we see in society. So there are lots of very an animal welfare and legal implications about understanding the nature of volition and freewill for what that says about when we hold people responsible. Consciousness science has moved from being a kind of armchair the main for philosophers to something that's of immediate practical importance in basically every area of human society. So there is an urgency to understanding it a lot better. Exactly how that is accelerated is a very, very good question. I think for me, the the area of greatest productivity at the moment is in taking the different theories that we have and figuring out how to make them more precise, more predictive, more explanatory, be able to test them against each other. So that we can begin to finesse down begin to get a clearer picture of what where the theories might agree and where they might disagree in the space of conscious AI and the safety and alignment arguments. And back to your points Allison about evaluate or tests, as I like to call them using actual language. Is, is we need to we need to think about measuring the human biases as well. You know, when we're when we're making an email or test for consciousness, we need to know whether we're testing a property of the thing or a property of our tendency to attribute I don't know how many people have seen x max and Alex Garland film. So there's brilliant if you haven't, you should see it, there's this great thing called the garland test. Now in philosophy it's about, there's this piece of dialogue where where he says the real test is to show you that she's a machine the robot and see if you feel she is conscious. So that becomes a test of the human not of the machine in the same way that the Turing test is a kind of a test of what it would take for a human to believe that a machine is intelligent. So characterize
it, so we're just running low on time. So I'm going to cut you off here. And we're gonna go through a couple quick things, and then take questions in the middle here. So if anyone who wants to ask a question can start lining up in the middle, that would be great. We, and when you do ask questions, please try to keep them succinct, because we don't have that much time for them. Don't don't just rant as we are the only ones allowed to do but ask. Ask a short, succinct question. Thank you. So, um, it a couple other things just to mention before we start to take questions. One of the things, for example, that Alison Forsyth Institute and my company a studio are both doing is encouraging people to take neglected approaches towards solving alignment. So that's why consciousness, for instance, and consciousness research itself is a neglected approach to solving alignment, it's something that most people might not think could be super impactful, but you might have an enormous breakthrough with. And so that's why we're studying things like self other overlap, and actually building with reinforcement learning, and other stuff with Michael Graziano. Actual small instantiations of conscious AI that seemed to hopefully be poised to lead to pro social behavior, and essentially, potentially make AI more capable by virtue of its alignment with humans, which, if that happens, would be pretty great. Who knows, we have to assume that any one of these things is low probability of working but very high impact if it does work. And so that's, that's why we want to sort of put you if you're wondering, what can you do, what can I do to try to help solve the alignment or problem or do something to do with consciousness? One, please go home tonight and figure out what consciousness is until the rest of us. That would be great. To incidentally, also, it would, it would be good to get funding generally increased both for consciousness research and AI safety, that it could be increased several orders of magnitude. And there would be plenty to do eventually, you can apply for grants to do creative approaches to AI alignment for me from foresight from others, you can nerd snipe or, like get interested, startup founders who are about to start their next startup, to do something that advances AI safety a bit rather than not at all. And also you can go buy a book from Anil in level three of the convention center after this talk. And finally, if you take into account the the research about how how kids are more likely to yell at people when they don't understand them right away. If they grew up in a household with Alexa, and Michael's research about how people actually are wind up, I think, happier and better off in certain ways. When they project consciousness onto replica AIS, which aren't actually conscious. We know, they wind up being better off when they the more consciousness they project on them. I take that into account and and say, Please do your Alexa tree chat GBT nicely, maybe. And that's that's, that's all we have time for right now in the talk. Questions, please go ahead.
To advance this. This thought on what is consciousness? And how do we know if something's conscious? I'm interested in the panel's views on whether a tree is conscious and whether an ant is conscious. And what's the basis for your opinion on that.
I guess I'll start I'll give you a very quick and it's very, very hard to know where to draw the line. We face this problem that usually in biology, you want to study the simplest instantiation of a phenomenon. But we can't do that with consciousness because you don't know what that simplest instantiation is we don't have like Drosophila or simple bacteria or something. For me, consciousness depends on having an in a nervous system of sufficient complexity. We know this when for humans, because we can lose consciousness quite easily under anesthesia or three brain damage. Trees do not have a nervous system. I do not think they are conscious. There's no reason for them to be conscious. If you look at what consciousness does for us, insects and much more tricky case, you know, they behave, they do things. Perhaps the way in which they behave in the world doesn't need this integration of a rich scene with lots of possibilities for flexible behavior, which is what consciousness seems to do for us. So I'm still inclined to think that ants can probably get by without without being conscious. I start to worry when we talk about fish and bumblebees. And for me, it's very, very hard to know that's a very gray area. To talk about without falling prey to our biases, again,
we think about like just the question like, how much do we care about that? Right? There's like one the question of like, how probable is it that something's conscious? And then maybe also, if there's different, you know, like, let's say, if it's a spectrum of consciousness, like how conscious is that system? And let's say, for example, we may even she's faced like a, you know, artificial super intelligence. And we may have low credence, or like, we have a low probability that is conscious, but if it was, it would be like, hyper conscious. There's like some interesting philosophy papers on this. But there's even there's these two questions and your questions, I think we need to kind of like, also untangle those. And just, you know, wrapping up one thing that I still want to say, and they I point is that I think there's an orthogonality thesis three ways. So, you know, we usually like an AI safety, we usually try to disentangle intelligence from alignment. And I think we also have to disentangle it, someone's from consciousness. So I think if one, if anything, now that we can have these other systems with which we can cooperate, and on which we can do some kind of empirical tests, we at least will be able to, for us to kind of disentangle this notion of consciousness, intelligence, and kind of like, human alignment, which usually goes together for humans. But now we finally have a chance to actually like, get more scientific about that and investigate it and possibly other systems. So I think, if anything, we'll learn much more hopefully, as empirically as possible. Next question.
Thank you. So I kind of want to put the question of the definition of consciousness over here. And say that there's, there's, there's a whole bunch of space of things that we could learn before that that could be really helpful for us. So my question is kind of about that. And there's a weird kind of paradox problem that's happened as computer technology has gotten better. We're getting more and more depressed and lonely. So what I'm curious about from your thoughts are, what are you excited about? That AI can help us learn about ourselves, and how we live so we can live? Well, not just so we can communicate information faster. And further.
I'll try it see, we've done a little bit of research on human interaction with AI and how that impacts human interaction with other humans. And this, in my mind is probably the biggest immediate impact of of AI. I mean, technology, in general tends to separate people, right from the automobile, which separated families over long distances. But in in AI. Right now, with these really sophisticated chat bots. And companion bots, there's this really weird phenomenon, where people who are otherwise very lonely or cut off find a consistent partner to talk to, right. And what we find is that people who don't do this, who don't rely on chatbots, for companionship, look at it very negatively. And that's probably most of the people in this room, you're probably thinking, Oh, my God, that's awful. That's like a terrible descent of humanity into some, you know, dystopia. But the people who do it, the people who actually use companion bots for companionship, report this enormous benefit to their mental health and their ability to relate to other people. Right. So in society, a lot of the problems we have are caused by Lonely people, people who, who aren't connected to other people, they get angry, they get frustrated, and then bad things happen politically and socially, right. And now we have this weird tool that allows people to feel socially connected to an agent, regardless of whether it is conscious or not, they think it is. And for some people that's like their lover, they have like lover conversations with these agents. And, and this really weird way for a large chunk of population, we may have hit on this, this partial method that addresses a mental health need that we have right now. So I would say is very complicated. And it's not all bad. And there are some really interesting islands of good things going on.
So one, one interesting thing to take into account that has been core to how I've built my company is that interestingly, it is better business sense to think long term and build products and services that do things to increase the end user of your agent, you're the end user agency of your your end users, people tend to think to short term and think about quarterly profits are showing certain certain numbers to users. But if you actually increase the agency of your end users that makes them like you more and want to use you more and spend more money and refer their friends to do the same thing. And this is why the average tenure of a company in the s&p 500 is something like 12 to 13 years, because they stopped doing this at a certain point. And they get disrupted by startups who are forced to actually innovate and care about their end users to get any users at all. Unfortunately, people start to tend to think to short term about this again, the startup that we built and sold it was based on this thesis there's a lot of value hiding in plain sight by thinking about increasing agency of end users And it's it's, it's interesting because as AI makes us all substantially risk richer, we will also be able to more trivially and more easily think longer term and build user experiences that are more agency increasing for the end user. Hopefully we just don't get value lock in of, of the values that we have to start with of the development of technology for the past 1015 years like this. But let's let's move on next question.
Thank you very much. panel's opinion of AI not as conscious or not, but as a bridge to other conscious in particular project SETI which is using this machine learning to decipher sperm whale Coda's? What if we do make that breakthrough? And they say, Nice to meet you? Can you please stop harpooning us and dumping crap in the oceans? Where would you find AI as that bridge to other biological consciousness? Thank you. Well,
there's project SETI, and then also the earth species Institute, for example. And they've also collaborated on a variety of different things. And I think usually back then, I don't know, I guess, 10 years ago, I always had this idea of like, well, we can probably learn a lot from animal minds. And we will be prepared for, you know, human consciousness to enter our minds. And I almost think maybe it's the other way around. Maybe by actually using AI and engaging with that with AI, we can actually learn a lot more about our relationships to other animals, and perhaps, like, take that a little bit more seriously. So in principle, I'm, like, pretty excited about it project SETI, it's not the s. But it's the C for SETI, just in case you want to look it up. It's really wonderful. I'm really excited about many more of it.
Hello, I have a two questions, but they're both very brief if that's all right. All right. The first question is for a new, you said that consciousness can be turned off. I don't really know if that's the case. In my experience, one of the very first questions that I have, if I'm in a new place is where am I? So can you tell me about consciousness being turned off?
Have you ever had general anesthesia? Yeah,
I've just fully asleep, I guess.
Well, I think it's interesting for me, this is this is the existence super interesting non experience to go through. It's very different from sleep, despite what an anesthesiologist might tell you to reassure you. In a state of sufficiently deep general anesthesia, you your experience is completely interrupted. So in my experiences of anesthesia, you're out, and then you're back. And no time at all seems to have passed, it could have been one minute. And it could have been 50 years, that when when people say they're put into a medically induced coma, they're just kept under continuous general anaesthetic, it's the same thing. This is very different from sleep. When you fall asleep, and you wake up. Even if you're confused about what time it is, as I am here, with some jetlag, I still notice roughly some hours have passed, and not all of which I was dreaming in. But general anesthesia, you are turned into an object from a person and then back again, into into a person. And it's not the it's not that there's an experience of absence, it's that there's an absence of all experience. So for me, it's the closest to what being dead or not being born yet, is like, and I find there's something actually quite reassuring about this, the novelist Julian Barnes has a beautiful book with the title, nothing to be frightened off. And that has a beautiful double meaning I think, when it comes to the nothing, the oblivion of non existence,
things, let's move on to the next person. The second question, were no, no quick, no second question. Sorry.
This may be for Mike. Have you ever consulted the experience of mystics in trying to figure out about consciousness because when I've done a meditation, seven days Zen retreat, my consciousness is so different just by doing that? And are there any clues there?
So we actually study meditation, and we put people in MRI scanners, while they meditate, we're terribly interested in it. In particular, focused attention meditation, which we think is altering these self models as I described them, and especially models of one's own attention. So if you think of the conscious experience are the stuff going on in your head is this you know, the, your self image yourself, they are self models, these things are modifiable. Suddenly, you have an explanation for how these things change. I'll give you one example. Really simple example. There are people who for centuries of course, have been talking about out of body experiences. Well, part of yourself model is computing your, the spatial location of your mind, right? And if that computation gets messed up, you get really weird illusions essentially. And this is inducible in Turns out, you can electrically stimulate a spot in the brain and induce these out of body experiences, because you're scrambling a self model. And so we think there's a really close association between the brain constructing models of itself, these models being a little bit modifiable, and being in what appears to be these different states of consciousness. And we find that terribly fascinating. So that meditation connection is really interesting to us.
Yeah, my question is pretty quick. Just assuming you guys create a product that is conscious? How do you take into account something that like, understands people at a fundamental level, and can probably likely predict their behaviors? Like how do you incorporate in user privacy in your product, so that it's not, like selling us a bunch of stuff, and like, really getting
there. So this is something that concerns me a lot, because the whole point of much of my work is improving, is building AI, that's not sociopathic, by giving it the ability to really empathize with at a deep level to to if that's the right word, I don't know. But to to understand, in effect, people have the same way understands itself. And so I think this is fraught with all kinds of ethical considerations. And one of them is you give it that ability, and you're also giving it the ability to manipulate other people in ways that are really unethical. And so now we have to get into regulation, and we have to get into some kind of policy. But that may be better than having machines that consider people that treat people the way we treat Nanga, nonconscious, nonliving, things like we don't want machines that are so sociopathic, they just step on us.
But also importantly, I do think it's worth noting that the goal, there's no goal of creating a product that is a conscious AI owned by Michael Graziano. That's the that would not be the case that the intention is is AI safety, work. r&d research. It's not it's not. It's not that. Yeah,
I mean, this question applies just as much to things that give us the powerful impression of being conscious whether or not they are, you know, if we, if we can't avoid projecting consciousness into a system, then we expose the same vulnerabilities as if it was actually conscious, there's a bit of a really different reading, which is why we should not be building conscious AI.
Well, I mean, that's that make money at the end of the day, right? I mean, right, for, I can't
have any investors.
So it will, there's also a real possibility that we move rapidly into a world where like, AI, all of the economy is AI, transacting with other AI and humans are left behind too, if you're thinking for far enough out. Okay, I think we have one last question. And then we're gonna wrap up here. Awesome.
Thank you, everyone. And I just have a quick question. If consciousness could be considered a network of neuro components that work together to create experience, how would the discovery of an impersonal awareness that precedes consciousness impact our current research on consciousness?
I'm not sure I understood the question.
So basically, if if consciousness is the end all be all. And that's what we predicate all of our research research of being sentient, or however you want to describe it. What if there was an awareness that preceded consciousness that did not involve the brain at all?
Just I don't think we're smart enough to understand such a smart question is that so I think we're gonna just end on this note. Thanks, everyone for for coming out. It was great having you all here. Thank you.
