Robin Hanson, George Mason University : Value Drift | Mark S. Miller - Toward Paretotopia

11:03AM Feb 6, 2021

Speakers:

Keywords:

world

alice

bob

values

people

governance

diagram

technologies

voluntary cooperation

cooperation

talking

explains

decision

share

drift

descendants

preferred

robin

extreme poverty

preferences

Welcome, everyone. So now that we have so many folds already 40, and I think still counting, again, for me an official Welcome to the second meeting, and

I am,

can you still hear me? Yes. Okay, good. Okay, good. I'm very, very happy to see so many of you here. Also, so many of you that we've already had the chance to meet at the last meeting, I'm trying for this and to be much more decentralized and to take myself out of the corporation loop. This is why I share with you a form in the chat this form, and I'm going to share it again, because I don't think zoom actually saves them. So this form allows you, if you fill it in, it will automatically populate a spreadsheet. And that spreadsheet will be shared with everyone to be used as a collaboration doc throughout this year to surface common pieces of interest. So in case you want to collaborate on something, and that's what this document is for, you know, I think the book that we're writing is like one, one kind of solution, or like one aspirational attempt for us to try to push the technologies that you're working on, along, but there's probably a lot more other things that we could be doing, and a lot more and things that could be shortcuts. So please, if you're interested in then fill out the form, it populates the spreadsheet, the spreadsheet has an entire the spreadsheet has an entire share of the whole entire program of this year with each individual section. It has a member registry, and it has a section in which you can recommend and propose topics as well. Okay, lovely. Well, without further ado, welcome to the second meeting, we shared chapter before this meeting in advance, which I'm going to share again in the chat. So that today what we're hoping to discuss is the topic of firm value diversity to voluntary cooperation. It's based on the second chapter in the book that I shared. And it's basically in which it is a more philosophical chapter. And we're wanting that about that in the introduction of the chapter. So I'm expecting lots of fun, controversial discussions today. But I do think it's kind of important to set the context for the technologies that we're discussing afterwards. And so in this chapter, we are making a few claims, we are kind of going from, from the premise that, that there's a path from biodiversity to voluntary cooperation. And first, you know, by discovering that voters are really quite diverse within us, and that even by the time that we can reflect on them, we already have some values. And we can sometimes make progress by bringing them into more coherence if we reflect on them. But we have really imperfect insight as to what those are, and also remaining conflict, internal conflict, maybe more feature than bug. And so with so little insight into our own values, epistemic humility is advised for addressing the caring structures of others, especially as we as a civilization get much more diverse over time. And as we experience more environments, and as we grow, and as values are drifting. So across creatures, we recommend a path of voluntary cooperation as perhaps a principle for interaction. And then mark will discuss a little bit on what exactly that entails. So we'll have Robin first with value drift and biodiversity, and perhaps a few words on future key before Robin, before Mark takes over with voluntary cooperation as a coping strategy for that. Alright, we have 15 minutes each about and then we have a discussion. I'm expecting lots of fun, controversial discussion. And I will be taking questions and comments already in the chat. I don't want to take any further time away from you, Robin. Please go ahead. And then we'll have a mock follow along. So welcome, Robin, thank you very, very much for joining us today. It's really a pleasure to have you.

Hello, everyone. I am sick with COVID, which I've now had symptoms for 10 days, which is apparently the day you're allowed to leave from a quarantine but I'm still sick. Ah, I bailed out on my lectures for the last two days. So I'm being brave here and still coming and doing this here today. The topic coverage here is kind of crazy ambitious. I'm supposed to discuss both value drift and few turkey all in 15 minutes all as an intro to big grand topics, which seems really hard. But let me try a bit. And you can ask more in q&a. As you all know, ordinary decision theory separates values and facts and beliefs about facts as the two components and decisions. And in ordinary decision theory. The second part is the thing that's supposed to change. And the first part it's not supposed to change. You're supposed to learn about the world and change your strategies and change your beliefs. But you're not supposed to change your values, because that's the anchor that's trying to use to derive all your actions. So in that context, if people think about values have to sounds like a terrible, terrible thing, because values are supposed to do this. They're supposed to be the thing the anchor. But when we hear people talk about values, like in a mission statement or diversity is our strength or you know, we value, you know, less materialism, whenever people talk about values, they seem to move into this mode where they're not necessarily talking about the most fundamental stable features of themselves. And they talk sort of aspirationally and often in sort of a coalitional politics sense of trying to set up symbols that they're going to share with other people and that they can decide to agree on to go together on projects. And Coalition's and so in that sense, value drift is much less shocking or even surprising that in the sense of values, meaning, you know, what, what are symbols that we're going to orient ourselves around lately, or, you know, in this group, then it's not about our fundamental values, it's more about our recent local strategies. And that is the sense in which people usually use the word values. Although fundamentally, of course, in decision theory, you might say, human values haven't fundamentally changed, we're selfish, we get hungry, we get sleepy, we want to be respected, we want to, you know, have more descendants, we, you know, etc. But our values do seem to have changed in the sense of you look at the kinds of things that we put in political statements or mission statements or things like that. But over time, our values have changed in the sense that we like different kinds of music and different kinds of foods, and different kinds of important issues that we would like to talk about. And so from all those points of views, we say, we have value drift, and there is some extent to which actions do depend on those things. And to that extent, many people, I was really surprised to learn at least a decade or two ago that people who focus SOC started to think about AI would say, well, in the past, humans just had to accept that their descendants would have different values from them, your children don't, you know, decide to do everything you want to do, and your children will decide to act different. And that's just the way it is children decide that different than their parents. But once people said, Oh, and our descendants might be robots or AI, then they said, Oh, no, no, no value, there must be a law allowed at all, that would be terrible. Those things can't possibly deviate from our values, we must make sure they never never change. Any values we set now for now and forever and 2 billion years in the future thing that's never changed or not. And that seemed to be pretty ambitious in the sense that we'd never succeeded that with human descendants, how would we expect to succeed with other sorts of sentience, but many people were very insistent on that. And they felt very strongly that that was really important. And basically said, Good luck. With that. I've also basically said, Look, the way we usually keep our peace among ourselves isn't by making sure we all share the same value. So so maybe within a tight startup, or a family or something, you try to make sure you share it all the same values. But in larger societies, we mostly keep the peace, not by sharing values, but by sharing law by sharing larger norms and law that keep the peace among ourselves. And that's basically how we do it. And I've recommended that in the long run, that's probably more robust strategy to keeping the peace among ourselves and with whatever dependence we create, is to try to share law, including property rights norms, and other sorts of ways that we use to keep the peace. So that's my sort of large intro to values and value drift quickly. And then they also wanted me to mention a few Turkey, which is an attempt at solving the generic governance problems. So and I have some comments. But so the key idea is if you think of governance as making good decisions, then you could say, well, what we really need to do to make the decisions is to aggregate information together about consequences of our decisions. And so we need to sort of set an objective and then aggregate information about which action would best achieve the objective and then take that action. And from that point of view of what governance is about, we could say that, basically, decision markets or decision conditional betting markets would achieve that purpose very well. That is, we could set our outcome measure relatively, you know, without that much controversy, say if we're a firm, we could set our profits as the objective measure or some other thing, we could set another one. And then we could have decision conditional markets, where they were estimating which of the many actions pose be the one that would best achieve it. And arguably, that would work very well. And the interesting thing is, most people who look at that say, yeah, that should work pretty well. And they're still almost no interest in it. And I think it highlights the fact here as with value drift, that basically, we're not entirely honest with ourselves, when we're talking about these things. governance is actually a way in which we struggle for power. And some of us come out on top and the rest of us submit, and pretend like we thought that was right all along. And because of that, we are very indulgent of people we give power to and we allow them to get away with quite a lot.

And I think that's because in fact, we are in the habit of pretending that we uphold higher standards than we really do about our governance systems. So for example, it's an ancient observation that people across the border they have tyrants and we have kings. What's that mean? Well, it means because they have a title who's eligible? Right? We could go take over? And that would that would be okay. Because, you know, they they would want us to do because they don't like they're tired, do they, but we have a king, we wouldn't want them to come over here and take over because, you know, we have a legitimate ruler. And you know, you most people think that politicians in general are bad, but they're politicians, okay. Most people think that bosses in general are bad, but their bosses, okay. And in general, we just have this habit of submitting to our boss of finding a way to accept our bosses legitimate enough that we submit, and we claim that we hold them to higher standards of legitimacy, and the rules are supposed to follow the principles are supposed to follow. But I think that's mostly bullshit. We mostly Submit. And I think, you know, that explains a lot of sort of features of our world and the way we react to governance, which means that it'll be it's always been possible to write abstract scripts and treatises about how we could better govern ourselves. But when it comes to the actual government that we put in front of ourselves, we tend to throw that away quickly and play politics and accept whoever seems to be in charge for fear that if we seem to oppose them, they will crush us which they might. And so I can say more about, you know, how this our hypocrisy about governance tends to explain some behavior. But I think that's probably used up my time here, and perhaps my energy for the moment. So let's hear mark.

All right, lovely, I think Does anyone have a question right up front. Otherwise, we'll go ahead and mark, and we'll leave it for the discussion afterwards. Okay, then mark, this data is yours.

So when I think about the dynamics of cooperation, there's several pictures, several diagrams that are in my head. So I thought I'd draw those diagrams and share them so that we have a common visual language for talking about some of the dynamics of cooperation. But first one slide on the history of the world since 1820. The red on this diagram shows the number of people living in extreme poverty. And the green shows the number of people not living in extreme poverty, we see that from 18 to 22, the present number of people not living in extreme poverty has gone up much faster than the people living in extreme poverty. But recently, something even more wonderful has happened, which is, even as the total population of the world has been zooming up past 7 billion, the total number of people living in extreme poverty has been plummeting in absolute numbers. So as Steven Pinker says, we've been doing something right, it would be good to know what it is. While this good, this global civilization that has this dynamics is emergent from the choices we're all making. So let's try to reason about the choices. And and what kind of dynamic of civilization can emerge from does emerge from those choices.

So over here, we've got a diagram showing states of the world, the red.in, the center, is presumed to be the current state of the world. And we're going to organize it by Alice's preferences and Bob's preferences. But we need to remember that we're actually talking about a 7 billion dimensional diagram. But for most purposes, we don't need to think about all 7 billion dimensions at the same time. Any particular interaction involves only a bounded, bounded set of aspects of the world and only a bounded number of participants. So for each one of those, there's a separate diagram like this, in which we can organize the possible states of the world each.on. This diagram represents a possible state on the world with the red dot being the presumed current state, we can organize them vertically by Alice's preferences, where the vertical dimension where the ones that are up are the ones are the states of the world that Alice would benefit in. And the states that are below are the ones that she would do worse. And likewise, we organize the states horizontally by by Bob's preferences you'll often hear the the terms positive some and negative some, this is what they're talking about. The green area above the diagonal line is the positive sum area. But the whole notion of some presumes that there's something you can add together. So we invent this, this abstraction of utility, we make this assumption that utility can somehow we can think of it as a quantity, we can think about comparing it, we can think about adding it together. And a utilitarian would say that the entire green region above the diagonal is the area of greater total utility. It's the area in which the world is better off. So any motion, there is a motion that a utilitarian should favor, we can divide up this positive sum area into three regions. In the upper right quadrant, we have the the peredo preferred states of the world, the ones that you can get to by voluntary cooperation. So the blue.is peredo, preferred to the red dot, because some at least someone is better off at the blue dot, and no one is worse off. So everybody involved in this particular transition, has no reason not to agree with it to make that transition. The blue.in, the upper left is one in which, let's say Alice suggests, putting in a plan of action that would result in the blue dot that we see on the left, in which Alice is much better off, but Bob is worse off. Well, Bob doesn't like that plan. So Alice, being a good utilitarian explains, I'll be more better off than you'll be worse off. However, for some, for some reason, despite Alice's impeccable utilitarian logic, Bob still doesn't like this plan. So we have a conflict. Not only do we have a conflict, but Alice expects that there'll be this conflict Bob expects that Alice will expect the conflict. And this cascade of mutually expected conflict results in the hub z and trap, where the mutual expectation of conflict creates a creates a pre emptive conflict leads to players to a pre emptive conflict. And one particular terrible form of danger that we had faced was the first strike instability during the Cold War. And the future speaker in the series, Daniel Ellsberg, in his book, the doomsday machine, explains in a quite terrifying manner, just how close we came to destroying the world because of that particular HUBZone trap.

So let's go back to this upper right quadrant, the peredo preferred quadrant. And still not the case that any deal between Alice and Bob can reach any point in that quadrant. And some of the reasons are human psychology.

So, for that particular point, that would be a deal where Alice is much better off and Bob only gets a few crumbs. And Bob might resent that outcome and experimental economics, economics experiments have shown with things like the ultimatum game, that even though Bob is better off, by a few crumbs, he will still not agree to a transition there because of that resentment. We are going to be entering into a world in which many of the much of the cognition of that world many of the entities populating that world are our artificially intelligent descendants that we'll be coexisting with. And we should expect that these systems will have mind architectures that are incomprehensibly different than our own. So, this notion of utility that that is a metric that can be compared and added together was already a problematic notion. Once we are coexisting with mind architectures, incomprehensibly different, that notion just becomes incoherent. What we're left with is to reason in terms of revealed preference, that as long as the entities are in any sense, goal seeking, then the choices they make An outcome of those goals, and they reveal a ranking of preferences, but they don't reveal a metric, we only have a reference, we only have a ranking when each dimension and the absence of a metric, there's nothing to compare. And there's nothing to add together. So the diagonal red line disappears. Envy in sexual world will also be less coherent. But nevertheless, the green cone in the middle is still the sweet spot for what Alice and Bob can achieve by making deals. And the reason is that our that Bob can still reason about what seems to be Alice's preference ranking. And hold out for a better for a better deal. But still, the green arrows, the edges of the peredo box, are still important. Those are still reachable by other means. And to understand why, we need to remember that these are just two dimensions out of a seven out of a 7 billion dimensional diagram. So let's rotate Bob out of the diagram and rotate Carol in. And we're assuming that Carol and Dave are just in some other part of the world. They've never heard of Alice and Bob have never heard of Carol and Dave. So this dynamic that we showed before, where Alice and Bob make a deal, from the point of view of the Alice Carol diagram that has just that has, Alice has worked with Bob, to bring about a world in which she's better off the Carol's completely indifferent to it, Carol's probably even completely ignorant that that's happening. And this is actually most of the world is just independent activities going along concurrently. Most people don't know each other most activities that are happening the world have no strong connection to most other activities. So while that's going on, concurrently, Carol is working with Dave, to bring about a world state in which Carol is better off. So even though each of these individual transitions, hug the edges of the parado box, collectively, what they're doing is they're taking step by step, they're taking orthogonal steps into the parade or preferred area. And this shows the the also the other important thing about volunteerism. In order for the right to choose to cooperate to be meaningful, you also need the right to choose not to cooperate, the right to choose not to interact. And this diagram, and voluntary independence is actually most of the 7 billion dimensional diagram. There are many more pairs that are simply independent of each other, then there are pairs that are interacting. But let's go back to our Alice and Bob diagram.

And ask, in the absence of any technologies of cooperation, how well can Alice and Bob move into that peredo preferred area? So let's take the worst case in terms of lack of technologies cooperation, which is a one time interaction where Alice and Bob are anonymous to each other. But let's also take the simplest possible cooperative interaction, which is a simple trade. And as we'll see, in this scenario, there's effectively a barrier preventing Alice and Bob from progressing there. So let's say that, that Alice and Bob have a scenario where Bob first pays Alice 10 shekels and then Alice would give Bob a gourd. So Alice gives Bob gives Alice 10 shekels Alice is 10 shekels richer Bob is 10 shekels poor, Alice no longer has a reason to give Bob a gourd. So Bob says, where's my guard? and Alice, quoting Hobbes says, For he that performance first has no assurance the other will perform after because the bonds of words are too weak, and then Alice runs off with the 10 shackles. So Bob, of course, anticipating this outcome doesn't give Alice the 10 shekels in the first place. And there's simply stuck behind the barrier not able to make progress. However, technologies of cooperation, like reputation, like iterated games, that our future speaker Robert Axelrod, in his book, evolution of cooperation has taught us the power of arrangements like escrow. All of these things are various technologies of cooperation. And with those technologies, Alice and Bob can arrange to cross the bridge together into the world in which they're both better off. So the world that we're in may have solved the simple individual exchange problem. But a tremendous number of the problems that we have, that we spend lots of time trying to deal with, are, are, are basically the same phenomenon rich law writ large, these collective action dilemmas might be one in which the there's 1000s, or millions of players interacting simultaneously, the array of the cooperative arrangements at stake rather than being simple trades, might be complex arrangements that unfold over time. And the Nash equilibrium trap is what we saw with Alice and Bob, which is the individual players can't incrementally make moves from there to the world, they would all prefer, because the as we saw, attempt to, to go around the barrier, with leave Bob better off in a way that he might not recover from. And 1000s of years of the evolution of human institutions have conventions, of norms, of notions like the rule of law, notions of contracts. All of these things have given us more and more bridges more and more technologies of cooperation, enabling larger numbers of people to engage in some kind of credible commitment, so that they can cross these barriers into the world, they would all like better.

The world of blockchain and smart contracts gives us a whole new technology base for building these bridges. We have future speakers in the series. Speaking about Balaji Srinivasan on the network state. Marc Stiegler talking about crypto title. And many more speakers consult the schedule. But we'll be exploring a large space of possible arrangements. Robin Hansen's few Turkey is an example of something that could be implemented in the in the world that we have but is much more likely to be implemented in block and in the world of smart contracts on blockchain. So let's return to our original problem, we've been doing something right, it would be good to know what it is I want to introduce the notion of a tropism. plants have a phototropism. They grow towards the light. And we've investigated and found what the actual mechanisms are that respond to the light and lead the plants to grow towards it. What I've tried to do in this talk, is explain how the emergent outcome of us all interacting with each other and making voluntary choices, and also otherwise leaving each other alone for voluntary independence, that that that phenomenon all together lead civilization as a whole to climb parado preferred hills to go into that upper right quadrant, which is actually has 7 billion dimensions, and continue the progress towards worlds that are generally better for everyone. Now, I don't want to be pollyannish. We're making progress there in an imperfect way. The actual story is way more complicated, in more ways than I can name. But nevertheless, to my mind, that's the overall dynamic and explains a lot of what that progress is about. But we're just getting started. We're introducing whole new technologies including technologies, new technologies of cooperation and Hopefully we can amplify the tropism and accelerate our progress into worlds that are better for us all. And that's it.

I believe Fantastic. Thank you so much, Mark. And okay, I will stop sharing your screen. I hope you don't mind. Thank you. Okay, great. We have already a lot of questions collected in the chat. What I'm going to do now is I'm going to turn off the recording. So from now on, we can