This podcast is brought to you by the Albany public library main branch and the generosity of listeners like you. God daddy, these people talk as much as you do! Razib Khan’s Unsupervised Learning.
Hey, everybody, this is Razib Khan. I am here with Dr. Cory Clark, who is affiliated right now with University of Pennsylvania. Cory, can you tell the people out there a little bit about yourself what you're interested in how you define yourself, what you identify as and what your pronouns are.
I identify as a tiger. My name is Cory, I guess I'm technically a social psychologist, but I call myself a behavioral scientist, because social psychology, the reputation has plummeted over the past 10 to 15 years. But that's what my PhD is in. I'm at University of Pennsylvania in the psychology department and management department. And I'm the executive director of the adversarial collaboration project, which is a sort of science reform initiative where we're trying to get enemy science to collaborate with each other rather than write mean replies to one another's work and call each other assholes at conferences.
Yeah, at the tail end of this, I might want to like talk a little bit about that, and how science works in the United States, how it's gonna work in the future, how it doesn't work. And that's an interesting topic, for scientists in particular. So I'll - I'll get to that at the end. Because I think there's a lot of normies out there that don't really care. They’re just like, show me the results.
Cory: Yeah, I don’t really blame them.
Is it true or not, you know, that's a -
They don’t need any of this philosophy of science bullshit.
Nah. But um, so actually, let me ask, before we get to the paper that I want to talk about with you. What is up with social psychology? Why was it in particular of the fields in psychology vulnerable? Why do you think it was vulnerable to the replication crisis and all those things?
Well, some people, the pro social psych folk would say, like, well done social psychologists, you know, we figured out our own mistakes and are cleaning up our own mess. And other disciplines have replication crisis issues too, which is all true. However, in the handful of other fields, people have looked at replication issues in other disciplines, it does seem like social psychology is particularly terrible. And my best guess for why that is, is because the, like the whole concept of social psychology, might be a false positive, in that, these little the things that social psychologists love to do, is manipulate these little teensy, tiny environmental things that last a second or five seconds, and it has some radical change on some really important outcome. People are more ambitious, they're happier, they're kinder, or they're more aggressive. And I think those kinds of effects these little, or you put someone in particular mindset, and then they do better in school, like these little teensy, tiny, low cost interventions that social psychologists love to do. I think most of them are bullshit. And so it was sort of inevitable that eventually we were going to figure out that none of these little things worked. And instead, a lot of the correlational stuff replicates. And that's the kind of stuff that social psychology thought it was too good for.
Well, and I do have to say correlational, and the the nudges and all these other things, you know, behavioral economics, which was a massively sexy field, right after the financial crisis, in particular, where there was all these ideas that oh, yeah, economists are wrong, because, you know, they use a rational actor model and like, we have all these cognitive biases. And I mean, that field is having a massive comeuppance the woman at Harvard, of Italian background, she, I mean,
Cory: CD Francesca Gino
yeah, radioactive, like it's going to destroy the field. She's on so many papers. And now, Dan Ariely is finally - they're coming after him again the second time. And so with Ariely and some of you might know him from his media production. He used to be on NPR and stuff all the time. He wrote, predictably irrational, and then he wrote a book about lying. And it turns out that almost certainly some of the data that he has used, like, you know, insurance data and stuff was fabricated. Now the question is, whether he's responsible or not responsible. There is a real thing with / in some of these fields, where the original data does get lost or deleted, that does happen. You know?
I think the APA guidelines was like 10 years you have to hang on to … I mean, now, I think the normal practice is to hang on to it forever. But when people were holding on to paper surveys, you know, collecting dust in the storage unit, I think people said 10 years so like, that's all plausible, but those findings, even though maybe they would call themselves a different discipline, that's social psychology kind of stuff, you know, these little manipulations, sign at the beginning sign at the end, and it has these big changes for honesty. Like, those are all social psychology type findings. And those are the type of findings that don't replicate. And if you're going to be a successful scientists who's publishing those kinds of findings, you might feel compelled to cheat because you can't get them to work. And you got to publish papers. So even the fraud cases, I don't think that's, I don't think most scientists did that level of you know, it's that's not even p hacking at that point. But they didn't engage in that, but
I think we call it pulling a Stapel,
Cory: pulling a staple?
Like Diederik Stapel
I think that's Stapel?
Okay whatever. We live in America. Yeah, we live in America. If English is good enough for Jesus Christ in the Bible, it's good enough for me. You know. So, but yeah, whatever that guy - he literally made up datasets when people were thinking about how do we explore this? And he's like, I have a data set, and he would go and make it. And so, you know, that's doable.
He wasn't the only one. But thankfully it was pretty rare,
Well, so there's a guy, I think he's an immunologist that just came out. I mean, this is everywhere. Like when the replication crisis hit. A lot of people were saying like, it's like, in biomed, in pharm, there's massive issues, you know, and a guy like checked some immunology, some immunologist, I think, checked a bunch of papers. And wherever he found, wherever he found the raw data, he said that there was like 25%, like, fraud or massive, massive error that was basically like, just invalidates the paper. You know, what, that's a lot. When he only found summary statistics, he could only find fraud, or invalidate the paper in 1% of the time when they didn't report all the data.
Cory: hmm, I see.
So he's concluding from that like, No, it's not that those people have better data. It's that they're not reporting the data. And so you can't even confirm that.
Cory: We can't know.
Yeah, probably 25% of those cases, you can't really confirm that it's fraudulent or it's in error. You know?
I mean most people who committed fraud would probably try to destroy their data to minimize their chances of getting caught. So that's why it's sort of surprising to me that some people who were committing fraud still went ahead and made their data publicly available. It's pretty bold.
You know, what, when you're that big, they let you know, but I mean, that's kind of, you know, like, I don't want to, you know, I'm not okay, this is not libelous, but I mean, it's just like, if Dan Ariely was not Dan Ariely if he was not a very prominent, you know, professor at Duke, if he was junior faculty, or if he was grad student or something like that. He would have been done a long time ago, you know,
well, maybe or maybe not, because I think people like to target high status people because people like to see the high status people tumble, you know, if that paper had been published in a journal, no one cared about, then no one would have bothered double checking it because they wouldn't have been jealous. You know?
Wait was your psychologizing there. Is that a robust model? I mean has not been replicated?
No, that's an intuition. It’s a series anecdotal experiences.
No, but that's a fair enough point. That's a fair enough point also. Yeah. And people that have been in situations where they've been touched by fraud. They're paranoid after that, too, because I've known of people that were in collaborations and then they found out something about their collaborator, and then they freaked out. So I'm going to move on to like the paper, but I will tell a quick anecdote, a story. I think I've told this before, but anyway, I'm not sure if I have on this podcast, but it's just, you know, stuff happens in science, and I'm not trying to make science seem bad, but it's weird. So I have a friend, and he ended up getting a postdoc in the European Union. I'm not gonna say what institution and it was like a great opportunity, supposedly. And then he's in the lab, and he realized they're fudging things all the time. And he just like asks his mentor in the lab, not the primary investigator, not the scientist lead as is like, I think something's going on here. And like, The guy looked at him, and he was like, Wait, you didn't know that we just made stuff up? And my friend was like, ‘Wait, what?” he's like, Oh, everyone knows we make stuff up in this lab.
Cory: what?
Yeah, and it was a big PI. It was a big it was a big lab.
Cory: Did they get busted?
I don't think so. But the story doesn't end there. And so they just assumed that he knew he was okay with the fraud and that's why he joined. It was a - he had moved out of field. I guess I can say it was it's a plant lab. He wasn't a plant person. And anyway,
Not even the plants are replicating?
Yeah, I don't want to get into it. I don't want to give too many details because I don't want to be sued. But so, okay, so he, he eventually had to leave. But he didn't narc because they were like your career will be destroyed. What you need to do is make a pretense why you have to leave the EU. And so he did. It was a family emergency. So he left. And then he got another postdoc somewhere else, and he's fine. But I was like, what I was like, That is crazy. And he's like, Yeah, but everybody knows, everyone in the field knows, but this person is publishing. And then later, I was hanging out with a friend of mine. And she was at that same institution, in the same field, different organism, but whatever. And I was like, you know, so and so I'm gonna just make up a name. That's generic, but that sounds like him. Mike, Mike told me. Mike told me that that postdoc, that lab he was in. And she was like, oh, yeah, the one that produces fraudulent data. And I was like, what, so it's just, it's a big, it's a big professor, and everyone knows it's kind of like an open secret, I guess. And nobody wants to call out this person because it will destroy there's a coordination problem. I think I've told this story. I think I've this story
Clearly someone is competitive enough with this PI that they would be happy to watch him or her go down.
Yeah, maybe. I mean, maybe it has happened this i This happened 10 years ago. Yeah, 10 years ago, and I talked to her nine years ago. So ,
I mean, it is it is professionally reputational and costly to be the person to tattletale, you know, on other people. - sometimes if you're right about it, like,the Datacolata guys, I think are pretty respected by people.
Yes, yes, they are. They are. But um, so, you know, I'm just, I'm telling this story, like how science sucks. But, you know, it's great to, you know, this is why I'm talking to you, this is why we're in the game, but, you know, you hear things and you know, things. It's a human enterprise, and it's a social enterprise. And so, you know, sometimes, for example, like, just concretely, like, you know, you'll see, I don't know, there's like a trash paper. And it's in Nature. And, you know, it's in Nature, because the last author, bla bla, bla, bla, bla, bla, bla, bla, but the paper itself is trash, you know, there are cases where it's like, okay, this paper should be in a very, like, you know, low impact factor, you know, it's not fraudulent. But this is not a real Nature paper sometimes. But it's like, statistically, or it's sophisticated enough that outsiders would think it was a Nature paper. And so that those are the kinds of things that's like, makes me always suspicious when people are like, well, it's in a high impact journal, because it's like, if it's not my field, I don't really know if it should be in a high impact journal, because a lot of times, it shouldn't. And later, it turns out that like, oh, the science paper had like massive problems. And well, it's because, you know, I mean, it's a glam journal, and, you know, glam PI's glam professors go to it, they can get away with stuff, they can push stuff,
they're like, at least two things that are are going to predict a really impressive package of studies are really impressive results. That is, you have really impressive results, or you made up really impressive results. And it's really hard to actually find very impressive results, because scientists have been doing science for a long time. And there's not a whole lot of new stuff to discover at least in you know, disciplines that aren't totally changing their methodological procedures these days. So you probably do get more fraudulent papers in the top tier, you get better papers, but you also get more fraudulent papers and more P hack papers. And if I recall, I do believe there was a I think there was an analysis of replicability in social psychology or maybe with Psychology Journals and impact factor. And I think I don't think the better ones were more rough, replicable, I'm not sure if the worst ones were more replicable. Or if it was just completely random. Some of the top ones were the lowest and replicability. And again, because like if you're gonna get published on these top tier journals, you have to be so perfect across so many studies. And the odds of that happening naturally are pretty low. And so people, you know, file drawers, some studies and they do some like little subtle P hacking thing. And then they get their paper in JPSP. And then, you know, five years later, people can't replicate the results because you got lucky once and file drawer and a bunch of other studies.
Yeah, yeah. Yeah. So let's talk about censorship, censorship in science. So you have a preprint scientific censorship by scientists, and it's got a bunch of authors, as we talked about, before we were recording, but, you know, this is an interesting topic. So you have a taxonomy in the paper, you know, hard versus soft what do you think of censorship?
Oh that paper, I thought we were talking about the taboos paper. That totally changes. That's okay though . I talk about the taboos paper and that paper, so it'll be fine. Yeah, hard versus soft? Yeah, well, we kind of define - So a lot. So when you think about scientific censorship, the sort of, I guess first thing that comes to mind is like government's trying to censor scientists are like you think about Galileo, like scientists suffering truly horrible consequences for their inquiry, but that's not what's going on in the US today, like a lot of the censorship, you see, is actually peers, causing their peers to self censor by like intimidation, and like threats of, you know, I might try to ostracize you or I might call you out on Twitter, you get a petition to get your paper retracted or get you fired, or all of these more, they're like, you know, not as big of a deal consequences. But there's still a big enough deal, that a lot of scientists are self censoring, and they avoid studying certain topics, they specifically avoid trying to find certain conclusions that aren't going to be, you know, friendly to the sort of left leaning academia. And, and yeah, so the soft censorship is just the sort of, like more subtle fear of consequences, where people avoid topics because they're afraid of like, social consequences that they're gonna get from their peers.
Yeah, and that, I feel like the soft censorship is actually okay, I don't actually know if this is true, but it seems like it's, it's a bigger deal now than it was, say, 40 or 50 years ago, because the viscosity of like, you know, it's like, information flows much faster, immediately, at the speed of light on social media, emails are being I mean, think about, like, you know, the 1970s, very few people had email, you want to call someone, you had to write a letter to them, maybe you got involved in a collaboration, that, you know, if there was Twitter, they'd be like, you can't collaborate with that person. They're bad. You know, I mean, I'm just, I'm just trying to think of like, how people the things that I have seen, like how people would have done some of the things that I've seen, do them them do back in the era before computers, and you know, conferences, and calling people up. And writing letters.
Yeah it would have been harder, you can't just like start a petition and post it on Twitter, and then get 1400 signatories in 48 hours, and intimidate somebody into doing whatever it is you want them to do. So I think like the fact that people can coordinate much easier on social media makes people do these kinds of, sort of like vigilante censorship moves, where they think they know what's right, and they want to get a paper retracted or a person fired, or they don't want somebody to get hired, or whatever it is, they can go out and get a big enough group to come together, that's going to put pressure on whatever it is the administration or the department chair, the editor of the journal. So that's happening. There's also this direct contact between scientists and everyday people that I think you didn't have before, which is also happening on social media. So now all these everyday people are seeing scientific findings and reacting to them. And they're not, you know, maybe trained and how you're supposed to interpret scientific findings. And you don't just leap to the worst possible application when you read a finding you don't like. And then the third one is, is just in this in this other paper, I see pretty big differences between men and women and among psychology professors and their endorsement of these kinds of behaviors. And women are much more supportive of, you know, retracting papers or punishing scholars or ostracizing scholars, if you don't like if you if you think there are scientific findings are like morally dubious?
Well, I think I saw the figure on social media, what was the proportion difference
Cory: of what
Men and women men versus women?
Oh, I have a lot of different questions. So like, for example, there was one, which was if pursuit of truth and social equity goals came into conflict with should scientists prioritize, and men say truth. And women say it's complicated. Those are the most common responses, if I do, should scholars be free to pursue research questions without fear of punishment? Like or should scholars be able to do that? Men say yes, women say it's complicated. If I say, imagine a scholar affords this controversial conclusion what should happen to him or her? Most people just say like nothing other than criticism, but women are more likely to support pretty much everything so they're more likely to say we should ostracize them. We should call them names on social media. We shouldn't hire them. Even if they meet typical standards. We shouldn't publish their papers even if they meet typical standards of merit at the journal those kinds of things. So, so the fact that women are now dominating academia at all levels, and especially in the social sciences, and they have these very different standards for, you know, what science should be doing, and they specifically, are more likely to think science should be pursuing, like, what is morally good, and, and then scientific findings that don't pursue something morally good. We don't need them, like, too risky, get rid of them. So I think those three things, the social media, and how quickly people can, you know, amass a crew of people to put pressure on institutions, the fact that scientists are interacting directly with people like on podcast, for example. And then the fact that we have more women in science are like a combined series of changes over the past 20 years that have created conditions where a lot of scientists are afraid to study certain topics, or say what they believe is empirically true. And we have data on this. Almost all scientists are reluctant to say what they believe is empirically true, at least some of the time.
So is this explaining some differences? When…. Is this this explaining some differences between disciplines, then in terms of censoriousness already, like, would we be able to see this? Like, say psychology versus engineering? I mean, I don't know what engineers would, I don't know, like nuclear engineers, like nuclear power, nuclear weapons, this is a whole thing DoD contracts, you know, I mean, it could be a whole thing.
There are a lot of correlated factors. So it is true. Like, for example, Fire has been keeping track of cases of scholars who have been targeted for their research or teaching. And scholars are more likely to be targeted, targeted in the social sciences and humanities than they are in the hard sciences. But there are two things happening there. One is the social sciences and humanities often touch on topics that people care about more, even if you might say, maybe they should care more about like the potential for nuclear war. They care about stuff like sexism and racism. But then also women are more likely to be employed in those disciplines. So it's, again, it's I don't know which one's more responsible for that. Probably both. I don't know which one, which one, if either is more than the other. But yeah, they're like, you know, you don't even hear that much in the social sciences, about gain of function research, for example, and concerns about that. It's all like, Could this research finding spread negative stereotypes about a vulnerable group of people? Like, that's a really big concern to some people in the social sciences. So
Yeah, I mean, and so this is, I mean, this is why I think viewpoint diversity and Jonathan Haidt been talking about it, because like, who is vulnerable? Like, let's say, like, we find out that evangelical Christians are less intelligent than the rest of the population, on average. I think most people, I think most people in academia would say they're not vulnerable, like they have power, you know, but if you -
Or they’d just say, Who cares? Like it's the truth, publish it.
Yeah. But Evan Jellicle Christians, you know, they're not. I mean, they're not. I mean, there are some in academia, they tend to keep a low profile. Yeah. But they're not excited about that, finding. You know, they'll argue with you. ‘Well, actually’, you know, there's going to be some reason you don't, I'm saying, like, Well, did you like check Calvinists? Maybe they're smarter, you know, people always like, I think, like a friend of my lawyer, friend. This is like, it's arguing the hypothetical where it's like, you can, you know, do a hypothetical counter argument, like kind of like, you know, logic chop your way. But, so I always feel like these ideas of harm reduction, it's a who whom question about who's the in group and out group, right?
Yeah. Or even like, dude, not all people consider this thing to be harm in the first place. Some people, like some people might say, you know, if you lower support for immigration, that's a harm other people would say, No, that's good. So yeah, like who's controlling the scientific decision making? And yeah, right now, it's mostly people on the left, I mean, plenty of people have argued for the viewpoint diversity, I'm not like persuaded that that would be particularly helpful. You just have more conflict, and probably more people trying to censor more stuff, like what I would prefer is, let's keep the activism and the science separate. And like, I even think it may be would be a good idea to try to split some scientific disciplines up into people who were like, No, I'm like a descriptive scientist, I say what is true, and then other people like I take science, and I tried to make the world a better place with that science. Because when the two are, like operating at the same time, you know, you can distort the truth because you don't find a particular finding to be, you know, socially helpful. And then that can undermine trust in the whole institution of science which is which is all already happening?
Yeah, I mean, it's happened a lot in the last 10 years, the polls are just crazy, but they're not shocking. And I, you know, I remember, some of the listeners will know Heather McDonald, a friend of mine, I remember in 2010 was 2010 2011 I just did in graduate school. And I remember she was saying that, you know, I can we trust these scientists and stuff like that. And I was just like, well, you know, they're all liberal. But, you know, they're not like, activists like in women's studies or something like that, you know? And
Who are you talking about? Some of them are activists.
Well they they are now.
I’m sure some of them were then. Probably
But the social norms were different in terms of well, you know, you know, it's like, now it's, it's basically like, you can't even be conservative.
People will just, yeah that - but they'll also just say out loud, like, I'll have reviewers just say, I'm worried how this finding could undermine liberals noble agenda to do X, and like, the fact that they're willing to say that in the review, that, that tells you something, they think it's totally legitimate to criticize a paper because they don't like the political implications of it. But whereas I think 10-15 years ago, if scientists had that concern, they at least tried to hide it.
They would mask it, like it would give a different rationality
Be a good scientist and lie to me.
Yeah, well, I mean, the issue is, like, you know, you do have to put a pretence on certain things sometimes just to maintain the vigor and the rigor of an institution. So let's say, you know, like, I will defend the masking in a way, because if you have the pretense of objectivity, you'll start you still going to draw in many more people who are interested in objectivity. So I can tell you for a fact, I know young scientists who are leaving the field, or leaving academia, and it's because of the culture of academia, not because of the science, because, you know, it's like, they have to look at themselves and like, think like, do I care about the science, or like to, like, endure these people candidly, you know, and it's because of this, like, massive politicalizarion that's happening, the constant activism, a lot of these people are not themselves even very conservative or anything like that. But there's no space for people to just, you know, let's just talk about the science and not, you know, get obsessed with like, you know, whatever, you know, DEI requirement for the department now, and all these other things. And, you know, oh there's another anti Israel petition, and I'm gonna look bad if I don't sign the anti Israel petition, like these sorts of things. They were on people, but what happens is, the people that aren't interested in that leave, and then all of a sudden, the department is all people into activism. They're like, mono culture, and then then what's going on? Like, then what happens? I don't know, in terms of, like, what the what the face of the field is, I mean, I said this on social media, so I'll say this on the podcast, because, whatever. You know, I have a friend and he just left academia. I'll tell you, he's a white male, okay. But he's been on faculty searches. And he was just one day he was like, he's like, I really think faculty searches. It's almost like dating white women don't like Asian men. Like, it's like, like, all like, he's just like, they're very consistently negative. And, you know, it's not really racism, but it's like, you know, a lot of these Asian guys, especially if they're from Asia, they don't know all this DEI stuff. They don't know what to say. They don't know. Like, they kind of try sometimes they've obviously been coached, but it's not natural. And so the culture, the culture, the culture, divergence is massive between those two groups is the way I would explain it. But like, you know, my friend was in a hard science. He was not even in a social science. But the demographics he's, he's saying, like the demographics if you look at the young faculty, the activists are reproducing themselves now. And people like him are leaving. And, you know, they put like, really strong vetoes on people that aren't enthusiastic enough about diversity and politics. And, you know, he's an he's an applied Life Sciences. Okay. You know, it's not like, he's in social science or anything like that. And so, you're, you're creating this group of people that basically agree on everything. And, you know, I don't see a way out of this equilibrium. And in terms of like, self censorship and soft self censorship, the problem with that is, it's soft, it's less visible. You know, I mean, like, you can't go to granting -
You can't, you often can't see it at all. And so it's easy for people to not deny that it's even happening, which is why I've been trying to collect the data on it to be like, Look, it's happening people, but I have mixed feelings about it like I you know, I mentor so younger folks, and the ones that aren't super woke, they already when they're young, they're already having a hard time because they don't necessarily agree with everything that's being said in a seminar. And if they open their mouth, then they become, they're immediately ostracized from everyone else. And then when I'm trying to advise people, like, should you stay in academia? Should you try to get a job, you know, at a university. On the one hand, I don't want to contribute to the problem, and people becoming even more like the population of academics becoming even more activist. But on the other hand, it is it's a, it's a pretty unpleasant place to be. If you're not, you know, in that club, I'm not in that club. And like, when I go to conferences, I'm like, on the verge of a panic attack the whole time. And I just like, mostly look for a couple of people that I'm friends with, talk to them, and then get the hell out of there. Because like, I know, a lot of them hate me. So part of me is like, well, maybe we should just let them have it. And then all of these other institutions are popping up, you know, you're gonna have think tanks and you know, different kinds of startups and organizations that are like, hey, if we can get the funding, we can do science, too. We don't need to be at a university to do it. And then we can get rid of all this extra bullshit that goes along with it. So maybe that will happen. That would be quite interesting. And then the academics will have some competition to deal with
Yeah, yeah. I mean, you know, it is a very highly funded institution, though. I mean, the academia as a whole, I mean, for now. So, you know, it's really hard. They have students paying a lot of money
Razib: For now, yeah, we'll see how much longer this goes on. So I do think a little bit of a little bit of a monoculture is one of the reasons that they're like, being a little - they're getting a little taken aback by, you know, people on the right, going at the institution, because they're like, we're sacrosanct, you know, you know, they expected the - to last forever and ever, because money grows on trees. And, you know, it's a little bit of a bubble and the bubble existed partly to cultivate independence and intellectual inquiry. But if that's not happening, no one has the buy in, you know, it's not like scientists have ever been like the population there, like, like a lot, they've always been more atheist, they've always been more liberal, et cetera, et cetera. But you know, it was a price of the people that were not of that persuasion, they were willing to pay to fund this stuff, because science is important. But now if you're not even doing, you know, science, that's important. Science is objective science is producing real results, then the whole, the whole enterprise is going to fail. So let's talk about adversarial collaborations. So you have it's a preprint, or no, it's, it's a paper, it's out.
I have a few papers on on adversarial collaborations. But yeah, essentially, for those of you who aren't aware of the sort of typical norms of publishing scientific papers, well, first, like, the big incentive is like, you have to say something new and exciting that differentiates you from everyone who came before you. And then you have to convince, you know, employers that you have this novel idea, and then that becomes your identity. And then you spend the rest of your career defending it a lot of the time, not all the time. And so what typically happens with disagreement is, you know, a scholar forwards some theory, and they become sort of, you know, academia famous for that theory, another scholar comes along and says, I have this new one that's totally different from that guy. And here's how, and here's my data. And then they just debate each other for years or decades. And it almost never happens that these two decide to work together and actually get to the bottom of what's going on. And they both just continue to pump out more and more data that supposedly supports their side of the supposedly contradictory theories. And it just creates all this chaos in the scientific literature where there are, I would guess, maybe I probably shouldn't put a number on it. But tons of academic papers have other academic papers contradicting them. And, and we almost never get to the bottom of them, or at least the people who are affording these papers almost never get to the bottom of them. And then like, sometimes they die, and we kind of figure it out. But the adversarial collaboration in ideas just, you know, we need to hold scientists to a higher standard, and say, if you're going to be publishing research that contradicts another scholar, the first thing you should do is, you know, call that person up and say, Hey, this is so odd, like, let's figure what's going on here and work together, and then forward, a more perspective that actually accounts for all of the data. And typically, what you find and this is what I've been finding, and all of the adversary collaborations I've been doing is you end up with just like more nuanced claims, you know, neither is neither sides big, broad, like super impressive claim is right 100% of the time. It's like, this person's right in this small way, and this person's right in this small way. And so a lot of these apparent contradictions aren't even contradictions. It's just both scholars are incentivized to frame their findings as excessively broad and important.
Well, I mean, yeah, you It's interesting. You're talking about, like, these contradictory findings or whatever in the literature, supposedly. And when someone says studies show or there's a study, I mean, it's kind of a punchline now, because you just find the opposite study, you know, and it's so it like, lacks. You know, I mean, if you're in the field, you kind of know, or you can, like, you know, look at, like, who's being cited and how but even then that's not necessarily, you know, valid. And so it basically, like, just undermines the whole purpose of having the academic literature if it's just, like, filled with all these people like talking at cross purposes, no one knows what's going on, you know?
Yeah, people like trust the science like, well, which science ? Yeah, because there's a lot of science that says the opposite thing too. Yeah, that's something that's funny, like, even scientists will say that and I'm like, You do not like you more than anyone understand what science looks like. It's a mess. It's like, we have discovered some things, but a lot of stuff is a mess.
It's a form of it's a form of Gell-Mann Amnesia, you know, that thing you like you read a paper, and it's in your field, and everything is wrong in a journal, but then you read about, like, some international thing or another field, you're like, oh, that's, I didn't know that. You know and so it’s like, within science. It's like, oh, yeah, who cares if that's in PNAS, or, you know, whatever. This is why it's wrong. And like, these stats are shady, oh, they didn't change their parameter values in the model. And all of this, like stuff just like, jumps out at you as like, suspicious. And then you read this, I read an awesome paper about, you know, the distribution of, you know, wildebeest in Africa. And Iike the maps are very beautiful. And I know so much more about why there's so many of them Tanzania now. But like, I don't really know if I know that because like, they're telling a good story. That's what you're taught, right? You're told, told to tell a good story. So I don't know. I mean, I'm not a postmodernist, obviously, I think knowledge and truth can be attainable. I think replication using a common language is actually a big deal. In terms of like, economists have a certain formal mathematical language, and so they can talk to each other really fast. And they can double check and cross reference each other, I think some of the problems in the more descriptive fields is like, Well, what do you do to like, cross check their interpretation, you know, or like, you know, are you gonna go live among this tribe yourself? You know, because, like, the way they the way they like, summarize their findings, sometimes in some parts of cultural anthropology is just, like, so impressionistic, that I don't even know what to say about this, you know, I mean, they're not like measuring like, you know, how many sticks they have, like, you know, we're whatever spears, you know, and so, you know, the method, the language, like, a lot of these things can make it clear. But I think one of the things we're getting at here with adversarial collaborations, you know, this idea, what are we? What are we here for, like, what are we actually doing? And I do worry that sometimes science so I, I saw this guy on social media, as, as we do, we see people on social media. And, you know, he's, he's a statistical geneticist, I'm not gonna lie, I'm not a big fan of his for various reasons. I'm not a big fan of his mentor. They're bullies, you know, whatever. But I, I pay attention to him, because I know he's a bully. I know, he's from a bullies lab. Like, there's bully behavior that they all do. And he said, The reason I am in the field, in the field of statistical genetics is because of the community. And I'm just like, No, no, no, no, no, no, no, no, that should not be why you're in a field of science, because of the community. If you want to, if you want to be part of a community, go join an NGO, go join a church, go join a lodge. Yeah, like, just like, there are communities out there. Like, the reason that you are studying statistical genetics is to know statistical genetics. It's not to go to the conference and go to the parties or, you know, be like, supported in whatever issue you have in your life. And so maybe this is gendered, he's obviously I mean, I'm using a he, him pronoun for him. It's a dude, but the way he was speaking, I just want to scream because I was like, this is the problem. This is the problem. You're forgetting why you're doing this. You're forgetting why you're foregoing a private sector job and spending your life, you know, submitting papers to journals that very few people read, you know, you're doing it because you care about the truth, you know?
Well, I mean, ideally, I think like, I think most scientists will tell you that they're in science to pursue truth, and that's what they care about. They care about science, they care about the truth. But how much of that is actually true? I mean, science is sort of like status system and people want to get status within their community. They want to get cited and they want to get in the prestigious journals and they want to win awards and like, do they care more? about publishing true information? Or do they care more about publishing fancy papers that get them a lot of attention and praise? Like? I don't know, I would, I would guess it's at least 50/50. Right? So, yeah, well, that's part of the adversarial collaboration is to try to. And part of it is to kind of try to shift the incentives. I want people to respect people who participate in adversarial collaborations because it demonstrates a commitment to truth and a willingness to update your beliefs, even if it contradicts something you published before. But also, my long term hope, is that if these became super normal, people were doing them all the time. And it became normal for scientists to update their beliefs and say, you know, I published this paper before, and I made this broad claim it was a little bit too broad. Now, I believe that in this context, it's true, but not over here. And over here, this other thing is true. And instead of that being like, reputationally damaging, we could be like, good job, like, we're proud of you for changing your mind. Like if people could get status for that, instead of for digging in their heels and being like my theory is the best theory. And it's the right one, and everyone who disagree with me is a fucking idiot. Sorry, am I allowed to say the F word?
Razib: yeah, you are
an idiot! Okay? Instead of like giving those people of all of the awards, give the awards to the people who like make these contributions to science, and they increase over time, and they changed their minds when they should. And then if that really became the thing, we like, rewarded scientists, for more people would do it, they don't do it. Because people try to shame each other, when they get something wrong. They try to humiliate each other, like when a paper fails to replicate, and like, maybe they should, in cases of you probably P hacked. But if it was just a sincere mistake, or something, then maybe like, people shouldn't be so judgy of those people. And that's just not the way things are. Right.
Yeah. So, um, you know, you have an empirical example of like, is how incentives matter? This is this is a repeated issue that has happened in science. So the President, I think he's a president of Stanford just resigned because of some issues with fraud. And, you know, this goes back 20 years, he's a big dude. He's a big, he's a very, very influential scholar. But it goes, you know, there's problems in his lab repeatedly. And one of the issues that came up, and this is an issue in fraud that has come up repeatedly, is a lot of
I thought they said he didn't commit fraud, but he made like, a really bad, I think that there
I think there is fraud in some of the papers of his juniors, but that he didn't bring it to light or something, you know, what I'm saying? Like he himself, he himself, I'm actually pretty sure he himself probably did not commit any fraud, and did not knowingly sign on to fraud. So, you know, just cover myself from libel, you know, there are allegations of fraud by his - against people who work for him. But what he did, which I this happens, this has happened before, in these labs, as he has a system where it's basically sink or swim for the postdocs, and the postdocs that produce the good results are the postdocs that go up and get the good the recommendations, and the postdoc still produce good results are out. Alright, so what are the incentives?
I've heard of, I've heard of mentors like that, where they praise the person who gets these totally implausibly significant findings. And then like they’re their star student. And then meanwhile, these other ones who are just like, working hard and doing a lot of studies, but they're not getting significant results are like neglected. And yeah, I think it's not a good environment, if what you're if what you care about is, you know, figuring out, what’s true
And this has happened, I mean, just, you know, like in life science, biomedical science, or some famous cases of like, manufactured figures, and, you know, stuff that was touched up, and all this, you know, all of these issues have a come out of labs that are extremely competitive. And the incentive then is to produce by any means possible, right. And so I think, I think that's something I don't know what the solution is, because you want people to be competitive, you want people to do the best they can, because otherwise they're not going to do anything, you know, and there are different academic cultures in different countries. I have a friend who is from Italy, and she didn't want to go, she has like a job offer in Italy, but she want to go back. Because they also told her like, Look, if you come back to Italy, you're not gonna work that much. Like literally, like, we're gonna get mad at you if you work too much.
What does that mean?
They don't pay them that much. So they don't work that much. And so the culture is -
They get mad at you. If you work to much?I think a an assistant professor in England for a couple years. And they didn't they didn't expect you to work that much. But they wouldn't get mad at you if you worked a lot.
Well, because the issue is the issue is you because they said look, we know we've looked at your publication record. It's impressive. If you come over and just start keep publishing because you work a lot, then we're going to look bad. You know, and so they were just like, if you willing to like,
Cory: that’s bizarre
Yeah, no, there was straight up about it. Like, I totally find it plausible because I have a friend who is in the National Police in the Carabinieri. And he got the same actually feedback because he's a very diligent worker. Yeah, he got he got a call from Rome, Roma. And they were just like, the guy was like, Look, you do a good job. But you know, people have been like talking about you that, you know, you're out there working, solving crimes. I mean, I don't know what, what I mean, like, imagine the town. Yeah, it was a weird stuff. And then like, I still remember this. He said,
There’s going to libel case against Italy.
No But he said, Look,
,like, Don't exert yourself, you know, he's like, like, I've been, I've been, I've been police officer for 40 years, the only time I exerted myself, at work is when I'm on the toilet. You know, I'll be like, get some roughage into your system, bro. But yeah, like, there's weird things like that. And that's just a culture. It's a culture, like in the United States, the culture in academia is -
So don’t get murdered in Italy, is what you're saying?
Well, I mean, or, like, you know, as, what's her name? I forget her name now, but, you know, don't be adjacent to someone that murders someone or you might get in prison. Right? You know, I mean, because, you know, you know, the, the justice system. But here in the United States, we have, like, I think, an opposite problem, where people work really hard and to be candid. Like, I don't understand why some people at research tear universities are working so hard on research, because,
Well, I think, they like it like research, I, well, you know, different professors like being in academia for different reasons, some really love to teach a lot, some love to mentor, even some love admin, but like, I think for a lot of people, the research is their favorite part. And so they're willing to put in overtime in that domain. But, ya know, it is true that expectations are really quite high in the US, and I known, you know, grad students who will pull all nighters to get grants in and then you have to ask yourself, Okay, maybe we're maybe we're overdoing it a little bit. But
Well so Richard, I want to I want to I want to actually like end, because, like, with the ‘The Evolution of Relentless Badassery’ because I like the title.
Cory: Thank you, that was all me
That does seem like you. But um, you know, it lacks dignity. No, but. But I do want to say, so, you know, Dick Hanania. Online, shit stirred. He has a funny story.
Cory: Do people call him Dick?
I'm trying to make Dick happen. But he has a funny story. And I've heard this story before, like this sort of story, like not the exact details, but the general gist, someone at a mid tier research one university, and cancelled academic and, or, you know, and so one of the graduate students was like, she was like, you know, I like you, but like, we can't be seen talking to you, because that will jeopardize our career opportunities. And he he was thinking,
Cory: talking to him?
Yes. Well, yes. Talking to him. And the guy who was counseled was thinking, he didn't have the heart to tell her that. She's at a mid tier r1. University. She has no career opportunities.
Mid tier r1 is okay, it's okay. Yeah,
I mean, it depends on it depends on what your department is. I don't know the deal. It has to be social sciences, if the guy was talking to Richard. But the point is, there are universities where the faculty went to Harvard. You know, at the graduate students, okay. There'll be lucky. You, I mean, except for a few exceptions, most of the graduate students, if they stay in academia are going to be at a lower, considerably lower rank. And so this whole system is a little bit of a Ponzi scheme in a way.
Cory: It’s true
And I don't know, but I've heard stuff like that from professors where they're, like, you know, like, I don't even know why I have graduate students, because like, they can't, they're not competitive with people, you know? And so like, what's going on here? What am I doing, you know, but whatever. So ‘evolution of relentless badassery’ Like, what's, what's the deal with this? So was this a bet? Did you have a bet to write a paper with this title you know
No, it was we were invited to write a chapter for the book is called “The Godfather and Philosophy”. And I had this theory about like, error management and how it applies to blaming people and essentially, like, if you if you don't blame someone when you should have then you're a sucker. You know, you're like the idiot who you know doesn't get revenge and you become a target. But if you punish people when no one deserves to be punished, then you're a badass like Don Corleone from the Godfather and like people are scared of you. And so we're essentially arguing that people should have evolved a bias toward punishing and blaming people, when maybe they shouldn't. And that's the Don Corleone principle. And then we talk about the evolution of Michael as a relentless badass, and how he comes to accept this principle over the course of the Godfather series.
Yeah, I mean, I do think the Godfather is I mean, I think, though, especially the first movie first couple movies, I think it's very interesting, because there's, I don't know if it's purposeful, if it’s just because the directing, there's a lot of like social and cultural depth to them in terms of what they illustrate about social and cultural dynamics. I think one of the most interesting ones to me is that scene where a man's daughter has been raped, and you know, you know the scene and he comes to the godfather. And, you know, he just like, he like goes on this, like, little bit of a rant that, you know, you're a respectable person. And normally, you wouldn't give me the time of day. And now, like, you interrupt me, on the day of my daughter's wedding, because you want me to commit you want you want you want violence committed against this, man,
Cory: you don't know of me friendship.
Yeah, there was just like, the whole thing was like, well, there's like, some serious stuff going on here, you know, but it's like, legit, true. And then of course, you know, he's like, okay, like, it'll be done. Like, you know, leave or whatever. And the guy is, like, Thank you Godfather. And so I think it's interesting. I think it's interesting, because, you know, there's a whole theory of these sorts of behaviors, where it's actually just like, what the early state was, like, you know, and, and the mafia does come out of, you know, Paris state institutions, you know, state was weak, the localities were weak in parts of southern Italy. And the mafia, you know, emerged, as you know, in the in the anarchy that ensued, to take over state like roles. And so I think it's interesting films like The Godfather, yes, they're entertaining. But I think one reason that they appeal to us is there are a lot of conventional dynamics operative in a criminal context, you know.
Yeah when you can't rely on institutions to handle those kinds of things, which for most of human history, there were no institutions to depend upon, then you would want to be loyal to someone who could, you know, get your justice for you. And, you know, give them status so that they'll be on your side.
Yeah. And I do think, you know, with the Pinker idea of ‘the decline of violence’, which is like an empirical observation, right. It's interesting that people that are badasses are kind of I'm saying this word - those people like, it's, I think we're much more ambivalent about them now. You know, because they can cause a lot of damage. And also, like, you know, you don't want you don't, you know, like, okay, let's say, let's say your friend is bullied. You know, I don't know the details today. But it's like, I think like - violent retribution on your part is going to cause problems for you, Like the authorities will come after you because what they don't want, they don't want vendettas, you know, cycles of vendettas and stuff like that. So this is a different social equilibrium. So the evolution of this characteristics culturally, biologically, cognitively, could be a holdover from this earlier time when it was necessary. Just like, you know, you know, if you have like total transparency in your institutions, you don't need constant cheating detection.
So I mean, this, these are, these are things that are going to that are going to change. And as the United States becomes less and less violent, let's hope it stays, you know, not super violent. And, you know, we're gonna get until anything big wars, I think, this sort of Michael type character, we'll seem more and more strange. And, you know, when you read the Iliad, when you read the Iliad, like, it's just, it's like, In the beginning Achilles. Yeah, he's having a dispute, because he had his sex slave taken away. You know, like, thinking about what's going on here, you know, he killed, he killed, he killed her brother, and he killed her father, and enslaved her, you know? And she was his, like, you know, sex slave. And Agamemnon was mad, and he took it off, you know, and like, that is like the cause that is like that the injustice is that, you know, and it's like, we look to the ancient Greeks, and they totally, totally, you know, like, how dare you sir? I didn't even buy her. I actually stole her after killing her family, you know? I mean, we're laughing because it's like absurd, but I do think like these sorts of commentaries are interesting because -
But I do think like what The Godfather indicates like it's, I think you're right, like it'll seem you know, even when I Talk about this, this sort of like theory, like, you know, we, you know, we respect people who are punitive and vindictive, because we're sort of afraid of them. And you know, we don't want to cross them. But, you know, even when I talk about that people are like, no, like, we hate people who are really, you know, hotheads and these kinds of people, but I think, yeah, it's a, it's like a product of our culture where those temperaments or personality traits are no longer valuable. But you could imagine in a world where all the sudden, we didn't have the institutions we had, like, how quickly would it go right back to that, I think, probably in almost no time at all, like, I don't know if you've ever been in a sort of, like, emergency situation, but I've been like at a gas station when a hurricane was hitting. And like, all civility goes out the window. Like, everyone's like every man for himself. Like we're not polite. We're not waiting in lines, there's no norms are no rules. It's just, you know, get what you can and get out. So, I think in circumstances, these, like, these psychological tendencies are there. It's just like, you need the right situation to bring them out.
Yeah, right. Right situation. And like, you know, the groups that adapt to the new situations will also do better, you know, institutionally, you know, in terms of when there's anarchy. You know, I mean,
You’ve got to find your leader, you gotta get with your group, you got to coordinate. Sometimes you got to be violent.
Yeah. So, what do you what do you, what are you looking to for the future as we wind down now? I'm just curious, like, what's, what's your, what's your biggest project that you're focused on right now?
Um, I've got a lot one that I like, a lot that I'm finishing up right now looks at how politicisation undermines trust towards institutions. And the cool thing we see is that both people on the left and right, don't trust institutions that are politicized, even if they're on the left or right, so even liberals hate when liberal institutions get political. And even conservatives hate when conservative institutions get political. And we see this spreads out to like skepticism toward them willingness to support them financially willingness to defer to their expertise, it's just negative across the board. And then we even see like, if you have a sub institution, so our sub organization, so we have like a fake organization called like Economics Professors of America, and if they have like a Democrat or Republican Speaker at their convention, not only do people trust Economics Professors of America, less, they trust all economists less, and they don't want to defer to economists in general. So like, you can have this one little group get involved in politics, and they sort of represent the whole broader professional group and screw it up for everyone. So yeah, I didn't expect to see such big effects. Even when it was an in group institution, I expected to definitely see it when it's an out group institution, but nobody likes it, which I guess then my next project has to be like, Why do organizations and institutions constantly get involved in politics, when it's harming their reputation with the public in a, in a pretty big way So far as I can tell?
Well, I mean, isn't that that? But that’s the iron law of institutions, right. Where it’s like, in group institutional status, is different than the institutional effectiveness and the institution status. Right. So I mean, I mean, I guess, like, I'll just make an observation. As I'm closing, I was like, you know, I've been, like, I have been warning scientists that, you know, going, like, you know, mask off, like, you know, we want to put, you know, our political enemies, you know, like, you know, on the street or whatever, you know, I mean, sometimes more extreme, it can make you seem badass to your in group, you know, but other people are watching. Yeah, then the affected. So, for example, like, let's talk about public health during the pandemic, like they start to be a bidding war of like, how, like, hardcore, you were, I think, in a lot of ways. And then there was also during BLM, all of a sudden, the mask and the public gathering, opposers just flipped for political reasons. And, you know, they signed a petition, like, we believe the social justice, blah, blah, blah, all this stuff, right? Well, I mean, the ones who didn't sign up, they were they were in trouble. They were being attacked within the institution. But actually, everyone knows that that totally destroyed. The broader the broader public acceptance of that whole field, you know, in terms of like, no conservative, was ever going to take anything they said seriously ever again. Because, you know, when it came to like a political signal, they were totally like as one,
right? Yeah, there's the question of like, so take a social psychologist, for example, they could do some pro far lefty thing, and maybe they undermine the credibility of social psychology and among the public, maybe even among the left leaning public, but do they get a lot of credit among the left leaning social psychologists, which is their most immediate in group, and that's all they care about. They don't care if they undermine social psychology, they care about that their own self rising through the ranks of their, like, smaller. But I do wonder, so like, when I presented this at a talk, I have the slide, if you use it, you lose it. Like, I do wonder if people use their positions of authority within these institutions that have power and influence in order to push a political agenda and sure, that gets them status within their community. But in so doing that, they undermine the status of their community. And so the group of people that they want to have power, like, among that group, might itself fall down a few tiers. And so the thing that you were trying to accomplish is actually not worth as much as it was before you did the thing that you did. So it's like, is it actually a rational move? I don't know. But But that's also a relevant question, of course, like, what are people trying to accomplish? And what are they actually accomplishing?
I think this is, I think it's a coordination problem. It's a coordination problem, because if everyone's on the same page, or like trying to stay above board, the institution has high status, you know, and you gain out of that, but, I mean, this is like an economic game, whereas, you know, there are societies where, you know, people are competing, and then everyone is, you know, everyone is poorer. Right? So it's like this, there's, you know, we could do like Game Theory, Nash equilibrium, whatever, there's all sorts of things going on here. But, um, yes, what you're alluding to is, the pie gets so small, that, that the winner is poorer than they would have been if they were the loser in the within institution game.
Right. And is that a better outcome for them? Because they only care about their relative status compared to their immediate peers? Or, you know, I guess if they could somehow climb the status hierarchy without doing that and taking the institution's reputation that would be preferable, I would think, but maybe it's not possible. I don't know. But yeah, so maybe that's maybe that's where I'll go in the future. If any of your listeners have ideas or interesting questions, please. I don't know you can email me or something.
Yeah, I'll put I'll put I'll put all the all the contact info will be in the show notes. So yeah, I mean, you know, you have a lot of different interests. It's cool. I always enjoy talking to you have not talked to you as much as I should. So I'm glad that I got to catch up. Maybe we'll see each other IRL at some point at a conference or something like that I'm you know, I'm flying around the country doing bizdev guys so you know, if you got if you have money I'm raising for my startup and you know, I will take selfie with you if that's what needs to be done.
I’m if you have money, I’m looking for it st the adversarial collaboration project too to give it to me instead of Razib
Well, okay. She wants a grant I'm offering equity that might return
I’ll do selfies. I'll buy you an old fashion.
All right. So this is this is a this a - now we're getting into a competition and I think like I think a selfie with you would be more valuable
I’m cuter I’m sorry.
Yeah, yeah, I think a selfie with you would be more valuable so Okay, I think we're gonna end it here because we don't want the institutions reputation to drop any further.
Yeah. Wise.
All right. All right. Take it easy.
Thanks for having me.
Is this podcast for kids? Thisis my favorite podcast
