1:05AM Jun 20, 2019
David Epstein. David is a writer and researcher extraordinary and the author of two great books. His second range is out today and I highly recommend it. We discuss the pros and cons of both the generalist and specialist mindsets in detail and go down many interesting trails along the way. Please enjoy our conversation. David, thanks for being here with me, we've got a whole ton to discuss covering your two books, a lot of your other writing, I know you're interested in a million things, we're going to go all over the place, since the antecedents for your latest book range really are buried in your first book, the sports gene, I thought it'd be fun to start there. And really focus on what you discovered in the first books research that led you to get so interested in this idea of specialization versus kind of broader set of interests. So maybe talk about that what you uncovered the kernel that you uncovered in the sports gene that's led to this later research.
First of all, it was I noticed that some countries that were sort of turning around to their national sport programs like at the Olympics had embraced what they called talent transfer programs, which was taking adults who've been practicing with a history of broad history of sports participation in bunch of different sports, were trying to make national teams weren't making in whatever they were training in and said, Hey, why don't we let you try these other sports before we kick you out of the developmental pipeline, and all the countries that we're doing that like Australia, the UK, not only gold medalists, but world record holders out of adults who had never participated in their sport at the Olympics prior to when they won the gold medal. And this sort of went so contrary to the 10,000 hour rule thinking right where you have to, you'll get behind if you switch, that I started thinking more deeply about that. And eventually, when the book came out, obviously, I critiques the 10,000 hour rule in the sports gene. And I got invited the MIT Sloan sports analytics conference to debate Malcolm Gladwell 10,000 hours versus the sports team. It's on YouTube, even though we have some common ground for sure. And I'd never met him before. Very clever, and I didn't wanna get embarrassed. And so fear motivated researching. I anticipated what I thought he would argue one of his arguments, I figured would be that head start in deliberate practice, or highly technical, focused practice, and whatever you're going to do is like an insurmountable advantage. And so I said, Okay, if that's the hypothesis, let me go look in the research and see what we know about the development of athletes who go on to become elite from childhood. And it turned out to be very much the opposite. So the athletes who go on to become elite, have early on what's called a sampling period, they do a wide range of sports, they gain a breadth of general skills, they learn about their own abilities, their own interest, and they systematically delay specialization later than their peers. So I go to this debate with Malcolm, and I frame this as the Roger versus Tiger problem. Tiger Woods is the most famous maybe ever developmental story, walking in six months golfing in 10 months. And Roger Federer where we don't really know that story, so much played, like a dozen different sports as a kid, mother was a tennis coach refused to coach him, forced him to continue playing badminton, basketball, soccer long after his peers had specialized. And he turned out okay, and so my question was, which one of these is the typical developmental model, and it turns out that in almost all sports, it's the Roger model, not the tiger model. And so that became the afterword of the sports gene. And furthermore, golf is a really bad model of almost all things that humans want to learn. And so it sort of grew out of the sports gene. And then out of this debate that I prepped for with
Malcolm, so in both books you cover a ton of different ideas will cover a lot of them today, I'm always interested in books like that there's not just one core idea. It's sort of a collection of things that often afterwards, the public will glom onto a couple core ideas from the book. So before we leave the sports gene, I would just love to hear your perspective on what the things are that the public really paid attention to. And then maybe what the things are that maybe they tended to gloss over what you most enjoyed about the first book,
I was not good at predicting what people would glom on to necessarily so I thought, the chapters about race and about gender would be by far the topics of discussion. And in fact, it was the 10,000 hour rule. The sex and gender chapter is actually kind of coming back right now because of debates in sports about how to separate the female and male classifications and sports for people who have differences of sex development or transgender athletes. But it was by far the 10,000 hour rule stuff. So the 10,000 hour rule is explained in sports gene is based on a study of 30 violinists who were so highly prescreened, they were already in a world famous Music Academy. This is a problem that statisticians call restriction of range, you restricted your subject range based on something correlated to dependent variable. And when that happens, you can have all kinds of weird results. So I liken it to if I were going to do a study what causes basketball skill, used only centers in the NBA in my subject sample and said, Oh, okay, they all practice a lot. Therefore, only practice got them where they are not practice plus being seven feet tall, because I've squashed that range. And in fact, I've done the analytics on this. And some years, there's a really high positive correlation between height in the American population and points scored in the NBA. But if you restrict your range to only NBA players, it turns negative. So if you are scientists to do that, studying NBA players, you give parents the advice to have short children, for them to score points the NBA if you don't recognize the restriction of range. And so the other thing that 10,000 hours was that so the top 10, violinists in the school had practiced on average 10,000 hours by age 20. But there was tremendous variability, tremendous, his average kind of squashed all that variability. And so I just explained the background of this study, where really came from what you could really infer from it looked at studies that tracked development in a more systematic way, and in a more systematic way. And larger music studies and saw that that wasn't the case. In larger studies, people still had sampling periods and things like that, and told this story in the book called The Tale of Two high jumpers, where one high jumper basically spends 20,000 hours practicing and one like randomly discovers that he has this talent, and they end up competing against each other. And that became like one of the most popular stories, this story that I used to display the variability around the 10,000 hour rule. So that one, and the last chapter about the Finnish cross country skier who has genetic mutation that basically causes his blood to be naturally like what Lance Armstrong's was with technology, those two things, people really, really caught fire in a way that I didn't expect. Some of the things that were the closest to me, there was a chapter about sensitivity to pain and the extent to which that is modified by your nature and your environment. And in that chapter, there is a section about sudden cardiac death and explains the sudden death of my own training partner at the end of a race. And that is the reason why I got into journalism to write about sudden cardiac death and athletes. I was GLG grad student living in a tent in the Arctic, you know, I was going down a different path. And I decided to merge my interest in sports and science and try to write about sudden cardiac death and athletes. And in fact, that's sort of what led me to become the science writer at Sports Illustrated. But that was some of the stuff closest to my heart. And that is most traumatic to me and went totally under the radar in the book. So shows my bias of what like I focused on
in the process from and maybe you could touch on some of the what you felt were going to be the popular chapters in the race and gender thing in terms of kind of this nature versus nurture. argument from when you started doing the research for that book to when you ended? What was your move on that spectrum? And be curious what you what your kind of your priors were of the nature nurture debate, in sports, specifically, I guess, but we're going to talk about that more generally to in skills, and how much the needle moved during the process of writing that first book.
Yeah, it moved a lot from me, to me, the answer ultimately becomes things that I thought that were completely nature, like the reflexes, it takes a 200 mile per hour fastball turn out not to be that at all, to be totally learned, and other things that I thought were completely volitional, the will to train turned out to have these incredible genetic components. And so it turned out most of the things I wrote about were contrary to what my intuition was. And I really want to go back and read my book proposal because the books that I likened my book to in the proposal were like outliers, and the talent code and talent is overrated. And they in the end, my book turned out being said exactly in opposition to those books. I think there's a lot more nuanced than that. The seat of that book was a 7000 word article I wrote for Sports Illustrated about genetics, that basically interpreted a lack of certain types of evidence in genetics for evidence of absence, that genetics matter, right. So this is like a little secret. And nobody's really called me on an Esports gene, which is that I cite one of my own Sports Illustrated articles as being wrong in my book, which is scary, because you can spend a couple months and it's not like there was a fact wrong, right? He passed through fact checking the published papers, were there the people with the PhDs who did them were telling the fact checker, but the problem was, they were making inferences from their data that their data did not support. And once I had a year, so my process for writing a book in the first year, I don't write try to read 10 journal articles a day, every day for the first year. And after I did that, I realized that they were making conclusions they could not make with their data, and they were ignoring other data. And so the book ended up focusing, I think it was still very much balancing nature and nurture, but focusing a lot more on nature than any other book had. Because I started to realize that evidence had just not been discussed and not kind of plumbed with the same depth
that a lot of the nurture evidence had been, we were talking before we hit record about the interesting analogy between like health and sports and investing, and this constant desire to intervene, everyone reads these books and wants to know, okay, like, what's the couple sentence summary of what this means I should do for myself, for my kids from my friends, you know, whatever it might be. And obviously, there's always more nuanced than that, right? And people want the snack versus the full meal, which I get one interesting idea in the sports scene is maybe a simple practical outcome of your research is to practice what you're naturally good at. Maybe talk a little bit about that, that blends the practice and the nature, that's true. So I'd say right after the 10,000
hour, chapter, one of like, the next tier of the ones that were the most popular was one about train ability that suggested this, like anyone who's ever been in a training group or gone on certain, like fad diet, or whatever, they noticed that and I mean, I noticed this in my college track training, we're all doing the same things. And in many cases getting more different, not more of the same. And it turns out that that's one of the same lessons like we've learned about for medical genetics, you might need three Tylenol I need one because of your genes involved in acetaminophen metabolism are different from mine, or maybe it just doesn't work for you at all. The same things basically true about the medicine and exercise or training, the American College of Sports Medicine slogan is exercises medicine, I think that's very apt for that reason, because no two people have the same exact response to that given stimulus. So what you really want to do is find what fits best for you, where do you get the best response. And not only that, but in some cases, we always judge talent based on what we see right now. And I think one of the revelations in the sports team for me was that in some cases, like the more important talent is train ability. And in some cases, your ability to improve with training is uncorrelated from how good you are a baseline. So it's a really different. So in one of these studies, where people were being trained for endurance, the researcher said, okay, on day one, here are 10, most talented people put everyone identical training, and missed 100% of the people who ended this study looking the most talented, because we have this differential response to training. And so I think in the book, one, more people are screened out of elite performance now in sports by either their nature or their nurture than ever before, because it's so competitive. And I think one, you need a sampling period to figure out where you fit, you want to delay specialization, because when we pick before people are, go through puberty, especially the more likely put the wrong person in the wrong slot. And also, then you need to be able to experiment to find the best training environment for yourself. So I think when we are directing people to particular sports before they have a chance to learn about themselves and their own abilities, we are minimizing the chance that they find the right fit. And that's why these talent transfer programs that like Australia and the UK, and to some extent China started, have produced Olympic champions from people who have just started their sport is because there is so much room still to gain with fitting people into the right sport. But they won't do that for themselves. Because the feeling is you'll be behind if you switch. And so just giving them that liberty to go exploring, using what they've learned about themselves and see if they can fit into another sport, has produced Olympic champions, even given how competitive things are today,
you mentioned something in your own research process, which is fascinating, which this idea of reading 10 journal articles every day for a year,
a lot of journalists make it every day. But I mean, most days Yeah,
I think most great books on creativity start with this process. Like it's just a collection exercise early on without judgment or bias, just like get information. I would love to hear a little bit more about that process. I'm surprised that there are that many journal articles to read on a topic like this. So talk through that, is there anything you're looking for? I think this could stand as analogy for anyone trying to get into anything right that they want to be in this collection phase and cycle exploration phase early on any special insights from that period of your research?
It's interesting, you mentioned that because that you were surprised there are that many journal articles. The fact is sometimes I end up down rabbit holes that turn out I surface like a week later and say How the hell did I ever think that that was going to be useful. And I used to really chide myself for that, because it's so inefficient. And I'm not sure if I've just come around to feeling self set to like a self serving message, or it's true. But my thinking now, and partly, this thinking is influenced by the research that went into my new book ranges, that that is actually one of my competitive advantages, the time and the willingness to have this expansive personal search function, where I find and connect things that other people don't, and that I cannot eliminate the inefficiency there without eliminating some of like my competitive advantage. Basically, the other important thing I do is, there's a psychologist and statistician, who I, I essentially keep on retainer, sort of he would do it because it's like, we just really like doing this, I think it's fun for him. But I don't want to like abuse his time. I mean, I do want to abuse this time, but I wanted to feel good about me abusing his time. And basically, so that he'll just if I have a question about methodology of a paper and things like that, we'll talk anytime. So in that process, we talked constantly, basically, while I'm trying to figure out Am I seeing this right, just another brain basically, to weigh in on that. And I started to notice, as I was thinking about this quotes from people I respect like Erik Larsen, Chris Nolan, where they would say, well, between projects, I really just need to read without any purpose whatsoever. And I sort of feel that way about it, too, is that we want to set aside some time where I think its recommendations, algorithms and things are great sometimes, but also they can kind of, they're comparing you to a bunch of other people that they think you're just like, right. And sometimes they can really track you into a certain set of interests. And if I go off into like, a library or rare books archive, or I just follow one journal to another, what I would do for a while is I'd go to I had an alumni reading car to Columbia University. And there were four computers that were logged, simultaneously logged into every journal that the universe had access to. So I'd go there USB drive, read the article, all the hyperlinks in the citations, I can download 300 articles in a day or something like that, and take it back. And there, you can follow it right away, because they're all hyperlinks. So I would just go sit there open till close. And again, I think the difficulty is, it's seen as very inefficient. And I think I'm very fortunate and privileged to have gotten the space to be a little inefficient and find things other people's don't. And I think my last job was at pro publica, and I left to finish this recent book. But I think one of the sort of bits of genius about pro publica, whatever you think about the individual stories was they brought people in, say, get your arms around as big a project as you can. And they accepted that that means some of them will fall through. Like when I was there, there was someone who worked on one project for a year that fell through, they had nothing to produce after a year. And they were okay with that, of course, the writer was horrified. But I think you have to sort of tolerate that. And that going off. And being an efficient in your search, you land on things that other people don't, because you're looking in different places. But it's hard to convince people that that's okay to do. One of the characters in range this, this world renowned scientist says everyone brings like lunch back to their desk now. And really like I want them crashing into each other, even though that seems really inefficient, basically,
is there a single article or discovery as part of that process that was most surprising in your memory, thinking back on either book, or on any research project? Actually,
for sure, when I started reading the studies that weren't the foundation for the 10,000 hour rule school of thought. And I was like, all these very famous writers have written about this, I must be wrong. And I'm looking at this stuff. There are no measures of variance. The accounts of the practice hours are inconsistent on multiple attempts, which normal but very inconsistent. What am I missing here? This doesn't look like rigorous research to me. And then I started to say, Okay, let's start at the most basic thing. If only hours of practice matters, then how come I broke the women's world record in the 800 meters after two years of practice, this is a genetic difference to matter. So let's just start the most basic. And I started talking to a prominent scientist in the 10,000, our school about that. And I remember she sent me a paper. And we were kind of arguing in a very civil way, she sent me a paper that said, Look, this even holds for other organisms. If you look at racing dogs, racing Whippets, they have equivalent to their lifespan 10 years, 10,000 hours or 10 years rule of practice before they become elite. And I'm going through the citations of this paper and one of them notes that 40% of those dogs also have this incredibly rare genetic mutation that causes them to be really fast and have more muscle. And so I'm like, these people have told a winder, I can't believe she would send me this paper as evidence of this. And so I started to have this turnaround, where I said, Gosh, I've written some things that are wrong, a bunch of other people are writing some things that are wrong. And there's a real opening here for someone to come in and kind of dissect this work. And so that was a big surprise to me. When I decided I wasn't, I wasn't just missing something. And in fact, you know, I was right about that.
This idea of narrative violation is really interesting. And one of the lines in your writing, I think it was in the story on stance in hearts. There's a line that says something like our culture rewards discovery, not replication. And it's a huge problem again, both in health probably in sports and investing, you get these ideas that then 10,000 hour rule has this inertia, like it just won't go away. And I'm sure even after listening to this, people will probably still believe that and I probably you know, there's probably elements of myself that will still believe
Well, there are good things up from it too, right? Like the emphasis on practice and effects matter. No, but it uncontroversial. The practice matters.
Yeah. Let's bridge now to your new work in range. And I think this is an incredibly powerful set of ideas. There's a lot of things I want to go into. You already laid out the tiger versus Federer kind of framework, maybe flush that out a little bit more with this idea of Martian tennis, and how this analogy is helpful, but it's not perfect. And that the real world is neither of these people. And neither of these sports,
right? This was a tricky thing to get to because when again, when I first did the book proposal, the proposal was called Roger for range was called Roger versus tiger. So in the book opens with that thing that I call the Roger versus Tiger problem, I tell the Tiger Woods story again, incredibly precocious, father's media training him at age three, because he realizes he's, he's got something special. Whereas Roger is totally opposite. One of my favorite little anecdotes about Rogers early life was his parents were as a sports illustrated writer called it pulley not pushy, and they were trying to get him to take tennis less seriously, even once he wanted to focus. And once he got good enough to be interviewed by a local reporter, the reporter asked him what he would buy with his first check if he ever became a pro. And he says, a Mercedes, has mothers aghast and asked the reporter Can she listened to the recording, and she listens. And what Roger actually said was mere CDs in Swiss German, he just wanted more CDs. And she's like, all right, it's okay, we've still got them under control. But so my plan was to contrast those developmental paths and say, let's look at different domains. When is it best to be a Roger and when is it best to be a tiger? And what I realized was that in some ways, those were insufficient, because all of sports is a very limited analogy, which again, is sort of a problem because I think of the ways that I've used sports as an analogy wants to sports teams sort of took off and I got asked to speak about it in certain ways. Having done range, I'm more tempered in the way I'm willing to use sports as an analogy now. So Robin Hogarth, the psychologist, you mentioned Martian tennis, characterized, he was trying to address this question of when do people just get better with experience. And when they just get better with experiences when they're in what's called kind learning environments, a kind learning environment is where the next steps are clear, all the information is available goal is right in front of you. Every time you do something, feedback is immediate, and perfectly accurate. And so if you just do it and are cognitively engaged, you get better golf, people take discrete turns, those sorts of things. Tennis is somewhat less kind, because you have to deal with teammates and things like that. And you have to use what's called anticipatory skills where you have to judge what other people are doing and things like that. But it's still more on the kind end of the spectrum, the information is available, you get quick feedback. On the other end of the spectrum, the far end are wicked learning environments, which are actually in the wider world, where most of us are, we're not in a game that has lines drawn that we have to play between, we can't see all the information, we don't wait for the next person to take their turn, etc. Information is hidden from us. Feedback is delayed, if given it all, sometimes inaccurate. And one of the examples Hogarth uses is this famous New York City physician who was renowned for his ability to predict the patients would get typhoid. And he could do this time and again, he would help aid their tongue around their tongue and predict weeks before they had a single symptom. Over and over again, typhoid became very famous for this. And as one of his colleagues, another physician later pointed out, he was a more productive carrier of typhoid than even typhoid Mary, because he was giving these patients with his hands, typhoid. So in that case, the feedback he was getting was reinforcing exactly the wrong lesson. Okay, so that's like the extreme of a wicked environment where your feedback teaches exactly the wrong lesson. Most things aren't as wicked as that. But most of the things outside of sports are more toward the wicked end of the spectrum. And so Robin Hogarth said, usually we're not playing golf, we're not even playing tennis, we're usually playing Martian tennis, you see people out there playing, something's going on, you don't know the rules, it's up to you to introduce them. And they could change at any moment without notice. And that's the situation that we're actually in for most of the things, the complex things that most of us care about,
there's a great term in the book, which starts to bridge into this idea of maybe avoiding early specialization is a smart thing to do, which is the cult of the head start. And I'd love you to weave that together with this notion of how we want to set up our own feedback loops. Because obviously, everyone likes a tight feedback loop. And I think in many ways, getting tighter feedback loops in life would improve our lives. But in some ways, it's actually really bad. So talking about the cult of headstart and the notion of feedback loops.
In a more general sense, the culture that had started was this, I started to see in different domains that the easiest way to give someone an advantage is to teach things, they're usually called closed skills. So things that are technique that you can teach very quickly and see an advantage. But those fade out, or they appear to fade out because everybody actually learns them. And then the people that we're learning these sort of broader frameworks for skills will surpass those people. But tell me not to if you don't want but you asked before what the most surprising study was in the sports gene. And this question just prompted what the most surprising study was for range. Okay. The study at the Air Force Academy in the fourth chapter where I apologies to Danny Kahneman, titled learning fast and slow. So this study at the Air Force, the Air Force has an incredible setup for a kind of natural experiment that you'd never be able to replicate no lab, really. And they wanted to study the impact of teacher quality on math learning among their cadets. And every year, a freshman class comes in the Air Force, and they have to take like three levels of math calculus one, calculus two, and then they go on to more advanced math. And every year, it's like 20 kids are in, in each calculus, one section, they are randomized two professors, and the test scores and all that end up everything, the skills that came in with our randomized cross classes after that class, they all get the same exact test. The tests are graded by groups that just do like a certain question. So no teacher can like favor their students, the test is like the whole grade. And then those kids are re randomized to Calculus to same thing. And then they are randomized, again, to follow on courses. And the same thing happens in the engineering courses there re randomized every time. And see they do this with thousands of students over the years. And what these researchers wanted to see was, okay, what's the effect in a given class of a professor, and what's the effect on the students trajectory. And what they found was that the professors who were the best at causing students to do well in their own class do well on the test, and by Well, they meant over performing compared to the baseline characteristics they came in with systematically undermined those students performance in the follow on courses, which is incredible. So the professors out of like 100, professors, they tracked the one who was sixth in performance, students in his own class, performing above what they be expected and 17 student evaluations, because the students evaluate them Well, usually, when they do well, or they feel like they're learning was dead last and what they call deep learning, which was how well his students did going forward. And so what happened was, the teachers that taught them more narrow curriculum, were better at prepping their students for the test they would face. But because they didn't give them this broader, what's called Making connections framework, where they would connect ideas in math, they were systematically at a disadvantage for all the classes going forward. And the bigger problem was that those students because they see their progress so quickly, because they're learning this narrow set of skills, they rank those professors that undermine their long term development really highly. So there was a negative correlation between deep learning or how well the students would do going forward and how well they rank their professors. So that's really wicked feedback.
Did they get into what was causing? Maybe I don't know if it was the exact inverse, were some of the teachers that weren't having students do as well in their own class, but then setting the kids up for greater success long term? What those teachers were doing, right?
Yeah, basically, part of it was they were ignoring what was on the test, essentially. And by ignoring what was on the test, they ended up teaching a much broader conceptual curriculum. So what they did was they use more, what researchers called Making connections knowledge unless using procedures knowledge. So using procedures, it's like there's a certain kind of problem, you know, why was formula or whatever, exactly more like that. And making connections is much more like, let's try to figure out if this formula always works? Or why does it work? Or can you find a place where it doesn't work. And this stuff is kind of frustrating, and you try to pick single problems that have multiple modes of attack, and the using procedures, knowledge, you get really good at that. And if you see the same problem with using procedures is great. If it's golf, where you're going to see the same situation over and over again. But for these follow on courses, what you want is someone to have knowledge that they can apply to situations they've never seen. And in that case, you want this broader conceptual knowledge that's making connections knowledge, what you really want to teach them is how to match a strategy to a type of problem. And that's what those teachers who are doing better, we're doing a better job of teaching in ways that proves so many different scenarios, the students got frustrated. But they started to learn develop these more abstract models of how do I match a strategy to a problem and they showed up and other types of research in the book, like, in that same chapter, there was training for people to respond to naval threats, and people were being trained were split up into two groups and digital simulations. One group trained on a certain type of threat over and over and over and over and over and over again, and they make super progress. Other groups, same number of training rounds, but mixed up all crazy, they're never seen the same thing twice. Group aid is doing the same thing makes tons of progress feels really good race through learning experience highly all that stuff group to didn't perform well feels like crap breakthrough learning experience low when they bring them back like a month later, and give them new scenarios that none of them have ever seen. group two destroys group one. And that's what you want. That's like kind of a theme of the book is when you want people to be able to apply knowledge, whether that's physical or cognitive, to situations they have not encountered before, you need to teach them early on in this much broader way. So they build these conceptual constructs. What do you think this batch of studies
and then other ones that you found, say about our early education system, and maybe our entire education system up through college? Does it suggest that we should have more of a liberal arts broad kind of education or my sense is that we're moving more towards managing to the test managing to the sap managing to whatever, and away from no specific goal in mind Don't teach to the test type education.
So I very consciously stayed away from making the arguments for a liberal arts education of Fareed Zakaria as a book that does that. And I wanted to stay away from that, because I don't necessarily view that is inherently more valuable than you know, a lot of people get a liberal arts education and don't know anything about science. And I wouldn't consider that a well rounded education. So I think more than about what they should study, it's about how we study the things we study. And by the way, there have been other studies now showing that Professor evaluations, this may be different for like, middle school and elementary school, but at the college level that Professor evaluations are offered and like inversely related to in like STEM courses, especially how well the students end up doing later on. And I think it is a problem, when we incentivize professors and students, when they're full incentive basically looks like doing as well as they possibly can on a short term test, then what you get are strategies that produce short term skills and learning that evaporates when they start facing these more novel situations, what you really want is information to stick and to be flexible. So ultimately, you want them to be able to apply it in situations they've never seen. And teaching narrowly, and using procedures, knowledge is basically a way to undermine that. And so I think there's, we need methods to evaluate, right. But I think if we are too focused on these narrow sets of tests, that can be a big problem. Speaking of ways
to evaluate, you talked a bit about this idea of the Flynn effect in the book, and I'd love for you to recount the idea there. I was surprised by you show that classic picture. I don't know what it's called the test is called you ask people to identify whether one ball is bigger than the other, you could describe what that is. And I was surprised by the kinds of people that make certain choices. So talk to that idea.
Yeah, that's called the house illusion anyway, but that chapter, the Flynn effect is the finding that IQ scores around the world are going up at a rate of about point three points per year, or three points per decade over that starting in the 20th century, steadily. And it's not just the bottom of the curve moving up, or it's not just the top of the hour. It's like the whole curve shifting to the right. And in fact, it's not only that, but it happens where was least expected. And the most abstract portions of tests. So there's parts of tests, cognitive tests that are on things you might learn in school have not moved so much. But tests like the Ravens Progressive Matrices is this test is totally abstract. This was created to be the test. That's what's called totally culturally reduced, meaning nothing you learned in life or School Matters, you get these abstract designs ones missing, you have to produce the rules Martian tennis, and figure out what pattern is missing. And this was the test that if Martians landed on Earth, this was the one that could tell how clever they are. Because you don't need to have learned anything. It's just pure reasoning skills. And what Flynn saw James Flint, or the Flint effect is named after is that this is a test where the biggest gains were occurring over the years, the one that was supposed to stay stable. And why is that? And it turns out, what he suggests is that we have moved to a world where we are used to classifying things to grouping things instead of being stuck with lots of concrete knowledge and, and factual knowledge. We face a world that constantly a work world, especially he found most of the effect was from modern work as opposed to modern School, which seems to have some effect, though. But we're in a world that forces us to constantly be playing Martian tennis and trying to deduce rules when we've been given on whether that's learning how to play a video game or operate a computer without an instruction manual, or how to coordinate work with people that are far away, we constantly have to do these abstract things that allow us to, to classify sets of knowledge and transfer it between domains. And when he looked at research done on what are called pre modern people, in some of the studies where they are no worse, like one way of life is not better than the other, we're just adapted different things. But people who are like subsistence farmers, classification doesn't really make any sense to them. If you give them a hammer and X, a knife and a bird and ask them which one doesn't belong. In these studies, they'll say like, well, there's no use for just like the hammer in the axe unless you have the bird because you need something to cut up. And so you could go this way, or could go that way. Like we would say, like, well, three are tools in one are, you know, is an animal, and we're just kind of classification is very native to us. And so of course, we would suck in their domain. But for our world where we're constantly laterally translating knowledge to different areas, we've never seen this ability to have knowledge that we don't have from hands on exposure is really important. And that's shown up on these tests, where we do much, much, much better at abstract thinking, which is a signal that we really are well equipped to move our knowledge between domains in a way that our forebears were not
on come back to this feedback idea. And he already mentioned the chapter titled learning fast and slow, which I really liked, and the conclusion that we should want to learn probably slower than we do. So I'd love to hear that kind of translated to maybe advice is the wrong word, but how people should maybe change their behavior based on your findings in this part of the book,
because I've changed my behavior a little bit based on those findings. And here's like the concrete, we just went from a very abstract on here's like, the more tips from learning fast and slow. There, I get like an at least a monthly basis, probably more often press releases about learning hacks. There's basically like five learning hacks that are really supported rigorously by research, and largely ignored in popular media. And three of them that I focus on in that chapter are testing, spacing and inter leaving. Basically, testing is actually which you know, I just denigrated is wonderful for learning. In fact, you want to test people before they've had a chance to study because it actually turns out it Prime's your brain for when you then hear an answer to retain it, even if you get something, especially if you get stuff wrong. So there's something called the hyper correction effect, where if you're quite confident about an answer, and turns out you're wrong, you're more likely to remember the right one when you get it. So testing before you're ready spacing, or what I call deliberate not practicing, is you want to leave space between bouts of practice of the same thing. So again, if you practice the same thing over and over, and over in one session, you'll see progress right away. But what you really care about is how long does it stick. So give an example of spacing. There was one study, one famous study where two groups of people were learning Spanish vocabulary, one group got like eight hours intensive on one day, the other group got four hours one day, and then the other four hours a month later, they had the exact same studying one was just separated by a month, eight years later, they brought them back, and the group that had the spaced practice intervals, remember 250% more with no studying in the interim. And so you have a certain amount you want to study, it becomes more efficient, if you space it out, you take time of deliberately not practicing. So testing, spacing and inter leaving means just are mixed practices like mixing up the things you want to learn. And this again, has to do with forming these conceptual models for knowledge. So if you're teaching kids how to learn a certain type of math problem, instead of giving them the same type over and over, let's say actually, you have the same 20 math problems that has five of one, kind five, another five, another five another, you give them, what we usually do is you give them five of type a five of type B, five, type C five, type D, you'd be way better off mixing those all together. So again, they will struggle more, they'll take them longer, but they'll learn again, more how to match a strategy to a type of problems. This shows up in all sorts of learning studies. If we were going to go over to the museum, we're like, not that far from MoMA. If we wanted to recognize Basquiat, and Picasso, and Brock or something like that. I mean, those guys aren't so hard to tell apart, but, and we wanted to recognize paintings we'd never seen before. And we had a stack of flashcards. Instead of looking at all the baskets, and all the Brock and all the Picasso, we'd want to shuffle those all together, then try to quiz ourselves on them. And we would be much better when we then go to museum and identifying paintings who painted them, even if we had never seen them before, because it forces you to build these abstract models that you can then apply going forward.
It reminds me of this past guests had this format called stress plus rest equals growth, and kind of some similar concepts baked in there, like a lot before talking about the struggle a little bit more, because I think that's an important word for learning. What are the remaining two that have good empirical foundations in terms of hacks for learning?
Well, one of the other ones, it's also in the book, but not in as like tippy of a form, I guess, is using making connections questions. So that's using type of questions that force someone to bring together multiple concepts. So it sort of disables them from being able to do just using procedures, knowledge, they have to draw in multiple concepts. And then one of the others had more to do with tailoring learning, basically, which was like a little little more like practical on the grounds for classroom stuff.
So expand on this word struggle, because it seems like I think the quote in there somewhere was struggles better than repetition, which you've kind of just laid out. But for those thinking about how they might harness the power of struggle as a means of getting better at something. Talk about that a little bit.
Yeah. So Nate Cornell, this cognitive psychologist, I quote a bunch and range had this phrase I like where he said, basically, difficulty isn't a sign that you're not learning, but he is. So he studied these things like testing, spacing, and are leaving, and he says, If something's getting easy, you better mix it up, mix it with more things, or stop doing it, go do something else. And wait until it's hard. Again, he said, You should wait. There's actually these interesting papers that calculate how long you should wait. If you want to information be maximally sticky. They're like no one time horizons how long you should wait, basically, you want to wait until you've just forgotten it again, so you can't retrieve it. And the reason that seems to work is because it makes it hard again, and every time you practice it when it's hard, you have like very steep learning. And so you can sort of incorporate these things, you don't have to like not be doing anything. But if you have a certain amount of things you have to learn, you can mix together into leaving and spacing, because by inter leaving, like mixing the tasks, you have to do shuffling them up as it were, you effectively space them anyway. So it's not like you have to just go to sleep and waste your time. But you should take different tasks and mix them together. Because it It causes you to space them and into leave at the same time, basically. So I sometimes like I give talks sometimes like that started that was a thing I didn't know was a thing until after the sports gene. And I memorize, I'll memorize our long talks like word for word, and I'll improv off of that. But I like to have it memorized so that I can then improv. And every time I give a talk, people come up to me and say, give a photographic memory. If I put my keys down and spin the circle, I still lose them, I definitely don't have a photographic memory. But I've totally incorporated that stuff into how I learned. So if I make my slides, I want to talk into slides. Like before they get there, I'll test myself on them right away before I've memorized them. And then I make sure I leave some space and interval no cramming. And I'll mix the slides in a different order, not even the order, I'm going to have to practice them in case maybe I want to add a little. It's been amazing. So I look like I have a photographic memory when I definitely don't.
What's your take on the role of passion and grit and all of this? So two very different things. I think you would probably argue that those are good things, generally speaking, but maybe sometimes misapplied or misunderstood. So talk about the role of passion and grit in terms of finding what you want to do, should do are good at and how much when perseverance is good.
And when it's bad. I'm curious if this will end up being one of the things people glom on to in the books of this chapter, titled, The trouble with too much grit and grit is, as a lot of people have probably heard and what exactly you think about it probably depends on where you heard about it. But this incredibly popular psychological construct most associated with a researcher Angela Duckworth. And it comes from a survey that basically asks a half dozen questions about your personal resilience and your perseverance. And then another half dozen questions about your consistency of interests, which is sticking to the same thing that you're doing, and the most famous study. And you can stop me from progressing too much here. But the most famous study is the one that Duckworth and colleagues and colleagues did at West Point at the US Military Academy, and cadets going through what's called beast barracks, which is the six week orientation, where like high school kids are taken, and this is time to convert you into officers and training. So physically and emotionally rigorous, you know, they get gassed, and they don't get to sleep a lot, and all that stuff. And what she found was that the whole candidates score, which is the most important thing for admissions at West Point, it's like test scores, and some athletic tests and leadership experiences, stuff like that was not so good at predicting who would make it through beast. In fact, it was very bad. Although it is quite good at predicting how well people do in the academy overall. And the grid survey was better at predicting who would make it through beast. And so then she started applying this to other things like who in the finals of the National Spelling Bee would advanced to later rounds, it was great if people had high verbal IQ test scores, that was good. But if they were lower, they could make up with some of it for grit. And if they were lower on grit, they can make up with some of it with higher verbal IQ test scores. So what she was finding was that this, this construct, it measures your passion and resilience and also your kind of stick to it. Ignis in terms of the same idea had some independent predictive value of how people perform separate from any other skills. And I think that's a powerful notion. But I also think it's been wildly misinterpreted in some ways. So we've talked about restriction of range a little earlier. So this famous study of beast barracks, that's extreme restriction of range, you've already siphoned off this tiny sliver of humanity who are quite similar on whole candidate score from the rest of the world. So all your other variables better look like they're more important because you flattened one of them. And most every cadet gets through be spirits. And the amount of variants that grit explains, in any given paper that I've seen is usually between about one and 6%, which is probably not what like companies they're testing for it think they probably think it's a lot more than that. But I think the bigger problem is that all of the studies that are done on grit like that with people in the National Finals, the National Spelling Bee or at the specs, they're looking at people who have a very narrowly defined goal for a very short time, those people are trying to get through six week orientation. People who are already in the finals of the National Spelling Bee or just trying to get farther in the National Spelling Bee, like the goal has been restricted to this incredibly narrow thing for a very short time. And what I thought was interesting was looking at, so grid has some predictive value for cadets to get through these barracks. But since the explosion of the knowledge economy since like the early and mid 90s, especially those gritty cadets that get through beast and then get through West Point have been dropping out, almost half of them have been dropping out the first day they can from the military. And those rates have gotten higher and higher and higher and higher. And so when we zoom out and look at a longer term goal, it's gotten so high that this is high ranking officer suggested defunding West Point because it's an institution that quote teaches its cadets to get out of the military, which of course, is not the case, right. But what's really happening is, as we've entered gotten less into this sort of corporate salary, man, upper out kind of economy, these cadets learn transferable skills, they learn things about themselves in their early 20s, age 18 to late 20s, is the time of the most personality change. That's not just my intuition that's in psychological research, and they decide to go do something else. And is that a loss of grit? Or is that because they learned something about themselves in those intervening years, they learned valuable skills that they can transfer, and they change their goals. So here you have these cadets, who would score lower on grid when they're 27 than they did when they're 18. Because Kris, not a stable construct of people, there's no evidence that it is. And what those cadets are doing is they're on the hunt for what economists call match quality, which is tried some things they've learned about themselves, match quality is a degree of fit between your interests, your abilities and the work that you do. And I don't think it's a failure of grit for them to go hunting for match quality, because it turns out that that's incredibly important to your overall performance, how will you fit with what you're doing? This goes back to something we were discussing in sports initially. And so I think a better construct, or a more holistic one, because I think grid has, is very useful, and feels, to me intuitively, a ton, is to look at this more holistically. And this aspect of personality study called if then signatures, where it's like, if David is at like a giant rave, then he's an introvert. But if he's with his small team and work that he's an extrovert, and those things are true, and it turns out, that's how personality functions where we don't just have necessarily these stable traits, we have traits that are consistent in a given situation. And I think a better way to think of grit is when you help someone get the right fit, it will look like grit. And that shows up in some of like the music research that I say in range, where you see kids that the practice a little bit, they don't like that much. And then they switch instruments. And when they find an instrument that really fits with them, then their practice explodes. And so when you just look at the end results of the study, you're like, well, these kids just love to practice. But then if you look at the whole developmental trajectory, it's actually that they cycled through some instruments, when they found one that fit their interests and what they like to do, then the practice explodes. So it's getting to the right spot that causes that to happen. So I think we need to be conscious that the studies of grit have been in these very contained very short term environments. Like if you're in the finals of the National Spelling Bee, not deciding, you know, you don't like the spelling bee anymore is an advantage. But would it be a bad thing, if you decided that like memorizing rude words, you don't have any idea what they mean, but you know how to spell them is maybe not the best match for the way to spend your time. I don't think that's a bad thing, necessarily.
A lot of these concepts, especially this range of personality change with the said, 18 to the late 20s. I'm curious what the drivers are behind that, and how that matches up with what I guess given all your research you would suggest people do more of in that time period and take advantage of that will call it malleability early in life early in the career pre kids like this kind of this formative years of adulthood that people do to make sure that that personality change, I guess is in the right direction.
Yeah, I mean, the drivers, I'm not sure how to answer it. I'm not sure anybody knows their things, we suspect. But the correlation between personality traits between the teen years and like middle age is usually about point 2.3. So like, low to moderate, which means like between your teen years and middle age, there are traces of the teenage you that are distinct in the middle ages to you. But mostly, you're a very different person. And the biggest change there it goes on from 18 to 28. What those drivers are, I think it's hard to say, are they natural? Are they just that we're having all these it's the first time a lot of people are like truly autonomous? And so I can guess it that I don't know the answer. I just know the empirical findings. But I think what it means is that if you're matching early, yeah, I think it adds more fuel to the Roger model, because you may be like Teigen, you may find your thing right away. And that happens, I think it's just a lot less likely, because you're in the position of having to choose if you're like, if we thought about careers, the way we think about dating, we would never tell people like settle down so quickly, if you'd be like, go get more data before you like make a commitment here. And we may spend as much time working as we do with our partners are more, at least waking hours. And so I think when we're like rushing people to specialize, I think it's with good intent, but you're putting them in the position of choosing a match for a future to them who they do not yet know, in a world that they cannot yet conceive, basically. And so the approach that the people who find better match quality taken there's sort of two sections on this one called the Dark Horse project. And then this woman her many of you, Barbara, that studies people, she is one of my favorite quotes. In the book, she studies how people find good career fits in heart and change careers, was we learn who we are in practice, not in theory. And what she means is, we like to think that we can just introspect, and and like, like the commencement speech advice, actually, Paul Graham had a great I guess, never delivered but but he wrote like a commencement speech telling people to ignore the commencement speech address, and he posted online. And what he basically says is totally in line with her mini bar, which is,
most of the commencement speeches will tell you picture who you want to be in 20 years, and like confidently marched every step toward that. And he says, like what computer scientists call that is premature optimization. Basically, you shouldn't do that what you should do. And what are many bars suggest is, all this research shows what we know about ourselves and our skills. And our potential is constrained by our roster of previous experiences, which are really limited when you're a teenager especially. And so I think one of the reasons why personality changes is because we're just expanding our experiences and learning more about ourselves and adjusting to the world. And we learn who we are in practice means we do things and then we reflect on them. And that turns out to be the way we learn about ourselves, as opposed to just being able to introspect and have a theory about ourselves and be accurate. And so what she says the way to go forward, then is to do things that are essentially little experiments, say, I'm going to try this thing. Here's who I am right now, here are the skills I have right now, here's the things I want to learn here, the opportunities in front of me, I'm going to try this one. And maybe a year from now, I'll change because I will have learned something about myself. And maybe I'll have different opportunities. And they keep zigzagging until they find better match quality than the next guy. And that improves their performance. So in this dark horse project at Harvard that similarly too many borrow looked at how people find their best match quality, which improves their fulfillment and their their performance. Short term planning. A focus on short term planning was like the main trait that most of them had. But when they all came into the study, they would say, it was called the Dark Horse project, because all these people would view not everyone again, it was most of them would view themselves as having come out of nowhere, because they would come from some other field or have some weird MySpace experiences. So they'd all come in and say, well, don't tell people to do what I did. Because like, I'm an oddball, but that turned out to be the norm is that they would bounce between things learning about themselves, really, it's an investment and learning who you are, and how you can better fit. And so Robert Miller in the book, economists modeled this and statistician as what he called a two armed bandit process. So one armed bandit is a slot machine, his nickname for a slot machine. And the two armed bandit processes are multi armed bandit process, because he looked at multi not just essentially, career matching a model is if someone's sitting in front of like a bank of slot machines, and each one has a different reward. And each one has a different probability of reward. And they're tasked with pulling different levers trying to extract some information about where to focus. And then when they find the one that has the best probability of reward, they stick with that. And he found that to be similar to the way that people test careers is they go around, you try some things, and you start to like, figure out where you should, should focus. So he advocates, he thinks like the young and stupid, you know, people who dive into like high risk careers is not stupid at all, because you get high information signal from those careers very quickly. So he advocates or what his model suggests is that, that younger people should dive into these high risk high reward careers where you get lots of feedback quickly, early, because that's the best time to try it out and sort of get these high information signals, you don't want to go into like a low information signal Career Early on,
there's a tremendous amount of advice for the individual and everything that you just described, and kind of everything we've talked about. As the book progresses, there's also some ideas about applying this idea of range, I guess, like a more mature or static or business like environment. And one of these is this notion of using the outsider. So I'd love you to expand on this idea of how I guess businesses is kind of what I'm thinking, but maybe we'll use the Gameboy example as one fun one. But the ways in which an outside perspective, in my world, the buzzword for this is cognitive diversity in your investment process, or something can be used is really harnessing the powers of range for more productive outcomes.
Yeah, that gets there's like several chapters that touch on that in one way or another, I'm guessing the your listeners are kind of pretty familiar with the outside view where so maybe I won't go like too in depth on that part. But the Gameboy example, for example. So I think one of you know, actually listened to one of your previous episodes, because I'm always interested in like I said, we talked a little bit before, and I know Josh Wolf, and I think he's like a very interesting guy. And to put it mildly. And one of the things that he mentioned, I think, when you were talking to him was that he diverges. I'm going off on a tangent again, here, go go go.
Do the format that he disagrees with Robert Gordon economist who sort of says Robert Gordon is is very compelling point where he says, if you took someone from 1860, and put them in a house in 1940, they wouldn't even know how to use it, because it would have indoor plumbing, and like a refrigerator and electric lights, and a telephone and a car. Whereas they would have had like an outhouse, and a horse and like whale oil lamps or whatever. So that's 80 years before 1940. But if you go 80 years, from 1944, which is us, we wouldn't know how to use all that stuff. And his suggestion from that is like we haven't made that many leaps, like the big leaps are behind us. And I think that's interesting. But one of the things Josh mentioned is that he thinks communication technologies kind of diverged from that, and allow this sort of dissemination of information more widely and collective action in a way that we haven't had before. And I tend to agree with that. And I think and again, I'm seeing this through my own range lens, of course right now, because I mean, it looks about to come out, and it's on my mind a lot. So that's my bias, obviously, range colored glasses. But I think one of the ways that he was right about this really changing the game, is that because specialized information can be disseminated so much more quickly. And because there's so much available, it's a lot easier to be broader than a hyper specialist now, because you can take this information very quickly from all these different domains. And there's so much out there, there's so many opportunities to recombine things in new ways that people who are in a more sort of narrow focus or or what one of the scientists in the book I love called a system of parallel trenches where everyone's in their own trench and not usually standing up to look over at the next trench even though that might be where their answer is. You can be broader now because there's so many chances to combine thing and information is disseminated so quickly. And one of the guys who you're alluding to I think, is a great example of that as this guy gun pay yo boy sort of profile in the book, by the way, speaking of another research advantage to go back to our other topic was he had some great books that had not been translated from Japanese. So I hired someone to translate them from Japanese. Right? So that's great. So that's a great advantage because he laid all this stuff out, but nobody had translate to English. Similarly, for sports, Gina had some old like German sports journals translated and it didn't end up being as big a part of the book, but it's still like, I kind of look for stuff like that. And so gunplay, yo Koi was he studied electrical engineering, but he wasn't so good at it, frankly. And this was at a time when, I think as one of his one of his colleagues said, The technology was changing faster than snow melts, and sunlight. So he just couldn't keep up. And all of his peers who did really well on these tests, went to big companies in Tokyo. And he didn't do well on the test. So he didn't get Tokyo interviews. And so he had to settle for going to this small company in Kyoto that had started in the 19th century as a wooden shop that was selling playing cards, so called hundred Buddha cards. And when he got to this company, were about 100 employees. At this point, they were making these playing cards, and he was hired just to maintain the playing card machines. That's it. But there wasn't much to do, because it wasn't that big of a company. He'd always been a tinkerer, he just like love get involved in tons of different stuff. And he started tinkering with machines one day, and the company was having a lot of trouble playing car business like wasn't going that well anymore. The one good thing they had going was they had machines that helped the process be cheaper. But one day he cut these like crisscrossing slats of wood using some of the company equipment and made like an extendable arm that you see popping out of like a robot stomach in the cartoons. And he started blazingly grabbing stuff that was far away. And the company president, who was in a tizzy about turning the company around, sees him doing this coming to my office, he thinks he's going to get fired, who's supposed to be maintaining the machines not playing with them. And the company president says turn this into a toy, we're going to market. And this turns into what's called the ultra hand, like grab things. And this was the company's Nintendo. And this was Nintendo's first toy. And now he starts they've been trying to make at the time they'd convert some facilities into making in like a desperate attempt to save the company while the playing card business was deteriorating into facilities that made like cartoon branded noodles, basically. And so they got rid of it didn't work very well, they got rid of that and turned it into toy development. And Yoko started trying to develop different toys. And he realized that he wasn't so good at cutting edge stuff that first year, he works on another toy that's like I called drive game where it has an electronic motor and you have a real steering wheel and you guide a little plastic car over like a moving like the motor moves the road underneath the car. And it turned out that the mechanism was complex. And that was like you needed to work with some people in new, more cutting edge stuff. And it was constantly breaking and defects. It was super expensive total flop. He says never again, none of this cutting edge stuff. And he developed a theory he calls lateral the translate to lateral thinking with weather technology, with weather to technology, lateral thinking with weather technology, and what he means with our technology. He means stuff that is cheap, well characterized easily available, other people have overlooked it at this point, they've moved on lateral thinking taking it into some use or some domain where it hasn't been before. And he starts doing that with everything. Remote controlled cars were big at the time, you know, 1970s, but they were expensive. So only adults could get them in the biggest expense was multiple radio frequencies to control things, power of the motor turning etc. And he says, and everyone's racing to make more and more features are getting more and more expensive. And he says forget that. I want to democratize this. I think the barrier to this being more of a success is not more features. It's more people being allowed to do it democratizing innovation. And so he stripped it down to the bare minimum one radio channel. So the car can only turn left calls, it left the RX markets at a 10th of the cost of all the other ones now kids can buy it, and kids don't care. The left turn their way out of trouble. No problem and so they're democratizes so now he sticks with so we started getting momentum for this lateral thinking with with their technology one thing after another just repurposing things from different domains. One day he's riding the bullet trains he's a salary man trying to curious boredom with a calculator playing with his thumbs and things cash. What if I could make games that you could hold like play with your thumbs. And he randomly has to play show for for the company president for a day pitches is idea. president's going to a meeting with sharp makes calculator screens. And the President goes there and says Oh, we've got this guy, he's got this this idea yo quite thought like, it was going nowhere. Next thing you know, as they bring the sharp calculator, the sharp calculator people are engaged with like Casio and this calculator wars where they're both like just trying to make better and better screens and prices going down. And Yoko says are you guys make me can you make me a screen really small, like the size of a business card that I can display some graphics on. And I'll take these like old processors that aren't like cutting edge at all anymore, but cheap, and your screens that are getting cheaper. And we'll put those together and we'll make small games. And they're like, now we don't think we can make screen that small. But we'll try and they sort of making a prototype. And the problem is the stuff is crammed together so tightly that it causes this thing called Newton's rings, where there's distortion on the screen, because pieces inside the screen are touching. And so he says, gosh, we need to separate those like this is a problem and the calculator. The sharp people are like, this is like a law of physics, you're screwed. And he thinks will what other industries he starts searching other industries make small separations on something the size of business card realizes and credit cards, they're embossed, right, they like raise the lettering a little bit. So he goes he gets someone to help them tweak the machines, so they can embossed like a credit card. And he just embossed his little dots on the screen components is separated enough, Newton's rings are gone. He creates what's called the Game and Watch, which is and this is Nintendo now getting into the game sphere. So it's like just little games that you can play, play with your thumbs and they explode. I think their goal is to sell half a million units and they sell like 11 million or something like that right out the gate. And so he gets emboldened in this philosophy of lateral thinking with with their technology and just one hit after another. And then eventually, he decides that they're going to go even farther back, you know, in the technology is progressed a lot in the 80s. He says we're going to take processors that are even older, cheaper, I want to step up our hand handheld games, things like Nintendo Entertainment System have exploded now. So lots more people have had video games in their house. And there's this competition on graphics and those sorts of things. And atari and Sega are developing these color handhelds, and he says we're going to go old processors that would have been cutting edge 10 years ago, we're going to have four grayscale shades in ours, right looks like screen color of like old spoiled alfalfa graphics smeared across the screen when you move fast, because doesn't move fast enough. The processors not that good. And some of his colleagues are sort of like, well, you're crazy, but at this point, yeah, he's earned it. And they've paired him with some more specialized electrical engineers who can sort of it's like it's sort of like the jobs well as the acting like Satoru Okada was the guy who sort of will politely say like, your coin was not good at electronics. But he but he sort of backed him up, you know, made up for some of these deficiencies. And they put these things together. And I remember on one of the day that when a Gameboy is about to come out, so this turns into the Gameboy. Of course, what I'm talking about. One of your quiz colleagues comes and says, We're in trouble. Sagan Atari just came out and Yoko says, Are they color? And the colleague says, Yes, they are. And then we're fine. Don't worry about it. And it turns out the Gameboy absolutely blows them away becomes the best selling console in the 20th century. What it lacked in color, and graphics. It made up for in durability, you can drop it like in the reporting of this, I went to my basement, my childhood basement and found mine. The batteries had had had expired at two batteries that had expired in 2007. To the expired 2013. I flipped it on and played Tetris for a couple minutes, one of the batteries had exploded, and it was still fine. And it was covered in grime. Right. So I've made sure take video and I turned it on. So if I can use it in presentations and stuff. So it was durable. Like even if it died because it got wet. You could try it out who's coming in could fit in a big pocket, you could play for weeks on batteries. And because the tech was so simple, people started making games for it inside and outside of Nintendo like crazy, almost like app developers, right? It's like he drew people onto his team by using similar technology that they understood. And it just totally blew away the competition. And I've seen in some articles I had translated more recently, Nintendo is still embracing this philosophy, the President, an article I had translated, he said we was very much out of gun pit company, okay, passed away in a traffic accident in the late 90s. Unfortunately, and the President said this is very much out of Yoko's lateral thinking was with their technology philosophy, because the barrier to entry again, was not graphics, it was complexity of gameplay. So we made it simple and democratize this right, like Queen Elizabeth started playing and all these things, and what they just did the thing Nintendo lab or label or you say, it's like cardboard that people build around their switch or whatever. And so I think they're still succeeding with that philosophy. And Yoko is feeling was a, he said, If you draw two circles on a blackboard, people will know it's a snowman, they can feel the color of the snow, they don't necessarily have to see it. And, you know, he just felt that once there was a competition for graphics or screens like that was for the specialists to compete. That was like a zero sum thing, where like, one company was going to blow another one out of the water, eventually sharper cast, or whatever. Whereas there were many more opportunities to take this stuff that was already well known that everyone was looking past and recombine them in new ways. And I think it's a good symbol of where we are, and why some of these companies like you know, centive and candle that I talked about in the book, have had such success with farming out problems that stumped NASA that stumped Eli Lilly to just like, random people out there. And they get some success, because I think we're in an age because of communication technologies, specifically, where it's easier to be broader than a specialist and still have an impact. That was like a really long, I really, it's an
absolutely incredible story. With so many interesting takeaways and lessons, we've seen it firsthand where you put people who are talented in their own ways, and sick them on the problems that we've been trying to solve for a long time. And often, very quickly, alarmingly quickly, they come up with different takes that lead us down very different paths. So I think that that's very broadly applicable, this idea of the outsider and combining all of the old technologies piece to I'm curious in the reporting, did you find any other favorite examples? Not not at Nintendo, but elsewhere, of where their technologies being applied?
Oh, yeah. in that chapter, actually, I talked about three m, basically quite a bit. And this research that shows that, I should caveat this a little bit. So there are when I looked at research, they look at patents for technological innovators. In areas where sort of next steps were more clear specialists were better, where you just had to drill down into a certain technology in areas where like, the next steps were less clear, people who had worked across a large range of different classifications, according to the US Patent and Trademark Office were the ones that made the bigger impacts. So I think you want an ecosystem with both the specialists and the generalist. So it's kind of depends how well characterized your problem is. But one of the things I liked about three M was that they had a lot of examples of this. And they make it really easy for people to do that, because they have, they have this thing they call the periodic table of technologies. And it sort of helps everyone see like, what else is going on outside of their field and like all the technologies that they own as a company so that you can go back to old stuff and sort of recombine it. So even things as simple as like the post it note, which is one of their blockbusters came from basically a reusable adhesive that they had found like no use for and one of the inventors was have trouble like marking pages in his hymnal when he was like at church on Sundays, and he wanted to be able to like on mark and remarket. And so we basically combine a bookmark with this old adhesive that they had just abandoned him with post it notes. That's a pretty whether technology or I mean, one of my favorite inventions mentioned the book is is multi layer optical film, which is, in some ways cutting edge in the sense that it's like in your computer in your cell phone, recycling light that's bouncing around in there, so that you need less battery power to get a bright image. But the technology that sort of convinced the guy who led the team that invented it, that it was possible was the plastic of a water bottle just wasn't in his area. He wasn't a materials scientist. But he knew that the plastic and water bottle had certain properties of light that people just hadn't, it's like in front of you every day. But the specialist in the field just hadn't thought about using it for these applications of like modifying reflectivity. But probably my favorite example overall, of lateral thinking with with their technology because of how weathered it is, was to use the first and only woman in Chinese national to win the Nobel Prize and sciences. She's called the professor of three knows because she has no postgraduate degree, she has no foreign research experience. And she has no membership in any like National Academy or anything like that. Or at least she didn't. And she an interest in science and interest in history. And so she went back to these like archival old, centuries old manuals of Chinese medicine to look for hints at treatments for malaria just for hands, she wasn't going to do like fake dietary supplement. Because this thing has been used forever, it must work. She wanted to see hints of like just old notes and found in this like several centuries old manual, a hint about something that someone thought might like help with certain treatment for malaria. And she starts testing all these but long story short, it leads to this extract called artemisinin, that she gets the clue again from this centuries old text. And it turns out to be the world's most effective treatment for malaria. So a recent study credited artemisinin based therapies with between like 2000 2015, preventing 146 million clinical cases of malaria in Africa. And so she won the Nobel Prize for that. And I just liked that example, because technology doesn't get much more weather than that she found in like an ancient Chinese text researchers before her had tried almost a quarter million different compounds looking for something like this. And she just went back to things that had been overlooked and looked for hints that could target her searching. And and she won the Nobel Prize,
would it be fair to characterize kind of the arc of your work as starting with this interest in performance? And like what affects performance? And it seems like so much of the examples you've given today are more about innovation and creativity and how range intersects with those two things. And it makes me think of the James cars, you know, finite versus infinite games. And seems like you kind of started with the finite games and performance in those and it moved on to a much more open ended playing field. very
astute observation. Yeah, totally, totally. And part of that was just my own interest. I mean, some of my you know, I guess the not so secret secret of, of both of my books is I'm just investigating things I'm interested in, right, like sports gene was like a bunch of questions I had in the back of my head of like, Well, why can't baseball hitters hit a softball pitcher, when the transit time with the ball is longer and wired certain athletes coming from certain places, you know, and then and similarly with range? And? Yeah, no, you're definitely right. You're definitely right. And that reminds me of something sort of semi related, which is, in looking at some of this creativity and performance, I tried to pick studies, sometimes it could it could count for certain types of survivor bias, because when I was reading some business books, I think there's some amazing ones. And there are also some that suffer in the extreme from certain forms of survivor bias. A lot of them Yeah, yeah, for sure. Not just business books, books. So I particularly liked sort of studies like this one on comic books, that looks at what causes comic book creators, whether they are individuals or teams to do well on average, and to be more likely to have a blockbuster innovate. And the scientists like to this comic books had to have a very like, defined area of the start of creative explosion when there was this doctor who's kind of convinced Congress that in like the 60s that comic books were like making kids deviants, he fabricated some of his work, actually, but and so the, in order to avoid regulation, the comics industry started self censoring. But then in 1971, the government asked Stanley, they were worried about drug addiction and overdoses. And they asked him to make a spider man narrative about that address drug addiction. And so he made one where by Peter Parker's best friend, ODS, and the Comics Code Authority, the self censorship body didn't approve it. And they published anyway. And it was it was a hit. And suddenly the censorship went away, and floodgates are open. Now you start seeing complex characters with emotional problems, diverse characters, all these sorts of things. One is series I love love and rockets, where the characters age diverse cast at ages in real time with the readers. And so they studied starting at that point of creative explosion and tracking the value of comic books because you can track them easily whether they went up or down, which I think is a very important thing and the variants of, of creators, whether they made things way below their norm or way above. And so I tried to pick studies like that, that can track things if they go bad, too. And what that one of the interesting findings of that study was, the scientist made predictions about what would cause someone to be better on average, and more likely to produce a hit. And they predicted resources of the publisher would be more likely to produce a hit years of experience in the field, number of reps, number of creations they had done, and they could track individual creators. That would be true if it was like an industrial task. But turned out none of those things were true years of experience had no effect repetitions, had a slight negative effect when it got really high, maybe a sign of overwork or something, resource of the publisher, no effect. But what did was the number of different genres that a creator had worked across, they characterized comic books into 22 different genres, you know, whatever fantasy, adult drama, comedy adventure, and the number of different genres a creator had worked across, improve their performance, on average, the value of their comic books, on average, in the likelihood that they would make like a massive blockbuster that was way above their norm. And the really interesting thing to me, so I liked that because it followed people down and up. But really interesting was they asked the question will, can you recreate a broad individual with a team? Like if we get three people who have each worked in one different genre and put them together? Is that as good as one person who's worked in three different genres? And the answer was initially, yes. So if you have two people that work in one genre, each combined, they do better than one person who's only worked in, in one genre, on average, and more likely to innovate. But after four genres that crosses over an individual who has worked in more than four genres, becomes better than a team who by platoon has worked in the same number of genres. And so one of their conclusions is that the individual in some ways is the best unit for integrating information. And as that genre experience goes up, the individual becomes more and more, the broad individual becomes more and more valuable, even compared to the diversified team. So they titled The paper, Superman or the Fantastic Four. And they said, What do you want for creation? Well, you want the one Superman has worked across many genres, if you can, if you can't get that you want a fantastic team of broad individuals, basically, yeah,
I love that. I mean, it's it's a great closing story for the idea and the return on range as a concept and for people to encourage people to really collect things and try things that I'd love to close with something fairly unrelated. But just fascinating to me in reading your work, which was the story about I think it begins about implanting stents in people to high blood pressure with heart problems. And just this broad literature in the medical world of intervention, which has really no basis in factor empirical data, that let's say cardiologists, in this case, nonetheless, continue to do spending huge amounts of money and every year. And the reasons why. And you wrote this really thoughtful piece for pro publica that I recommend people read. And this is a an adjacent topic, and obviously different from what we're just talking about. But I'd love your thoughts, thinking back on the research for that one. And again, the obvious analogies in the investing world of very often, we're doing stuff that we think seems right to do, or we need to do something because things are going wrong, when very often these things are unsupported by the evidence, and maybe the best action is something totally different.
So in some ways, my interest in that topic, grew out of my interest in specialization, because the first thing I saw was, so I knew this certain type of repair of the meniscus, you know, this piece of fiber in the knee was one of the most popular if not the most popular orthopedic surgery in the world. And I started seeing these studies in Scandinavia, particularly, that would do placebo controlled trials of surgery, which is crazy, right? So they take people in who have torn meniscus, some of them get the surgery where it's repaired, or shaved back into like its crescent shape. And some of them get Sham surgery where they make an incision, bang around, make some surgical noises, so them backup and send them to physical therapy. And it turned out that those people did just as well. In fact, a year later, the surgeons couldn't even tell them apart from the people who had actually had surgery. And so I started to see that and say, like, that should be a sea change. And then I look at the number of times that procedures being done and like still going up, and then look at the specialist journals, and the ways that they would write off that kind of research. And they would write off that research with their experience. I've done this on patients, I've seen them get better, which is not a question the people in the study got better. Right? That wasn't the question, but seems like it's the PT, or the time Aha, for whatever. And so I became sort of alarmed at that where you would see. And then that led me this book called ending medical reversal, where these two doctors basically just catalogued a huge number of standard practices that were implemented based on either intuition, or low levels of evidence. And when better studies, the zoomed out, looked at what we really care about, it showed that these things didn't work. And yet, they continue to become more popular because specialists are so tied to them, even when their compensation isn't necessarily tied to them. So removing compensation from being tied to procedures has been something like the Cleveland Clinic has tried, and they still end up over treating with things like stents, which are, you know, these like metal tubes that did open an artery. And one of the reasons for that, I think, one is partly like the when all you have is a hammer, everything looks like a nail problem. It's like when people come in with stable coronary artery disease, or because a stent can save someone's life, and if they have a heart attack, but when they have stable condition, and they come in with chest pain, they're probably going to end up getting a stent if imaging shows that they have a narrowed artery. And over and over again, there's something like 12 randomized trials that show that it will open up their artery. But what you really care about is heart attack or stroke or dying. And they have heart attacks and strokes and diet exact same rate. The same thing shows up with one of the most popular blood pressure medications in the world that I mentioned that article where, because we've gotten so specialized in medicine, everyone usually is looking at surrogate markers, they're not looking at the outcomes you care about, they're not zooming out. So they're looking at did I lower blood pressure numbers, that's a surrogate marker, again, for stroke, heart attack, and death. And so some of these drugs, they lower blood pressure, and people die at the exact same rates of the exact same stuff with lower blood pressure numbers. So we need those specialists, because knowledge is exploding. But you also need people that are zoomed out looking at the bigger picture and looking at the outcomes we actually care about. And once I got to that book, ending medical reversal, and realized how common so many of these things that have been overturned, this should not be in practice anymore, are I kind of feel I've really, really got to write about this.
Is there anything that you think might change this trend? So this idea that we do stuff, we studied the stuff, we find that it doesn't work? And then it keeps happening? I mean, we try to write about it to stop that. But then it keeps happening again, Are there new strategies, because my interest in this is both health but also the investing parallels are so obvious that you don't even need to go through them? This just relentless amount of information and strategies and ideas that are clearly useless or wrong. Yet perpetuate. What do you think we're doing wrong? Trying to debunk this stuff? Why can we not beat the inertia?
I mean, I think in one of the reasons, like I use the last chapter in range of talking about doctors and scientists is partly because those people are viewed from the outside is like the epitome of specialists. And I wanted to get to say, okay, even within these things that compared to the world at large, are quite specialized, how do we harness some of the advantages of breath. And I think our total cost of all is one of the characters in that last chapter, who's arguably the most renowned immunologist in the world, went to Johns Hopkins, to try to start a new education program, where instead of teaching all this specialized techniques to future doctors and scientists, he starts with how to scientific thinking work more broadly, how do we evaluate evidence? How do we know what is true? And that I think that program is going to have a huge impact, partly because of his own renown. But I think I confess in the book that I when I was a grad student, I committed unknowingly statistical malpractice. Talk about a wicked learning environment on the project that got me a master's degree from Columbia, Okay, first, I was embarrassed about that. And I'm just mad about it. And it's because I was shuffled into this very narrow set of knowledge before I even knew what the statistical program I was hitting buttons on actually even did in a deeper sense. And I think recognition is important. I think it's gonna be generational change, I think it's going to take some generational change. But I think changing the way we educate doctors and scientists to teach, as Arturo says, All The World's knowledge is available on your phone, that people walking around with all the world's knowledge and no idea how to integrate it or evaluate it. And so he's de specializing education of doctors and scientists and saying, Let's start with how information is evaluated and how scientific thinking works. And I saw him on a panel saying that the editor, editor of the New England Journal medicine, was on the panel and said, No, you can't add all these classes about scientific thinking and stuff to doctors training. There's already too much time and New England Journal medicine, by the way, one of the most retraction prone journals in the world. And our true said, Yeah, get rid of all that other stuff. They can learn that stuff, like in practice, they need to learn how to think. And I think he's really hit on something. And he's noticed that we have no ability to correct because our specialists, not just our specialists in general, we don't have a good ability to evaluate new information. And so I think some de specializing of education is going to help, I think, but I think some of these problems are devilishly hard to the stent issue. So all these studies show that students don't work for stable disease, but they do work for heart attacks. So there have been examples of hospitals where the doctors are so sure that stents still work, right? They're so committed to this idea, because it's bio plausible. This is a term that Mike Joyner, a physiologist at the Mayo Clinic sort of uses that I really like bio plausible meaning. There's a clogged artery, how could opening it up not work. It's got to work except it turns out the body's much more complicated than like a kitchen sink, and we didn't design it. And it's the disease is much more diffuse. And so some of these doctors started saying show up making emergency room appointments, they'll say show up in the emergency room tomorrow at 3pm will give you a stent, because maybe insurance is starting to not cover it for stable anymore. So
they find ways around it. I think that's
tremendously difficult. And I think we need some of these what Freeman Dyson calls birds as opposed to frogs, right? He says we need birds in frogs, he's talking about math and specific, the frogs are down on the ground looking at like a very narrow area of the ground, the birds are up. They don't have a good definition on the ground, but they see the bigger picture. And I think we need to make the medical ecosystem more friendly to some of these birds who are looking at the outcomes we actually care about, not just those surrogate markers of did I fix the meniscus? Did I open the artery
Well, this has been just incredibly interesting. And I'll use this as a conversation especially to give to young people as sort of an ode to tinkering and range and curiosity and exploration, especially early in life and career. So thanks for all the fascinating examples in the great conversation.
It's my pleasure. And I'm sure, I mean, you can gather from talking to you that you're like a big reader, right? So obviously, you do this in your own life. And I'm sure it's I'd be curious what your background is and like what you studied because you obviously consume a lot of information have arranged yourselves.
I appreciate it. And we'll do that offline and hopefully do around two sometime. Great.