How do we know which of our successes and failures can be attributed to either skill or luck? That is the question that investment strategist Michael J. Mauboussin explores in his book The Success Equation: Untangling Skill and Luck in Business, Sports, and Investing. Wharton management professor Adam M. Grant recently sat down with Mauboussin to talk about the paradox of skill, the conditions for luck and how to avoid overconfidence.
An edited transcript of the conversation follows.
Adam M. Grant: Michael, we’re delighted to have you here today to talk about your book The Success Equation…. I would love to hear you speak a little bit about this paradox of skill that you have discovered and the relationship between luck and skill.
Mauboussin: Let me first tell you what the definition of the “paradox of skill” is. Specifically, it says that in activities where skill and luck define outcomes, as skill improves, luck becomes more important in determining outcomes. [By that definition,] more skill means more luck, which seems very paradoxical. It’s not my idea. I learned about it from Stephen Jay Gould, the very eminent biologist at Harvard. He talked about it in the context of Ted Williams, the last player to hit over 400 in major league baseball, which he did in 1941. Gould was wondering why no one has been able to achieve over 400 since that time. He looked at [variables such as] maybe because the guys play at night, or they travel too much. Really, none of those things checked out. Then he said, maybe Williams is just this amazing player — an immortal among mortals…. But if you look at every other sport, for example, things measured against the clock — there has been absolute performance everywhere you look, so that doesn’t seem to be the case. Then he thought about it more carefully, and he realized the actual result is because everyone’s gotten better, and as a result, the standard deviation of skill has actually narrowed. If you think about batting average for your season and your player, some level of skill plus some level of luck gives you your outcome. What’s happened generally is that the standard deviation of skill has gone down. Why? Because you’re recruiting players from the world now, versus from just parts of the United States. You’re training better. You’re coaching better — all those kinds of things….
The point is this paradox of skill. We’ve seen the differential skill narrowing. We see it really all over the place. We see it in the world of investing. We see it in the world of business. I think it is very interesting. As skill improves, especially in competitive markets, luck becomes more important determining outcomes.
Grant: It’s incredibly interesting and much the opposite of what most of us want to believe. [How do you apply this] as a business professional or a leader?
Mauboussin: Well, a couple things are possible. One is to think about finding fields where there is differential skill. If you see something that’s very highly competitive, you really need to have something completely different to get you on the right side of the tail of the skill distribution. The easiest thing is probably to think about things where there is differential skill or to try to attack a more skillful player using an unusual tactic, for example.
Grant: That makes a lot of sense. Obviously, there are some outliers who don’t fit this picture. I think of Miguel Cabrera winning the Triple Crown this past year. Is that luck? Or is there a way still to cultivate skill, even though you’re dependent on luck more than before?
Mauboussin: Athletics is a great example. One of the points I would make, which I think is pretty common sense once I say it, is that whenever you see an outlier in sports, it is always a combination of really good skill and really good luck. One of the best ways we can measure that is through streaks. For example, Joe DiMaggio had this 56-game hitting streak, but if you look at all the players in major league baseball history who have had 30 or more hitting streaks, their career batting average is over 300. They are about one and a half or two standard deviations away from the average. To say it differently, not all skilled players have streaks, but all streaks are held by skillful players. You can almost be assured that whenever you see really, really good results, it’s skill and luck combined. By the way, you almost never see it on the other side, which is really bad luck and really bad skill, because those people either metaphorically or literally have to die off in the population. We mostly see the outliers on the positive side versus the negative side. We don’t see the failures so much.
Grant: Are there ways that you can improve your luck?
Mauboussin: Maybe I should step back and define luck because I think it’s actually a fascinating topic. When you think about luck or read about it, it really spills into philosophy, — moral philosophy, very rapidly. By the way, there are tons of aphorisms about luck. Luck is where success meets preparation, and you make your own luck. Those aphorisms, while they have an important sentiment, are actually not accurate. I’m going to define luck. Luck exists when three conditions are in place. Number one is that it happens to an individual or organization, so it could be you or your team or your company. Second, it can be good or bad…. The third, and I think really essential [condition], is that it’s reasonable to expect a different outcome could have occurred. When those three things are in place, there is luck. Now, by my definition, another, simpler way to think about it is: What’s in your control versus what’s out of your control? Luck would be what’s out of your control. When you hear people say, well, you make your own luck, what they’re encouraging you to do is work harder or be persistent or be gritty. Those are all really important things. But if that’s within your control, in a sense I’d put that into the skill bucket.
How do you manage luck? I’ll share a couple of ideas. One is that there’s a really simple heuristic that when you are the favorite, the stronger player, you have positive asymmetric resources, you want to simplify the game. When you’re the underdog, you want to complicate the game. That would be one example. Of course, the canonical example is David versus Goliath, which is really a wonderful story if you’ve read the whole story. David comes up, and he’s delivering stuff to his brothers, and there’s this ruckus going on. What’s going on? There’s this guy, Goliath. He’s a 6’5″ dude, 130 pounds of armor, threatening everybody. David says, “Yeah, I can take this guy on.” Originally, they put him in armor. He’s going to go toe to toe with Goliath. He’s this little skinny shepherd boy. David gets this quickly. [He realizes that] this isn’t going to work. He immediately takes off all the armor. Of course, [he draws on] his famous sling shot, and takes five stones from the creek. Then he goes out and uses his own technique.
In business, that would be an example of disruptive innovation. Rather than going straight at the leader, you come at [him] with a flank strategy. In warfare, it would be guerilla strategies versus, again, toe to toe. In football, it might be trick plays versus running straight down the middle…. You [might] say it’s common sense, but it’s remarkable in business and sports and even in military how underutilized that strategy is.
Grant: That raises another interesting question. You’re familiar with the really robust evidence for overconfidence. David Dunning and his colleagues have done some of the most interesting [studies]. About 90% of any people you would ask would assume they are above average on any given attribute — intelligence, skill, and so on. What you’re saying basically requires people to assess whether they are a favorite or an underdog. We know people are biased toward thinking that they’re favorites. How do you temper that level of overconfidence to make good judgments about how to change the rules of the game?
Mauboussin: That’s a really interesting [question]. One of the frameworks I like [to use] is going back to the Daniel Kahneman-AmosTverskyidea of inside versus outside view. You’re very familiar with that. The inside view says that when we’re trying to solve a problem or tackle something, the typical way that we do it — and this is where overconfidence plays into this — is that we gather lots of information about the situation, we combine it with our own inputs and then we project into the future. It’s almost — “idiosyncratic” might be strong, but it’s your own point of view.
The outside view, by contrast, says, “I’m going to look at the problem as an instance of a larger reference class. I’m going to ask what happened when other people were in this situation before.” That’s one of the ways to temper that overconfidence. Rather than looking at this as my own unique situation where I think I’m above average, I’m going to look at what’s happened to everybody else who has tried this before. Now, I actually am ambivalent about this argument on one level because if you’re an entrepreneur, we know that a high percentage of them are going to fail. But we know that some small percent are going to succeed and create an enormous amount of value for society and so forth. You want the entrepreneurs to get out of bed in the morning and say, “I’m going to go take the mountain.” But if you step back and said, “Oh, statistically, that’s probably not a great [chance]” — I’m a little bit ambivalent about the argument, but that would be one of the ways to try to temper that.
The other thing I’ll mention quickly on this is the “under-sampling” of failure — which is that people, a company or team will pursue a particular strategy, and they’ll succeed wildly. Another team or company will pursue a very similar strategy and fail. But, of course, the failures go away. What happens is you walk along and say, okay what strategy works? You see that strategy and you see success and so you say, “Oh, that’s got to be great.” You under-sample failure. That’s another way to temper some of the thinking and say, “I want to understand the entirety of what’s happened with this strategy,” for example. Those might be some ideas about how to mitigate the overconfidence.
Grant: That’s very helpful. It actually connects to one of your other really interesting points in the book, which is about using better statistics. Can you talk to us a little bit about how you would do that as a business leader?
Mauboussin: We’re awash in statistics. You watch a ball game or you read the business page of a newspaper, and we know that they’re not all created equally. So what makes for a useful statistic? What I basically argue for — I got this from the sabermetrics guys, the sports statistics guys — is that you really want two things. One is that you want persistence and the second is that you want predictive value. Persistence simply means that the actual statistic is correlated from one period to the next. For instance, if I know your batting average for 2012, it would correlate highly with your batting average in 2013, or how well it would correlate would be a measure of persistence.
The second thing is predictive value. You want that statistic to actually correlate highly with the objective you’re trying to achieve. For example, in baseball, on offense you’re trying to generate runs. The question is how well does batting average correlate with run production? Let me tie this back to [the film] Moneyball. There are a lot of different themes in Moneyball, but one of them was a simple one, which is that one base percentage is a better measure of performance than batting average. What they found was on-base percentage has a higher correlation from one season to the next, which means it’s more indicative of skill than batting average, so it passes the persistence test. Secondly, on-base percentage actually correlates higher with run production than does batting average. That’s going to say that is a superior statistic because it’s more persistent and it’s more predictive. I should have backed up and said that high persistence is almost always indicative of high skill, and low persistence typically [is indicative of] lots of luck.
Grant: Let’s take an exception to that rule, which is when you think about the interdependence of different players on a baseball team, or for example in a company, too, you could assume that on-base percentage is attributable to skill, but then you end up batting right after another highly talented batter, and that’s going to increase in general your on-base percentage, right? How do you decouple the individual scale from the context in which you find yourself?
Mauboussin: Super difficult, right? When I did the work on on-base percentage, I actually did try to size up the problem you just articulated. I did it on a team level versus an individual level. Of course, they’re going to roll up on some level, but you’re right. It’s a major step in the right direction, but in some ways it can be a fairly blunt instrument. The really careful statistical people try to understand exactly if you’re batting in a different order what impact will that have, and they try to measure that out and extract that effect.
But this also leads to another point that’s broader…. For example, take sports that are very simple. Tennis is one on one. We have a large sample — if we play a five set match, there’s a large sample size, right? So we pretty much know by the end of the tennis match which of us has been more skillful. But you get into a football game, there are a lot of players, a lot of interaction, and identifying those effects of individuals is a vastly more challenging task. Again, it doesn’t mean you shouldn’t try, or try to gain some insight into doing that. But, you’re right, the degree of difficulty goes up as you add complexity — and of course, with corporations, it’s the same thing. There’s a lot of moving parts; it’s very difficult often to say that this, that or the other caused one thing or another.
Grant: You mentioned Danny Kahneman’s work earlier. He’s added another really interesting variable to the equation, which is, are you working in an environment that’s stable and predictable or much more turbulent? He’s made the point that you can rely on your skill and your expertise and your intuition much more in a predictable environment. But most of us don’t have the luxury anymore of working in these very predictable environments. How do you think about navigating a more uncertain world?
Mauboussin: This is such a fascinating topic. I wrote a chapter about this in my prior book and I called it the “expert squeeze.” I basically said the main way to think about expertise is to think about precisely that continuum you just laid out. In some fields, the environment’s stable. I like to say it’s stable and linear, so cause and effect are very clear. In those realms, you can train your subconscious to be really good. The challenge is that increasingly, especially in a business setting, computers are taking on those tasks, professional tasks, and can do them very efficiently and cheaply. Experts are good at those [tasks], but increasingly there’s an encroachment from technology. That’s a challenging situation.
The opposite extremes you point out are environments that are unstable and non-linear. There, we know that experts are very poor predictors. There’s really no way to train your system one, your subconscious. I always love to make this distinction — especially since I’m in the finance business — the big deal between experience and expertise. People often think that experience and expertise equal each other, and that’s true on the stable side of the continuum. But when you’re on the unstable, non-linear side of the continuum, you don’t really have a predictive model that works. The key to expertise is having a predictive model that works. There we know that experts do very, very poorly in their predictions. By the way, this is early 2013, this is the time of year everybody’s making predictions about what’s going to happen, and of course, most people who keep track of those things know that they’re notoriously very poor.
What’s interesting, though, is what we’re seeing in some cases called the wisdom of crowds, that collectives can be more effective than experts in making judgments in those kinds of areas under certain conditions. I call this the expert squeeze, because experts are getting squeezed on this unstable, non-linear side by wisdom of crowds properly harnessed. They’re getting squeezed on the other side by computers and technology. There’s less space in the middle for the experts to navigate than they used to have. This is the fundamental first question to ask: Where is the problem I’m trying to think about on that continuum from stable [and] linear to unstable, non-linear? That really dictates a lot about how you should think about solving it and by what means and techniques as well.
Grant: To build on that, it seems like unlearning is a big part of the equation there. There’s some work that Nancy Rothbard here at Wharton was involved in showing that when people move from one company to another, they end up getting hurt by experience because they carry a lot of baggage with them about what worked in a particular context that’s no longer relevant to their new context. Do you have any wisdom to share about how to unlearn some of those things?
Mauboussin: No, I don’t, but obviously this is a big theme in all of social psychology, which is the context of where you are is incredibly important in shaping the decisions that you make. You have certain experiences; you’re socialized within an organization a certain way. Those experiences and even imprintings … deeply carry you through your decision-making for often the rest of your career. It’s a really hard thing to unlearn. But it can be at the same time very useful just to be mindful that while we’d love to think of ourselves as rational and objective and fact-based in our decisions, social context — be it new or old organization or whatever’s going around you — is deeply influential in how you decide. That inserts a lot of humility but maybe raises awareness to help people get more effective at making their decisions.
Grant: You work in the world of finance. How do you take all this knowledge and apply it to achieve success in your own job?
Mauboussin: There are a number of different angles on that. One is what I like to call macro-aware but macro-agnostic, which is to say spend as little time as possible predicting big things in the world. You have to be aware of what’s going on obviously and how those things may in fact have an impact on various scenarios that might happen for a company or an economy. But try to be macro-aware, macro-agnostic. The second thing is just thinking about what statistics are useful. For example, which money manager is likely to succeed in the future? We know that past results are typically an ineffective way to anticipate future results. But an isolation on process — a manager’s process — might give us a better insight. There are statistical ways that we can start to get a glimpse at process that can be very helpful.
[Also,] you mentioned overconfidence before…. That’s also rife obviously in the investment business. Even as an analyst trying to anticipate a company’s performance, one of the classic ways that that shows up is people project ranges of outcomes, for example, sales growth rates or profit levels, that are vastly too narrow. In other words, they’re overconfident in their own ability to understand the future, so just getting people to widen out those ranges, to think more robustly about that can be very, very helpful. There’s almost no facet of finance where these ideas don’t touch and can’t help. Very hard to do, but awareness and tools and techniques to try to manage it, to minimize the mistakes, is of great value.
Grant: You manage to do all of this and keep up to date on the latest evidence that might inform these practices. How do you juggle these two things simultaneously?
Mauboussin: Part of it is that I’m pretty bad at everything. A lot of it is just a natural curiosity. Also one of the things that’s been very helpful for me is teaching as an adjunct, so I’m not a real professor like you are — a heralded professor — but being an adjunct for me has been very helpful. In part, the way I think about it is to try to take the very best of what academics bring to the table and the very best of [what practitioners bring]. What academics do that’s very helpful is tend to be rigorous, using the scientific method to understand and explore ideas. But they’re not always totally practical. What the practitioners bring to it is [to say], “Hey, we have to make money and we have to have a practical angle on it.” Taking the best of both of those worlds and trying to combine them is what’s been for me the most satisfying aspect of this.
Again if there’s something you can draw from the world of academia that can improve your performance in some way, that’s great. I’m about to start my 21st year teaching at Columbia. When I started there, there was really no behavioral finance program. In fact, I often recommend the students take negotiation courses, because that was the closest you got to sort of tapping into some of these ideas from what we now call behavioral finance. That’s obviously come a long way. These are extraordinarily useful ideas, but shockingly there are whole generations, including my own generation, of people who never learned this in the classroom. Unless you go out on your own, in effect, and learn these things now and try to put them into your process, you have a blind spot in a lot of your decision making. That’s also a fascinating thing. A lot of people running corporations have never learned about these ideas, and they have this blind spot. Trying to fill that in a little bit has been a really fun activity.
