Why do some scientific breakthroughs take off while others languish in the shadows? In their paper, “Breakthrough Recognition: Competing for Attention,” Wharton management professor Anoop Menon and Sen Chai, a management professor at ESSEC Business School in France, analyzed more than 5.3 million research papers published in the life sciences between 1970-1999 to see what might be hindering some ideas from gaining wider recognition. Menon recently sat down with Knowledge at Wharton to discuss their findings. (Listen to the podcast using the player above.)
An edited transcript of the conversation follows.
Knowledge at Wharton: You have some previous research that looks at the nature of breakthroughs in the space industry. This latest paper looks more at research in the life sciences. What was the inspiration for this paper?
Anoop Menon: The inspiration comes from the observation that time and time again, we see great, really novel ideas come out in terms of just the raw quality of the idea. But then nobody picks it up. It makes no impact, no splash. It just gets lost….
My co-author, Sen Chai, who’s a professor at ESSEC, Business School in France, and I were really fascinated by that question. Why do some ideas get picked up while others do not, even if the quality is very comparable? So that’s what got us started.
And within that stream, [we focused] on the spread and recognition of idea, not the creation. That was the starting point.
Knowledge at Wharton: To study this question, you analyzed more than 5.3 million research publications, focusing specifically on citations. For people who aren’t in the research world, can you explain how those work?
Menon: The best way to think about it is when you write a document and you are building on or referring to a preexisting body of work, you have to give them their due. So that’s where you source them, or in the research world it’s called citing. And at least within the research community, that is considered a good indicator of the impact of your idea. Because if many people are citing you in their work, and building on your idea, then you can claim that your idea is having a big impact. So the more forward citations — the more people that cite you and your work in the future — the more impactful your work is considered to be.
Knowledge at Wharton: How did you take that huge data set and analyze it?
Menon: Yes, this was the entire corpus in the life sciences. So, we just went to the PubMed database which had this. There was a little bit of a data wrestling match involved because we had to do a fair bit of juggling with this data. It’s not as if we could just take it in as is and run analyses on it. we actually had to create a few variables which took those 5.3 million documents, and in each of them we identified the areas that it covered. And we then found all the combinations of the areas as well.
“Time and time again, we see great, really novel ideas come out in terms of just the raw quality of the idea. But then nobody picks it up.”
So, all of the sudden the complexity is blowing up. We’re dealing with billions and hundreds of billions of combinations. Yes, some computers broke down.
We had to come up with new, clever ways, of, let’s say, handling this big data. It was kind of fun.
Knowledge at Wharton: Did you have an idea of what you were looking for? How did you isolate these different factors to determine how breakthroughs were spreading or what might be hindering certain ideas from spreading?
Menon: Based on the intuition of having done research and having seen some ideas go through and what some ideas not go through, we had a rough sense that this notion that ideas compete for the attention of researchers was an important thing. But that hadn’t received much attention in the literature so far.
That, and coupled with our prior work in this similar space, there are these things called simultaneous discoveries where the same idea or breakthrough happens at multiple locations. Multiple research teams can come up with the same idea at the same time. There are some other really good studies which have studied that. And from there we know that, if the idea tended to happen in an area that people had already been doing a lot of work in, it might get lost in the crowd. While if this was in a relatively uncrowded area, it got picked up. So that was, an indicator that that might be something to look for.
Knowledge at Wharton: In your paper you cited one example where two sets of researchers found something, but based on the models used to find it, one group got a lot more attention. And I think there was a potato involved, somehow.
Menon: There was a potato and there was a worm, too. Yes. This is kind of a modern classic in the simultaneous discovery literature that I was telling you about just now. In the early 1990s there were a few teams trying to study RNA interference in gene silencing. So, there was a big discovery when the trigger mechanism to RNA interference of gene silencing was discovered. Two teams came up with the idea simultaneously.
One, as you said, was studying potatoes. And in that area the teams had been looking for similar things for a while. There were already a lot of publications in the keyword areas that that domain already looked at. While the other team was studying the worm called C. elegans as their model. And in the animal research world this was not something that a lot of people had been studying.
So, one was a crowded space while the other one was not. But it was the same breakthrough; at the deep conceptual level it’s the same idea, happening simultaneously in both places. Because the plant space was so crowded, the idea did not get picked up as much. In the animal space, it gets picked up, gets a lot of hype and ends up winning the Nobel Prize in the 2000s.
“Because the plant space was so crowded, the idea did not get picked up as much. In the animal space, it gets picked up, gets a lot of hype and ends up winning the Nobel Prize in the 2000s.”
Knowledge at Wharton: So, worm beat potato.
Menon: Worm beat potato, that’s right.
Knowledge at Wharton: Could you explain the two key effects that you found in this paper?
Menon: There were two main channels or biases, if you would, that we are empirically demonstrating. One is bias against novelty, which the prior literature has already done a pretty good job of finding. We replicate those findings. And here the idea is that the more novel or unusual a certain idea is, as reflected by how unusual a combination of ideas that topic is, the less likely it is to be picked up by somebody, because it doesn’t conform to paradigms. And then it gets dropped away. So that’s the bias against novelty perspective. We replicate that finding from prior literature.
The other one is what we call the competition for attention view, which is the notion that if you come out and publish in areas that are already very crowded — there are a lot of other researchers working on those topics — then somebody who is looking for your paper will be swimming in a lot more other papers that they need to search through. And thus, the odds of picking you are lower. And their attention is now split in many different ways. So, your paper has to fight or compete for that researcher’s attention. And that’s the second bias, which is if you publish in an area that is already very crowded, the odds of your ideas getting picked up are much lower.
Knowledge at Wharton: You had some inkling that maybe competing for attention and novelty might be having an effect here. But were there any findings that really surprised you?
Menon: So, a bit of background: We are not, by far, the first to study the spread of ideas. But a lot of the attention so far has gone into how the novelty of an idea affects the spread of ideas. The idea there is, there’s this concept called the bias against novelty. We all love to claim that, oh, really novel combinations of ideas, they’re wonderful, that’s where breakthroughs come from. True, in the creation of the breakthrough. But in terms of the spread of the breakthrough, if it is a bit too out there, it’s a bit too novel, it actually gets discounted. The chances of it getting recognized and spreading actually decreases if it is too novel. There is some really good work which has studied that.
But what was not studied was the attention view. And why this is interesting, why this was a bit surprising for us is, if you think about it — at a surface level — you can think about novelty and the crowdedness of a space as literally the opposites of each other. But we actually sliced it apart. We think of novelty as combinations of ideas. So, if you see really weird or rare combinations of ideas in a paper, then that would be considered a highly novel paper.
“In terms of the spread of the breakthrough, if it is a bit too out there, it’s a bit too novel, it actually gets discounted.”
On the other hand, we proxied for the crowdedness of the field by looking at how many times each of the keywords was being used at the individual level, meaning if there is a lot of chatter on a certain topic, that’s a crowded area versus not. Once you slice these apart, what was surprising is that both effects simultaneously held. You have, on the one hand, this bias against novelty coming through very strongly. But on the other hand, you also have this crowding-out effect of ideas also happening, and being very strong as well, to the extent that if on average a topic had 10,000 more citations — people talking about it — the likelihood of you popping up in the top 1% of impactful papers, or even the likelihood of you getting citations, dropped by over 10%. There were big effect sizes in both effects simultaneously. That was a bit surprising.
Knowledge at Wharton: What does this mean for the research and development space? Obviously, everybody can’t decide to focus their research on something that’s going to not get in the way of these two effects, which means you have to deal with it. What do we do? Does this mean a lot of potential breakthroughs are just getting ignored because of these two effects?
Menon: Some of the ways we try to think about that question are: Look, these effects are there. This is part of human nature and the sociology of research, how we do research, at least as of now. But as you said, that also implies there might be really good ideas that are already out there that have just not been picked up. Perhaps both these biases in combination could give us a compass, a map to then go look at what areas would these effects, these biases have been strongest in? And that might give us some hint as to where to go and look for previously published papers that are just not being picked up yet.
Another way that we thought about it is, yes, first be aware of the biases. And then maybe with all the advancements we’re making in AI these days, perhaps we can complement our search and recognition capabilities of researchers with some AI tools that can help beef up on our weaknesses. So being aware of the weaknesses might help us build some tools that can help us, at least reduce, if not overcome, those weaknesses.
Knowledge at Wharton: Are there also lessons here for policy makers, anybody that is allocating funding to different ideas or different research lines?
“Being aware of the weaknesses might help us build some tools that can help us, at least reduce, if not overcome, those weaknesses.”
Menon: Yea, absolutely. It’s a bit tricky, right? Because usually if you’re say, NIH (National Institutes of Health), in our case, and the health space, or more broadly, where would you want to allocate a bunch of research and a bunch of resources? It would be where a lot of the excitement seems to be. That would imply that would be in some of the more crowded areas, while the backwaters will not get much money, much funding.
Now put yourself in the shoes of a young researcher who is trying to decide, “What should I go into, what direction should I go?” Well, the crowded areas have a lot of money. On the other hand, the odds of my ideas getting picked up might be lower. That’s a tough place to be. So perhaps, being aware of these biases and effects might help us to spread out the research dollars a bit better so that that hard choice can be avoided.
Knowledge at Wharton: That is to say, maybe I shouldn’t ignore these more novel ideas or maybe I should be drilling down further in these spaces that are very crowded to find some things that aren’t always getting noticed?
Menon: Exactly. There is a caveat there though, which is that we also find that if you publish in areas that are not very crowded, yes, the odds, the mean effect of you becoming a blockbuster paper are higher. On the other hand, the variance is also higher — meaning, the risk goes up. So, there’s a risk/reward trade-off. If you go into those less crowded areas, you have a higher chance of hitting a home run. On the other hand, you also have a higher risk of just completely flopping too. Again, something to think about.
Knowledge at Wharton: This paper is really about the spread of ideas. So, are there also applications here for, say, marketers when they’re trying to understand how consumers make decisions?
“If you go into those less crowded areas, you have a higher chance of hitting a home run. On the other hand, you also have a higher risk of just completely flopping too.”
Menon: Oh definitely. In fact, some of the work that we’ve built on and where we drew some of these ideas are actually from the marketing space as well, and consumer psychology, where they have been really thinking deeply about attention, consumer attention, for a long time. And we know that these effects exist. And hopefully we can now bring together that line of research, which has focused a lot on the competition for attention, and less usually on the bias against novelty, which has more been in the domain of scientific research, and perhaps bringing the two together and in that way contributes back into the marketing literature as well.
Knowledge at Wharton: I guess that’s one possible future line for the research. Are there other future lines that you plan to explore?
Menon: We thought about this paper as a first foray, just to demonstrate that this other channel also exists called competition for attention. Spread of ideas is not just affected by novelty, but there is at least this one more thing. But there could be other things that we don’t know about yet.
Other things we also found which were interesting, is that the effect of this crowding out of the competition for attention effect — its potency — actually changes over time. It is much more of a problem in the first few years that a paper comes out. But by the time you’re 15 years out, that effect starts to wind down because I guess the quality of the idea has really started to come through anyway, so you don’t need to rely on these other channels. But that would be an interesting area to think through and get more detailed data on, like how these effects change dynamically over time.
And also, as you said, we have on the one hand a bias against novelty, on the other hand you have the competition for attention. How do the relative weights of these two effects change, because in some ways one drives in the opposite direction of the other, right? Are the relative effects more significant early on in the lifecycle of an idea? Or, when an idea matures, will one effect start to dominate the other effect? Those would be very interesting, at least for us, areas to try to probe further into.
