How Data Analytics Can Drive Innovation

Data presents an invaluable opportunity for firms to innovate, but only if they know what to do with it. In her latest research, Wharton professor of operations, information and decisions Lynn Wu looks at how different organizational structures influence the use of data analytics to spur innovation. Her paper, “Data Analytics Supports Decentralized Innovation,” is forthcoming in the journal Management Science and was co-authored by Wharton operations, information and decisions professor Lorin Hitt and Wharton doctoral candidate Bowen Lou. Wu spoke with Knowledge@Wharton about the research. (Listen to the podcast at the top of this page.)

An edited transcript of the conversation follows.

Knowledge@Wharton: Let’s start by talking more generally about the ways in which data analytics can help fuel innovation.

Lynn Wu: There are so many great examples. [There are] lots of analytics technologies, especially fueled by recent advances in machine learning and the vast amount of digitalized data. For the first time, I think a year or two ago, a machine powered by analytics was able to win against the best human player in the world in the game “Go.” We also have self-driving cars [that rely on] the large number of digitized images that improve vision recognition systems dramatically. We have seen IBM’s Watson culling through lots and lots of [research] literature in digital records and find six new cancer suppressors in two months. That would have taken researchers years to find. Even in the realms of art and music, for first time AI analytics is creating art that people are willing to buy and music that people are willing to hear. So, we do see lots of data-driven analytics creating a lot of really cool innovations around us.

Knowledge@Wharton: In terms of how companies can use data analytics to spur innovation, what was the central question or problem that you were trying to address with this research?

Wu: I just gave you some really good examples of [analytics-based] innovations that we are all aware of. But if you look at innovation statistics, economists have documented that we have been spending more and more money per capita on research, but we are actually having a decline in the rate of innovation we’re generating. We’re spending more money, but we’re getting much less in return. That seems like a paradox, right? We’ve seen lots and lots of cool, data-driven innovation, yet we don’t see any innovation statistics.

“Analytics is really great at finding these linkages or hidden patterns we may not easily observe by mining through a ton of data.”

That makes me think about a famous quote by Nobel laureate Bob Solow from three decades ago. He said that we see computers everywhere, but not in productivity statistics. If you replace “computers” with analytics, and you replace “productivity” with innovation, we have the same paradox. What I’m trying to do in my research is to see whether we can use the same set of frameworks to explain the analytics innovation paradox that we are observing today.

Knowledge@Wharton: In your paper, you note that there are many factors in an organization that could affect innovation, but you specifically focus on two ways that firms handle innovation: decentralized and centralized. What is the difference between those two?

Wu: There are definitely tons of factors that might mediate the relationship. I purposely picked decentralized and centralized innovation because there’s lots of work that’s been done on [the questions] what is the advantage of decentralization, and what is the advantage of centralization? What I define as decentralized innovation structures is based on their collaboration networks. You can think of inventors creating the same patent and working together, so there’s a link between them. When I describe a decentralized innovation network, it’s really about how concentrated the innovators [are when they collaborate]. You may see a big cluster with lots of inventors all working together, or you can see a very much decentralized or dispersed — the opposite of a concentrated — structure where there are lots of small clusters of individuals and they are loosely connected together.

The reason I picked those two structures is that neither one is absolutely better than the other. We’ve seen throughout many industries that very innovative firms have both types of structures. If you look at Apple, it’s very much a centralized, concentrated cluster with a small group of people responsible for a vast majority of innovation. But if you look at Google, you see a small group of clusters, and they are loosely connected. They are also very, very productive in terms of their innovation. You see that in pharma as well. You see Sanofi and you see Roche. Sanofi has a much more decentralized or dispersed structure, and Roche has a more concentrated structure. My question was, do these structures play a role in how they use analytics to innovate?

Knowledge@Wharton: You say that one’s not better than the other, but what did you find in your research in terms of how these structures impact innovation?

Wu: That’s getting into the key of what analytics can do for innovation. What I find is that analytics can really drive the creation of recombinations, or combining a diverse set of existing technologies in a new way. Each individual technology already exists, but how do we recombine them in some ways to create a new innovation? Or reuse something that we know solved one problem, but apply it to a different domain? Analytics is really great at finding these linkages or hidden patterns we may not easily observe by mining through a ton of data. That is really the key to driving decentralized innovation for several reasons.

“Analytics can really drive the creation of recombinations, or combining a diverse set of existing technologies in a new way.”

Decentralization’s advantage is that there are small clusters working on a problem, so they really know what the problem is in that domain. They see what exactly they can do to solve that problem more closely than a centralized structure, which is bigger but much more coordinated. A decentralized structure lacks the coordination. They know very well what they do, but they don’t know what other people are doing. Centralized structures know what everyone is doing, but they don’t know details for each individual problem in the domain, unless they have the capability to comb through lots and lots of data finding hidden patterns. That’s exactly the disadvantage that decentralized structures have. In that sense, decentralization does not easily find other people’s work. Analytics finds a way to cull through that and find you a new combination, a new way of solving your problem that you may not have easily found without analytics. That capability, of course, can also help a centralized structure. A centralized structure already has that search and coordination mechanism built in; it just doesn’t have as much higher marginal benefit as a decentralized structure would have.

Knowledge@Wharton: As data analytics becomes more ubiquitous, will it pay off for firms to move towards a more decentralized innovation structure?

Wu: That’s a great question. I think that depends on what your innovation goals are. If you were a decentralized structure and you really want to do innovation that combines existing technologies in a new way, or reuses technology applied to a different domain to solve a different problem, analytics is great for helping because you get both advantages. You deepen the problem domain and get diverse knowledge from outside.

But centralization is great at looking at bigger pictures and creating novel or de novo technologies that could act like a building block for future recombinations. They’re foundational technologies. That’s difficult to create with big data. [Centralized structures] don’t necessarily need to have big data to create that kind of technology.

Knowledge@Wharton: Like Apple, for example?

Wu: Exactly. Lots of it is creativity or human intuition that’s kind of hard to digitize. If you were in that type of innovation work, then having analytics or having a decentralized infrastructure wouldn’t necessarily help you. It depends on what your goals are. Because analytics is making it so much easier to recombine new technologies in a new way, that raises the value of the new foundational technology. Once I create it, it can be quickly exploited to make new combinations. There’s a trade-off between the two.

Knowledge@Wharton: Are there any contexts where you found that data analytics might impede innovation?

“We can use lessons learned from past generations of IT and analytics technologies to inform us about what the future could look like.”

Wu: We didn’t have any conclusive evidence that it impedes, but we definitely find that analytics does not help with building or creating de novo innovation that is foundational and can act like a future building block for future combinations. That is something that analysts are not great at. If you think about it, if something’s so new, it probably didn’t exist in data yet. So, there’s not much you can do with data analytics to help you find that pattern.

Of course, recombinations can be radical innovations; they can have a lot of profound impacts. Lots of innovations are recombinational, right? In that sense, I think analytics are really moving forward in what we could do to speed up the innovation process.

Knowledge@Wharton: What’s next for your research on innovation?

Wu: We are on the cusp of a really great change in technology, especially with the rise of AI and machine learning that is dramatically changing employment and how work is organized. I’m looking at analytics, including AI and the subfield of deep machine learning, to examine how we can use them more effectively to innovate. That is an emerging problem. We can use lessons learned from past generations of IT and analytics technologies to inform us about what the future could look like.

I only mentioned centralized and decentralized innovation structures, but there are lots of different ways of innovating. If we think of analytics or the large field of artificial intelligence, it’s a general-purpose technology. It will take decades for us to really understand how to apply it. Think about electricity. It took a long time for us to learn to use electricity effectively…. This really provides a ripe opportunity for us to examine what cutting-edge firms are doing with these technologies and how they innovate. And that can provide some key lessons for the future about what other firms can do to leverage analytics better to innovate.