What is the next big idea or market opportunity? The question plagues CEOs of all growth-hungry companies as they race to scoop their competition with a product or service that no one has thought of before. But how do companies know what products and services their customers want — and will be willing to pay for — next?

Henry Ford famously said, “If I had asked my customers what they wanted, they’d have asked for a faster horse.” In other words, the road to true innovation is rarely illuminated by customers telling you what to do next; they may often not know what they want next. But, experts from Wharton and Dallas-based George Group Consulting warn, unbridled proliferation in the hopes of “hitting it right” can lead companies into the trap of having too much complexity, which consumes existing resources and ultimately harms returns. It is a difficult tension to manage and one which requires a level of “ambidextrous” thinking. The key, they say, is for companies to identify the unmet and unarticulated needs of the customer and align their innovation processes to those insights. Companies must discover what innovations customers are willing to pay a premium for, identify their own competitive strengths and free up innovation capacity by removing or managing complexity within the organization’s products, services and operations. The potential reward is a better bottom line and increased visibility with customers, as companies invest in understanding customers’ needs while shedding the excess clutter that can bring down their rivals.

Ford’s words resonate even today with Dan Chow, senior vice president at George Group and leader of its Fast Innovation practice. As an example, he says there was no sure-fire way to know that customers “needed” an iPod, Apple’s MP3 player that has taken the market by storm. “While you might not be able to come up with the specific thing called ‘iPod,’ you can reliably generate iPod-like ideas by using different and varied sources of innovation fuel,” says Chow. “Challenging conventional wisdom and understanding core capabilities tend to be the fuel that helps companies generate out-of-the-box thinking to push them to new ways of uncovering customer needs.”

For example, says Chow, the real innovation with the iPod is its business model — tying music to great user design to great brand cachet. And behind that success is the fact that deeply understanding the customer is the first and most important capability a company can have to drive innovation, he says.

To illustrate that idea, Chow points to the example of a school supplies company that was challenged with commoditization of their product in the office and discount channel. Their product was purchased most heavily in the back-to-school season by students. Looking beyond the features and functions of the product in the hands of the student, to the frustrations, delights, wishes and concerns of students and their parents, led to an insight: There was an opportunity for the company to differentiate itself by addressing the broader needs of parents during what was an anxiety-ridden time of year. Armed with this idea, the company started selling complete solutions — integrated packages of school supplies — so that parents could do away with their lists and a bit of their anxiety. It also gave the parents more time — a precious resource for which they would happily pay a premium.

It is only through identifying these unmet needs that companies can continue to secure premiums in the market, Chow says, adding that this approach contrasts strongly with what is often a knee-jerk response to higher levels of commoditization: increased proliferation of features, attributes and product variants.

Distinguishing Innovation from Product Proliferation

But discovering what customers need isn’t always easy, and this is exacerbated by the fact that many companies often go down the wrong innovation path due to internal biases. Kevin Werbach, Wharton professor of legal studies and business ethics, says that Apple’s innovation strategy is distinguishing itself from conventional MP3 players by being “scored in simplicity” and resisting the influence of internal biases. Too often, consumer technology product makers tend to “over-feature” their innovations. “These are geeks who want to add the latest new thing and historically have a tendency to make their products too complex,” he says. “They are technology experts, and their tendency is to build the product they can use, which is not the mainstream product.”

Werbach says iPod is a “big counterexample” of that trend. “They didn’t try to put in every feature; they stripped it down and focused on a really great design and user interface, even though it by no means was the first portable digital music player.”

A simple and functional design similarly helped Google, says Werbach. “There were plenty of search engines before Google, but sites like Yahoo! and others got so clogged up with features that tried to get people to buy ads and buy other services,” he says. Google, he notes, stuck to a simple search offering. Users loved the plain screen with just a couple of links. Eventually, it paid off handsomely, notes Werbach: “As it turned out, Google figured out a way to do it and still become more profitable than all those other [rival] companies.”

In some senses, Google had a “me-too” offering, but it clearly did not replicate its predecessors. David Reibstein, professor of marketing at Wharton, says unforeseen perils await those who blindly follow their competitors. He recalls how PepsiCo followed Coca-Cola into the seemingly promising market for “half-the-calories” beverages. “The problem was that there was no market,” says Reibstein. “Pepsi went chasing Coke into a dead end and ended up wasting millions of dollars.”

Reibstein says companies often launch new offerings because they are afraid they’ll miss the boat. He recalls how a few years ago there was heightened speculation about the emergence of a paperless society, where credit cards would replace cash and so forth. He also talks of similar expectations in recent years of video-on-demand technology replacing video stores. “Markets are much slower to change than people would ever imagine,” he says.

Experts at George Group note that without a strong understanding of what value new products are providing, it is very easy for companies to drift off track; they may be simply ‘going through the motions’ in their innovation practices. A key question, they say, is: Are you innovating, or are you simply creating more SKUs (stock-keeping units)?

Much complexity — seen or unseen — is the result of going too far with what companies misread as innovation, says Matt Reilly, a senior vice president at George Group. “Some [corporate] leaders mistakenly confuse innovation with product proliferation,” he says. The problem, he explains, is when companies don’t really understand what their customers perceive as value and what they are willing to pay for. “Companies wind up over-featuring and over-developing to the extent that they become focused only on building something new, not necessarily building something profitable or building something valuable.” That gets further complicated, he says, when companies don’t get a grip on “how well they can actually execute that [innovation].”

Reilly notes that, unlike three to four years ago, the stock markets don’t value revenue growth alone; they now reward profitable growth. “Yet, a lot of these product companies are focused on driving top-line growth; the cost and the complexity of actually executing all these new offerings is typically not factored in,” he says. That situation persists in many companies, he adds, because “marketing people are often dangerously disconnected from operations and are rewarded for remaining that way.”

Finding the ‘Heart’ of the Customer

How can companies begin identifying customers’ unmet needs, and particularly those they are willing to pay for?

According to the book Fast Innovation: Achieving Superior Differentiation, Speed to Market, and Increased Profitability, by George Group’s Michael L. George, James Works and Kimberly Watson-Hemphill (McGraw-Hill, 2005), sources for analyzing customers’ needs might include ethnographic studies; face-to-face interviews with end-users and customers; diaries and intercepts; and expert advice and trend analysis on technology and markets. These help companies measure, explore and make tradeoffs among customer requirements, the authors write. Where differentiation in offerings is calibrated carefully to customer needs and fast-tracked to market, there is larger-than-usual opportunity to realize premium prices before commoditization.

What the authors found was that many companies spend too long in development time, and too little time and money in the upfront stage, leading to an inadequate understanding of customer needs. Companies are then compelled to commit more investments post-launch as they begin to understand customer responses and tweak their offerings.

Jason Santamaria, engagement director in George Group’s Fast Innovation practice, recalls an initiative to uncover customers’ underlying needs in a recent consulting assignment. The firm in question was a $400 million telecommunications equipment company that faced inadequacies in its own understanding of a particular customer’s needs. The customer — a telecommunications service provider — was equally lost about what it wanted.

But Santamaria’s client had one advantage: It possessed the expertise to deal with the highly technical and complex nature of its products, a capability that the customer lacked. “One of the ‘A-ha!’ moments was helping the [client] company realize that one, the customer was uncertain of its [own] needs, and two, it needed to work with the customer in a collaborative fashion to uncover those needs,” says Santamaria. The first step was to bring his client’s engineering managers into direct contact with the customer so there was no filtering of information between them. The telecom equipment maker and its customer adopted a “rapid prototyping” process in which the customer was presented with iterative prototypes of the product during development. In this case, it was a telecommunications traffic analysis tool.

According to Santamaria, “Instead of the traditional ‘waterfall approach’ where you receive a set of requirements from the customer and you go out and develop it and present the final product to the customer, we adopted a spiral design method where the engineering manager was in regular contact with the customer, and we repeatedly placed prototypes in front of the customer and got that customer’s feedback.” The end result, he says, was a final product delivered in a dramatically more accurate manner than the waterfall approach would have yielded. The takeaway, he notes, is that the key to customer insight development is to look across the entire value chain for insights, and not rely on your channel partners to do the consumer work for you.

Giving Customers the Faster Horse

Without a broader view, innovation efforts may be reduced to simply reacting to every little request from customers. In fact, argues George Group’s Stephen Wilson, co-author of Conquering Complexity in your Business, many companies are still responding to customers’ requests for a faster horse — to borrow the Ford analogy — without a view to the impact on the company’s costs and processes, nor to the long-term strategy of the company. Complexity creeps in, and these companies wake up to a sprawling portfolio with burgeoning costs and unintentionally find themselves unable to serve customers well.

One company, operating in a fairly mature market, “had built a strategy of having the broadest possible offerings and being very responsive to the customer,” he says. “But they were not necessarily being innovative.” The company was engineering-focused and tended to respond to every customer request for a product variation, Wilson notes. “They were so busy doing the incremental changes that they had no time to really innovate. Over time, it absorbed their capacity to innovate.” That also left the company vulnerable to competition, which scored better on speed to market and customer service. “By trying to respond to every customer’s specific requirements, they lost out on addressing customers’ more basic, fundamental needs.”

Moreover, he adds, the company was not able to generate premium margins sufficient to cover the incremental costs of complexity. “Decisions based on incremental revenue were driving massive amounts of waste in the organization,” he says. Inventories ran into “millions and millions of parts.”

In such a situation, Wilson recommends getting a clear perspective on what is really important to the customer and what they will pay premium prices for. Internally, management needs to adjust its focus from chasing market share to “value share,” which means getting an increasing share of the margins available in that market segment.

When it comes to matching customer needs with what could justify premium pricing power, Reilly points to a big disconnect that many companies fail to see: They often have detailed data on size of the market, pricing and trends, but don’t often have good end-user data. One reason for that, he says, is that sales teams working the distribution channels don’t always have an incentive to present the truest picture on the ground.

“If a competitor is gaining market share in a certain distribution channel, and if the sales team is meeting its quota, there isn’t a big incentive to report that,” he says. The situation gets worse particularly with consumer goods, where third-party logistics companies handle product deliveries at the retail level, which has the closest interface with the customer. “So companies end up making decisions at the macro level, not at the distribution channel or store level,” says Reilly.

Managing Complexity

Wilson points out that complexity can be an organizational drag, consuming resources, diluting focus and impacting profitability. In that way, it can be a drag on innovation efforts. But conversely, he notes, it is important to understand how the current innovation system helps or hinders the issue of complexity. “In many situations, the innovation system itself can be one of the drivers — a poor innovation system can lead to clutter and complexity,” he says.

Next, companies must get a grip on what causes that complexity. “Is it a lack of customer knowledge, or poor understanding of the economics of the situation?” asks Wilson. Additionally, he says, they need to get an accurate picture of the real effects of complexity.

There are corrective strategies for complexity, he notes. “One of them is to reduce complexity in your portfolio or in your processes. But reducing your portfolio is only one strategy, and it may not be the right strategy for your organization.”

Another strategy, says Wilson, is to “make your complexity more approachable for the customer and make the choices digestible.” Indeed, there exist ways to empower the customer to comfortably deal with the full range of a company’s offerings.

Wharton marketing professor Barbara Kahn says discovering that golden mean of how much is not too much is the trick. “If it’s too much, they won’t deal with it; if it’s too little, then they may be able to deal with it,” she says of customers’ buying patterns.

That’s where customer expertise comes into play, according to Kahn. “One of the factors that makes [a higher number of offerings possible] is expertise,” she says. “The more people become experts, the more they articulate their preferences — and the more they have a consumption vocabulary and know what the relevant attributes are, the more variety they will be able to take.” She also suggests “arranging [product] assortment in such a way that consumers just see what it is they want and they don’t have to see all that they don’t want. Websites are really good at that.”

Kahn likens the process of empowering customers with how salad bars help patrons navigate a mind-boggling range of options. “If you thought of all the different kinds of salads that you could make, and you presented [customers] all the different options, people wouldn’t be able to deal with that — there would be too much variety,” she says. “But if you do it the way [restaurants] do with salad bars, and divide salads up into attributes … they can deal with that variety because they can deal with those different attributes.”

A reference point in the form of an expert opinion could help in such situations, Kahn adds. “Even if you don’t take what they recommend, it gives you a starting point, and you don’t have to deal with the entire set of offerings,” she says. “You can tweak that starting point.”

Kimberly Watson-Hemphill, vice president at George Group and co-author of Fast Innovation, recalls one client that was able to successfully “tweak” a market niche for itself, thinking outside the box in its commoditized world and differentiating its product along a different dimension. The client, a pharmaceutical company that had a “me-too” product coming to market, was able to implement a process to allow customers to get the product covered by their health insurance policies. “They had a process by which they could frequently get fast insurance approval, when it would have typically been a time-consuming, uncertain process,” she recalls. “So customers would buy this product instead of their competitors’ products, which weren’t so differentiated on the basis of standard product performance.”

But for those that don’t find such unique fixes, the easy answer is not necessarily to throw out SKUs, warns Wilson. He says a flawed innovation system driven by internal processes — rather than by what the customer wants — could be generating those SKUs.

Companies that take the quick route to de-proliferate their offerings in an attempt to reduce complexity might end up returning to the same situation two years later, according to Wilson. That could lead to another danger, he says, of “cutting too shallow or too often.” He warns companies not to underestimate customers’ memory of portfolio changes. “The last thing you want to do is reduce some of the complexity, and then two years later tell the customer, “We didn’t do it properly the last time; we’re doing it again.”

Forecasting 2020 U.S. County and MSA Populations