In an article on Oct. 16, 2000, in the Financial Times’ Mastering Management series, Wharton accounting professors Christopher Ittner and David Larcker suggest that financial data have limitations as a measure of company performance. The two note that other measures, such as quality, may be better at forecasting, but can be difficult to implement. Below is the text of their article.
Choosing performance measures is a challenge. Performance measurement systems play a key role in developing strategy, evaluating the achievement of organizational objectives and compensating managers. Yet many managers feel traditional financially oriented systems no longer work adequately. A recent survey of U.S. financial services companies found most were not satisfied with their measurement systems. They believed there was too much emphasis on financial measures such as earnings and accounting returns and little emphasis on drivers of value such as customer and employee satisfaction, innovation and quality.
In response, companies are implementing new performance measurement systems. A third of financial services companies, for example, made a major change in their performance measurement system during the past two years and 39% plan a major change within two years.
Inadequacies in financial performance measures have led to innovations ranging from non-financial indicators of “intangible assets” and “intellectual capital” to “balanced scorecards” of integrated financial and non-financial measures. This article discusses the advantages and disadvantages of non-financial performance measures and offers suggestions for implementation.
Advantages
Non-financial measures offer four clear advantages over measurement systems based on financial data. First of these is a closer link to long-term organizational strategies. Financial evaluation systems generally focus on annual or short-term performance against accounting yardsticks. They do not deal with progress relative to customer requirements or competitors, nor other non-financial objectives that may be important in achieving profitability, competitive strength and longer-term strategic goals. For example, new product development or expanding organizational capabilities may be important strategic goals, but may hinder short-term accounting performance.
By supplementing accounting measures with non-financial data about strategic performance and implementation of strategic plans, companies can communicate objectives and provide incentives for managers to address long-term strategy.
Second, critics of traditional measures argue that drivers of success in many industries are “intangible assets” such as intellectual capital and customer loyalty, rather than the “hard assets” allowed on to balance sheets. Although it is difficult to quantify intangible assets in financial terms, non-financial data can provide indirect, quantitative indicators of a firm’s intangible assets.
One study examined the ability of non-financial indicators of “intangible assets” to explain differences in US companies’ stock market values. It found that measures related to innovation, management capability, employee relations, quality and brand value explained a significant proportion of a company’s value, even allowing for accounting assets and liabilities. By excluding these intangible assets, financially oriented measurement can encourage managers to make poor, even harmful, decisions.
Third, non-financial measures can be better indicators of future financial performance. Even when the ultimate goal is maximizing financial performance, current financial measures may not capture long-term benefits from decisions made now. Consider, for example, investments in research and development or customer satisfaction programs. Under U.S. accounting rules, research and development expenditures and marketing costs must be charged for in the period they are incurred, so reducing profits. But successful research improves future profits if it can be brought to market.
Similarly, investments in customer satisfaction can improve subsequent economic performance by increasing revenues and loyalty of existing customers, attracting new customers and reducing transaction costs. Non-financial data can provide the missing link between these beneficial activities and financial results by providing forward-looking information on accounting or stock performance. For example, interim research results or customer indices may offer an indication of future cash flows that would not be captured otherwise.
Finally, the choice of measures should be based on providing information about managerial actions and the level of “noise” in the measures. Noise refers to changes in the performance measure that are beyond the control of the manager or organization, ranging from changes in the economy to luck (good or bad). Managers must be aware of how much success is due to their actions or they will not have the signals they need to maximize their effect on performance. Because many non-financial measures are less susceptible to external noise than accounting measures, their use may improve managers’ performance by providing more precise evaluation of their actions. This also lowers the risk imposed on managers when determining pay.
Disadvantages
Although there are many advantages to non-financial performance measures, they are not without drawbacks. Research has identified five primary limitations. Time and cost has been a problem for some companies. They have found the costs of a system that tracks a large number of financial and non-financial measures can be greater than its benefits. Development can consume considerable time and expense, not least of which is selling the system to skeptical employees who have learned to operate under existing rules. A greater number of diverse performance measures frequently requires significant investment in information systems to draw information from multiple (and often incompatible) databases.
Evaluating performance using multiple measures that can conflict in the short term can also be time-consuming. One bank that adopted a performance evaluation system using multiple accounting and non-financial measures saw the time required for area directors to evaluate branch managers increase from less than one day per quarter to six days.
Bureaucracies can cause the measurement process to degenerate into mechanistic exercises that add little to reaching strategic goals. For example, shortly after becoming the first US company to win Japan’s prestigious Deming Prize for quality improvement, Florida Power and Light found that employees believed the company’s quality improvement process placed too much emphasis on reporting, presenting and discussing a myriad of quality indicators. They felt this deprived them of time that could be better spent serving customers. The company responded by eliminating most quality reviews, reducing the number of indicators tracked and minimizing reports and meetings.
The second drawback is that, unlike accounting measures, non-financial data are measured in many ways, there is no common denominator. Evaluating performance or making trade-offs between attributes is difficult when some are denominated in time, some in quantities or percentages and some in arbitrary ways.
Many companies attempt to overcome this by rating each performance measure in terms of its strategic importance (from, say, not important to extremely important) and then evaluating overall performance based on a weighted average of the measures. Others assign arbitrary weightings to the various goals. One major car manufacturer, for example, structures executive bonuses so: 40% based on warranty repairs per 100 vehicles sold; 20% on customer satisfaction surveys; 20% on market share; and 20% on accounting performance (pre-tax earnings). However, like all subjective assessments, these methods can lead to considerable error.
Lack of causal links is a third issue. Many companies adopt non-financial measures without articulating the relations between the measures or verifying that they have a bearing on accounting and stock price performance. Unknown or unverified causal links create two problems when evaluating performance: incorrect measures focus attention on the wrong objectives and improvements cannot be linked to later outcomes. Xerox, for example, spent millions of dollars on customer surveys, under the assumption that improvements in satisfaction translated into better financial performance. Later analysis found no such association. As a result, Xerox shifted to a customer loyalty measure that was found to be a leading indicator of financial performance.
The lack of an explicit casual model of the relations between measures also contributes to difficulties in evaluating their relative importance. Without knowing the size and timing of associations among measures, companies find it difficult to make decisions or measure success based on them.
Fourth on the list of problems with non-financial measures is lack of statistical reliability – whether a measure actually represents what it purports to represent, rather than random “measurement error”. Many non-financial data such as satisfaction measures are based on surveys with few respondents and few questions. These measures generally exhibit poor statistical reliability, reducing their ability to discriminate superior performance or predict future financial results.
Finally, although financial measures are unlikely to capture fully the many dimensions of organizational performance, implementing an evaluation system with too many measures can lead to “measurement disintegration”. This occurs when an overabundance of measures dilutes the effect of the measurement process. Managers chase a variety of measures simultaneously, while achieving little gain in the main drivers of success.
Once managers have determined that the expected benefits from non-financial data outweigh the costs, three steps can be used to select and implement appropriate measures.
Understand Value Drivers
The starting point is understanding a company’s value drivers, the factors that create stakeholder value. Once known, these factors determine which measures contribute to long-term success and so how to translate corporate objectives into measures that guide managers’ actions.
While this seems intuitive, experience indicates that companies do a poor job determining and articulating these drivers. Managers tend to use one of three methods to identify value drivers, the most common being intuition. However, executives’ rankings of value drivers may not reflect their true importance. For example, many executives rate environmental performance and quality as relatively unimportant drivers of long-term financial performance. In contrast, statistical analyses indicate these dimensions are strongly associated with a company’s market value.
A second method is to use standard classifications such as financial, internal business process, customer, learning and growth categories. While these may be appropriate, other non-financial dimensions may be more important, depending on the organization’s strategy, competitive environment and objectives. Moreover, these categories do little to help determine weightings for each dimension.
Perhaps the most sophisticated method of determining value drivers is statistical analysis of the leading and lagging indicators of financial performance. The resulting “causal business model” can help determine which measures predict future financial performance and can assist in assigning weightings to measures based on the strength of the statistical relation. Unfortunately, relatively few companies develop such causal business models when selecting their performance measures.
Review Consistencies
Most companies track hundreds, if not thousands, of non-financial measures in their day-to-day operations. To avoid “reinventing the wheel”, an inventory of current measures should be made. Once measures have been documented, their value for performance measurement can be assessed. The issue at this stage is the extent to which current measures are aligned with the company’s strategies and value drivers. One method for assessing this alignment is “gap analysis”. Gap analysis requires managers to rank performance measures on at least two dimensions: their importance to strategic objectives and the importance currently placed on them.
Our survey of 148 US financial services companies — a joint research project sponsored by the Cap Gemini Ernst & Young Center for Business Innovation and the Wharton Research Program on Value Creation in Organizations – found significant “measurement gaps” for many non-financial measures. For example, 72% of companies said customer-related performance was an extremely important driver of long-term success, against 31% who chose short-term financial performance. However, the quality of short-term financial measurement is considerably better than measurement of customer satisfaction. Similar disparities exist for non-financial measures related to employee performance, operational results, quality, alliances, supplier relations, innovation, community and the environment. More important, stock market and long-term accounting performance are both higher when these measurement gaps are smaller.
Integrate Measures
Finally, after measures are chosen, they must become an integral part of reporting and performance evaluation if they are to affect employee behavior and organizational performance. This is not easy. Since the choice of performance measures has a substantial impact on employees’ careers and pay, controversy is bound to emerge no matter how appropriate the measures. Many companies have failed to benefit from non-financial performance measures through being reluctant to take this step.
Conclusion
Although non-financial measures are increasingly important in decision-making and performance evaluation, companies should not simply copy measures used by others. The choice of measures must be linked to factors such as corporate strategy, value drivers, organizational objectives and the competitive environment. In addition, companies should remember that performance measurement choice is a dynamic process – measures may be appropriate today, but the system needs to be continually reassessed as strategies and competitive environments evolve.
