According to a June article in The Washington Post, a new report released by think tank the Commonwealth Fund ranked the U.S. “dead last” in terms of health care quality when compared with 10 other Western industrialized nations. Unfortunately, this will not be very surprising to those who follow such stats, as a number of other recent studies have also indicated that the U.S. is a lackluster performer in this area. Efforts to address the problem include the Affordable Care Act — the full impact of which is still to be seen — as well as a plethora of health care quality entities both public and private, such as the Consumer Assessment of Healthcare Providers and Systems (CAHPS), The Joint Commission, The National Quality Forum (NQF) and the National Quality Measures Clearinghouse.

Evaluating the performance of doctors, surgeons and hospitals, and releasing the information to the public in an attempt to help consumers make better choices, is a major focus of many of these efforts. According to Jonathan Kolstad, a Wharton health care management professor, as of 2006, 47 states had implemented some form of quality reporting system to assess health care providers.

But consumers are not showing much interest in the reports. Studies in Journal of the American Medical Association, Medical Care Research and Review and elsewhere have spotlighted the low usage of this information by consumers, who tend to rely instead on recommendations from family and friends or on another doctor’s referral. (Think about the last time you or a loved one looked for a physician — chances are you did not search the Internet to find one, or check online for a provider’s quality report.) Research has also identified barriers to usage for many consumers: Among them, language that may be too technical for those with inadequate health literacy. The response, according to Kolstad, has been increasingly to simplify the reports to make them appealing to the widest possible audience. But Kolstad thinks that overall, those trying to use quality reporting to effect a change in consumer behavior may be barking up the wrong tree.

In his paper, “Information and Quality When Motivation Is Intrinsic: Evidence From Surgeon Report Cards,” which was published in the American Economic Review, Kolstad documents some eye-opening findings that suggest that the most important audience for such reports may be doctors instead of patients.

“Knowing a lot of doctors, it’s clear to me that [people] go into doctoring for a whole bunch of reasons, money not being the only one, if even a major one.”

Kolstad was spurred to perform his investigations after noticing an unresolved question from initial evaluations on health care information release. He notes that “information release was seen as a real panacea for a lot of market problems and for addressing low quality in health care.” He saw studies indicating that while publishing assessments did not seem to change consumer behavior very much, they did seem to generate significant increases in quality of care. There was no clearly identified cause for this phenomenon, which Kolstad found strange and which made him want to “dig a little more.” He also wanted to see what applying methods from economics to that account of detailed data and consumer behavior might yield, as opposed to the broader program evaluations of prior work.

The Desire to Do Well

Kolstad wondered how much doctors’ intrinsic motivation — “inner” rewards such as the satisfaction of performing well and helping people, and feelings of professional pride — might factor into the equation as opposed to the typically measured extrinsic or “outer” motivations of financial gain. “Knowing a lot of doctors, it’s clear to me that [people] go into doctoring for a whole bunch of reasons, money not being the only one, if even a major one,” Kolstad says. He adds that the drive to be the best, and to do well by patients, is “ingrained in the medical profession.” What if doctors were discovering new information about themselves in these quality reports, and that was making them strive to provide better care regardless of profits?

To test his theory, Kolstad obtained data related to the introduction of quality “report cards” for surgeons performing Coronary Artery Bypass Graft (CABG) surgery in Pennsylvania. He analyzed data on surgeons’ performance measured by inpatient mortality rates. He also gained access to de-identified data on everyone in the state who had a heart attack and received a bypass surgery between 1993-1995, 2000 and 2002-2003, including the severity of each case and the outcome of the procedure.

Because he needed to measure profit motives as well as intrinsic motives, the CABG surgery data really fit the bill, says Kolstad. It was relatively easy for him to track the financial gain of surgeons who perform this procedure because they act as “freelance” workers not employed by hospitals, so they are compensated separately. The quantity of procedures a surgeon performs is a direct measure of profit. He adds that the fee-for-service compensation for CABG makes it one of the most profitable types of care in terms of both Medicare and private insurance reimbursement.

What was especially remarkable about the study results was that intrinsic motivation — the desire to do well — actually trumped financial gain.

Kolstad created economic models of surgeons’ beliefs about quality levels (both their own and that of their peers) corresponding to both before and after the public release of the report cards. He found, as researchers had before him, that the reporting had only a minimal impact on consumer demand. But he was able to identify and trace both an intrinsic and an extrinsic motivational response by surgeons, which led to improved performance — i.e., significant declines in risk-adjusted mortality rates. What was especially remarkable about the study results was that intrinsic motivation — the desire to do well — actually trumped financial gain by a notable margin. Kolstad found that the CABG surgeons’ intrinsic response to the report cards was four times larger than their response to profit incentives. Kolstad says he was somewhat surprised by “the degree to which information seems to impact things … how much larger the effect is than traditional market mechanisms.”

Kolstad says that a key to why report cards may represent valuable “new” information for physicians about their practice and that of others’ is that they are risk-adjusted. For example, when comparing mortality rates between two providers, a routine surgery on an otherwise healthy, relatively young patient is not given the same weight as a complicated surgery on an elderly person with multiple chronic conditions. “The introduction of risk adjustment … [gives] surgeons the ‘true’ difficulty of their cases — the expected outcome had the average surgeon in the state handled the case,” he writes, adding that, “If I’ve been practicing for 10 years, it’s very hard to get a good read on [my performance.]” For example, Kolstad says, we all know that lower mortality is good, but a doctor may not know what rate of mortality is “reasonable” given the severity of the cases he or she routinely handles.

In addition to intrinsically motivating the practitioners themselves, report cards, Kolstad notes, may be of value for physicians who make referrals. In the case of bypass surgery, “it could be that the cardiologist who’s been referring people thinks ‘Wow, I just got this useful information, and figured out that [so-and-so] is really a bad surgeon,’” he says. “Frankly, in the absence of [such reports], there are not often great feedback mechanisms in medicine, or at best they’re informal.”

Kolstad notes that since his study demonstrates that doctors are the real audience for quality reports, the ongoing trend to simplify the documents is probably not a good idea. It is important that they be detailed, comprehensive and clinically relevant. Or, perhaps eventually there could be separate physician-facing and consumer-facing reports, he suggests.

“The assumption is that every physician is reading every New England Journal article and synthesizing everything into their head instantaneously.”

The Value of Report Cards

Overall, providing doctors with more feedback about their performance is something that Kolstad thinks is an important, and generally overlooked, direction in the drive to boost health care quality. “The assumption is that every physician is reading every New England Journal article and synthesizing everything into their head instantaneously,” says Kolstad, stating that it can be challenging for doctors to keep up with the cutting edge of care. “We’ve tried in health care in the U.S. — for a long time — very traditional incentive schemes, which [means] just changing payments. We know that has an effect, but what we haven’t tried as much, at least in any systematic way, is providing quality feedback and information.”

Noting that his findings could also be applied outside of health care, Kolstad says, “I think surgeons are awfully similar to a lot of high-skilled workers who frequently don’t get great feedback — and especially not risk-adjusted feedback — which I think is very important.” The teaching profession might benefit, for example. “Certainly if you look at the high school level and the kinds of … test results that are being pushed in the education space, that’s a real opportunity for learning and for peer reference information to motivate teachers to do better.” Sales would be another potential area, he adds. “Typically [salespeople] are given very strong financial rewards, but people tend to ignore the opportunity to provide peer information as a motivator. [It’s] an opportunity to help people understand how well they can do, given what they’re trying to do.”