Warren Buffett is an undesirable board member. He’s too old, serves on too many boards and has insider ties to too many of the companies on whose boards he sits.

Sounds ridiculous, doesn’t it? After all, Buffett has a reputation for being one of the most astute and ethical investors of the last 30 years. Plus, the billionaire stock-picker is a bit of a good-governance guru himself, pushing, among other things, for companies to expense employee stock options to show their true cost.

But this is the sort of argument some corporate governance watchdogs make. Last spring, for example, the California Public Employees Retirement System wanted Buffett booted off the board of the Coca-Cola Co. Calpers, as the $145 billion pension fund is known, argued that Buffett had a conflict of interest. He sat on a committee that voted to let Coke’s auditor do some non-audit work for the company. At the time, Calpers was making a national campaign out of its hard-line stance on conflicts of interest, saying it would oppose the re-election of directors at more than 2,700 companies.

Also this spring, another watchdog group complained that two companies owned by Berkshire Hathaway, Buffett’s Omaha, Neb.-based investment holding company, sold Coke products. In theory, this, too, was a conflict for Buffett.

Wharton accounting professors David Larcker, Irem Tuna and Scott Richardson say this sort of check-box approach to corporate governance doesn’t work. Companies and their situations are too diverse. “The recipe book is big, and there’s a different recipe for each company,” Richardson notes. Even worse, the professors say, are consultants and ratings services that use formulas – which they typically refuse to reveal – to boil down a company’s corporate governance to a single number or grade.

“Lots of people are coming up with governance scorecards,” Larcker explains. “They’re coming up with best practices and selling this stuff. As far as we can tell, there’s no evidence that those scorecards map into better corporate performance or better behavior by managers.”

Larcker, Tuna and Richardson tried to create a magic formula of their own. But no matter how they sliced and diced governance data (consisting of more than 30 individual measures) on more than 2,100 public companies, they couldn’t find one. The three professors have released their findings in a working paper titled, “Does Corporate Governance Really Matter?” The title is intentionally provocative. They do think corporate governance matters, but after puzzling over reams of company numbers, they are not confident that anyone can measure whether one firm’s governance is better than another’s at least, not by using typical metrics.

As they say in their paper, “Our overall conclusion is that the typical structural indicators of corporate governance used in academic research and institutional rating services have a very limited ability to explain managerial decisions and firm valuation.”

What do they mean by structural indicators? An example is whether a company has a lot of insiders on its board. Another is whether a board’s chairman is also the company’s chief executive. In theory, these facts point to “bad” governance because they suggest that managers control a firm and can enrich themselves, with impunity, at the expense of outside shareholders.

Governance watchdogs also look to structural indicators of “good” governance such as whether a single outside investor holds a big block of a company’s stock, whether a company has a lot of debt or whether it has a lead director.

“The presence of a large block-holder is typically argued to be beneficial through the monitoring benefit of a financially sensitive shareholder,” the authors note in their paper. “The presence of debt-holders also offers additional monitoring benefit via external capital providers who have the incentive and ability to monitor firm activity to protect invested principal. The appointment of a non-executive director as a lead director is expected to create an additional monitoring benefit on incumbent management.”

Larcker, Tuna and Richardson evaluated whether these sorts of indicators made any difference in determining a company’s external valuation by the market or debt rating agencies, and whether they influenced internal decisions such as the level of CEO compensation. They folded into their analysis outcomes that governance studies haven’t often included, such as the propensity of companies to end up in class-action suits or to restate their financial reports. No matter how they parsed the data, they didn’t uncover any strong, consistent relationship. In many cases, they even saw the opposite of what one might expect, finding for example, that companies with big boards report smaller abnormal accounting accruals. “This is inconsistent with prior literature that suggests big boards are ‘bad,'” their paper points out.

“We set the study up to err on the side of, ‘The relationship [between governance measures and good performance] is there,’ Tuna explains. “And we can’t even find it when we do that. We biased the analysis in favor of finding something … The structural indicators just don’t seem to have that much ability to explain whether companies have to do accounting restatements, whether they’re selling at a higher multiple, whether they’re manipulating earnings and things like that.”

Not only are typical governance indicators based on shaky empirical foundations, they even can yield perverse results. Consider computer-maker Dell Inc. Its stock has returned 32% over the last two years, compared with 22% for the Dow Jones Total Stock Market Index. It’s considered one of the success stories of the ’90s tech boom. But some corporate-governance raters have criticized it, claiming insiders dominate its board. The list of top-performing firms that have been dinged in these sorts of ratings includes Wal-Mart and Southwest Airlines.

Contrast Dell with now-infamous Enron, which got good scores from some raters, Tuna points out. The energy-trader had implemented the kinds of procedures that governance gadflies like – separate people serving as chairman and CEO and a board stocked with high-profile outsiders, for example. And yet these so-called safeguards were cosmetic; they didn’t prevent executives such as Andrew Fastow, former chief financial officer, from bilking the company’s shareholders.

The researchers’ goal is not to minimize the importance of corporate governance. In theory, better-governed firms should perform better, they say. Yet it’s difficult to use strictly numerical tools to assess what really drives managers’ conduct. “We believe that if we had some psychological way to go in and assess people – ‘Is this a good person or a bad person? Do they have high ethics or low ethics?’ – that that would be extremely useful,” Larcker says.

“Maybe we could interview you and figure out if you had shareholder interests at heart or if you were a criminal,” Larcker adds. “Maybe we could develop a psychological map. But virtually all the work done on this by consultants and the preponderance of work by academics uses [measures] that are a level removed from that.”

In the absence of effective measurement tools, investors who are trying to assess firms’ governance have to do it the old-fashioned way. That is, they have to do their homework, examining companies one at a time, the professors say. They have to consider such things as whether the CEO’s cronies dominate the board, how much board members and managers are paid and whether managers have incentives to raise the company’s earnings and stock price. “Different things may be relevant for different companies,” Richardson notes.

For companies that are worried about their governance, the three professors’ message is far simpler: Don’t do anything rash just to appease a ratings service or because a popular consultant whispered his favorite formula to your CEO on the golf course. “A lot of well-known people and consulting companies seem to be espousing, ‘Here are the best practices. Trust me,'” Larcker warns. “Our message is that there’s not much evidence to back up those bold claims and simply benchmarking your governance to best practices isn’t going to be that useful.

Many companies, he adds, “want to improve their scores, so they are changing things. It’s costly to do that, and they may be making a mistake. You have to tailor your policies to what works in your setting. If you are going to spend big bucks and big energy on this, you want evidence, not rhetoric. Before you make these changes, see if you can find other companies that have made similar changes. See if their stock price went up.”

Does Corporate Governance Really Matter?