People like to think of the stock market as a person: It has moods, it can be ornery or exuberant, it can overreact one day and make amends the next, and so on. This makes for feisty television commentary, but can psychology really help us understand the financial markets better? Yes, say many academics, although not by putting the market on a couch. The subdiscipline of behavioral finance has gained ground over the last half-decade. The idea is simple: Investors are not as rational as traditional theory has assumed, and biases in their decision-making can have a cumulative effect on asset prices. This notion flies in the face of the established theory that markets are efficient. The efficient-markets theory suggests that assets are priced correctly because supply and demand reflect aggregate current public knowledge about those assets – and that the movement of stock prices cannot be reliably predicted on the basis of past results. Behavioral finance clearly isn’t a fad. “It’s accepted as a field in finance, whereas five or six years ago it wasn’t,” says
Behavioral-finance professors are also gaining professional ground. Even the conservative finance department at the University of Chicago has opened its doors to a few proponents, starting with Richard Thaler, whose research in the mid-1980s on the investment quirks and biases of individuals gave the field much of its practical impetus. To some, behavioral finance is a revolution, transforming how people see the markets and what influences prices. But not everyone agrees. “It’s too mainstream to be called a revolution,” notes Andrew Metrick, a professor of finance at Wharton whose research borders the field. “In some guise, this has been around since the beginning of finance research. It just hasn’t been codified and married to psychology quite so formally before.”
But all is not rosy in behavioral finance. Finance professor Eugene Fama of the University of Chicago, who is widely regarded as the initiator and principal proponent of the efficient-markets theory, has been the field’s loudest and most enduring critic – and he hasn’t softened his opposition. Fama’s views have spread – even among the ranks of behaviorists. There is now stepped-up criticism of the methodologies behind some behavioral-finance research and of the validity of behavioral models that purport to show how investor biases and cognitive errors affect asset prices. Seen in another light, however, the internalization of this criticism could be a sign of the field’s gradual maturation.
Fama’s criticism is significant. In a paper in the Journal of Financial Economics in 1998, Fama scoffed at the idea that behavioral finance – with its explanations of bubbles, panics, trends in asset prices and insistence that the market can be beaten – might replace or trump efficient-markets theory. He gave little credit to behaviorist explanations of trends and “anomalies” discovered in historical data of asset prices, arguing that data-mining techniques make it possible to locate patterns whose significance is nonetheless questionable, if not irrelevant. And ascribing behaviors to those trends is riddled with pitfalls.
Three years later, Fama’s views have not changed. “With the exception of the top people,” he says, “it’s easy to get articles done in behavioral finance. The field tends to attract people who are not very good statisticians. It has tended to attract people who basically just look for anomalies, without any rhyme or reason. And that’s not science.” In brief, Fama argues that behavioral finance hasn’t shown that the tendencies of individuals, when aggregated, have an impact on world prices. Fama also points out that behavioral-finance models frequently contradict one another.
Perhaps it’s a sign of the field’s robustness that its proponents do not feel obliged to accept its precepts wholesale – let alone huddle protectively around a few key ideas. Gervais, who has coauthored a number of papers that fall under the behavioral-finance umbrella, notes that he’s not a “perfect believer.” Some of the research that’s been done doesn’t impress him. “Some people have viewed behavioral finance as an opportunity to not talk about economics,” he says.
Gervais cites event studies, which focus on asset prices before and after an event such as an earnings announcement or merger. The weakness in this area, he explains, is that sometimes the market behavior displayed is attributed to overreaction and sometimes to underreaction. “People have taken the behavior that’s convenient for the particular study they’re doing,” he says. “They don’t try to see if the biases are the result of underlying economic forces. And right now there’s no kind of overall theory that will reconcile all these things.”
Even some of the field’s staunchest proponents are listening to Fama. Hersh Shefrin, a professor of finance at Santa Clara University and editor of the forthcoming three-volume Behavioral Finance, disagrees with Fama’s bottom line, but acknowledges the criticisms. “There are weaknesses in many of the behavioral finance papers that are being written now and that have been written in the last few years,” he says. “We don’t have enough professionals who have been trained in both academic psychology and traditional finance, so the models they’re putting together are ad hoc. They’re only thinly veiled by psychology studies of how people think, and they’re inconsistent with one another.” Still, he argues, the discipline is moving in the right direction. And it’s gaining momentum: “The paradigm is shifting. People are continuing to walk across the border from the traditional to the behavioral camp. Some of them are coming to peek; others are simply walking right across.”
But the argument is far from over. For many academics, efficient-markets theory does not explain every nook and cranny of the markets, but that doesn’t mean an intellectual blank check for behavioral finance. In A Non-Random Walk Down Wall Street, published in 1999, Wharton finance professor A. Craig MacKinlay and MIT’s Andrew Lo identified predictabilities in asset prices that were not easily explained by traditional economic models that assume rational behavior. According to MacKinlay, “That naturally led to the alternative explanations about predictability in asset prices, which are the behavioral finance explanations – that human nature does not always behave as we assume in our traditional economic models.”
So is MacKinlay a convert? Not really. “The major problem with behavioral finance is that the predictions with the models tend not to be specific enough that you can easily refute the theory,” he says. Mackinlay also brings up Fama’s data-mining argument that it’s always possible to find unusual patterns in historical data by virtue of having only one history to rely on. With increased data and computing power available, he notes, “it’s easy to do broad searches through the data and uncover unusual phenomena. But how genuine those phenomena are is an open issue.”
An example of this, Mackinlay says, is the premise that value stocks generally outperform growth stocks (although growth stocks recently had their day in the sun). Many of the value vs. growth arguments were based on data from 1962 to 1990, he notes, and that “happened to be a relatively good period for value stocks in my view, so those conclusions – even though a 30-year period seems long – don’t hold up as strongly.”
Metrick, too, doesn’t expect behavioral-finance models to replace traditional models of how the market functions. “There are some things going on that aren’t captured in the rational paradigm – those are the anomalies,” he says. “The models we currently have that assume rational behavior are never going to be bad descriptions of reality. They’re just not going to capture everything. I fall back on the rational paradigm and think it’s going to get me most of the answers in most cases.” Nonetheless, he points out, since bubbles and other anomalies have real economic effects – whatever their source – it is worth devoting time and effort trying to understand what drives them and whether their impact can be limited.
Meanwhile, one of the larger debates in behavioral finance concerns the kind of practical, hands-on trading insight it generates about the market. Fama, of course, dismisses the notion that behavioral finance can produce effective trading strategies. “There’s a big demand for behavioral finance among practitioners because nobody wants to believe the markets are efficient,” he says. “Everybody wants to think there’s money on the table. But the reality is, if there’s so much money on the table, how is it that every study of investment performance finds that professional managers can’t do any better than index funds?” And retail investors, he argues, maim themselves by thinking they can one-up the market.
Shefrin has an answer. It’s a misconception that behavioral finance means people can beat the market, he argues. “Behavioral finance doesn’t say, ‘There’s easy money, go after it.’ It says that psychology causes market prices and fundamental values to part company for a long time. And although there’s a potential profit opportunity there, it comes packaged together with additional risk – and smart money can’t or won’t take a large-enough bet to eradicate the anomaly.” This implies that smart money won’t be able to arbitrage away these anomalies and return the market to equilibrium – perhaps because it’s not possible, it would take too long, it’s too risky in the short run, or for some other reason. So the anomaly will persist.
“That’s the lesson about behavioral finance, and that’s the lesson that most academics don’t understand,” says Shefrin. He goes on to point out that retail investors who think they’re clever enough to beat the markets usually don’t even understand traditional ideas, and “should probably simply act as if Eugene Fama is right and follow a passive long-term strategy.”
David Hirshleifer, a professor of finance at Ohio State University whose work focuses on how the behavior of individuals influences asset prices, suggests that behavioral finance is only just developing its sea legs. In his view, because psychology-based finance is currently a growth discipline and not yet a mature one, there’s a lot of disagreement – but it’s productive disagreement. “People are generating new hypotheses, understanding better what the implications of different psychological effects for the market are, and subjecting these ideas to empirical testing,” he says. “We’re nowhere near a common new set of assumptions or implications for markets that everyone agrees on, but we do have a process whereby people are thinking about these issues systematically and trying hard to compare predictions to the evidence.”
The area of behavioral finance that steers clear of some of the more vexed arguments about asset prices is individual investors’ actual decisions and misjudgments about the market. “It’s not that there’s an ordinary finance and a strange, weird finance,” says Terrance Odean, a professor of finance at the University of California at Berkeley, and one of the few academics with a strong background in both psychology and economics. In his view, systematic biases in the decisions of investors or fund managers make a difference and can add to our understanding of how markets work – though those biases won’t always be equally important. “There are probably some behavioral finance theories proposed that don’t describe the world we live in,” he says. “But they won’t stand the test of time.”
For Odean (who has collaborated with Wharton’s Gervais on a few papers), research about why individual investors don’t behave the way efficient-markets theory says they should is on more solid ground empirically. In the late 1990s, using data from a large brokerage firm about the common-stock portfolios of individual investors, he found that people were much more likely to hold on to a losing investment and to sell a winning investment, relative to their opportunity cost, than they should have been. “That’s what the behavior theory predicted and it’s the opposite of what was normative behavior for tax reasons,” he says.
Shlomo Benartzi, a professor of accounting at UCLA who has coauthored a half-dozen papers with Thaler, suggests another reason to focus on the investment decisions of individuals. It has nothing to do with whether their biases might affect asset prices. Instead, he argues, “prices can be right yet every single investor could be making the wrong choice. It’s just that in the aggregate the mistakes are washed out.”
Benartzi’s research has grappled with how individuals make decisions about their portfolios in retirement savings plans, and how alternative programs can be devised to overcome biases and enable investors to make more rational decisions. He has shown, for instance, that the information and choices provided by a pension fund to its customers affect the decisions those individuals make. As a result of his work in this area, IBM has altered the menu of funds it gives its employees.
But there are more questions than answers about what influences the decisions of individual investors. Why, for instance, do people who have a lot of money invested in mutual funds, or who have money in a CD account, borrow heavily on their credit cards and pay high interest rates instead of paying the debt? “We have very little understanding,” Benartzi continues, “of how people decide how much risk to take, why people trade so much, how they think about diversification or rebalancing their portfolios, how they decide to exercise stock options, and so on.” These issues have an effect on the market but not necessarily on the prices of assets.
Many in behavioral finance are now extending this interest in the psychology of individuals’ decision-making to the field of corporate finance. According to Gervais and others, that’s where the payoff of behavioral finance is likely to lie. Gervais is developing a theory for why corporations hire overconfident employees. Shefrin says he’s shifting to the corporate finance side in order to explore how managers really make decisions in their firms. “It’s hard to teach individual investors how to avoid behavioral traps,” he points out. “But companies are always looking for new ways of doing things, especially when business isn’t going well. There are 12-step programs you can put in place to help management avoid major decision traps.”
Another reason corporate finance is an appealing area for study, suggests Odean, is that there are no arbitrage opportunities with corporate decisions. With stocks and bonds, if some investors have expectations that are deemed unwarranted by the market, others will step up to take advantage of those mispricings, restoring equilibrium. “What this means [for corporate finance] is that there are plenty of opportunities for people’s decision biases to have a meaningful effect on how the firm fares,” he adds.
Hirshleifer agrees that applying behavioral-finance concepts to corporate finance can pay off. If managers are imperfectly rational, he says, perhaps they are not evaluating investments correctly. Are they, for instance, making bad choices in their capital-structure decisions? Another set of questions, he notes, relates back to the market. If, say, Internet stocks are being valued very highly and the manager of an Internet firm believes the market has overvalued his firm, what should he do? What are the advantages and disadvantages of different strategies such as issuing more equity or acquiring another company that has real assets? Hirshleifer says he and his coauthors are beginning to tackle some conceptual issues that can help companies make such decisions.
As a field of study that offers a conceptual framework for examining how people think and act in the financial markets, behavioral finance clearly has legs. But MacKinlay is far from alone when he suggests that behavioral finance has probably been oversold. Few people realistically think behavioral finance will displace efficient-markets theory. On the other hand, the idea that investors and managers are not uniformly rational makes intuitive sense to many people. MacKinlay is also clearly not alone when he says of behavioral finance that “some of the underlying arguments are somewhat compelling — and consistent with the way I behave.”
