During the long bull market of the 1990s, Wharton finance professor Jeremy Siegel’s 1994 bestseller, Stocks for the Long Run, was the closest thing there was to an investor’s Bible, preaching the long-term benefits of stocks over bonds and cash. Then in 2000 Siegel’s friend and MIT graduate school classmate, Robert Shiller, warned of the risky, unpredictable nature of stocks in his own bestseller, Irrational Exuberance.

Just as Siegel’s book had seemed to predict – perhaps even to help create – one of the greatest bull markets in U.S. history, the book by Shiller, an economics professor at Yale, was dead-on in forecasting the stock-market plunge that began in the spring of 2000.

With two years to test these dueling views against real market data, which holds up best?

In a three-hour discussion June 14 with participants in Wharton’s Advanced Management Program, neither author conceded defeat, and each found his views supported by recent market trends. For Shiller, who started the discussion with a 45-minute presentation, the history of the stock market is a story of boom-and-bust cycles brought on by waves of irrational investor emotion. The enormous stock run-up in the 1990s, culminating in the soaring Nasdaq market, was merely the latest bubble to follow many others. “I couldn’t find a better example of a speculative bubble than this one,” he said.

A typical bubble begins with a “precipitating factor” such as a product, process or theory that strikes investors as revolutionary, Shiller argues. Then comes an “amplification mechanism” that seems to reinforce the view. Rising stock prices, for example, can convince more and more investors that a major opportunity is developing, causing them to bid prices up even more. Often, the news media contributes by spreading stories declaring the dawn of a “new era” in which old rules of stock valuation are said to no longer apply. As the bubble expands, investors seize upon ever more outlandish rationalizations to justify prices that defy previous standards. They may focus on data that supports their views, rejecting more valid data that undermines them.

“We have to look at what the psych department has been doing for the past 100 years,” he said, arguing that traditional market analysis fails to take account of behavioral factors.

The enormous run-up of the Nasdaq market in the late 1990s and early 2000 was largely driven by enthusiasm for the Internet, Shiller said. “You were using it every day, so it fueled your imagination like most inventions do not.” Investors dismissed the fact that most Internet companies were losing money, seizing on the idea that in this new era the old-fashioned benchmarks like the price/earnings ratio no longer applied or had to be drastically modified. Rising tech-stock prices drew more investors, who bid prices up even further, drawing more investors in.

To rationalize this bubble, Shiller said, investors embraced a theory that ever-higher P/E ratios made sense because technology would drive ever-increasing productivity that would push up corporate earnings.

But historical data does not show that rising productivity necessarily leads to higher growth in corporate earnings, and there is no sound reason for stock prices to continually grow faster than earnings, he said. “It wasn’t productivity growth,” he said. “It wasn’t a new era. It was a bubble… You get more and more people going in because the price keeps going up.”

Like all past bubbles, this one burst by a cascade of selling that began when a significant number of investors decided to cash out before the inevitable crash, Shiller said.

Siegel, in his introduction, agreed there was a bubble in the late 1990s but said “it was almost entirely in technology stocks … Bob and I agree, valuation is important,” Siegel said, referring to gauges such as the P/E ratio, which has averaged just below 15 to 1 since 1871.

Of the 500 stocks in the Standard & Poor’s 500 index, 81 were tech stocks in the late 1990s. Because this index is capitalization-weighted, the soaring tech stocks came to represent more than 30% of the index’s value. With P/E ratios averaging around 65, the tech stocks boosted the index’s ratio to a peak of about 50. Clearly, that is high by historical standards, Siegel said.

However, if the tech stocks were excluded, the index’s ratio remained in the more modest 20s. During the same period, the 30-stock Dow Jones Industrial Average, which did not have such a heavy weighting of tech stocks, did not experience a bubble, he said. “Really, the bubble was very narrow. I never thought the rest of the market was over-valued.” This was quite different, he added, from the bubble preceding the 1929 crash, or the “Nifty Fifty” bubble of the 1970s. In both of those cases, most stocks were overpriced.

Siegel agreed with Shiller’s view that productivity growth does not justify high P/Es. He argued, however, that Shiller is wrong to minimize the significance of long-term market patterns. From 1802 through 2000, stocks have produced average annual compounded returns of 6.9% above the inflation rate, Siegel said, though there were long periods when it did better or worse than average. (From 1966 through 1981, for example, stocks lost an average of about 0.4% a year.)

A “real” – after inflation – return of about 7% proves to be the return that investors demand for taking on the risks of owning stocks, Siegel said.

If stocks and bonds offered the same returns, investors would choose bonds, because they are safer. That would reduce the demand for stocks, causing stock prices to sink relative to corporate earnings. Investors would then conclude stocks offer the opportunity to share in earnings at bargain prices, and money would then flood from bonds to stocks, causing stock prices to rise. This constant readjustment has settled to provide an average real return of 7% for stocks, while safe long-term government bonds average 3.5%.

In the future, however, investors may settle for 5% from stocks, since the cost and risks of owning stocks has gone down substantially over the past 25 years, Siegel said. Brokerage commissions have fallen dramatically as the result of deregulation in the 1970s and the rise of discounters. Mutual funds now allow investors to inexpensively reduce risk by diversifying. Moreover, the economy and inflation are better managed and wages are more dependable and stable than they were during most of the 200 years Siegel has studied. That makes people feel they can afford to tie money up in long-term investments like stocks without expecting a large risk premium.

Accounting for all these factors, Siegel said the appropriate P/E ratio now and in the future should be in the low 20s rather than the historical average of about 15. That means today’s market, relative to analysts’ earnings projections for 2002, is fairly valued.

Asked why the stock market is performing so poorly this year even though last year’s recession has ended, Siegel said investors have been disappointed by the sluggishness of the recovery. Stock prices soared at the end of 2001 in anticipation of a stronger recovery and are now pulling back as investors realize they went too far.

In addition, he said, the market is suffering from worry about the accounting and corporate management scandals and international tensions over the Middle East and Indian sub-continent.

Shiller added that events of the past couple of years are typical of the period following a burst bubble, as investors turn on those they believe caused the bubble, such as dishonest analysts, greedy executives and pliable auditors. For the long term, he said, stocks will remain risky and unpredictable. “I think we are entering into a world of lower returns.” Only about 2% of his own portfolio is in stocks, Shiller added.

Siegel said he still invests in stock index funds, though he has moved more money to value-oriented index funds, foreign-stock funds and real estate investment trusts. Though stocks may return less than they did in the 20th century, he added, they should still beat bonds.