For months, the news for the hedge fund industry has been grim. Funds racked up abysmal results last summer, with August showing the biggest monthly loss since October 2008.

Then the prominent Bain Capital Absolute Return Capital hedge fund announced in early October that it would close, having lost money for three years. Soon after, the well-known Fortress Investment Group said it would shut its flagship fund after losing 17% for the year through September. The two funds used a “macro” investing strategy that bet on things like economic trends, currency moves or central banks’ interest rate policies – and trends had been unkind.

At the end of October, Carlyle Group’s Claren Road Asset Management LLC said that a heavy demand from investors wanting their money back would not be met immediately. Instead, two-thirds of the $2 billion in redemption requests would be spread over a number of quarters, a relatively uncommon practice used when heavy withdrawals are seen as potentially disruptive to fund investing strategy.

Poor performance, closures, investors rushing for the exits…. A casual observer might wonder if the industry’s best days are behind it. Indeed, naysayers have long argued that these high-fee investment pools for institutions and wealthy investors could not keep up in an era of low-cost index-style investing, where investors are content to simply match the market.

But, as the saying goes, reports of the hedge fund industry’s decline are greatly exaggerated. In fact, the industry’s assets under management recently set a record of over $3 trillion before pulling back slightly, according to the industry-tracking firm Hedge Fund Research. Many funds started to show better results in the fall, and from the start of the year through September, the industry, while losing 1.5%, beat the Standard & Poor’s 500 and Dow Jones Industrial Average by 3.7 and 7 percentage points, respectively — the widest margin since 2008. In this respect, hedge funds did what they are supposed to do: minimizing losses in a down market.

Many experts say that because stocks are risky and bond yields are near historic lows, sophisticated investors will continue to try to tweak their risk and returns with alternative investments like hedge funds.

“In general, I do not expect large, sophisticated clients to start a big exodus from the hedge fund space.” –Bilge Yilmaz

“, and there will be a demand for hedge funds,” says Wharton finance professor Bilge Yilmaz, who follows the industry and is a partner in Ada Investments, a portfolio strategy firm.

Zig vs. Zag

Key to hedge funds’ continued appeal is their ability to zig when broader markets zag, offering diversification that has become harder to find as other assets like U.S. and foreign stocks tend to move in lockstep more than in the past. “In general, I do not expect large, sophisticated clients to start a big exodus from the hedge fund space,” Yilmaz says.

“I think the industry is healthy,” says Mark Thomas, director of hedge fund research with CTC/myCFO, a wealth management firm that serves affluent investors. “There are always going to be some [fund managers] who take a concentration of risk and end up in the news, and that’s unfortunate.” Summer results, he observes, were hurt by the Chinese currency devaluation and the Federal Reserve’s mixed signals on raising interest rates, making the going tough for funds that thrive on interest-rate volatility.

Hedge funds are investment pools akin to mutual funds but restricted to institutions and individuals wealthy enough to be “accredited” – generally, those who have large incomes or at least $1 million in liquid assets. Pension funds, insurance companies and university endowments account for the bulk of hedge fund assets, partly because they can afford to tie their money up for years and partly because, as tax-free entities, they avoid the hit that taxable investors suffer with funds that engage in frequent trading.

Because sophisticated investors are thought to be capable of shouldering heavy risks, hedge funds are allowed to do things that most ordinary mutual funds cannot, like going “short” to bet that an asset’s price will fall, borrowing to enhance returns, or using complex formulas to buy or sell — or doing both at the same time — according to signals from a multitude of market indicators. Basically, if someone can think it up, a hedge fund is probably allowed to do it.

Experts say the diverse strategies make it misleading to focus on industry-wide performance. Under given market conditions, some funds will win big while others will lose.

“There’s no such thing as an asset class called hedge funds,” says Wharton finance professor Christopher Geczy, who studies the industry. While Fortress lost big betting on Brazil, other funds presumably made money betting against Brazil, he says. And although all investors want gains, not all choose every holding with an eye to beating a standard benchmark like the S&P 500. Some funds emphasize stability over market-beating performance.

Still, the numbers are rather stark. From 2010 through 2014, annual hedge fund returns averaged 4.5%, compared to 15.5% for the S&P 500, says Wharton finance professor Richard Marston. Hedge funds beat the S&P from 2000 to 2009. Even more revealing than the comparison with the S&P, he says, is the fact that hedge fund returns, independent of that stock rally, have steadily declined for 15 years.

“Why the decline in returns?” asks Marston. “Perhaps because there are now 10,000 hedge funds and almost $3 trillion in the industry.” There simply aren’t enough hot investment opportunities to go around, he argues.

“It’s a dangerous misconception to think hedge funds as a group are going to compete with a market like the S&P during a bull market in equities.” –Christopher Geczy

At the same time “it’s a dangerous misconception to think hedge funds as a group are going to compete with a market like the S&P during a bull market in equities,” Geczy says, referring to dramatic stock gains since the financial crisis. For most of the past five years, stocks have been on a tear, so hedge fund results often look weak by comparison.

“Most of our clients use hedge funds to dampen volatility [in the broader portfolio] and try to generate near-equity returns with less risk,” says Garb Mechigian, managing director with CTC|myCFO.

To hedge means to offset risk. A fund could hope that its stock holdings would go up, but at the same time purchase “put” options that would rise in value if stocks fall, reducing risk in the overall portfolio. But if stocks have a winning streak, the fund will trail the market because the puts will lose value. With a different fund, says Mechigian, investors may embrace extra risk in hopes of greater gains, and the fund may borrow money to buy more stocks. That would amplify both gains and losses.

Averages Hide Nuance

An industry-wide performance gauge masks these disparate goals. The start of October, for instance, was terrible for “quant” hedge funds, which invest according to computer models. Some lost 2% to 10% in a single week, Yilmaz says. But at the same time, long-short funds, which buy stocks expected to rise and short those expected to fall, did well, reversing a period of underperformance. “The bottom line here is that … across strategies they have quite different returns,” Yilmaz says.

Many of the most troubled funds suffered from downturns in energy, commodities and emerging markets like Russia, China and Brazil, and those difficulties could persist, as, for instance, there’s not much reason to think the oil and gas glut from fracking will end anytime soon. But Wharton finance professor Nikolai Roussanov says that doesn’t necessarily spell disaster for funds that specialize in these markets and assets, because hedge funds have the flexibility to change course and bet that prices will fall.

“If you are long-only, it’s probably going to be a protracted difficult time in emerging markets,” he says, “but a lot of hedge fund strategies are not long-only.”

The debate over hedge fund performance is complicated by questions about whether the industry’s data is reliable, says Marston. Much of the performance reporting is voluntary, and data that covers many years often leave out funds that are no longer in existence at the end of the period – a problem called “survivorship bias.” “Backfill bias” is another issue, Marston says. That’s when fund managers refrain from reporting performance in their early years, when many struggle.

One study, he said, showed a very large performance gap between the top 25% of managers and those in the bottom half. While a difference is to be expected if managers are ranked from best to worst, the performance range would be very narrow in a plain vanilla holding like a stock index fund, so that choosing one fund over another would make little difference. The extreme variation in hedge fund performance means that to end up a winner an investor must have access to the top funds, and often the doors to new investors are closed. “So investors have to ask the question: ‘Can my advisory firm get me access to the top-tier managers, or am I stuck with the third quartile?’” Marston says.

“So investors have to ask the question: ‘Can my advisory firm get me access to the top-tier managers, or am I stuck with the third quartile?‘” –Richard Marston

Studies have produced some evidence that hedge fund investors tend to do worse than the industry’s performance figures indicate, adds Wharton finance professor Robert F. Stambaugh. That can happen if investors have more money at risk when a fund does poorly, and less invested when it does well. To some extent, this is similar to what happens to ordinary mutual fund investors, who tend to buy funds after they have been on a run only to ride the next down cycle.

Other studies have shown that hedge funds that become very large have a hard time finding enough good investments to maintain the performance that made them large in the first place, Stambaugh adds. And big funds must trade in such large volumes that they can alter the balance between supply and demand. A huge buy of a bargain stock, for example, will create enough demand to drive up its price until it’s no longer a bargain, making it difficult to acquire enough shares to make the bet pay off.

Fees Questioned

Though the industry remains successful in attracting new investment, there are some pressures for change. “The question of fees looms large,” says Roussanov. Typically, funds charge an annual fee of 2% of assets under management, plus 20% of any investment profits. If the broad market returned 6%, the fund would have to return more than 8% — or 33% more – just to match the market, for instance. It’s hard for an asset manager to beat the markets that handily year after year, which is why many investors, including many who do dabble in hedge funds, put the lion’s share of their assets into low-fee index products designed to mirror broad market sectors like large- or small-company stocks. Fees on many indexed products come to less than 0.2% of assets.

In the face if this competition, “a lot of investors have been able to extract concessions from [hedge fund] managers to reduce both the 2% and the 20%,” Roussanov says.

Though hedge fund assets have recently set records, the industry does face growing competition in what Geczy calls “the liquid alternative space.” These include actively managed exchange-traded funds and some mutual funds that engage in some of the strategies hedge funds are known for, like short selling or betting on commodities, currencies, real estate or interest rates. The difference is that these new alternatives have much lower fees and allow investors, who in many cases need not be accredited, to buy and sell at will.

“There is a new battleground…. Things are changing fast,” adds Yilmaz. “But will this cannibalize hedge funds? I don’t think so.” It may, however, increase pressure on hedge funds to lower fees or make it easier for investors to take money out, he adds.

“I don’t think this will be the end of hedge funds. I just think the business will change…. Hedge funds will be successful as long as you look at the long term.”