Microfinance's Elusive Quest: Finding an Accurate Measure of Social ImpactPublished: April 28, 2010
Two new studies have raised doubts about long-held beliefs in microfinance. The studies -- which used randomized controlled trial methodologies -- did find that microloans helped poor entrepreneurs boost profits in their businesses. However, the studies found little impact on health, education, average consumption, women's decision making or self-reported well being.
The unexpected results, which were quickly picked up by The Economist and other media, created a stir in the microfinance community. "They don't capture what we believe is the real impact or dimensions of microfinance, which sometimes takes years to demonstrate," says Bruce MacDonald, senior vice president of communications at Accion International, a Boston-based non-profit involved in microfinance since 1973. "We would definitely disagree that microfinance doesn't have an impact on poverty because we have seen the impact over the past 30 years."
One problem with previous research, microfinance's critics respond, is self-selection -- that is, studies only survey clients who successfully take out loans. The new studies differ from the past in that both attempt to measure the impact of microfinance by comparing two groups: one with access to microcredit and one without.
In the first study, researchers from the Massachusetts Institute of Technology (MIT) worked with a microfinance lender in India to look at 104 slums in Hyderabad. Half of the slums were randomly selected to receive a loan branch, while the other half were not. Fifteen to 18 months later, researchers interviewed 6,850 households in the community -- about 65 per slum -- in an attempt to gauge changes in economic and personal well-being.
The second study, by Dean Karlan of Yale University and Jonathan Zinman of Dartmouth College, used a similar method to analyze microfinance in the Philippines, but instead of communities they looked at individual borrowers, tracking a total of 1,601 marginally credit-worthy loan applicants in Manila. Half were randomly offered a loan; half were denied. Researchers interviewed the sample group 11 to 22 months later to see if anything had changed. In both cases, results were mixed.
"While microcredit succeeds in affecting household expenditure and creating and expanding businesses, it appears to have no discernible effect on education, health or women's empowerment," states the India study. "Of course, after a longer time, when the investment impacts (may) have translated into higher total expenditure for more households, it is possible that impacts on education, health or women's empowerment would emerge. However, at least in the short term (within 15-18 months), microcredit does not appear to be a recipe for changing education, health or women's decision making. Microcredit therefore may not be the miracle that is sometimes claimed on its behalf, but it does allow households to borrow, invest, and create and expand businesses."
The Empowerment of Women
Alex Counts, president of Washington, D.C.-based Grameen Foundation USA, calls the studies "preliminary" and believes the timeframe is too limited to be meaningful. He cautions against making sweeping generalizations about microfinance on the basis of two studies, given how environments can vary from country to country. A better measure, he argues, is research his organization commissioned in 2005 that analyzed 90 major impact studies and found that microfinance had positive effects. Counts also questions the Manila results based on the lender involved. "They studied a microfinance institution that is in no way typical of good microfinance practice in the Philippines," Counts says.
For microfinance practitioners who focus on women's empowerment, the results of the studies were particularly surprising. The India study found no impact on women's decision making, while the Manila study found some evidence that microcredit helped men more than women.
"It surprised me but I also felt that it was leaving something out," says Mary Ellen Iskenderian, president and CEO of Women's World Banking. The 30-year-old New York-based institution, which works with 40 microfinance lenders in 28 countries, has "solid data" from numerous studies around the world that shows women who take out microloans experience a decrease in domestic violence. Women's empowerment is "infinitely harder to measure but every bit as important as the economic change," says Iskenderian, who sees the empowerment factor come up in many studies. "It's borne out by data as well as speaking to the women themselves."
Indeed, it may not be practical -- or even possible -- to measure poverty alleviation or profound social changes with randomized controlled trials, some say. Microfinance is too complex, its programs too varied. Loans are all different sizes for different types of businesses, and villages and families are never exactly the same. As microfinance grows and expands, it has become more difficult to isolate a "control group," especially in urban areas, because communities not served by one microfinance institution may still have access to another. Iskenderian recalls hearing of one attempted study that had to regroup after researchers discovered women in the control group (the village denied microfinance) were secretly sending their daughters to apply for loans in the next village, where credit was available.
Despite controversy over methodology, the two studies reflect a recent push in the microfinance industry to better measure social performance, says Wharton management professor Keith Weigelt. In recent years, as an increasing number of for-profit lenders have entered a market once dominated by non-profits and aid agencies, experienced practitioners have become concerned that microfinance is straying from its original mission of social development -- what those in the industry call the "double-bottom line."
Meanwhile, microfinance institutions and their donors have stepped up efforts in the past few years to improve transparency and impact assessment. The Grameen Foundation and the Consultative Group to Assist the Poor (CGAP), the microfinance research arm at The World Bank, commissioned the development of the Progress out of Poverty Index (PPI) to help microfinance institutions measure social impact. In 2008, the MasterCard Foundation, a large microfinance donor, gave $740,000 to the Microfinance Information eXchange (MIX) to help launch a web platform to track performance. And large donors such as The Bill and Melinda Gates Foundation have financed increasing numbers of studies on microfinance. "It's an evolution of the industry that we are seeing more studies being done," according to Weigelt. Finding an accurate measure of social impact is "the holy grail."