Big data and climate change share one important characteristic: Both are changing the course of history. Carbon dioxide levels have not been this high in 800,000 years, and the amount of data being generated today is unprecedented.

The question at the recent Wharton conference on “Sustainability in the Age of Big Data” was how rapidly advancing information technologies can be brought together to forestall the worst ravages of global climate change. As Gary Survis, CMO of Big Data company Syncsort, IGEL senior fellow and conference moderator, noted, “It is rare that there is a confluence of two seismic events as transformative as climate change and big data. It presents amazing opportunities, as well as responsibilities.”

Coming to terms with the scope of big data is a challenge, but the promise is enormous. Big data has the potential to revolutionize the two industries that generate the most carbon dioxide — energy and agriculture. Machine-to-machine communication can help reduce energy demands and increase the viability of renewable power sources. On farms, data from the molecular level may help give rise to a new green revolution, and sensors in satellites, farmland, trucks and grocery stores promise to reduce waste industry-wide.

Important questions remain. Can big data be used to influence people’s behavior without manipulating them? Can private enterprise capitalize on big data’s possibilities without riding roughshod over the rights of those who generate the data? And can the high-tech innovations already underway in the developed world help solve the problems of those most in need?

How well we answer these questions will determine whether we can realize the historic potential of “Sustainability in the Age of Big Data.”


What Big Data Means, and What It Can Mean for Sustainability

More data is now generated in a couple of days than the human race produced in the past two millennia. The volume of information is great, as is the variety. As much as 90% of the data generated by the Internet of Things is unstructured — sounds, images, sensor readings, and mismatched and unaligned metrics. Most vexing of all, perhaps, is the need to verify all this data. IBM estimates that poor data costs the U.S. economy $3.1 trillion each year.

Yet those who attended the recent conference on “Sustainability in the Age of Big Data,” hosted by the Initiative for Global Environmental Leadership (IGEL), heard convincing evidence that, while obstacles remain, Big Data is likely to have a big impact on the environmental problems threatening the planet.

Big Data and Energy: A Clear Synergy

Utilities and energy companies are finding that there are big savings in Big Data. The flood of new information is occurring just as reducing demand and increasing fuel efficiency (with attendant climate benefits) has become a top priority for government and industry. Fortunately, harnessing information has already yielded big energy gains, and considerably more are promised.

For instance, companies like Virginia-based OPower are using Big Data to allow homeowners to measure their consumption against their neighbors. Cities are optimizing the timing of traffic signals to reduce congestion, airports are communicating with planes to increase the efficiency of waiting ground crews, and building managers are using data analysis to cut energy use by 10% to 20%. And tomorrow’s smart thermostats will connect to the web and give consumers remote access to managing their electricity use. 

Can Big Data Feed the World?

The challenge: feed the one billion people who are now going hungry plus the nearly two billion more expected to join their ranks as world population grows 24% in the next 35 years — all while rising seas and increasing droughts destroy arable land; pollution and crumbling infrastructure deplete usable water reserves, and the use of energy from traditional sources compounds both problems. The UN’s Food and Agriculture Organization estimates that the global food supply will have to increase 70% in the next 35 years if famine is to be avoided. 

The role of Big Data:  By harvesting the vast amount of information coiled up in the DNA of seeds, forecasting weather and soil conditions from one end of a field to another and formulating optimal seeding plans for each field based on its particular characteristics, scientists can help farmers increase yields using less water, less fertilizer and less energy. And once those crops are harvested, they can be tracked and traced all along the supply chain, reducing the waste that now consumes 40% of the food in the developing world before anyone ever gets a chance to eat it.

The Dark Side of Big Data

As with so many innovations in the past, the benefits of Big Data come with significant risks. Practical issues have to be addressed, including the cost-effective transmission of huge data streams and a huge increase in the number of skilled engineers and data scientists needed to harness the power of all that data. Privacy, too, is a growing concern — not just in the worlds of espionage and online consumerism, but also in the ways Big Data may most foster sustainability. 

Big Data is poised to play a powerful role in sustainability, but that power can be abused, and it can remain cloistered in the developed world, providing scant benefit to those who most need it: the exploding population of the developing world.