By Sonal Shah & Hollie Russon Gilman, Stanford Social Innovation Review |

Big data has become a buzzword for private, public, and social sector organizations. For the social sector, there is a belief that “big” data is the new panacea to solving our greatest social challenges — whether criminal justice, health care, education, or international development. On the other side, there is concern about the cost of collecting data, the type of data we collect, and the real questions of privacy and ethics of data use.

We believe that data has the potential to help governments (local, state, and national) achieve real outcomes, but we need to ensure that we are collecting useful data, and governments need to put in place some practical safeguards before asking the public to invest in new systems and data collection. We need to examine the value of transparency of big data; understand the types of data needed to achieve outcomes; differentiate the differences between data, evidence, and judgment; and ensure that citizens are included in the conversation.

Why data matters

In the sciences, and increasingly in the social sciences, data has been a critical part of understanding, testing, and proving theories. It has the potential to more-effectively address critical challenges in our society — to target school interventions, improve health care, or help people find the right job training. Our ability to collect, analyze, and better understand data has become increasingly easier and cheaper. Even with limited resources, we can now collect micro-level information in real time, detect early warnings, and provide insights for effective, targeted interventions. Community- and heat-mapping techniques, for example, provide a wide range of valuable information, helping us better understand crime patterns and isolate hyper-local health conditions. In Chicago, data is helping Chicago Health Atlas identify health trends and provide hospital information. And Foodborne Chicago is using sentiment analysis (determining whether a piece of writing is positive, negative, or neutral) from social media and location-based 311 reports to detect food poisoning incidents. Data can help government provide better and more-effective services for its citizens.