In 2009, Google published an important paper in Nature. The paper explained how Google could “predict” the spatial spread of winter flu in the US, prevalent at that time, by analysing what people were searching in the internet. The paper was remarkable, because it attempted to address a public health problem in a manner no one could think of, while other options were simply unavailable.
Many similar examples indicate that a new information revolution is brewing, and it foresees use of data for solving human problems in a way hitherto unheard of. Big Data Revolution, as it is called, uses huge volumes of data generated by daily human activities to find answers to complex problems. Many businesses, governments, academics, and international organizations have realized the potential of big data analytics and want to use it to their advantage.
So, what is this big data? There is no rigorous definition. Big data is referred broadly as “the ability of society to harness information in novel ways to produce useful insights or goods and services of significant value” (Mayer-Schönberger and Cukier, 2013, 9). It has been heralded as, “the next frontier for innovation, competition, and productivity” (McKinsey Global Institute, 2011, 1). McKinsey Global Institute (MGI) has defined big data as, “datasets whose size is beyond the ability of typical database software tools to capture, store, manage, and analyse” (2011, 1). Every day, an avalanche of data is created by interactions of people with billions of devices, financial transactions etc. Big data analysts try to harness information in these huge volumes of data to produce useful insights which may result in goods and services of significant value. The idea is to use already available information such mobile data usage, banking transactions, buying patterns in supermarkets etc. to find out answers about the well-being of the people.
Big Data for M & E
So, what does the big data revolution mean for monitoring and evaluation in the development sector? The main aim of monitoring and evaluation exercises in the development sector have been to find out whether any intervention was effective. Traditionally, monitoring and evaluation (M&E) relied on extensive surveys to find out what worked and why. To be honest, data collection through surveys can be expensive, and, despite the perceived rigour of the statistical design, the findings remains a guestimate, at best. Sometimes, the impact evaluation exercise may end up using more efforts and resources than the actual intervention. Factoring in miscellaneous costs and overheads, any robust evaluation design ends up costing around 5-10% of the actual value of the project. It necessitates a relook at the approaches employed in M&E, and calls for a search for alternatives. The big data revolution provides an opportunity to use huge volumes of freely available data and obtain valuable insights about the interventions. This opens the door of a new approach, which may be termed as digital M&E.
Digital M&E using Big Data: New Data Sources, New technologies and New Approaches
Big data revolution presents some new sources of data, in addition to the conventional ones. Two types of sources: active sources like blogs, tweets, Facebook posts and likes, web-clicks etc. where the users actively interact, and passive sources transactional data, mobile signals, sensor data etc. Given the huge volumes of data generated from these sources, it requires adoption of new computational techniques such as machine learning, distributed processing, data mining and sensemaking etc., hitherto unused in the realm of M&E. This calls for the changes in the approach of the M&E regime. In traditional approach, much emphasis is placed upon the statistical rigour of the M&E framework and well-defined indicators are used to quantitatively measure change. However, in the big data approach, statistical robustness is not a major criteria. Instead, an intervention is better understood by interpreting and extrapolating the pieces of information that comes from a combination of diverse sources.
0 comments:
Post a Comment