Evidence and Gap Map of Developmental Impact Evaluations in India (India Country EGM)

Knowledge brokering has a crucial role in addressing the challenge of using research evidence, including evaluation findings, in policy implementation. It can help to bridge the gap between many different types of knowledge, such as contextual knowledge or equity knowledge. In the field of knowledge brokering, ‘evidence mapping’ is emerging as a systematic and replicable methodology that allows an understanding of the extent and distribution of evidence in a broad sector area such as development impact evaluations, highlighting both what is known and where gaps in evidence exist. As highlighted in the blog (Making evidence accessible and reducing waste in research) previously, mapping is an approach that can be applied to a range of research question and is not restricted to intervention studies. Our megamap of child welfare includes over 300 reviews, another map in high income countries on homelessness included process evaluations. A recent innovation has been Uganda country EGM that included all the impact evaluation, process evaluation and formative evaluations conducted in Uganda. We followed the same methodology to develop the pilot phase of the India country EGM.

Evaluation of the effectiveness of the public development programmes adds to the outcomes achieved and impact created by them. In India, timely evaluations of such programs are limited as currently there are no reliable resources to help identify what we know and what we don’t when it comes to the evaluation of development programs. Evidence is scattered and there is no way for decision-makers to get a quick overview of the existing evidence base. It also makes it hard for research funders to ensure that limited resources on evaluating programs are spent effectively. Evidence mapping can help provide a comprehensive overview of existing knowledge in this area and enable the purposeful and targeted commissioning of future impact evaluations.

What is the India Country EGM of Developmental Impact Evaluation?

An Evidence Gap Map (EGM) is a visual tool that aims to map existing research or lack thereof, across the different dimensions of developmental impact evaluations. It is essentially a matrix that maps evidence at the intersection of a row and a column. This map is based on Campbell conduct and reporting standards checklists for evidence and gap maps.

The India Country EGM will be an interactive online webpage that allows users to explore evidence on various dimensions of development impact evaluations in India as well as to detect gaps that reflect the lack of any existing evidence. The map was created after reviewing and systematically organising academic literature from 3IE database and will be extended to a full search of impact evaluations to other databases.

Figure: Snapshot of India country EGM

Based on the efforts to align the NITI Ayog ‘Strategy for New India’ with India’s commitment to the United Nations’ Sustainable Development Goals published ‘Strategy for New India @ 75’, this evidence map has identified we identified four major interventions areas (shown as rows) that were mapped to ten effects/ outcomes (shown as columns). For the rows, we considered infrastructure interventions, inclusion interventions and governance. Based on the interventions or characteristics present in the literature, we divided the rows into four broad categories such as –

1. Drivers
2. Infrastructure
3. Inclusion
4. Governance

Each of these rows and columns is further divided into multiple context-relevant sub-categories. The EGM is colour-coded (See Figure 1) where the colour indicates the completed and ongoing EGMs.This evidence map identified and mapped 267 impact evaluations conducted in India. The main finding of the map is the evidence is unevenly spread across the sector and the administrative divisions of India with the majority of studies coming from Southern Zonal council of India. Health is the most evaluated sector and Governance being the least evaluated.

This is a first edition of the map. Over the next month, we will seek to identify impact evaluations from other databases and grey literature and also adding more studies we may have missed.

Learn More about Campbell Evidence and Gap Maps here

This map is commissioned by Campbell South Asia and Public Health Evidence South Asia (PHESA)