Humanitarian impact evaluations: WFP and J-PAL build commitment for a common cause

This blog was co-authored by staff at J-PAL and the World Food Programme (WFP). A variation of it can be found on WFP's website.

In 2019, as the World Food Programme (WFP) published its impact evaluation strategy, Professor Abhijit Banerjee stepped onto a stage in Stockholm, Sweden, to receive the Nobel Prize in Economics. 

In his Nobel lecture, Banerjee described his work on randomized controlled trials (RCTs) in development contexts. These studies, first met with “reactions of puzzled tolerance” in the mid-90s now a number in the thousands, with 2,400 evaluations conducted by the J-PAL network alone 

Last year, on a stage in Rome at the Global Impact Evaluation Forum, WFP and J-PAL found themselves exchanging ideas on this very topic. 

 The world looks dramatically different than it did in 2019. An upward development trajectory would change course and development. Humanitarian practitioners are having to balance debilitating funding cuts on the one extreme, with mounting humanitarian crises on the other. 

In his closing remarks, WFP Deputy Director Carl Skau summarized it: This world demands hyper prioritization.Rigorous evidence from impact evaluations, coupled with the operational partnerships to make them happen in practice, are well suited to answer some of the hardest humanitarian questions during the hardest of times.

‘The presumption of knowledge is dangerous’

Prof Banerjee’s keynote conversation at the Forum opened with a story from his early work using RCTs in education: doubling teacher-student ratios. Everyone thought it would work. It didn’t. Often, he said, ‘common sense’ interventions, like providing textbooks or flip charts, that ‘felt right’ failed when tested rigorously. Banerjee argued that at times, development and humanitarian systems frequently acted on instincts, on ideology, or “how we’ve always done it,” and urged the audience to keep humility and curiosity as presumptions, apart from being ineffective, may even be harmful.

RCTs also prove – very effectively – what is working over the long run. Banerjee cited the Graduation Approach, pioneered by the Bangladesh Rural Advancement Committee (BRAC) in the early 2000s.

The study empirically showed that even the poorest households can become self-sustaining if given an initial, carefully designed boost.The model provided families with productive assets of their choice, such as livestock, and income support until their assets started showing a return, coupled with 18 months of intensive training and encouragement.

Large-scale RCTs in seven countries – Bangladesh, Ethiopia, Ghana, Honduras, India, Pakistan, and Peru – have consistently shown how households had better financial outcomes and increased psychosocial well-being even three years after the support ended. Researchers have tracked the same participants and found persistent effects of the program 11 years later. They also found intergenerational benefits as children started contributing to higher income streams. 

With an upfront cost ranging from $330 to $1,900 per household, depending on local costs, the long-term benefits outweigh the up-front investment nearly everywhere the program has been evaluated, and in certain countries, the economic gains generated were nearly four times the program’s cost a decade later.

Evidence – a ‘public good’ in humanitarian settings
 

The more collaboration that exists between funders, UN agencies, NGOs and academic partners across countries and contexts, the more feasible it becomes to build long-term portfolios of evidence such as the Graduation Approach.

In recent years, RCTs and other rigorous impact evaluations have gained momentum in contexts previously believed to be unnecessary in urgent crises, or even unethical or impossible in resource-constrained environments that can change overnight. 

Through its humanitarian workstream, WFP, alongside partners like J‑PAL and others across the humanitarian system, has demonstrated that these hurdles can be overcome. 

Recent examples include a randomized comparison in the DRC showing that community‑based targeting can match the precision of data‑driven proxy means test models at lower costAnother evaluation in Northern Ghana demonstrates that a simple change in transfer timing (lump‑sum versus monthly) can significantly boost agricultural investment and short‑term food security impact evaluations in Nepal and Bangladesh show that anticipatory action delivered before forecasted floods leads to higher food consumption, fewer negative coping strategies and improved wellbeing compared to equivalent post‑shock transfers. 

Insights from J-PAL’s evidence effect campaign highlight three key lessons on: 1) cash transfers in humanitarian settings and the importance of effective delivery, 2) supporting long-term recovery and inclusion via improving social cohesion, and 3) the importance of providing quick financial support to families in advance of extreme weather events in improving their resilience and recovery.

These examples are contributing to the foundation of humanitarian evidence that Banerjee described as a“ public good”.

J-PAL, with its partners, has invested significantly in advancing rigorous research in fragile and conflict-affected settings through dedicated research initiatives (CVI, HPI, DLI). Through these initiatives, J-PAL has supported a growing network of researchers working to address the methodological and operational challenges specific to research in crisis contexts, while accompanying and partnering up with local and international implementing organisations to generate new evidence for the sector.

The forum in Rome affirmed that UN agencies operating in humanitarian and fragile contexts share this commitment.

Now, it needs operationalizing. 

This year, WFP aims to continue efforts, in collaboration with other agencies and partners, towards a unified inter-agency agenda for impact evaluation and rigorous learning in development and humanitarian settings. 

Such an agenda would focus on identifying small, scalable design improvements that can strengthen crisis programming, often without increasing cost or complexity.

Partnerships between humanitarian agencies like WFP and J-PAL will play a fundamental role in driving this momentum forward. With complementary expertise – WFP’s global operational footprint and humanitarian mandate and J-PAL’s methodological leadership and academic network – the partnership provides a tangible framework for implementation.