Project Generasi: Conditional Community Block Grants in IndonesiaPDF version

 

There has recently been an increased drive among governments, international organizations, and NGOs to strengthen the link between development aid and performance. In Indonesia, researchers tested a pilot program that used incentivized community-based block grants to improve health and education outcomes. The evaluation found that linking community block grants to performance improved indicators of health, but had no impact on education indicators.

Researchers: 
Ben Olken
Researchers: 
Junko Onishi
Researchers: 
Susan Wong
Partners: 
World Bank
Fieldwork implemented by: 
World Bank
Location: 
Indonesia
Sample: 
3,100 villages
Timeline: 
2007 - 2009
Themes: 
Education
Themes: 
Health
Policy Issue: 
Community Participation
Policy Issue: 
Early Childhood Development
Policy Issue: 
Immunization
Policy Issue: 
Nutrition
Policy Issue: 
Student Participation
Policy Issue: 

Recently there has been an increased drive among governments, international organizations, and NGOs to strengthen the link between development aid and performance. For example, all foreign aid given out by the U.S. Millennium Challenge Corporation is explicitly conditioned on recipient countries meeting 17 indicators of good governance, ranging from civil liberties to immunization rates to girl’s primary education rates. While linking aid to performance may create incentives to improve effort and mobilize additional resources, there are potential downsides as well, such as multitasking problems, where effort is allocated toward targeted indicators at the expense of other goals. There is also a risk that performance-based aid will drive budgets to be directed to richer or otherwise better-performing locations.

Context of the Evaluation: 

In 2007, the Indonesian government began a pilot program to test a new approach to improving health and education: incentivized community-based block grants. Under the program, known as Generasi, villages received annual block grants which they could allocate to activities improving 12 health, nutrition and education indicators. The 12 indicators represented health and educational activities that were within direct control of villages - such as the number of children who receive immunizations, prenatal and postnatal care, and the number of children enrolled and attending school – rather than long-term outcomes, such as test scores or infant mortality.  

Details of the Intervention: 

Researchers partnered with the Government of Indonesia to evaluate the impact of the Generasi program on the 12 health and education indicators. A total of 264 subdistricts were randomized into either a comparison group or one of the two versions of the Generasi program: the “incentivized” version with a pay-for-performance component, or the otherwise identical, “non-incentivized” version without pay-for-performance incentives.

In the first year, both treatment groups received program funds based on the number of target beneficiaries in each village (i.e. the number of children and the expected number of pregnant women). In the second year, funds were allocated in the same way in non-incentivized villages, but in incentivized villages 20 percent of the funds were distributed based on village’s performance on the 12 indicators during the last year. The purpose of this bonus was to increase villages’ effort at achieving the targeted indicators, both by encouraging a more effective allocation of funds and by stimulating increased efforts to encourage mothers and children to obtain appropriate health care and increase educational enrollment and attendance.

The block grants averaged US$8,500 in the first year of the program and US$13,500 in the second year of the program. To decide how to spend the grant money, trained facilitators helped each village elect an 11-member village management team, as well as select local facilitators and volunteers. Consultation workshops were then held with local health and education providers to gather information and technical assistance, and coordinate the use of the funds.

Results and Policy Lessons: 

Impact on health indicators: Over the two years of the program, the targeted maternal and child indicators, including prenatal visits, delivery by trained midwives, childhood immunizations, and growth monitoring, were an average of 0.04 standard deviations higher in incentivized areas than in non-incentivized areas. This effect was driven primarily by increases in the number of prenatal visits, which was 8.2 percent higher in incentivized areas than non-incentivized areas, and regular monthly weight checks for children under five, which were 4.5 percent higher in incentivized areas than non-incentivized areas. While these differences are modest, the impact of the incentives was more pronounced in areas with low baseline levels of service delivery: the incentives improved the targeted maternal and child health indicators as much as 0.11 – 0.14 standard deviations in the poorer, off-Java provinces.

Impact on education indicators: There were no differences between incentivized and non-incentivized areas on the four education indicators examined (primary and junior secondary enrollment and attendance).

Adverse impacts of incentives: Researchers found little evidence that providing incentives had adverse effects. There was no evidence of a multi-tasking problem, or that immunization records or school attendance was manipulated.

Mechanism of impact: The results suggest two main channels through which the incentives may have had an impact. First, the incentives appear to have led to a 16 percent decrease in spending on school supplies and uniforms, and a 6.5 percent increase on health expenditures. Despite the reallocation of funds away from school supplies and uniforms, households were no less likely to receive these items and they were of no lesser value, and were actually more likely to receive education scholarships in the incentivized areas. The results suggest that the change in budgets resulted from more efficient spending. The incentives also led to an increase in the labor of midwives, who are the major providers of maternal and child health services in the area. By contrast, there was no change in labor supplied by teachers.

Cost-effectiveness analysis: Researchers calculated the total benefits of the program as the total number of “bonus points” the program created, using the weighting scheme assigned to each of the 12 indicators to calculate a village’s performance. Researchers calculated that the Generasi program as a whole cost about US$8-US$11 to generate one additional bonus point. Translating bonus points back into outcomes suggests, for example, that the implied cost of preventing one malnourished child was US$384-US$528, and the cost of enrolling one more child in primary school was US$200-US$275. When the additional costs of the performance incentives are isolated, the cost of the incentives themselves comes out to US$0.60 per point (e.g., US$30 per additional malnourished child prevented). This suggests that while the program itself is not particularly cost-effective, providing incentives is a way to make it more cost-effective than a normal block grant program. Adding similar performance incentives to other, pre-existing block grant schemes (holding the total amount of money spent) could be a very cost-effective way to improve aid effectiveness.

Related Papers Citations: 

Olken, Benjamin, Junko Onishi, and Susan Wong. "Should Aid Reward Performance? Evidence from a Field Experiment on Health and Education in Indonesia." Forthcoming, American Economic Journal: Applied Economics.