Combining In-School and Out-of-School Programs to Improve Learning in India
- Children
- Primary schools
- Students
- Student learning
- Pedagogical innovation
- Tailored instruction
Despite significant increases in the number of children enrolled in primary school in low- and middle-income countries over the past two decades, learning levels have not improved as much as expected. Researchers conducted a randomized evaluation to test the impact of the combination of an in-school learning program with an out-of-school learning program on primary school math and reading levels. The combination of the two programs led to a significant increase in children's test scores in both math and language.
Policy issue
Primary school enrollment numbers have dramatically increased in low- and middle-income countries (LMICs) since the 1970s. The World Bank now estimates that 90 percent of children in LMICs are enrolled in primary school.1 However, the rise in enrollment has not been accompanied by an equivalent increase in learning levels. For example, in India, more than 96 percent of children ages 6-14 are enrolled in school. Despite this, the share of children in grade 5 that can read or do math at a grade 2 level is below 50 percent.2 Further, these reading and math metrics have not improved in the past ten years, suggesting that work is needed to improve the quality of education.
Efforts to improve student learning outcomes have largely focused on interventions and improvements within school itself. One such in-school approach, called “Teaching at the Right Level,” had previously been shown in other settings to be effective at increasing student learning. However, little research has been dedicated to understanding the benefits of combining in-school pedagogical programs with after-school activities, conducted within the community.
Can combining an in-school intervention with an out-of-school program improve student learning in math and reading?
Context of the evaluation
The evaluation took place in the Nagaon district in the state of Assam in northeastern India. Assam has a population of 31 million and shares borders with Bhutan and Bangladesh.3 While school enrollment numbers in Assam are strong, only 34 percent of children in grade 5 can read a grade 2 level text. Further, only 14 percent of children in grade 5 can solve a simple division problem. These statistics rank Assam below the national averages in India.
The intervention was implemented by Pratham, the largest non-governmental organization in the education sector in India and a long-time partner of J-PAL.
Details of the intervention
In order to test whether a combination of an in-school learning program and an out-of-school learning program could boost student learning, researchers partnered with Pratham to conduct a randomized evaluation in primary schools.
The two components of the evaluation were the Learning Camps implemented in the schools and the Study Groups implemented in the villages.
Learning Camps: Pratham’s flagship program builds on the pedagogical approach known as "Teaching at the Right Level," or TaRL. TaRL consists of customizing lessons to the level that children are actually at, rather than simply teaching the content for their age or grade level. The Learning Camps represent Pratham’s standard delivery method and consist of short, intensive periods of teaching and learning activities conducted in school that are focused on language and math. Pratham first tested all students in grades 1 through 5 on their math and reading skills. Then, the children were grouped with others of their level and taught math and language for 3 hours each session. The Learning Camps were administered by Pratham staff with the support of local teachers and volunteers in some locations. They took place over three periods of ten days each, totaling 30 days spread over one teaching term of five months.
Study Groups: Pratham designed Study Groups to give parents and community members the tools to support the education of their primary school children in the villages, and these Study Group meetings were conducted outside of the school setting. The Study Groups were managed by community members and mostly mothers. Staff from Pratham only visited the Study Groups to provide learning materials. Each Study Group consisted of a single adult volunteer, often a mother, and around six children. Study Group leaders were given flexibility in what activities they arranged and the schedule of the group.
200 villages and their corresponding 200 primary schools were included in the evaluation. Each primary school was randomly assigned one of the following four groups:
- Learning Camps & Study Groups: 50 primary schools and villages received both the in-school Learning Camps and the out-of-school Study Groups.
- Learning Camps: 50 primary schools received only the in-school Learning Camps.
- Study Groups: 50 villages and primary school students received only the out-of-school Study Groups.
- Comparison Group: These 50 villages and primary schools received research visits but no interventions.
Due to capacity constraints within Pratham, half of the 100 total Learning Camp villages were randomly selected to receive the Learning Camps immediately after baseline data was collected, while the remaining half received the Learning Camps approximately 5 months later.
To measure whether the learning interventions were effective for children, the researchers collected data from three different sources: head teachers, children themselves, and their primary caregivers. The head teacher of each public primary school was surveyed by researchers. Next, a random sample of children was chosen from each school, resulting in selecting 29 children from each school. The researchers gave the children a test to measure their reading and math skills, as well as a short survey. Finally, 23 randomly selected caregivers of children from each school were interviewed to collect basic information about household characteristics of education-related activities. The researchers collected this data both before the intervention was implemented and approximately 16 months after the program started. The baseline data was collected between May and August 2018. Post-intervention outcomes were measured between November 2019 and January 2020.
Results and policy lessons
The full program, a combination of an in-school intervention with an out-of-school intervention, increased children’s test scores.
Children in villages assigned to both the Study Groups and the Learning Camps improved by 0.11 standard deviations in math and 0.9 standard deviations in language. These effects reflect an increase in the proportion of children who achieved minimum standards in mathematics of 20 percent, and a 13 percent increase in the proportion of children who achieved minimum standards in language. 30 percent of children in the comparison group achieved minimum standards in mathematics, as compared to 36 percent of children in the intervention group, yielding a six-percentage point increase. 31 percent of children in the comparison group achieved minimum standards in language, while 35 percent of children in the intervention group achieved the same, resulting in a 4-percentage point increase.
To add context to these numbers, a review of 270 educational programs in LMICs found that a 0.1 standard deviation increase in learning outcomes was approximately the median effect across these programs.4 Thus, the impact of the combined Study Groups and Learning Camps falls in the middle of other educational interventions in LMICs.
The researchers found that children in schools that held the Learning Camps earlier in the year (timing of the implementation was randomized) than other students scored approximately twice as high on their tests, suggesting that the timing of the program is important. Additionally, the researchers found that there was no gender difference in how the program affected students.
While the combination program improved learning outcomes, the Study Groups and Learning Camps implemented individually did not have any impact on children's test scores. The absence of an impact from the in-school program stands in contrast to previous findings about the same type of intervention in Uttar Pradesh, where researchers found significant improvements to learning outcomes, but is consistent with the large heterogeneity in the impact of targeted learning interventions observed in the literature across settings and studies. The researchers suggest that this discrepancy could be caused by the fact that most schools were already using some of the core principles behind the TaRL approach. For instance, most schools were regularly grouping together children of different ages and grades for pedagogical purposes. The researchers highlight that continuous program monitoring and adapting to local context is crucial.
Cost effectiveness: The researchers also examined the cost-effectiveness of the programs. They found that Learning Camps are more expensive than the community-based Study Groups, an expected result. The average cost for the Learning Camps is US$15.90 (INR 1318) per student, while the average cost for the Study Groups is US$10 (INR 829) per student. Combining the Learning Camps with Study Groups, with an average cost of US$17.20 (INR 1426) per student, can result in cost savings. These cost data show that the marginal cost of including the Study Group component is fairly low (US$1.30 or INR 107) per student) in settings where Learning Camps are already being implemented.
Overall, the researchers estimate that the cost of increasing learning outcomes by 0.1 standard deviations using this combined approach is between US$15.00 (INR 1243) and US$18.30 (INR 1517). In comparison to 27 other education program evaluated by J-PAL, this intervention falls right in the middle.
Policy implications: The combination of an in-school pedagogic program with an out-of-school Study Group program can effectively increase primary school children’s learning. The Learning Camps program currently expanded and implemented across India and in some parts of Africa was found in this study not to be sufficient to improve student outcomes, when implemented alone. However, combining it with an out-of-school Study Group program effectively increased primary school children’s learning. Given the popularity and expansion of the TaRL model, this study shows that combining it with a community-managed out-of-school program offers policymakers a relatively cheap option to enhance and adapt the original model across new contexts.
World Bank. 2018. “School enrollment, primary (% net) - Low & middle income.” Last modified February 2020. https://data.worldbank.org/indicator/SE.PRM.NENR?locations=XO.