Field notes: The power of community partnerships for scaling learning interventions
In Muzaffarpur, a district in the eastern Indian state of Bihar, a small group of 10-year-olds sat in a circle around their teacher. Schools were off for the annual summer break. But unlike their friends, who could choose to spend their summer mornings playing games or watching television, these children were spending theirs attending special lessons to practice reading and learning the rules of basic arithmetic.
These lessons were part of CAMal Ka Camp, conceptualized by NGO Pratham to complement the existing education system and delivered with the Government of Bihar. Inspired by Pratham's "Teaching at the Right Level" (TaRL) approach, CAMal Ka Camp focused on intensive instructional periods to improve foundational skills.
The world has made tremendous progress in getting children into schools. Not so much in making them learn. In India, which has one of the biggest school systems in the world, nearly every eligible child receives primary education. But almost half of 10-year-olds struggle to read a simple story by the time they finish primary school. The ASER 2023 report found that more than half of children in rural India between 14 and 18 years of age cannot solve simple three-digit division problems, typically taught in third and fourth grade.
Ending this learning crisis requires pedagogical innovations with proven effectiveness—and then rapidly expanding their reach for maximum impact.
The Government of India has turned its attention to improving the basic literacy and arithmetic skills of children, also referred to as foundational learning. It revised the National Education Policy (NEP) in 2020 to emphasize strong foundational education for children between three and eight years of age. In 2022, the Ministry of Education launched the National Initiative for Proficiency in Reading with Understanding and Numeracy (NIPUN) Bharat Mission with the goal of achieving universal foundational learning by 2025.
Pratham has been spearheading the movement to improve foundational learning among children in India for three decades. The NGO developed TaRL, a pedagogical technique that teaches children based on their learning levels rather than age or grade. Six randomized evaluations across seven Indian states by J-PAL affiliated researchers found that TaRL has led to some of the largest learning gains for children in grades three to five, among rigorously evaluated education programs. Today, TaRL reaches 60 million students in India and 4 million students across 12 countries in Africa.
This spirit of collaboration, exploration, and learning between Pratham, J-PAL affiliated researchers, and J-PAL South Asia is perhaps one of the biggest reasons behind the global success of TaRL. In this journey, they have benefited greatly from the enthusiasm of their partners in government and civil society, including school administrators, teachers, and community volunteers.
Community volunteers: Catalysts for success
In Muzaffarpur, Pratham recruited and trained young men and women from the local community who had volunteered to deliver remedial education to children over six weeks as part of its CAMal Ka Camp program.
With the government as the implementation partner, CAMal Ka Camp has reached approximately 3.5 million children across Bihar, Madhya Pradesh, and Uttar Pradesh by mobilizing more than 300,000 volunteers with support from various government departments. The program is one of Pratham’s latest initiatives to scale up TaRL-derived approaches across India, aiming to reverse the significant learning losses caused by the Covid-19 pandemic.
This community-oriented strategy is in line with NIPUN Bharat’s goal of actively involving all stakeholders—including teachers, parents, students, and community members—to lay a solid foundation for lifelong learning.
For over fifteen years, J-PAL affiliated researchers and Pratham have been evaluating different models to strengthen children’s foundational learning. Their research shows that volunteers from the local community, with some initial support, can play a significant role in overcoming challenges in expanding the reach of targeted pedagogical approaches such as TaRL.
One of TaRL’s precursors, Balsakhi, had a woman from the local community deliver special lessons to children using a standardized curriculum designed by Pratham. These women tutors were trained by Pratham and received a small fee for delivering the lessons.
In 2005, the Pratham team mobilized a clutch of villages in parts of Uttar Pradesh to identify children who needed extra support. It then trained volunteers in these villages to use a pedagogical technique developed by Pratham for teaching basic reading skills to deliver special classes to these children after school.
Randomized evaluations by J-PAL-affiliated researchers found that both of these volunteer-led models led to significant improvement in learning outcomes. Today, community volunteers have emerged as one of the driving forces behind TaRL's expansion in Bihar and beyond. In Muzaffarpur, it will not take long for someone to realize that its residents highly covet good education. In fact, several CAMal Ka Camp volunteers are young men and women studying or looking to enter college.
The modest schools didn’t dampen the spirits of the children in the learning camps. With laughs and giggles, they shouted the words they could think of using letters of the Hindi alphabet. The ultimate task was to form a full sentence with these words. Most were able to accomplish it.
The volunteers, for their part, made these camps fun through activities and games they played with the children, sometimes inside the classrooms; at other times in the school courtyard. The sweltering summer sun didn’t seem to bother anyone.
Keeping the wheels turning
It is practically impossible for an intervention to succeed at a large scale unless the government backs it.
In CAMal Ka Camp’s case, the Government of Bihar’s State Rural Livelihood Mission (JEEViKA), an agency working on rural poverty alleviation, and other local NGOs provided Pratham with a large pool of eager volunteers. Approximately 100,000 community volunteers in Bihar alone delivered these special lessons to children in grades 4 through 6.
But most of them had no formal teaching experience, just like those in other states. Pratham trained officials from government and NGOs, who subsequently trained the volunteers in Bihar and elsewhere so they were able to run the classes in a fun and engaging way.
Training sessions were held at district and sub-district levels, supplemented by a wealth of materials. The CAMaL Ka Camp volunteer booklet detailed activities, stories, letter charts (Barakhadi), assessment tools, and data collection formats. Quizzes, audio stories, known as Kahani Train, and other digital training materials delivered to the volunteers via WhatsApp equipped them to conduct the classes in an engaging manner.
All community volunteers received training from Pratham as well as from the Government of Bihar’s District Institute of Education and Training (DIETS) on TaRL.
It takes a village
Covid-19 gave the Indian government’s efforts to strengthen foundational learning a fresh impetus — and brought the significance of remedial education into sharp focus. Pratham intends to expand CAMal Ka Camp at an even bigger scale in the coming summer months.
Pratham and J-PAL South Asia’s experience with TaRL makes it clear that the local community can be an important participant in rapidly expanding the reach of evidence-based remedial education programs.
Equally importantly, it is a reminder that long-term, multi-stakeholder partnerships with governments, NGOs, and researchers are critical to take scientifically proven policies to as many people as possible, in as many places as possible.
This piece also appears on Pratham’s blog
Reorienting instruction has improved learning opportunities for over 60 million students in India and Africa.
Teaching at the right level (TaRL) is an approach developed by the Indian NGO Pratham that aims to build foundational skills in math and reading for all children before exiting primary school. At the instructional level, the approach works by assessing children’s learning levels using a simple tool; grouping children based on learning levels rather than age or grade; using a range of engaging teaching and learning activities; focusing on foundational skills rather than solely on the curriculum; and tracking children’s progress.
When TaRL is implemented within government systems, Pratham helps ensure that teachers receive strong academic support through mentors who are part of the government system. Ongoing, onsite monitoring and support, as well as reviews at different levels of the school system, all contribute to the effectiveness of the program.
A series of randomized evaluations by J-PAL affiliated researchers over the past fifteen years have shown that TaRL consistently improves learning outcomes when implemented well and has led to some of the largest learning gains among rigorously evaluated education programs.1 The iterative process of innovation and evaluation in collaboration with J-PAL has helped Pratham to refine and adapt TaRL over time, which has now reached millions of children in India and Africa.
The Problem
Children around the world are in school but not learning.
Most low- and middle-income countries have dramatically increased access to schooling in recent years. However, record high enrollment rates have often not translated into improvements in learning for all students. For example, in rural India, enrollment rates in primary schools were over 96 percent in 2018, but only half of children in grade five could read a grade two level text and 52 percent could correctly do a numerical two-digit subtraction problem with borrowing–a skill that children are expected to have by the end of grade two.2 Similarly low learning levels exist in Sub-Saharan Africa: in 2015, only 40 percent of grade five students in Uganda could read a grade two level text in the local language.3
School systems in India and other countries are not always designed to address the evolving needs of students, many of whom may be the first in their families to attend school. In practice, many national curricula target only the top students and fail to provide support to the majority of children who fall behind. Factors at school and at home contribute to this problem.
School-level factors:
- Schools are usually organized by age and grade, with children progressing into the next grade regardless of learning levels.
- Teachers are expected to complete the prescribed curriculum, which becomes more and more difficult each year.
- There is often no system in place to assess children’s progress or foundational skills in early years in primary school.
- School systems often do not provide learning support to children who fall behind.
At-home factors:
- Many children come from families where parents have had little schooling and cannot provide learning support even though their aspirations for children’s educational attainment can be high.
- Children have not had any preschool exposure.
The Research
Pratham’s Teaching at the Right Level has consistently produced large and cost-effective gains in learning outcomes.
Pratham, one of India’s largest education NGOs, developed Teaching at the Right Level (TaRL) in the early 2000s to provide a solution to this particular problem. At the instructional level, children are assessed using a simple tool and then grouped according to their learning level rather than their age or grade. Instructors teach each group starting from what children already know. This approach works best with children in grade three or older because they have some experience in school and are prepared for the activities. For each group, there are activities and simple materials designed for helping that group move ahead. There are activities that children do in big groups, small groups and individually.
As a combined result of these elements, children can progress quickly to the next group. Throughout the entire process, teachers assess their pupils’ progress through ongoing, simple measurement of their ability to read and do basic arithmetic. TaRL classes break free of the “chalk and talk” practices commonly found in primary school classrooms across the world by using engaging, fun, and creative activities focused on building foundational reading and mathematics skills.
Since 2001, J-PAL affiliated researchers—Abhijit Banerjee (MIT), James Berry (University of Georgia), Shawn Cole (Harvard Business School), Esther Duflo (MIT), Leigh Linden (University of Texas Austin)—and colleagues—Rukmini Banerji, Rachel Glennerster, Harini Kannan, Stuti Khemani, Shobhini Mukherji, Marc Shotland, and Michael Walton—have partnered with Pratham to evaluate the TaRL approach for scale. This process began with early proof of concept evaluations that showed the effectiveness of TaRL and continued with subsequent iterations to understand the effectiveness of different delivery models when implemented by village volunteers, Pratham instructors, and government school-teachers.
Six randomized evaluations in seven states of India show that the TaRL approach is consistently effective when implemented systematically and has led to some of the largest effect sizes rigorously measured in the education literature. For example, the TaRL Learning Camps in Uttar Pradesh doubled the number of children who could read a paragraph or story. The long-term partnership between J-PAL and Pratham demonstrates how findings from an evaluation can provide important inputs for continuously evolving the program which in turn is rigorously evaluated.
One key learning from decades of research is that TaRL works best as a holistic approach that reorients education systems towards focusing on learning outcomes especially for foundational learning. The series of evaluations in India have shown that simply training teachers in the approach or providing the teaching materials alone does not improve learning outcomes. However, when teachers were guided by clear goals, helped to understand data on children’s learning, supported by strong mentors who provided ongoing on-site help, and brought together to share learnings and challenges, learning outcomes improved. Moreover, the research suggests the importance of governments collecting data and using these data in regular review meetings.
Pratham’s TaRL approach has inspired a broader set of programs that use tailored instruction to promote learning. A growing body of evidence suggests that these types of programs that align instruction to students’ current learning levels are among the most effective and cost-effective ways to improve learning. For more information on this research, please see J-PAL’s policy insight on tailored instruction.
From Research to Action
Pratham’s TaRL approach has scaled throughout India and Africa to reach over 60 million students.
While several other delivery models had been tried in the past, starting in 2012, Pratham began to focus on two implementation models:
- Learning camp model: Pratham instructors work directly with children in "Learning Camps." Learning Camps are intensive periods of instructional activity that usually last ten days. Children (generally in grades 3 to 5) are re-grouped according to learning level rather than age or grade for two to three hours per day. Three to five camps are done through the year for a total of 30 to 50 instructional days, often with a gap of roughly ten days between each camp. When they are not in a Learning Camp, children return to their regular grade classes. Learning Camps are carried out during the school day with the permission of local authorities.
- Government partnership model: Government teachers are trained and supported to implement TaRL in their schools. In these models, teachers re-group children in grades 3 to 5 based on learning level for one or two hours per day to focus on basic skills. Usually the program is led by mentors or “leaders of practice” who are part of the government system but have carried out “practice classes” to implement and experience the TaRL approach first-hand. The leaders of practice then train teachers and also provide ongoing, onsite support. Drawing on learning from randomized evaluations in Haryana and Bihar, Pratham helps ensure that teachers receive strong ongoing mentorship support and that monitoring and review systems are integrated into existing educational systems.
Pratham’s efforts to implement these two models at scale, strategically support governments aiming to improve learning, and continuously incorporate learnings from rigorous research have led to the implementation of TaRL programs in many states in India.
Drawing on the success of the TaRL approach in India, governments and non-governmental organizations in Sub-Saharan Africa are adapting and implementing TaRL programs in several countries with technical support from Pratham, J-PAL, and other partners. Since piloting the TaRL approach in 2016, for instance, the Ministry of General Education (MoGE) in Zambia has expanded its Catch Up program to 1,900 schools in three provinces. After the program was first piloted in Nigerian schools in late 2019, TaRL Africa pivoted to supporting home-based learning when the Covid-19 pandemic resulted in school closures from April 2020 to January 2021. Since schools reopened, TaRL has been scaled to more than 800 schools across five states. In Côte d’Ivoire, TaRL Africa is supporting the Ministry of National Education and Literacy (MENA) to deliver and grow the Programme d’Enseignement Ciblé (Program of Targeted Instruction, PEC). Other organizations are working to support governments to scale TaRL-inspired programs in countries across the African continent, including in Botswana, Ghana, Kenya, Madagascar, Niger, Sierra Leone, South Africa, Tanzania, and Uganda.
In 2021, TaRL programming reached over one million children in 12 African countries, and TaRL Africa, which began as a joint venture between J-PAL and Pratham, registered as a Kenyan organization. For more information on TaRL in Sub-Saharan Africa, please visit the TaRL Africa website.
Recent years have seen growing awareness of a global learning crisis–reflected in the transition from the Millennium Development Goals, which focused on school enrollment, to the Sustainable Development Goals, which recognize that this increase in enrollment has not translated into an increase in learning.4 TaRL, along with the broader body of work it helped to catalyze on targeted instruction, has gained global recognition as an effective strategy to improve learning. As policymakers search for solutions, a number of education interventions–including customized learning technology in India, preschool pedagogy in Peru, low-tech interventions targeted at parents during Covid-19 school closures in Botswana, and tutoring programs in the United States–cite the importance of assessing learning levels and targeting instruction to learners.
A number of influential organizations now recognize TaRL as an effective approach to teach foundational skills and address learning loss. Citing TaRL evidence, an independent panel convened by the World Bank, FCDO, and the donor group Building Evidence in Education identified targeted instruction as an intervention for which there is good evidence of cost-effectiveness. The FLN (Foundational Literacy and Numeracy) Hub– a resource hub with guidance on improving children’s foundational literacy and numeracy skills, developed by J-PAL, Unicef, Pratham, and Delivery Associates in collaboration with the World Bank and the Global Partnership for Education–has a standalone section on TaRL. Unicef, drawing on its experience supporting Zambia to implement TaRL, identified Pratham as a key partner for inclusion in the Hub. The World Bank’s 2018 World Development Report cites targeted instruction as a way to support learners from falling behind in school on basic skills.
The TaRL approach has been particularly influential as education policymakers around the world have endeavored to address learning losses resulting from school closures during the Covid-19 pandemic. Researchers at McKinsey estimate that children lost eight months of learning,5 and the proportion of ten-year-olds in low and middle-income countries who could not read a simple story rose from 57 to 70 percent during the pandemic, according to the World Bank.6 In response, the Global Education Evidence Advisory Panel recommended that schools target instruction to students’ learning levels to help them catch up. The World Bank, UNICEF, FCDO, USAID, and the Bill & Melinda Gates Foundation issued a report calling teaching at the right level “the single most crucial intervention for reversing the decline in learning progress.”
This case study was published in September 2019 and updated in August 2022 to include more information on how the TaRL approach has been adapted and scaled in Africa, and to highlight how this body of research has shifted global thinking on addressing learning loss.
References
Banerjee, Abhijit V., Shawn Cole, Esther Duflo, and Leigh Linden. 2007. "Remedying education: Evidence from two randomized experiments in India." The Quarterly Journal of Economics 122 (3): 1235–1264. https://doi.org/10.1162/qjec.122.3.1235.
Banerjee, Abhijit V., Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2010. “Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India.” American Economic Journal: Economic Policy 2 (1): 1–30. http://www.aeaweb.org/articles.php?doi=10.1257/pol.2.1.1
Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukherji, Marc Shotland, and Michael Walton. 2016. “Mainstreaming an effective intervention: Evidence from randomized evaluations of ‘Teaching at the Right Level’ in India.” NBER Working Paper No. 22746. https://www.nber.org/papers/w22746.
Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukerji, Marc Shotland, and Michael Walton. 2017. "From proof of concept to scalable policies: challenges and solutions, with an application." Journal of Economic Perspectives 31 (4): 73–102. https://doi.org/10.1257/jep.31.4.73
Banerji, Rukmini, and Madhav Chavan. 2016. "Improving literacy and math instruction at scale in India’s primary schools: The case of Pratham’s Read India program." Journal of educational change 17 (4): 453-475.
Banerji Rukmini. 2019. "Banerjee and Duflo's Journey with Pratham." Ideas for India.
Banerjee, Abhijit, Rukmini Banerji, James Berry, Esther Duflo, Harini Kannan, Shobhini Mukherji, Marc Shotland, and Michael Walton. 2016. “Mainstreaming an effective intervention: Evidence from randomized evaluations of “Teaching at the Right Level” in India.” National Bureau of Economic Research Working Paper No. w22746, October 2016.
ASER Centre. 2019. “Annual Status of Education Report (Rural) 2018.” New Delhi: ASER Centre.
Uwezo. 2016. "Are Our Children Learning? Uwezo Uganda 6th Learning Assessment Report." Kampala: Twaweza East Africa.
World Bank. 2018. “World Development Report 2018.” https://www.worldbank.org/en/publication/wdr2018; Bhula, Radhika and John Floretta. 2020. “A Better Education for All During–and After–the COVID-19 Pandemic.” Stanford Social Innovation Review. October 16, 2020.
“Covid learning loss has been a disaster.” The Economist. July 7, 2022. https://www.economist.com/international/2022/07/07/covid-learning-loss-has-been-a-global-disaster
“The State of Global Learning Poverty: 2022 Update.” World Bank, UNESCO, UNICEF, USAID, FCDO, Bill & Melinda Gates Foundation. June 2022. https://www.unicef.org/reports/state-global-learning-poverty-2022
Over the past decade, many developing countries have expanded primary school access, but improvements in school access and enrollment may not always translate into improved learning outcomes for all students if the quality of education is poor. Researchers evaluated the impact of the Balsakhi Program, a remedial tutoring education intervention implemented in schools in Vadodara and Mumbai, India, on student learning. The program significantly improved student test scores in both locations.
Policy issue
Over the past decade many developing countries have expanded primary school access, energized by initiatives such as the United Nations Millennium Development Goals, which call for achieving universal primary education by 2015. However, improvements in school access and enrollment may not always translate into improved learning outcomes for all students if the quality of education is poor. Current research has identified several cost-effective ways to increase student attendance, but much less is known about how to improve education quality and student learning in a cost-effective way. Many schools rely on rote learning and memorization, but can lessons which are more tailored to children' s learning level improve achievement? How important is a pedagogical approach which adapts to the level of the child?
Context of the evaluation
A 2005 survey found that 44 percent of Indian children age 7 to 12 could not read a basic paragraph and 50 percent could not do simple subtraction even though most were enrolled in school. Even in urban India, the learning levels are very low in Vadodara, a major city in the Indian State of Gujarat, only 19.5 percent of the students enrolled in grade 3 could correctly answer questions testing grade 1 math competencies. Ironically, the difficulty in improving the quality of education may be complicated by success in getting more children to attend school, as in many cases neither the pedagogy nor the curriculum has been adapted to take into account the quantity and characteristics of the influx of new children.
Details of the intervention
In conjunction with education-oriented NGO, Pratham, researchers evaluated the Balsakhi Program, a remedial education intervention implemented in 122 public primary schools in Vadodara and 77 schools in Mumbai. A tutor (balsakhi), usually a young woman recruited from the local community and paid a fraction of the cost of civil-service teachers (US$10-15 per month), worked with children in grades 2, 3 and 4 who were identified as falling behind their peers. The instructor typically met with a group of approximately 15-20 of these children who were taken out of the regular classroom into a separate class for two hours of the four-hour school day each day. Instruction focused on the core competencies the children should have learned in the first and second grades, primarily basic numeracy and literacy skills. The instructors were provided with two weeks of initial training and a standardized curriculum that was developed by Pratham.
In the 2001 school year in Vadodara, approximately half of the schools were given a tutor for grade 3, and the other half were given a tutor for grade 4, while in Mumbai during that same year, approximately half of the schools received a tutor for grade 3, and the other half received a tutor for grade 2in both cities, which school received which tutor was randomized. In 2002, the schools were given a tutor for the previously untreated grade. In determining program impact, grade 3 students in schools that only received a tutor for grade 4 were compared to grade 3 students in schools that had tutors for grade 3, and so on. Academic achievement was measured through two annual tests, administered at the start and end of the academic term.
Results and policy lessons
Impact on Education: The program had substantial positive impacts on children' s academic achievement. In both Vadodara and Mumbai, the Balsakhi program significantly improved overall test scores; by 0.14 standard deviations in the first year and 0.28 standard deviations in the second year, with the largest gains in math. Moreover, the weakest students, who were the primary target of the program, gained the most. Researchers estimate that the entire effect of the program was due to a very large (0.6 standard deviations) improvement in average test scores among the children who were sent for remedial education. In contrast, there was no measurable impact for their classroom peers, who did not receive remedial tutoring, but were "treated" with smaller class sizes and a more homogenous classroom.
Balsakhi Turnover: There was rapid turnover among the Balsakhi tutors, with each tutor staying on average for just one year, typically until they got married or got another job. Despite the high turnover among tutors, the program still resulted in significant gains in student learning, which suggests that the success of the program did not depend on a handful of very determined and enthusiastic individuals.
Cost-Effectiveness: The Balsakhi program was very inexpensive, since the main cost of the program was the tutors' relatively small salaries. Overall, the Balsakhi program cost approximately US$2.25 per child per year, significantly less than the cost per child of a Computer Assisted Learning program, evaluated by Pratham at the same time. In terms of cost per improvement in test scores, researchers estimate an attractive cost-effectiveness of about US$0.67 per standard deviation increase in test scores.1 The Balsakhi program has since been adapted, re-evaluated, and scaled up across India.
In J-PAL's comparative cost-effectiveness analyses, the Balsakhi program led to a 3.07 standard deviation improvement in test scores and did not have an impact on years of education per $100 spent. Reducing class sizes did not have an impact on test scores. For more information, see the comparative cost-effectiveness analyses.
[!1] Please note that the associated paper for "Remedial Education, India 19" has a typesetting error on page 1263. The cost-effectiveness calculation indicates $0.67 per 0.1 standard deviations, not $0.67 per standard deviation.