Do Students Benefit from Blended Instruction?

Fieldwork by:
Location:
Haryana, India
Sample:
240 public secondary schools; 2 cohorts of students in grades 9 and 10
Timeline:
2018 - 2022
Target group:
  • Secondary schools
  • Students
  • Teachers
Outcome of interest:
  • Student learning
  • Service provider performance
  • Take-up of program/social service/healthy behavior
Intervention type:
  • Digital and mobile
  • Training
  • Community monitoring
  • School-based inputs
AEA RCT registration number:
AEARCTR-0003665
Notes:
This evaluation was started while Andreas de Barros was a predoctoral fellow at J-PAL South Asia
Partners:

High-quality teaching is an important determinant of student learning, but limited evidence exists on how to increase instructional quality in low- and middle-income countries. In partnership with Avanti Fellows, this study evaluates the impact of the Sankalp program, which provides teachers with resources and training to blend their instruction with video-based learning materials, on student’s math and science test scores.

Policy issue

Despite great improvements in student enrollment and attendance over the past two decades, student learning levels remain low in many low-income countries. As suggested by a growing body of evidence from low- and middle-income countries,1 ,2,3,4 high-quality teaching is a key determinant of student learning. However, teachers in many low-income countries may lack the necessary skills to teach effectively and often use teaching methods ill-matched to their students’ needs. For example, a study across six African countries found that less than ten percent of surveyed teachers had a minimum proficiency in language skills.5 While there is some evidence from high-income countries, little evidence exists on how to improve instructional quality in low-income countries. Can equipping classrooms with education technology and training teachers to enrich their instruction with video-based learning materials improve teaching practices and student learning?

Context of the evaluation

A partner for this evaluation, Avanti Fellows, is a large NGO in India that aims to provide affordable, high-quality education to disadvantaged students aspiring to study at India’s top universities. One of Avanti Fellows’ programs, named Sankalp, works to provide the best quality math and science education by integrating information and communication technology (ICT) into the curriculum, in close collaboration with the government. 

In Haryana, where this evaluation takes place, students’ mastery of math and science skills remains a challenge. According to the 2017 National Achievement Survey, less than half of grade 8 students mastered grade-level mathematics and science (37 and 42 percent respectively). These scores were slightly below the national average of 42 and 45 percent. At the same time, there appears to be no gender gap in learning outcomes for grade 8 students, as girls slightly outperformed their male peers in math and performed equally in science.

Details of the intervention

In partnership with Avanti Fellows, the study evaluates the impact of the Sankalp program, which provides teachers with resources and training to use video-based learning materials, on student’s math and science test scores. The evaluation hypothesizes that the program will increase student learning by improving the instructional quality students receive. They are also interested in understanding if the full Sankalp program is more cost-effective than a similar, but “light-touch” model that removes the technology component. Additional cost-effectiveness analyses assess the potential for the program to scale.

The two-year program began in the 2019-2020 academic year with grade 9 and grade 10 students, with another two-year cohort beginning in the following academic year. From a sample of 240 public senior secondary schools, researchers are randomly assigning schools to one of three groups: 

  1. Full Sankalp program: Schools receive two “smart” classrooms equipped with information and communication (ICT) infrastructure. Additionally, schools receive digital content to supplement instruction, printed practice workbooks for students, and a training program for teachers to improve their teaching in math and science.

  2. Workbooks only: Schools receive printed workbooks for students and a training program for teachers to improve their teaching in math and science. This program is similar to Sankalp but scaled down by removing the technology components.

  3. Comparison group: Schools receive no additional facilities, materials, or training programs. 

To measure the impact on student learning, the study relies on paper-based tests in math and science (a baseline assessment and yearly follow-up assessments). The study also uses monthly classroom observations to understand how the program affects teaching behaviors and instructional quality.  

Results and policy lessons

Ongoing study; results forthcoming

1.
Buhl-Wiggers, Julie, Jason Kerwin, Jeffrey Smith, and Rebecca Thornton. “The Impact of Teacher Effectiveness on Student Learning in Africa.” Working Paper, April 2017.
2.
Araujo, M. Caridad, Pedro Carneiro, Yyannú Cruz-Aguayo, and Norbert Schady. 2016. “Teacher Quality and Learning Outcomes in Kindergarten.” The Quarterly Journal of Economics 131, 1415-1453, https://doi.org/10.1093/qje/qjw016.
3.
Bau, Natalie and Jishnu Das. Forthcoming. “Teacher Value-Added in a Low-Income Country.” American Economic Journal: Economic Policy.
4.
Azam, Mehtabul and Geeta Kingdon. 2015. “Assessing Teacher Quality in India.” Journal of Development Economics 117, 74-83. https://doi.org/10.1016/j.jdeveco.2015.07.001.
5.
Bold, Tessa, Deon Filmer, Gayle Martin, Ezequiel Molina, Christophe Rockmore, Brian Stacy, Jakob Svensson, and Waly Wane. “What Do Teachers Know and Do? Does It Matter? Evidence from Primary Schools in Africa.” Working Paper, January 2017.