How to Teach English in India: Testing the Relative Productivity of Instruction Methods within the Pratham English Language Education Program

Researchers:
Fang He
Margaret McLeod
Fieldwork by:
Location:
Mumbai, India
Sample:
5,317 students in Thane; 9,745 students in Mangaon, Raigad
Timeline:
2005 - 2008
Target group:
  • Students
Outcome of interest:
  • Student learning
Intervention type:
  • Recruitment and hiring
  • Computer-assisted learning
  • Technology
Partners:

In many low and middle-income countries, the official language of government and business is either French or English rather than the native language spoken by the population at large. While the curricula of many countries do indeed require students to learn official languages, the quality of instruction may be particularly poor. Researchers partnered with the Indian NGO Pratham to evaluate whether different technologies and implementation methods can increase students’ English test scores. Overall, the interventions were effective at increasing students’ knowledge of English.

Policy issue

In many low- and middle-income countries, the official language of government and business is either French or English rather than the native language spoken by the population at large. The returns to knowing the official language can be quite high, and proper instruction is of importance. While the curricula of many countries do indeed require students to learn such languages, the quality of instruction may be particularly poor. This study evaluates a project that focuses on improving students’ acquisition of this crucial second language. In this regard, this research fits squarely within the branch of education which does not solely seek to understand how to get children into school, but also what it takes to cost-effectively improve school quality. Many schools rely on rote learning and memorization, but it may be more effective to take an interactive approach which is tailored to a child’s level of learning. How do different types of technologies and implementation methods affect students’ learning outcomes?

Context of the evaluation

Despite the fact that English is part of the normal curriculum, only 10 percent of second and third grade students in this study were able to correctly identify pictures of simple objects when given the object’s English name. In government schools, teachers generally train children to recite the English alphabet and memorize vocabulary lists, yet conversational skills are rarely introduced. Many teachers skip reading the source text altogether, and teach only in preparation for exams. Therefore, the Indian public primary school system often fails to teach a language that is of crucial importance to a child’s subsequent employment opportunities, and in addition foregoes the “critical period” in which a child is able to acquire a second language up to native-like levels.

J-PAL staff help a young girl solve a maze exercise in India.
J-PAL staff help a young girl solve a maze exercise in India.

Details of the intervention

Researchers evaluated the effectiveness of a unique English language training program developed by the Indian NGO Pratham on increasing students’ English test scores. The program, called PicTalk, has two components: first, an electronic machine called the PicTalk is designed to be used individually by the student; the second is a set of specially tailored flashcards and teaching manuals designed to promote oral communication with the help of a teacher. Both methods aim to teach equivalent curricula to its students, yet through the use of differential teaching methods.

To evaluate the effectiveness of this program, two randomized evaluations were conducted with over 15,000 children in grades 1-5 in both rural and urban areas of India. The research design varied both the implementation technology and whether the intervention was delivered through externally hired tutors or the public schools’ own teachers and assistants.

Externally Hired Tutors: In the first study year, 97 schools in Thane were assigned to one of two research groups: (1) PicTalk class in second grade but not in third, or (2) PicTalk class in third grade but not in second. Therefore, every school served as both a treatment and a comparison group. English tutors were hired and trained by Pratham, and attended the schools daily to implement the program.

Teacher Implementation: In the second year, 242 schools in Mangaon were assigned to one of four research groups: (1) PicTalk machine classes only, (2) activities classes only, (3) both PicTalk machine and activities classes (as in the first year), or (4) neither PicTalk machine nor activities classes. All classes were taught by normal classroom teachers.

Results and policy lessons

Effects on Learning: On average, the machines and the student activities were similarly effective at improving students’ English scores. The externally implemented program increased students’ scores by 0.26 standard deviations while students subjected to the teacher implemented interventions improved their test scores by 0.36 standard deviations. Thus, each implementation method seemed to be similarly effective, regardless of the technology used by the instructor or whether the instructor was a local teacher or from an external agency. When implemented by local teachers and teaching assistants, however, the interventions increased not only English scores, but also math scores. This seemed to be due to the discretion teachers had over the intensity with which the program was implemented: when the program made teaching English more efficient, teachers were able to spend more time on other subjects. The results demonstrated improvements in math scores of similar magnitude as the improvements in English for those students in the teacher implemented interventions.

Differences resulting from technological choice: In comparing the flashcard-/teacher-based intervention versus the self-paced computerized implementation, the study found interesting differences between the benefits to subgroups from such interventions. Specifically, lower performing students benefited more from interventions that included teacher implemented activities, while higher performing students gained more from the relatively self-paced machine-only intervention. This is notable, since the norm in Indian classes is to focus on the better performing students. These results also suggest that significant gains could be achieved by targeting different approaches to individual children.

He, Fang, Leigh L. Linden, Margaret MacLeod. "How to Teach English in India: Testing the Relative Productivity of Instruction Methods within the Pratham English Language Education Program." Working Paper, Columbia University, July 1, 2008.