Researcher Spotlight: Shobhini Mukerji on her journey into development economics

This podcast and blog series brings together researchers reflecting on their experiences conducting impact evaluations in India. From working in diverse local contexts to engaging with government and civil society, each episode offers a window into the realities of producing policy-relevant evidence on the ground. 

In this episode of The Evidence Effect, host Sambhav Choudhury speaks with Shobhini Mukerji, Executive Director of J-PAL South Asia. From her transformative summer internship with education nonprofit Pratham to becoming a principal investigator on groundbreaking research, Shobhini shares her journey into development economics and experimental approaches.
 

[Transcript]

Sambhav: Hi everyone, welcome to an episode of J-PAL South Asia's Researcher Spotlight conversation series. My name is Sambhav, and I'm a Research Associate at J-PAL South Asia. 
Today I'm talking to Shobhini Mukerji, who is the Executive Director of J-PAL South Asia. Shobhini oversees the pan-India operations of J-PAL South Asia, which has partnerships with over 16 state governments and several central government agencies. 

Shobhini works closely with donors, policy makers, civil society partners, and research institutions to promote collaboration, to increase the use of evidence in decision making, and scale-ups of successful social programs in India. She wears many hats: fearless leader of a cutting-edge organization, education expert, lover of J-PAL staff parties, and a bonafide foodie. But in this conversation, we wanted to focus on one of your incredible roles: being a principal investigator on some of the most exciting education projects at J-PAL South Asia. Hi Shobhini, thanks for talking to us today.

Shobhini Mukerji: Thanks Sambhav, thanks for having me here, and look forward to the conversation.

Sambhav: So we would love to begin by asking you, what got you interested in development economics and the experimental approach?

Shobhini Mukerji: Back in 2001, when I was preparing to do an MBA in a college in India, I was studying for the entrance exams and I took up a summer internship with Pratham, one of the largest education nonprofits in India, which works with underprivileged children to build their foundational literacy and numeracy. And that changed the course of my life. It was my introduction to development and to the space of education more specifically.

So I've always been a numbers person and also quite computer savvy, and we're talking about the late 1990s when we used to have very clunky software and the dial-up BSNL internet was slowly making its way to India. And my father taught me my computer skills. He got me a Windows Dummy series, this black and yellow book through which I taught myself Excel, Access, and Word. And Access was the main programming software at that point of time.

And Pratham hired me for the summer of 2001 to set up the databases in Access for tracking out-of-school children in their bridge classes in the Delhi government schools. So I ended up working in Pratham for almost 5 years, from the urban slums in Delhi to the villages of Uttar Pradesh, Bihar, Rajasthan. And my main work was conducting surveys, doing pre-post assessments, and you know what we call dipstick surveys to test reading and math levels of children in government schools and in Pratham's community bridge classes. And I had a lot of conversations with parents, with teachers, with children on what education and learning meant to them.

I was also part of the first core team that launched the ASER surveys in 2005, which is now a seminal survey that happens annually across India to reach more than 3 million children to gauge their learning levels. And I remember at that time, around that time, when I was first introduced to the experimental approach—RCTs as we call them—through researchers like Esther Duflo, Abhijit Banerjee, and Rachel Glennerster, who were working with Pratham at that time with Rukmini Banerji, who's now the CEO of Pratham, to study the impact of community involvement and participation of local elected leaders in taking steps to improve learning in the village schools. And I was intrigued, to say the least, with the whole treatment and control approach, which was quite new to me.

Soon after this experience, I left to study MSc in Social Research Methods at LSE with a major in social and development policy. And it was here that I was properly introduced to RCTs as a methodology to study causal impact. And I was hired by Esther after I completed my MSc to join her and Rema Hanna and Michael Greenstone as their research associate in Orissa in 2007 to study the impact of smokeless stoves on the health of women and children through an RCT. And we were working with Gram Vikas at that point of time.
And I was quite fascinated by the precision at the ground level that it took to conduct an RCT, from the survey design to the questionnaire testing, piloting, and most importantly, the rigor with which it took to understand attrition, spillovers, contamination. And what struck me most about the approach of RCTs is that we can take seemingly large rhetorical questions like "How do we end global poverty?" "How do we improve the health of people?" and break it down into a number of smaller but more precise questions at an individual or a group level. And then we answered each of these questions through a specifically designed experiment that tells you whether the solution works or not, and it assigns causality, which is quite neat. And we've applied this to many fields, from learning to health to political participation to using nudges and incentives to improve service delivery and take-up, and so on.

Sambhav: Thanks Shobhini, that was quite an inspiring journey. So some of our listeners are budding economists and are contemplating whether or not to consider the PhD track. Your career as a PI is very inspiring. Could you please tell us how you perceive the value of PhD training in being a good researcher?

Shobhini Mukerji: I will confess that I don't have a PhD. After coming back from LSE, you know, each year I would go back to Rukmini and Esther, who have been my mentors, you know, whether I should do a PhD or not. And they would ask me, "Why? What is your burning question? What is your motivation?" And we would always conclude at the end of that conversation that my work between Pratham and J-PAL was akin to doing many PhDs.

And no doubt that PhD gives a very strong theoretical base. I've come to believe that in the development space, fieldwork is invaluable in giving a good practical grounding, and shadowing professors and working closely with them can also give you a strong theoretical grounding. So I'm not saying don't do a PhD, but commit if you're very certain and think deeply about the problem statement that you're trying to solve for.
I got lucky because I had an opportunity to work with Esther, Jim Berry, Mark Shotland as a PI on a project in Haryana on an evaluation. It's a learning evaluation actually, which laid the foundation for the teacher-led model of Teaching at the Right Level. And I learned so much from that experience that the PhD could not have taught me. And also, many of the projects that I worked on, they did not have the impacts that I hoped for, and there were some important lessons in there on the failures. Again, a PhD won't teach you that.

So I'll stop here and say that commit to a PhD if you're certain, have a problem statement you're trying to solve for. If you're clear that you want to be an academic, definitely do a PhD. If you want to join the World Bank or have a doctor tag which looks good on your CV, then that is your motivation. Recognize that. But it's a high price to pay, and PhD is a lonely road. It's an individual journey.

Sambhav: Thanks Shobhini, this is actually solid advice for me and my peers who are on the fence about doing a PhD right now. So the next question that we had for you was relating to your experience working with state governments and government agencies. What key factors or strategies have proven effective in forming partnerships with governments or other stakeholders for your project?

Shobhini Mukerji: Now, India is fortunate to have some of the brightest and most thoughtful policymakers in the world, both inside and outside the government. And our ability, in a large part, to carry out these large-scale experiments of the policy interventions relies on their willingness to hear us out, their willingness to give us a chance to demonstrate the impacts of the program through the experimental approach.
And the more that we have been working with policymakers, I think there are few things that we've learned along the way. I personally like to call it the snakes and ladders of evidence-informed policymaking. And perhaps the biggest ladder for all of us is finding the right champion. So when you find the right champions, that can take you a very long way.

And in fact, one of our largest, most extensive partnerships with the government has been with the Tamil Nadu government under the guidance of Mr. Krishnan, then the Principal Secretary in Planning in Tamil Nadu, along with Iqbal Dhaliwal, our Executive Director. And what we did at that point of time when we launched the partnership in 2013—it began with months-long policy dialogue that brought together a dozen professors, researchers from JAPAL, secretaries from various governments across different departments, and they discussed these key priorities of the state and the latest scientific evidence we have from around the world on these issues, which led to the design of very innovative pilots that address several challenges in the state.

Which brings me to the second most important ladder, as I like to call it, is listening and being responsive to their priorities and being connected at all times. We've done this through policy dialogues, and every engagement between a government and organization like J-PAL, you know, it helps both sides better understand how policymakers and researchers can partner to design, evaluate, and scale up these innovative solutions based upon the problem statements at hand.

And in the last decade, we've learned how hard it can be to align with the needs and constraints of policymakers and researchers, but yes, once you do align, then how beneficial such partnerships can be. So the alignment is the most important here, and it takes time and relationship building, trust to get the senior decision-makers to commit their time to our approach.

And then the last thing I'll say in terms of an insight that we've learned is, you know, putting down the essential nuts and bolts for building a successful bridge between researchers and policymakers through an institutional setup and a system of collaboration. So beyond the individual project or collaboration that we have is a larger commitment that gets recorded through an MOU, a steering committee, or an advisory committee that meets and debates and approves each project. And it has participation of senior officials from the Chief Secretary to the Principal Secretary of the nodal departments, department secretaries that we're engaging with, and the senior leadership from our end, including bringing in our researchers at the table as well.

Because one thing that—one of the biggest snakes that we've seen that pulls you down is transfers, and we're in it for the long game, frankly. So it's important to be able to stick it out and play that long game using some of these strategies.

Sambhav: This is quite interesting, Shobhini. So the next question that we had was about your thoughts on the broader perspective. So how do you see your work improving the lives of people and affecting policy decisions?

Shobhini Mukerji: J-PAL's mission is to inform policies based on evidence and in the most rigorous way. But rubber really meets the road when you take the evidence and you turn it into a program that has the potential to reach millions of people through a scale-up. And at JAPAL, we have devoted considerable resources over the past two decades to the science part of scaling.
So we've had several successes, now, whether it's sustainable livelihoods to break the poverty trap for a million women in Bihar, or whether it's improving foundational learning for over 60 million children in India or Africa. So today, the research that we've done has positively impacted over 600 million lives globally and 300 million in India alone.

So taking a step back, while these programs individually scale and improve millions of lives across continents, it is still not enough given the magnitude of the challenges that we face and will continue to face into the future. So there will be new challenges, and the solutions will be different—some new, some old—and we will need to have more rigorous evidence to inform these solutions.
So for our work to be truly impactful, J-PAL will have to find a way to have many seats on many tables. We have a lot to say, so I hope we can change that and do more on the scaling and policy influence equation in the future.

Sambhav: Thank you, Shobhini. Lastly, could you describe a memorable experience from your fieldwork that significantly shaped your research or perspective?

Shobhini Mukerji: Well, it's a lovely question, Sambhav, and there's not one particular experience that comes to mind. Each time I'm in the field, I come away with so many memorable conversations, meetings, insights from the field, and each of these has shaped my perspective. And of course, we input that into research.
So whether it is my time in Orissa chopping wood, rescuing frogs out of abandoned mud stoves, or conducting arm sambas on stray dogs in villages in Uttar Pradesh... So there was one time when I had gone on a surprise field visit in Madhya Pradesh, and we traveled all day to reach a remote village, and the school was almost getting over by the time I reached. And I won't forget what I saw: one teacher and one child in the school, deeply engrossed in teaching and learning. That is it.

I learned later on that the village had just one school-age child, and it was a single-teacher government primary school with just one child studying. And the teacher came every day to teach the girl, and she went every day to learn. So such experiences give you hope that when everyone around you tells you the teachers are not motivated, schools don't work for the poor, then you can say that you've seen otherwise.
But I've also had several uncomfortable experiences, frankly. I've been asked in different states, in different villages, I've been asked my caste and why I'm not married—and I was single at that point of time. And now that I am married and I have a child, someone also asked me in the middle of an interview as to why I stopped at one child if I have a girl.

So each experience gives you a small window into the lives of people, into how they think, their aspirations, their motivations, their struggles. And it also collects somewhere in the corner of your brain, and you can pull it out much like an instinct when you're working on the project or when I'm trying to convince a policymaker to adopt a seemingly good program or pitching to a donor to invest in.

Sambhav: Thanks, Shobhini. The story with the teacher and the child in Madhya Pradesh was quite wholesome. And like as a young field researcher myself, I also am regularly in these situations where I get asked questions that I do not expect. So yeah, looking forward to more such experiences. Thank you so much for this conversation. I enjoyed this a lot.

Shobhini Mukerji: You should keep writing your experiences. You'll remember them later.

Sambhav: Thank you, Shobhini. I'll definitely do.