Researcher Spotlight: Gautam Rao on co-producing evidence with policymakers

This podcast and blog series brings together researchers reflecting on their experiences conducting impact evaluations in India. From working in diverse local contexts to engaging with government and civil society, each episode offers a window into the realities of producing policy-relevant evidence on the ground. 

In this episode of the Researcher Spotlight Conversation Series, host Sambhav Choudhury speaks with J-PAL affiliated professor Sandip Sukhtankar, Professor of Economics at the University of Virginia. He discusses the unique challenges and rewards of conducting research with law enforcement agencies, offers practical strategies for maintaining government partnerships despite personnel changes, and explores the nuanced role of technology in social protection programs.

Want to learn more? Explore other podcast episodes in this series where we speak with Shobhini Mukerji on her journey into development economics, Ariel Zucker on applying economic theory to real-world solutions, Harini Kannan on scaling evidence-informed programs with governments, and Gautam Rao on co-producing evidence with policymakers.

[Transcript]

Sambhav Choudhury: Hi everyone, welcome to an episode of J-PAL South Asia's Researcher Spotlight conversation series. My name is Sambhav Choudhury, and I am a research associate at J-PAL South Asia. Today I'm talking to Sandip Sukhtankar, who is a professor in the Economics Department at the University of Virginia. His research interests are in development economics, political economy, and public economics, with a particular focus on corruption, governance, and the delivery of public benefits and services. Sandip's non-academic interests are as exciting as his research. He's an avid cricket fan since childhood and is now also a practicing ceramicist. Research associates that work with him are regularly in awe of how Zen he is. Hi Sandip, thanks for talking to us today.  

Sandip Sukhtankar: Hi Sambhav, thanks for having me.

Sambhav: The first question that we have for you is relating to your paper "Increasing Access to Security and Justice Through Women's Help Desks in Police Stations in India," in which you study the impact of women help desks on registration rates of cases of gender-based violence. Can you briefly tell us how it was working on this project in India and some of the advantages and challenges of running an intervention in this context?

Sandip: Sure. This has been a very, very interesting project for me. It's the first project in which I've worked with the police, and I knew very little about the police before I started working with them. So, for example, the fact that police in India generally don't get any leave—like during the week, Sunday is not off, holidays that other people get are not off. In fact, that means double time for them since they're working policing festivals or whatever else there is.

So it's a very interesting context, and I'm really glad that it worked out. And the way it worked out was also super interesting because it was not us reaching out to them, but it was the Madhya Pradesh Police Department reaching out to Iqbal, and then J-PAL and Iqbal and Abhijit bringing me on to meet with them.

It was super interesting to work in this context. I think it worked because of the stakeholder interest. Otherwise, I think it would be very challenging because even after we had very strong support within the police, there were also other factions who were not as keen on working with researchers and not as keen on sharing data or being open to running an intervention that might fail and then have us write a paper that suggests that there was something that didn't work.

Now in this case, some parts of the intervention did work, and the paper got published very nicely, and they got some nice publicity out of it too. So I think it all worked out, but there's lots of advantages and challenges of running an experiment with a sector like the police, which is very understudied and super, super interesting.

Sambhav: Thanks, Sandip. So often when we work on projects with governments, there are personnel changes within the department and new stakeholders that we have to work with for the same project. So in your experience, what key factors or strategies have proven effective in maintaining strong partnerships with government departments despite this barrier?

Sandip: Yes, this—as anyone who's worked with the government knows—this is one of the biggest challenges: the change in personnel. You'll have a secretary or joint secretary who's super enthusiastic about experiments or research and will be giving you all support, and you're just about ready to launch your project, and they get transferred.
So this is a situation that all of us have faced, and I think there's ways in which you can prepare, but I think we have to be open to the fact that this is just the arena in which we work, and if things don't work out, they won't work out.

But some of the things that we can do to avoid them—I think having written MoUs and a file in the government is always important. Government puts a lot of stock in paperwork, so an existing file that's been there on the project and sitting and signed off on somehow has some cache. I'm not saying it has all the cache, but I think it has some cache.

And then I think just building personal relationships with people who will be there for a longer term, maybe people who are either higher or lower in the hierarchy than, say, the secretary. So either way, you can use those connections and relationships to keep building, keep the continuity in the project going.
So I think those are probably the two things that are most effective, but nothing, of course, is foolproof.

Sambhav: You're also one of the researchers on the Payments and Governance Research Program that aims to understand the impact of direct benefits transfers models on public finances and the effectiveness of anti-poverty programs. In this context, could you throw some light on how technology has helped you in exploring these ideas?

Sandip: Yeah, so technology is very interesting. In fact, I just finished writing a handbook chapter on digital technology in social protection. And I think what I've learned, and what we as researchers who work in this area have learned, is that technology by itself doesn't really do anything good or bad, right? It's the protocols around its use, it's how it's being employed that really matters.

So, for example, take the case of using Aadhaar-based authentication to authenticate beneficiaries getting some kind of benefit. You could, on the one hand, it could reduce fraud and corruption. On the other hand, it could increase exclusion, right? The people who may not have Aadhaar or whose fingerprints don't get authenticated may stand to lose their benefits.

And so in this case, the success overall of the technology is based on what protocols do you have in place to, let's say, figure out, stop workarounds to and prevent corruption. What protocols do you have in place as backups for people who cannot authenticate? And these tend to be the most vulnerable, right? So how can you protect them?

So I think the big takeaway on technology is just that you need to be careful about designing the protocols, and it's how you use it that can lead to benefits or potential disadvantages.

Sambhav: Thanks, Sandip. So, given your extensive fieldwork experience in South Asia, what advice would you offer to emerging researchers or budding economists interested in pursuing a career in development economics?

Sandip: I think probably just be open to questions and be observant, and research questions can come from anywhere. You know, they're often not going to come to you from reading economics papers. They're probably going to come to you from interesting conversations that you have with random people, or something that you notice as a pattern when you're traveling.

So, in that sense, for me at least, I think there is no substitute to actually being in the field and talking to people and observing how programs are working or how they're not working.

So I feel, for example, I feel particularly energized—I just spent a few days in JPAL and in Madhya Pradesh doing sort of extending some of the research we've been doing as well as trying to start off new projects. So I ended up speaking to something like 300 newly recruited constables in the police force, and it was just really, really interesting to see their perspective and hear about their complaints as well as how they feel about working on the job, and then link that to what's going on with labor markets in India overall.

So yeah, I think something like this is fantastic for generating ideas.

Sambhav: Thanks, Sandip, that's quite fascinating. The final question that I had for you was: what research projects or areas are you currently excited about or planning to explore in the near future?

Sandip: Yeah, so that's a great segue since the project that I was trying to launch in Bhopal is something that I'm very excited about. It involves newly recruited constables in the Madhya Pradesh police, but from the angle of female labor force participation.

And these, as you might know, any government job is sort of super valued. So about a million people applied for 6,000 or so positions. But even after that, a fair number of them drop out for other lucrative government positions or other issues, especially in the cases of the women constables if they're posted far from home or don't have the support networks and things like that.
So some of the things that we're trying to do are looking at interventions that might help the police retain these women constables, help them improve job satisfaction, and of course, overall performance in the police.

So I think that's super interesting given the broad context of unemployment in India as well as female labor force participation in India. And I think there's lots of fascinating, just some fascinating descriptive work that we can do as well as the experimental work.

Sambhav: Yeah, that sounds quite interesting, and as someone who's interested in labor markets, I look forward to reading more about this project. Thank you, Sandip, that was quite interesting, and thank you for this conversation.

Sandip: Great, thanks.
 

This podcast and blog series brings together researchers reflecting on their experiences conducting impact evaluations in India. From working in diverse local contexts to engaging with government and civil society, each episode offers a window into the realities of producing policy-relevant evidence on the ground. 

In this episode of the Researcher Spotlight Series, host Sambhav Choudhury speaks with Ariel Zucker, assistant professor of economics at UC Santa Cruz. Ariel discusses her journey from an early interest in global health disparities to becoming a development economist specializing in experimental approaches. She shares insights from her research on health interventions for diabetes patients and groundwater conservation in India, highlighting how she balances theoretical frameworks with practical field realities to create meaningful, scalable solutions. 

Want to learn more? Explore other podcast episodes in this series where we speak with Sandip Sukhtankar on his research in India, Shobhini Mukerji on her journey into development economics, Harini Kannan on scaling evidence-informed programs with governments, and Gautam Rao on co-producing evidence with policymakers.

[Transcript]

Sambhav: Hi everyone, welcome to an episode of J-PAL South Asia's Researcher Spotlight conversation series. My name is Sambhav Chaudhury, and I'm a research associate at J-PAL South Asia.

Today, I'm in conversation with Ariel Zucker. Ariel is an assistant professor in the Department of Economics at UC Santa Cruz. Her research interests include improving health and environmental conditions for underserved communities, and she has experience working on projects that focus on countering behavioral biases in personal decision-making. 

Ariel is an outdoorsy person who loves running—in fact, she won 20,000 rupees when she participated in a half marathon in Coimbatore during one of her field visits.

Hi Ariel, thanks for talking to us today.

Ariel: Hi, great to be here!

Sambhav: We would love to begin by asking you—what got you interested in development economics and the experimental approach?

Ariel: Yeah, so I think I love that you asked the question in that order since my interest really came in the order of development, then economics, and then experiments.

I think I decided I wanted to work in development probably in high school. I was really lucky to travel a lot—even before high school and then during high school—to different places, and I met a bunch of people, a lot of them my age, who just had such a different life path. I got particularly interested in development and health at that age because I was meeting people who were my age—15 or 16—who already had HIV, elephantiasis, or TB. I was totally flabbergasted by how different the health outcomes were in different parts of the world.

So I decided to get into development, and especially investing in health in countries where health outcomes were not so great, at a really young age. Then when I got to college, I got excited about economics just because I loved my economics classes, and that motivated me to apply for a PhD in economics. It was really during the application process and then when I went to graduate school that I got excited about experiments.

I love experiments for a lot of reasons, but I think the main reason is that you get to design them yourself. You work with a lot of people—policymakers, NGOs, or other organizations—but you get to think about what you want to do in the future rather than just focusing on what has already happened in the past. I love this natural, forward-looking angle that experiments allow you to take, in contrast to a lot of other methods where you're kind of stuck with what's already been done.

Sambhav: J-PAL's mission is evidence-based policy and decision-making. In this context, could you shed some light on how you balance the theoretical aspects with field realities in South Asia?

Ariel: Yeah, sure. Most of my research tries to put policy first and theory second, just because that's what got me interested and excited about development economics. And second, because it's kind of my comparative advantage—I think some people are more theoretically minded than me.

In a sense, I'm balancing the field realities with the policy impact. Sometimes you're trying to evaluate a policy or a program, but it has a lot of infrastructure necessary to roll it out in the way that you think it should be implemented at scale. It's like you're trying to evaluate a model of the program, and you just make a sort of quick and dirty version.

For example, with my co-authors Rebecca Dizon-Ross and Shilpa Aggarwal, we were really excited about understanding how a policy that incentivized people to hit daily step targets would impact their walking behaviors and their health. We were targeting people with diabetes, and we thought that if this program or policy were ever implemented at scale, there would be some automated system to collect daily walking data—either using a pedometer on their phones or some Wi-Fi hubs where they could automatically sync their pedometers.

But that kind of infrastructure was too expensive for an experiment. So instead, we gave people Fitbits, which have a way to automatically sync and send data back. However, since we were working with people who didn’t have access to Wi-Fi and didn’t set up that infrastructure, we had to adapt. Instead, we had daily phone calls go out to people, and they would self-report how many steps they had taken. That’s what we used to incentivize them, and at the end of the experiment, we went back and verified their step counts to ensure they weren’t cheating.

That is not how we would expect it to happen at scale, but the field realities were that people didn’t have access to Wi-Fi, and we didn’t have enough funding to set up the infrastructure we would need at scale—so we had to adapt to what was already there.

Sambhav: Thanks, Ariel, that was quite insightful. Sometimes, projects don’t align with our initial expectations, but they can still contribute to learnings in that sector. Could you discuss such a project you were a part of and what lessons you learned?

Ariel: Yeah, I'll talk a bit more about the same project where we were incentivizing people to hit step targets.

We wanted to compare incentives for walking to other low-cost programs designed for people with diabetes. We found a promising approach in the literature—an RCT in South India that tested SMS text messages with motivational messages about healthy behaviors, like eating a healthy diet, tailored to the South Indian context.

We expected it to at least work a little, but it didn’t work for us at all. People who got the text messages actually walked less. They didn’t report healthier behaviors, and there were no improved health outcomes like lower blood sugar or weight changes.

The big lesson here is the importance of replication—just because something worked in one context doesn’t mean it will work in another. Publication bias is also a factor—studies that find positive results are more likely to be published, so it’s critical to keep testing and validating findings.

Sambhav: That’s quite insightful and helpful for field researchers. Looking at the broader perspective, how do you see your work affecting policy decisions and improving lives?

Ariel: I really hope to design projects aimed at evaluating programs or policies that could be directly implemented. For example, incentives for walking could be scaled up through government or large health systems in South Asia.

I also have another project on reducing groundwater usage. A lot of farmers in India overuse groundwater and electricity for irrigation because they don’t have to pay directly for the water. We implemented a program where instead of charging farmers for electricity, we paid them to use less. We worked with AKP in Gujarat to implement this model.

So far, it looks like the incentives were effective—farmers did respond by reducing their water use, even with small incentives. Now, we’re working on final outcome data to see whether reducing water use also affects yields or if farmers are just using water more efficiently.

Sambhav: That sounds like a fascinating project! I’m excited to see what the results show. Those were all the questions we had. Thank you so much for your time—this was a really insightful conversation.

Ariel: Thanks, Sambhav! So nice to talk with you.
 

This podcast and blog series brings together researchers reflecting on their experiences conducting impact evaluations in India. From working in diverse local contexts to engaging with government and civil society, each episode offers a window into the realities of producing policy-relevant evidence on the ground.

In this episode of the Researcher Spotlight Series, host Sambhav Choudhury speaks with Harini Kannan, research scientist at J-PAL South Asia. Harini shares her journey from being inspired by budget speeches as a child to pursuing a career in development economics. Drawing from her extensive experience collaborating with government partners across India, Harini offers valuable insights into building effective partnerships and scaling evidence-informed programs.

Want to learn more? Explore other podcast episodes in this series where we speak with Sandip Sukhtankar on his research in India, Shobhini Mukerji on her journey into development economics, Ariel Zucker on applying economic theory to real-world solutions, and Gautam Rao on co-producing evidence with policymakers.

Sambhav: Hi everyone, welcome to an episode of J-PAL South Asia’s Researcher Spotlight conversation series. My name is Sambhav Choudhury, and I am a research associate at J-PAL South Asia.

Today, I'm in conversation with Harini Kannan. Harini is a research scientist at J-PAL South Asia. Her research interests are in the education and health sectors. Apart from leading projects at J-PAL, she works with the capacity-building team, providing advisory services for various partners such as USAID¹, the Government of Haryana, the Government of Tamil Nadu, and Pratham.

On the weekends, you can find Harini baking or cooking. This year, she went all out and baked 24 cakes on Christmas for her friends!

Hi Harini, thanks for talking to us today.

Harini Kannan: Thank you, Sambhav. Thank you for having me.

Sambhav: We would love to begin by asking you—what got you interested in development economics and the experimental approach?

Harini Kannan: I think ever since I was a really young child, I was very interested in what happened around me. I was very interested in politics. At some point, I remember watching the budget speech with my grandfather, and it felt so powerful. I think it was Manmohan Singh who was delivering it. He was basically telling everyone how the money would be spent—so much on education, so much on this program or that program.

I still remember that when I was in maybe 9th or 10th grade, I decided that I wanted to be the Finance Minister of India because the budget speech just inspired me. That influenced my decision to study economics, which wasn’t something many people in Chennai pursued in the late 90s. It was a bit of a battle to convince my teachers that economics was something I wanted to do and why. But once I got into it, there was nothing that could pull me away from it—it was just so wonderful.

It gave me a way to look at the world, and my interest in public policy continued. I eventually did a PhD in public finance and policy. But I always knew that I didn’t want to work in academia. I wanted to work at the intersection of government, fieldwork, and research.

After my PhD, I struggled to identify where I could work because these kinds of opportunities didn’t quite exist in India in 2010. Then someone told me about J-PAL. Of course, I knew about Abhijit Banerjee and Esther Duflo, but I wasn’t aware of J-PAL itself.

It was a fateful meeting with Shrinivas, and I got into this work. It turned out to be the perfect fit for me because I get to do a lot of different things, and my portfolio keeps changing in ways that I both appreciate and enjoy.

Sambhav: That is quite inspiring!

The next question we have is about your paper Informal Math Games to Improve Children's Readiness for Learning School Mathematics in India. Can you briefly tell us how it was working on this project and some of the challenges of running an education intervention with children?

Harini Kannan: I think the best part of this project was working with Elizabeth Spelke, the cognitive psychologist at Harvard. She opened my eyes to a completely different set of literature—something I may not have been exposed to otherwise.

This project was all about how children learn mathematics, how innate skills exist among most humans (and even some primates), and how these skills can be leveraged to learn formal mathematics. Every conversation with Liz feels like a graduate class in cognitive science, and it has been absolutely wonderful.

What made this project unique was that we weren’t just evaluators—we were also implementers. And not just overseeing implementation, but actually designing the program ourselves.

I remember looking at these games developed at Harvard and thinking, This seems difficult. These games were originally designed for lab studies, where children played them for half an hour before an assessment. Our challenge was to turn them into a three- or four-month curriculum—something systematic, with scaffolding and increasing difficulty, while also keeping children engaged.

Piloting this was one of the most interesting experiences of my life. On the first day, I thought it was never going to work. My research assistant and I struggled because we didn’t know how to make four- and five-year-olds sit down. One child climbed onto my lap, another was pulling my RA’s dupatta—we were just completely overwhelmed.

From that chaotic moment in 2013 to now in 2023, where we are scaling up these programs across five different states, it has been a fantastic journey.

Sambhav: That’s amazing! In your experience, what key factors or strategies have proven effective in forming partnerships with governments or other stakeholders for your projects?

Harini Kannan: I think the most important thing is to listen to them. Sometimes, we may have a tendency to tell them, This is your problem, and this is how you should solve it. But what works best is letting them speak and then guiding the conversation toward solutions.

Building personal relationships is crucial. In many of my projects—especially those in Haryana—I’ve had endless cups of sugary, milky chai while sitting with government officials. I realized that these informal conversations help you understand the undercurrents and nuances of a department.

If your goal is to get approval for a project, of course, you engage with top-level officials. But it’s just as important to engage with people down the chain. Your IAS officers will move on, but your state civil service officials will stay for years. Getting them excited about the project and respecting their perspectives is key.

Many government officials may not have read Esther Duflo’s book or fully understand RCTs, but they are equal stakeholders in the process. Working across all levels of government hierarchy is extremely important.

Sambhav: That’s really great advice. Now, looking at the bigger picture—how do you see your work affecting policy decisions and improving people’s lives?

Harini Kannan: We all hope that our research projects will be so impactful that they get scaled up. I’ve been fortunate to be part of three different studies that are currently in different stages of scale-up.

I’ve seen the full cycle—starting with an idea, running an evaluation, piloting an intervention, and then scaling it up. What fascinates me is how challenges evolve across these stages.

Over the past few years, I’ve worked more on scale-ups, and I’ve realized that running RCTs is much easier than scaling up interventions!

Sambhav: That’s a great insight. As a final question—what research projects are you excited about in the near future?

Harini Kannan: One project I’m really excited about is reading games—companion games to our existing math games. We just started piloting them in Delhi. If the pilot goes well, we plan to turn this into an RCT.

Another exciting project is linking datasets from different government departments to track long-term outcomes. We’re still in early stages, but I’m looking forward to seeing where it goes.

Sambhav: That sounds fascinating! Thank you, Harini—this was an inspiring conversation. Best of luck with your upcoming projects!

Harini Kannan: Thank you, Sambhav! Hope to see you at J-PAL soon.

[1] This conversation was recorded in late 2023, prior to the closure of USAID.