PDF version

Improving Industrial Pollution Regulations in India

Localisation: Ahmedabad and Surat, Gujarat, India
Sample: 900 Industrial Plants
Timeline:
2009 to 2011
AEA RCT Registration Number: 
Données: 
Partenaires:
Question de politique 

Third-party auditing is used to enforce regulation and verify quality in a wide range of settings. Such audits, which make sure services are delivered and regulations enforced, may be particularly important in developing countries. There is a growing concern, however, that third-party auditors may face a conflict of interest between providing credible reports and maintaining business with their client firms. Many audit systems have adopted piecemeal reforms but not addressed the core problem – when firms directly hire their auditor, they can shop for a favorable report. This choice may undermine firms’ incentives to comply with regulation in the first place.

Cadre de l'évaluation 

The Gujarat Pollution Control Board (GPCB) is charged with enforcing environmental standards across one of India’s most industrialized states. Since 1996, Gujarat has had a third-party audit system, under which plants with a high pollution potential must receive a yearly audit, conducted by a private auditing firm hired and paid for by the plant. Auditors visit each plant three times a year, observe the plant’s environmental management practices, and measure their pollution outputs. The results of these visits are then compiled and sent to both the plant and GPCB. The audit system includes a number of safeguards for ensuring audit quality: auditors must be accredited by an audit committee, auditors are forbidden from taking on additional consultancy work for plants they audit, a given audit team cannot undertake more than 15 audits in a year, and auditors that are found to be inaccurate can in principle be decertified. Despite these safeguards, audit reports are suspected of being unreliable, and emissions of the water pollutants targeted by the system remain high.

Conducting an audit in Gujarat

Détails de l'intervention 

Together with the GPCB, researchers evaluated a modification of the existing audit system in order to increase the independence of auditors. This modification consisted of three changes to the existing system: random assignment of auditors to plants, payments to auditors from a central pool, and backchecks of auditor performance.

In 2009, 233 of the 473 audit-eligible plants were assigned to the audit treatment, while the remaining plants served as the comparison group. In the treatment group, auditors were randomly assigned to plants by the research team and were paid from a central pool of funds raised for the study, rather than being hired and paid directly by the audited plants. A random 20 percent of audit visits in the treatment group were also backchecked by technical staff of independent engineering colleges. Auditors were notified in advance that some of their work might be backchecked but the actual visits were unannounced. During the first year, auditor accuracy was measured but no consequence of poor performance was explicitly specified. However, in the second year, auditors’ pay was directly linked to accuracy, as measured by the backchecks.

Researchers collected audit reports for both treatment and comparison plants filed with the GPCB in 2009 and 2010. This information was supplemented by backchecks for treatment plants after 20 percent of audit visits in both years. At the end of year two, additional backchecks were performed across both treatment and comparison plants. The auditors were not notified in advance and, in the comparison plants, the results of the backchecks, did not impact auditors’ pay. Finally, researchers conducted an endline survey of pollution measurements roughly six months after the end of the intervention.

Résultats et conclusions politiques 

The results indicate that auditors reported pollution in treatment plants more accurately than in comparison plants. Auditors in the comparison group, hired by client plants, systematically underreported pollution readings, as measured by backchecks. In particular, in comparison plants, auditors systematically reported many pollution readings to be just below the regulatory standard.

The audit treatment eliminated this large bias in third-party reporting. Auditors working in the treatment group improved the accuracy of their reports by 0.21 standard deviations, on average, enough that by the end of the experiment they were statistically equal to the truth. Many of the same audit firms worked with both treatment and control plants and this difference in reporting accuracy is evident even when comparing the reports of the same auditors across the two sets of plants. The researchers measure the separate effects of two components of the treatment by looking at accuracy across audits before and after incentives were introduced and with varying exposure to prior backchecks. Both financial incentives and backchecks appear to have played independent roles in improving auditors’ accuracy.

The purpose of the audit regulation is not to produce accurate reports but to reduce pollution. The experiment measured pollution outcomes for air and water pollution emissions across all plants in the sample. Plants in the treatment group responded to independent audits by significantly reducing their pollution emissions. Plants subject to the audit treatment reduced pollution by 0.21 standard deviations. This reduction occurred mainly in water pollutants, which are a regulatory priority.

These results offer clear evidence that altering economic incentives can cause auditors to switch from biased reporting to truth-telling, and that this change can lead, in turn, to improvements in real outcomes.

Résultats 
When auditors were hired and paid by the firms they were auditing, as in the status quo audit system, false reporting and pollution were high.

Auditors in status quo plants were paid about 24,000 rupees per audit on average, which is well below the average cost of conducting a full audit at 40,000 rupees. This suggests that many auditors did not conduct all the tests needed to complete an audit properly.

Twenty-nine percent of audit reports in comparison plants falsely reported pollution as below the relevant regulatory standard. For particulate matter pollution, auditors reported that 7 percent of plants violated the standard, while in fact 59 percent were in violation. They also reported that nearly three-quarters of plants polluted just below the standard, but the independent backchecks reveal that only 19 percent of plants polluted in this narrow range. This shows that auditors systematically reported firms as being narrowly compliant with national pollution standards.

The new audit system led auditors to report pollution more truthfully and substantially lowered the number of plants that were falsely reported as compliant with pollution standards.

Relative to auditors in comparison plants, auditors working under the new system reported much higher pollution. They were also 23 percentage points (or 80 percent) less likely to falsely report a pollution reading as compliant with the relevant regulatory standard. Auditors working under the new system also reported that far fewer plants were polluting right below the standard. However, their reports still bunched a little beneath the standard, relative to the true pollution readings.

Since some auditors worked in both treatment and comparison plants, researchers were able to compare their behavior under both audit systems. They found that the same auditors reported pollution more accurately under the new system than they did in comparison plants that they were auditing at the same time. This shows that the increased accuracy was due to the new audit system and not to treatment plants having better auditors or auditing firms with more financial resources.

Industrial plants reduced pollution in response to more accurate audits.

Plants facing the new auditing system reduced pollution by 0.21 standard deviations on average. This reduction is driven by an even larger reduction in water pollution, which is a top regulatory priority for the GPCB. The pollution reductions came from the highest-polluting plants. In practice, the GPCB reserves the harshest penalties, like plant closure, for plants with readings that significantly exceed the standard. This is reflected in the fact that the dirtiest plants responded by reducing emissions the most.

Conclusions politiques 
When auditors are chosen and paid by the firms they are auditing, third-party audit systems may yield very inaccurate reports.

In Gujarat, when auditors were hired and paid by the plants they were auditing, they did not provide regulators with reliable information about pollution. There is evidence that many auditors did not even conduct all the tests necessary to complete a full audit. If they are to be an effective policy tool for enforcing regulation, third-party audit systems must be designed to incentivize accurate reporting.

Resolving this conflict of interest can lead to more accurate reporting.

Randomly assigning auditors to industrial plants, paying them a fixed fee from a central pool, and double checking their accuracy led auditors to report industrial pollution much more accurately.

When the environmental regulator received better information about pollution levels, industrial plants responded. In response to more accurate pollution audits, the dirtiest industrial plants reduced their emissions substantially.

This suggests that plants may also change their behavior if the regulator obtained more accurate information through other means, such as its own inspections or better emissions monitoring technologies.

Eliminating conflicts of interest for auditors could improve third-party audit systems in other sectors beyond environmental regulation.

The core problem in Gujarat’s environmental audit system—that auditors had poor incentives to report pollution levels accurately when they were chosen and paid by the firms they audited—exists in virtually all other third-party audit systems. This evaluation provides the first-ever findings on removing the fundamental conflict of interest that characterizes third-party audit markets. It seems reasonable to assume that a version of these reforms adapted to the particular institutional features of other third-party audit markets would produce similar results.

Duflo, Esther, Michael Grenstone, Rohini Pande, and Nicholas Ryan. "Truth-Telling by Third-Party Auditors: Evidence from a Randomized Field Experiment in India." Working Paper, MIT, March 2, 2012.

Duflo, Esther, Michael Greenstone, Rohini Pande and Nicholas Ryan. "What Does Reputation Buy? Differentiation in a Market for Third-party Auditors." Working Paper, MIT, January 2013.