Community-Based Monitoring of Primary Healthcare Providers in Uganda
- Health care providers
- Rural population
- Citizen satisfaction
- Mortality
- Transparency and accountability
- Service provider performance
- Take-up of program/social service/healthy behavior
- Information
- Community monitoring
Ineffective monitoring systems and weak accountability relationships between service providers and beneficiaries may result in poor public service delivery. Researchers conducted two randomized evaluations in the health sector in Uganda to evaluate whether community monitoring could impact public health worker performance and subsequent health utilization and outcomes. They found community monitoring, when combined with information on health provider performance and user’s entitlements, led to better quality and more frequently utilized health services, and ultimately improved health outcomes. However, community monitoring alone had little impact.
Policy issue
Nearly 11 million children under five die each year, many from preventable diseases such as pneumonia, malaria, and measles. Though prevention and treatment for such diseases is relatively inexpensive, many do not have access to necessary health services. Why are these services not provided? One possible reason may be ineffective systems of monitoring and weak accountability within public healthcare delivery. Poor incentives for public providers to deliver quality services may result in high absenteeism and low-quality patient care. Beneficiary monitoring of providers may be one important way to improve delivery, given that beneficiaries have the most to benefit from improved health services. However, existing research has yet to identify how public service providers’ accountability to citizen-clients can be strengthened: what factors underlie successful attempts to enhance citizens’ ability to hold service providers accountable? And can stronger accountability relationships translate into better health?
Context of the evaluation
Rural Uganda is characterized by poor public health service provision. Rural dispensaries, the lowest tier of the Ugandan health system, provide preventive, outpatient, and maternity care, along with laboratory services. However, on a typical day, approximately 50 percent of staff are absent and patients wait for more than two hours to be seen. The rural, public health clinics are usually staffed by one medical worker, two nurses, and three nursing aides. The dispensaries provide few incentives for their workers to increase their efforts. A number of government departments are responsible for supervising these dispensaries, but actual monitoring and enforcement are rare.
Details of the intervention
Beginning in 2005, researchers conducted a randomized evaluation in fifty rural dispensaries covering nine districts in Uganda to examine the impact of a community monitoring program on health worker performance and subsequent health service utilization and health outcomes. In a follow-up study from 2007 to 2009, researchers evaluated the longer-run impacts of the original intervention and conducted a second randomized evaluation of 25 dispensaries in the same districts to further study the effect of different components of the program.
Information and Participation Intervention: During the first evaluation, half of fifty dispensaries were randomly chosen to participate in the community monitoring program. The other half did not participate in the program and formed the comparison group. In the catchment areas around the dispensaries in the program, local NGOs facilitated three sets of meetings to kick-start the process of community monitoring:
- The first meeting brought together a large number of community members—both the disadvantaged and the elite—to discuss the status of their health services and steps they could take to monitor providers.
- The second meeting gathered local health staff to discuss their views on the key constraints in local health care delivery.
- The third meeting brought community members and health workers together to discuss patient rights and provider responsibilities, and to produce a shared action plan about how to improve local health care delivery.
During these meetings, the NGOs also disseminated report cards containing information about the dispensary’s performance, as measured through facility and household surveys conducted prior to the intervention. In addition, the cards contained comparisons with other health facilities and national standards, as well as information on health care users’ rights and entitlements.
After six months, the NGOs facilitated a half-day follow-up meeting with community members and health workers to review progress on their action plans. In 2007 and 2008, researchers again returned to program communities and repeated the same set of four meetings, but did not provide any new report card information.
Participation Intervention: To determine the relative effectiveness of the report cards compared to the facilitated meetings, researchers tested a participation-only intervention through a second randomized evaluation beginning in 2007. Among 25 newly selected dispensaries, half were randomly selected to receive the program. The other half did not and formed the comparison group. The program mirrored the information and participation intervention but did not provide dispensary report cards—the most expensive component of the program.
Researchers administered surveys to both service providers and a randomly selected subset of households in the catchment area of each dispensary prior to the intervention, again one year later, and following the interventions, in 2009. Thus, they measured long-run impacts of the information and participation intervention and short-run impacts of the participation-only intervention.
Results and policy lessons
The combined information and participation intervention led to significant improvements in the quality of care, facility utilization, and health outcomes. But without information, the process of stimulating participation and engagement had little impact.
Impact on Quality Care: In the short run, health facilities in information and participation villages were 32 percentage points more likely to have suggestion boxes and 16 percentage points more likely to have numbered waiting cards, relative to 0 and 4 percent of comparison facilities, respectively. Patients experienced a twelve minute (9 percent) reduction in wait time, relative to an average wait time of 131 minutes in comparison communities; facilities recorded a 13 percentage point (28 percent) reduction in absenteeism, relative to 47 percent absence rates in comparison facilities; and the overall cleanliness of treatment facilities improved. Four years later, many of these quality improvements, including suggestion boxes and waiting cards, persisted in information and participation communities. However, within the participation-only intervention, there was no evidence that the management or quality of the clinic improved.
Impact on Facility Utilization: Though the magnitude of improvements declined slightly over time, the information and participation intervention led to sustained increases in health facility utilization. Four years later, facilities recorded nearly 100 (16 percent) more outpatient visits per month, relative to an average of 650 monthly visits in comparison facilities. These facilities also performed 6 more deliveries each month, relative to 13 deliveries in comparison facilities (a 46 percent increase). In addition, patients in information and participation communities were approximately 20 percent more likely to obtain antenatal care and family planning services from local health facilities, relative to comparison communities. By contrast, the participation-only intervention had no clear impact on facility utilization. While households in this group were less likely to use traditional healers or self-treatment, relative to the comparison group, they were no more likely to use local health facilities.
Impact on Health Outcomes: The information and participation intervention led to sustained reductions in child mortality. In the short-run, the under-five mortality rate dropped to 97 per 1,000 live births—a 33 percent reduction relative to comparison communities, which averaged 144 per 1,000 live births. Four years later, under-five and infant mortality remained 23 and 28 percent lower, respectively, in information and participation communities. Furthermore, in the long-run, these communities experienced a 0.22 z-score increase in infant weight-for-age, a summary measure of current nutrition and illness, while this measure actually worsened in comparison communities. By contrast, the participation-only intervention only marginally improved infant weight-for-age and had no impact on other health outcomes.
The Role of Information: Providing information on facility performance helped communities to identify the most urgent areas for actionable improvement. In the development of the community action plans, information and participation communities almost exclusively identified local issues that the health workers or the patients could address themselves, such as absenteeism, opening hours, and waiting times. Without information, the participation-only group mostly identified issues that required third-party actions—e.g. more financial support and timely deliveries of medicines.
The Role of Community Heterogeneity: In the short run, the information and participation intervention was significantly less effective for ethnically heterogeneous communities and those with more income inequality. In these heterogeneous treatment communities, fewer patients sought childbirth deliveries, prenatal care, and family planning than in homogenous treatment communities. They also were less likely to actively monitor healthcare providers than homogeneous treatment communities. This suggests that, while the information and participation intervention resulted in a more engaged community that discussed health service provision, more ethnically fractionalized communities had problems turning this activity into action according to the short-run results.
These results suggest that assembling and disseminating relevant information is crucial in programs designed to strengthen local control and oversight, as it enables the community to focus on actionable tasks. However, more research is necessary in order to understand how the design—including the type of information provided and mechanisms for delivery—best supports local accountability efforts. While the information and participation intervention was significantly more expensive (approximately US$10,000 per facility/community over a four-year period —costs mainly related to data collection for the report cards), the findings of large and sustained positive health effects suggest that such investment in data collection and dissemination, while expensive, may be worth the cost.