Identifying and implementing incentives that give rise to a strong relationship of accountability between service providers and beneficiaries is viewed by many as critical for improving service delivery. How to achieve this in practice and if it at all works, however, remain open questions. Systematic evaluation of service delivery innovations to increase accountability can show what works, what doesn’t and why, a first step to scaling up success. This paper discusses one such attempt: a randomized evaluation of a Citizen Report Card project at the community level in primary health care in Uganda. The Citizen Report Card project collected quantitative information on the quality and quantity of health service provision from citizens and public health care providers. This information were then assembled in "easy access" report cards that were disseminated, together with practical information on how best to use this information, in community, staff, and interface meetings by local community organizations in order to enhance citizens’ ability to monitor the health care providers. The intervention improved both the quality and quantity of health service provision in the treatment communities: One year into the program, average utilization was 16 percent higher in the treatment communities; the weight of infants higher, and the number of deaths among children under-five markedly lower. Treatment communities became more extensively involved in monitoring providers following the intervention, but we find no evidence of increased government funding. These results suggest that the improvements in the quality and quantity of health service delivery resulted from increased effort by the health unit staff to serve the community.