Community Monitoring & Evaluation Print E-mail

Author: Olivia McDonald, Senior Governance Advisor, Christian Aid

Community monitoring and evaluation (CME) is a process through which communities measure the quantity and quality of public services and other government activities. The main objective of CME is not simply to collect data about the government’s performance. Rather, it is a process to strengthen the relationship between citizens and the state by informing people about their entitlements, the promises made to them by the government and the tools to ensure they receive both the entitlements and the promises.

Download the complete Tool as a PDF file or access the sections below.

You may scroll down to post your comments.


What is it?

What is it?

CME is more of an approach rather than a specific technique to involve citizens in gathering evidence of government performance and may include specific activities such as budget tracking or community scorecards which are described elsewhere in this toolkit.

The CME approach evolved from other techniques to involve communities in monitoring and evaluating the effectiveness of specific development projects. Such approaches have been used by many development agencies which has generated extensive literature on the subject. Much of this focuses on conducting monitoring and evaluation in a way that does not merely use the communities for data but also encourages them to voice and express their concerns, realities and the extent to which a given project has impacted and improved their lives.

The emergence of CME can be linked to two separate trends that emphasize the central role of government in development. First, the rights-based approach increasingly adopted by civil society organisations holds the government squarely responsible for service provision meant to secure people’s rights. Therefore, many CSOs have shifted their role from being service providers towards advocacy, civic engagement and public participation. Second, over the last decade or so, the donor discourse on development has increasingly emphasised the role of the state in development. The donors would like to ensure that the aid being channelling through governments is reaching the intended beneficiaries (poor communities), and whether it is leading to desired impact. So, as donor aid is increasingly supporting governments for provision of public services, NGOs have increasingly supported communities to appreciate the role of the state in the provision of public services and to enhance their capacity to participate in monitoring those services.

It is important to note that CME tools being developed and applied by CSOs are not merely limited to monitoring the access to and quality of services but also to monitor government’s response to human rights violations, tracking environmental degradation, monitoring and evaluating government contracts with private sector; and to track implementation of any public policy.

In general, the aims of Community Monitoring and Evaluation are:

  • Ensuring communities are aware of their entitlements and strengthening their capacity to hold governments accountable to secure their entitlements.
  • Strengthening the relationship between governments and citizens which in turn influences government to play a proactive role in the interests of citizens and respond to their needs.
  • Ensuring governments are working towards fulfilling their promises
  • Ensure that the actions of the governments are reducing poverty and making a real difference to the lives of poor communities.


How is it done?

How is it done?

The steps described here are adapted from ‘Monitoring government policies: A toolkit for African civil society’ jointly published by CAFOD, Christian Aid and TRoCAIRE (2006).

1. Step one: Build trust with the community and identify the issues for monitoring

Building a relationship of trust with the community is essential for securing their commitment, which in turn is necessary to ensure the sustainability and therefore the rigour of the monitoring work. So too is identifying the issue that matters most to them. This initial engagement and assessment should ensure the process you develop is pitched at the right level and fully inclusive. You need to consider power relations within the community. It is good to talk to the whole community but if you want to speak to the more vulnerable and less powerful members of the community such as women, children or people with disabilities for example, you may need to also speak with them directly. Two tools that are handy in this relationship endeavour are Problem Tree and Focus Group Discussion.

2. Step two: Identify the stakeholders and potential allies

Use a stakeholder map to identify those responsible for delivering the action or output that needs to be monitored. If the focus is local level, these could include local officials and councillors as well as traditional leaders. If the focus is on a national policy, these could include officials from the relevant line ministry and MPs. You can ‘surprise’ your targets with your findings at the end of the monitoring process; however, there is evidence that CME is more effective when government is engaged throughout the process. You may be more successful if you bring some other groups on board. If you are focusing on a national policy, working with a few communities in different parts of the country could show whether that policy is being implemented consistently like for instance, a national organisation wanting to identify local organisations who can work with the community to collect the data. It would also be good to strategically link with organisations that have influence with government officials, access to information, good campaigning skills, links to the media, expertise on your issue or technical skills in conducting research. The key tool to be used in step two is called Stakeholder Analysis.

3. Step three: Define the monitoring objectives

It is important to decide whether to focus on monitoring what government is doing what it said it would, or on whether these actions are making a difference. The following table explains policy inputs, outputs, outcomes and impact, and the monitoring objectives using the education sector as an example:

We want to look at:Our monitoring objective is to
The inputs the government is putting in like money to pay teachers.. monitor whether teachers are regularly receiving their full salary
The outputs produced as a result of the inputs, for example the number of children attending school.. see whether more children are going to school
The outcomes or changes or improvements resulting from the outputs, for example increases in children’s literacy.. monitor whether there have been improvements in children’s literacy levels
The impacts or ultimate changes in people’s lives as a result of the above, for example young people getting jobs or going to further education.. track the number of children leaving school and getting jobs

It may not be possible to monitor objectives in all four areas, so organisations may need to select priority objectives to focus on. It is important that monitoring objectives are Specific, Measurable, Achievable, Relevant and Time-bound (SMART).

4. Step four: Choose indicators and establish a baseline

Before beginning to collect evidence, it is essential to establish the unit of analysis, indicators and baseline:

  • Unit of analysis: Will evidence be collected about individuals, families (households), specific groups such as older women or the entire community?
  • Indicators: These are what the community would actually measure. For example, as part of their community monitoring programme the Uganda Debt Network identified the following indicators to track in water and sanitation provision:
    • incidence of diarrhoea
    • availability of clean water
    • distance from nearest water source
    • time spent at borehole or in queue at water source
    • number of boreholes in a zone.
  • Baseline: In order to measure change either in terms of improvement or deterioration, it is important to establish what the situation was prior to or at the commencement of a particular development intervention. Primary research or existing information like census data and government household surveys can be used. The tools for developing indicators and baselines can be accessed from the websites of CORE Initiative and Uganda Debt Network.

5. Step five: Gather evidence

To collect the data you can choose to use quantitative methods like household surveys, qualitative methods like focus group discussion or a mix of both. For instance, a survey could help establish whether there has been an improvement in, people’s perceptions about the length of time spent waiting at boreholes. At the same time, you could also ask community monitors to stand by a small sample of boreholes one day a month and simply count how long people wait.

The evidence gathered must be representative. It might not be possible to speak to everyone in a community, but interviewing two people does not provide rigorous evidence about that community. This is why sample size is so important. It is also important that research is legitimate, that there is openness, free agreement and respect for security on all sides. This is why building a relationship of trust with the community is important. Finally, involving communities in the analysis of the results is fundamental if you want to ensure the benefits of your initiative are in place long after the specific initiative has been concluded. The information on qualitative and quantitative methods can be accessed from the websites of Research Methods Knowledge Base, CIVICUS and Research Consortium on Educational Outcomes on Poverty (RECOUP).

6. Step six: Use your results

Take your results to those you have monitored, sharing any recommendations that have come out of the process. Try to involve communities directly in these activities perhaps ensuring community representatives attend meetings with local or national government. See if it is possible to take the relevant government representative to the community to discuss the findings. If you’ve built a good relationship with the government officials in advance, this should be easier to do. If you are concerned that government will not act on your findings, you may try to make a media splash to help generate more public pressure on the government. The tools for step six can be accessed from the websites of CARE and Water Aid

Benefits

Benefits

  • CME helps ensure people get the things that they have been promised by the government like for e.g. effective monitoring has increased government probes into attacks on Dalits2 in India
  • CME can increase people’s confidence to stand up for themselves and their community and recognise that the resources in question are ‘theirs’ rather than the governments.
  • CME can be socially transformative when the process gives marginalised groups in a community an equal voice with the powerful or when it brings together groups where there has been a tradition of conflict like for e.g. between Muslims and Christians in Nigeria.
  • In general terms, CME builds the capacity of civil society, though to be effective often requires strategic partnerships between groups working at different levels such as for e.g. communities, CBOs and NGOs and those with particular skill sets like research institutes, experts, media groups etc.

Challenges and Lessons

Challenges and Lessons

  • It is important to build a relationship with those being monitored, particularly if you want to get access to information you need and ensure people act on your findings.
  • However, this will not be possible in all contexts and in some contexts it may even be dangerous to monitor government in this way. Civil society organisations need to ensure they are aware of the risks to communities participating in these projects. Again, avoiding taking government by surprise may help overcome this.
  • 2 Marginalized communities at socially disadvantaged position in the caste system
  • Access to information is often a problem. This may happen because governments do not want people to see the information, or simply because governments have not got/cannot get access to the information required which is often the case at the level of local government.
  • Monitoring objectives and indicators should be developed in a participatory manner to ensure they resonate with the community needs, priorities and aspirations.
  • It can also be good if projects are conceived to align and coincide with national planning processes and focus on national goals such as MDGs. This, however, may directly conflict with the participatory processes highlighted above.
  • CME is more effective if it is done in partnership between different NGOs, but there can be tensions between groups caused by variations in the levels of capacity, approaches, political analysis and ideological perspectives, which can undermine the partnership process. Taking time to build trust between allies is important at the very outset.
  • Just speaking to the ‘community’ may reinforce power imbalances. Activities, such as community meetings need to find ways of giving those with least power an equal voice whilst all data collected should be disaggregated across gender, age, religion, ethnicity, and disability.
  • Since there are no legal sanctions, robust evidence alone rarely challenges bad government performance. Therefore, CME projects need to have a clear plan on how to advocate on their evidence, linking with the media and formal process to hold government to account, such as elections.

Key Resources

Key Resources

Anju Dwivedi. Strengthening Citizen’s Monitoring. A Power Point presentation. PRIA, India
http://www.intrac.org/docs.php/2109/Anju Dwivedi.pps

This Power point by Anju Dwidei dwells on some of the key challenges of
CME.

Bjoern, Arathi, & Nidhi. Qualitative Research Skills Workshop: A Facilitators Reference Manual. Research Consortium on Educational Outcomes on Poverty (RECOUP)
http://manual.recoup.educ.cam.ac.uk/resources/downloads/RECOUP_Manual.pdf

This manual is designed to assist facilitators, with some level of experience of teaching and practice of qualitative research and to train others to conduct qualitative social research.

CAFOD, Christian Aid and TRoCAIRE (2006). ‘Monitoring Government Policies: A Toolkit for African civil society’
http://www.internationalbudget.org/MonitorGovPol.pdf

This tool kit is a comprehensive guide for CSOs on how to use CME for tracking government policies, programmes, plans and public services.

Centre for Economic and Social Rights (CESR). Monitoring: A methodological toolkit for holding governments accountable
http://www.cesr.org/article.php?list=type&type=53

The tools describing in this methodological tool-kit are focused on tracking how far governments are delivering on their rights obligations through their policy commitments and actions. CESR works to promote social justice through human rights in a world where poverty and inequality deprive entire communities of dignity, justice and sometimes life.

Chambers, R. From PRA to PLA and Pluralism: Practice and Theory. Working Paper no 286. Institute of Development Studies. United Kingdom (2007)
http://www.gsdrc.org/go/display&type=Document&id=2786

This report captures the transformation of PRA (participatory rural appraisal) to the more inclusive PLA (participatory learning and action) as families of participatory methodologies which have evolved as behaviours and attitudes, methods, and practices of sharing.

Communities Responding to HIV/AIDS Epidemic (CORE) Initiative (2006). Participatory monitoring and evaluation of community and faith based programmes: A step-by-step guide for those who want to make HIV and AIDS services more effective in their community.
http://www.coreinitiative.org/Resources/Publications/PME_2nd/PME_2nd.pdf

Designed for use by local implementing agencies, this Guide demonstrates how using monitoring and evaluation can improve the impact of community interventions on HIV/AIDS. It is a step-by-step guide to making community-level HIV and AIDS services and activities more effective through community involvement in monitoring and evaluation

Davles, R & Dart,J, The Most Significant Change (MSC) Approach: A guide to its use.
http://www.mande.co.uk/docs/MSCGuide.pdf

This publication is aimed at organisations, community groups, students and academics who wish to use MSC to help monitor and evaluate their social change programs and projects, or to learn more about how it can be used. The technique is applicable in many different sectors, including agriculture, education and health, and especially in development programs.

Food & Agriculture Organization (FAO). Chapter 6: Community-level monitoring on the right to adequate food, in Methods to Monitor the Human Right to Adequate
Food - Volume I.
ftp://ftp.fao.org/docrep/fao/011/i0351e/i0351e06.pdf

This chapter looks at monitoring by the community for the community in relation to food security and then moves on to describe monitoring outputs that can be used for targeting communities for policy measures or community level decision-making and planning along with highlighting many case examples

Frode, A & Masara, C. Community Based Ecological Monitoring: Manual for practitioners. SAFIRE Southern Alliance for Indigenous resources (2007)
http://www.landcoalition.org/pdf/07_SAFIRE_Community_based_ecological_monitoring_Manual.pdf

This manual illustrates how CME can also be used to monitor environmental changes, either working with government as part of a formal monitoring process or using the results to hold government to account for its environmental, industrial and natural resource policies.

Governance & Social Research Development Centre (2008). Helpdesk Research Report: Community Monitoring of Service Delivery http://www.gsdrc.org/docs/open/HD508.pdf

This short report provides examples of community (participatory) monitoring and evaluation projects for holding government accountable for service delivery and expenditure management and includes examples of tools like Participatory Expenditure Tracking, Community Score Card and Report Card

HelpAge International
http://www.helpage.org/Researchandpolicy/MadridInternationalPlanofActiononAgeing/Resources

HelpAge International helps older people claim their rights, challenge discrimination and overcome poverty, so that they can lead dignified, secure, active and healthy lives. HelpAge works in over 75 countries and is strengthened through their global network of like-minded organisations, the only one of its kind in the world. The ‘Resources’ section of their website contains several reports on community monitoring and evaluation of various programmes for the aged across the globe

Overseas Development Institute (ODI), United Kingdom
http://www.odi.org.uk/rapid/Tools/Toolkits/Communication/tools.html

ODI is Britain's leading independent think tank on international development and humanitarian issues. The Research & Policy in Development section of ODI’s website contains a comprehensive description of various participatory tools such as Problem Tree Analysis, Stakeholder Analysis, Focus Group Discussion, Force Field Analysis etc.

Sprechmann, & Pelton, E. Advocacy Tools and Guidelines: Promoting Policy Change. CARE International (2001)
http://www.impactalliance.org/ev_en.php?ID=4303_201&ID2=DO_TOPIC

This manual outlines ways to contextualize policies, or the lack of them, in order to develop the most appropriate strategies to influence policy decision-makers. It recognizes the frequent need to incorporate advocacy into projects in order to tackle the root causes of problems faced by communities. These tools and guidelines provide a step by step guide for planning advocacy initiatives, as well as advice for successful implementation.

Trochim, W.M.K. The Qualitative-Quantitative Debate.The Research Methods Knowledge Base
http://www.socialresearchmethods.net/kb/qualdeb.php

The Research Methods Knowledge Base is a comprehensive web-based textbook that addresses all of the topics in a typical introductory undergraduate or graduate course in social research methods.


Uganda Debt Network (2002). Community-based monitoring and evaluation systems (CBMES) pilot test in Kamuli District.
http://www.udn.or.ug/Kamuli_Report.pdf

UDN is a Civil Society Organization (CSO) monitoring the utilization of Poverty Action Fund resources in seventeen (17) Districts in Uganda to ensure that services reach the intended beneficiaries. The report is a useful resource on the CME methodology describing the pilot carried out in Kamuli district
Vinuales, D. A tool against climate change....and hurricanes, a case study in From Poverty to Power: How Active Citizens and Effective States Can Change the

World, Oxfam International (2008)
http://www.oxfam.org.uk/resources/downloads/FP2P/FP2P_Nicaragua_tool_against_CC_CS_ENGLISH.pdf

This short case study describes how community participation, monitoring a nd evaluation can act to fight climate change in Nicaragua.
Water Aid (2007).

Advocacy Source Book
http://www.wateraid.org/documents/plugin_documents/advocacy_sourcebook_2.pdf

Excellent resource on advocacy strategies for CME initiatives and meant for CSOs involved in water and sanitation issues

Case Studies

Case studies

Older citizen monitoring in Tanzania
Age discrimination routinely prevents older people from accessing public services and participating in public life. Help Age International has attempted to address this through supporting older citizen monitoring in Tanzania, where older people helped develop the indicators they wanted to monitor. The indicators on quality of healthcare for e.g. looked at: resting/waiting place (chair, ground, standing); medical staff listening to older people graded as poor, average, or good; satisfaction of older person with the service in terms of yes or no; use of language or discussion graded as poor, average or good; greetings offered by the staff as yes or no); whether staff mocked at older person in terms of yes or no) Their findings of their initial monitoring on healthcare were:

  • 40% said the tone of language used by medical staff was mocking.
  • 94% were charged for the consultation.
  • 30% were unaware of how to apply for free health care.
  • Over one-third had to wait four to six hours to see a doctor.
  • Almost half had to pay for their own fare to get to the hospital.

As a result of monitoring and advocacy on the findings, the local government has now granted free health treatment to all vulnerable older people. The project has also given older people a sense of self-dignity and many have said that they felt that their concerns were being listened to.

For more information, visit: http://www.helpage.org/Worldwide/Africa/Keyprojects/OldercitizensmonitoringTanzania
http://www.helpage.org/Resources/Regularpublications/Ageways

Monitoring a water and sanitation project in Uttaranchal, India
As part of a World-Bank funded project to improve water and sanitation in Dharali village in the Indian state of Uttaranchal (SWAJAL water and sanitation project), a Village Water and Sanitation Committee was established to oversee the planning, implementation, maintenance and management of the village water and sanitation schemes. The committee did not function well while participation was ritualised and women were not adequately represented. The Participatory Research in Action (PRIA) along with Himalayan Action Research Centre (HARC), came in at this stage to engage the community and build their capacity to identify, discuss, analyse, investigate and develop their own solutions to the issues and problems concerning the project.

The community nominated a new monitoring committee which was more representative and legitimate. To facilitate this, meetings were run at times suitable for the community like for instance in the evenings during busy farming periods. Indicators of ineffective water supply were identified for tracking by the committee such as: regularity of water supply; functional stand-posts; quality of water;, condition of water pipes and contribution of the community.

After one year of activities, there was increased participation of the community members, and greater openness, transparency and accountability on the part of village and project officials. An unintended but positive consequence was the creation of a village development committee that is now responsible for the vast array of development work in the village. This shows how investment in one sector which, in this case, was water and sanitation, can be a very sustainable intervention that could have a spill-over effect on improvements in other sectors.

For more information, refer:
‘Beyond participatory rituals: Strengthening citizen monitoring in Dharali (India), Nandini Sen, PRIA
http://portal.unesco.org/education/en/file_download.php/5212f484843c6fc9026d12caceb38cf7Cit+Ed-India+Case+Study.pdf

 

Discuss on Civic Education


Have practical experience in implementing this PG tool? Share your lessons with others by joining a discussion group here : Note that you will need to be registered to the site.

or

View readers comments or post your comments here.

Contact us | Disclaimer | About PG Exchange | CIVICUS