Rigor or participation? Discuss issues and challenges in participatory M&E practices
Participation has turned into a key notion in development of late, with governments and donors world over pressing for adopting participatory approaches in executing programmes. This greater prominence of participation has led to an increased stress on adopting participatory approaches in monitoring and evaluation of development initiatives (Estrella and Gaventa 1998).
Participatory Monitoring and Evaluation (PM&E) engages key stakeholders as dynamic participants and demonstrates the viewpoints and aims of those most exactly concerned (World Bank 2010b).
Even as the “ideological case” for participation is broadly recognised, branding PM&E as ‘participatory’ does not essentially secure the participation of all stakeholders. The process, time and again, results in issues, such as, who participates and who is left out, which consequently leads to criticism of the “representativeness” of the results (GSDRC 2012).
Keeping in view the critics of non-conventional approaches, this paper aims to discuss the issues and challenges in Participatory Monitoring and Evaluation (PM&E) practices with a focus on participation, power relations and methodological issues.
Introduction
Monitoring and evaluation (M&E) is an established management exercise that considers “learning and change” when applied persistently. However, participatory monitoring and evaluation is not the same as conventional M&E, as it endeavors to embrace all stakeholders in all phases of the course of action. PM&E does not limit the participation of stakeholders to identifying the problem but also acknowledges their role in gathering, examining and interpreting the information for development and investigation. This makes PM&E a possible monitoring and evaluation process that integrates local information and develops analytical ability of the stakeholders (Holte-McKenzie, Forde and Theobald 2006).
Being an element of a broader “historical process”, PM&E infers from diverse participatory research backgrounds, such as, Participatory Action Research (PAR), Participatory Rural Appraisal (PRA) and Farming Systems Research (FSR) or Farming Participatory Research (FPR) (Estrella and Gaventa 1998).
International donor agencies and development organizations have already made the models of PM&E part of their policy making field. In addition to the area of development, PM&E can also locate its application in the private sector for the purpose of individual and organizational knowledge (Raynard 1998).
Since PM&E attracts numerous divergent observations about the nature and constitution of participation, it has no single and specific definition (Estrell 2000; Gujit et al. 1998). Therefore, a true PM&E can be defined as an approach that aggressively engages all the stakeholders, including, the associated staff and beneficiaries, in all phases of the monitoring and evaluation exercise, such as, planning, designing and execution, and in constructing shared decision relating to the outcomes and the follow up activities (King 2005).
Generally, PM&E attracts more consideration and significance in the perspective of development activities. There is also a growing call for employing PM&E so as to guarantee better accountability in the utilization of resources and to produce demonstrable impacts of community-based development programmes from different stakeholders, for example, contributing governments, donor agencies and the beneficiaries (Cracknell 2000; Parks 2005).
Core Principles of PM&E
There are four common principles which form the centre of PM&E. Participation is the first and most important principle, which allows various stakeholders to play a role in determination of what should be achieved and how it should be accomplished (Estrella and Gaventa 1998). Thus, participation is a fundamental aspect of the whole PM&E exercise, from identifying goals and information requirements to evaluating and employing results (Campilan 2000).
The second main principle in PM&E exercise is learning, which lays stress on “practical or action- oriented learning” of the stakeholders involved in a development initiative. The concept of learning works to stress the purpose that in PM&E, participants can achieve the aptitude to measure their own requirements, evaluate their own preferences and targets, and evolve “action-oriented planning” by means of their knowledge in the PM&E exercise. This process of learning is a way for capacity building for the local participants (Pfhol 1986).
The third key principle is negotiation. PM&E is constantly acknowledged as a “social process” wherein people engage in negotiation for their different requirements, expectations and viewpoints. Due to the inclusive nature of PM&E, negotiation is deemed to be important for developing trust among the participants and shaping their views, deeds and approaches, which impact the manner of their role in the project. Moreover, the negotiation practice is recognized by many researchers as a “highly political exercise”, which essentially tackles the “issues of power, equity and social transformation” (Estrella and Gaventa 1998).
The fourth core principle of PM&E is flexibility. Many researchers are of the common view that PM&E process is persistently changing and adjusting in line with project-specific conditions and requirements instead of sticking to one design. Therefore, flexibility in the blueprint and execution of PM&E exercise is necessary to ensure that the process represents the stakeholders’ needs. In particular, the PM&E design is required to signify local socio-economic, cultural and political conditions (Estrella and Gaventa 1998).
Issues and Challenges in PM&E
The above discussion suggests that PM&E is capable of producing great benefits to communities. However, there are numerous issues and challenges which make it difficult for PM&E to realize its core principles easily.
Participation
Participation is central to whole process of PM&E. However, putting the concept into practice is truly a challenge, considering the lack of universal understanding of the concept or degree of participation. For example, there is lack of agreement on minimum criterion to deem M&E as participatory due to different explanations in the case studies in Asia (Armonia, Ricardo and Campilan 1997).
Engaging the local people only in execution phase is not sufficient, as their involvement also in “management and decision-making” can make a project wholly participatory (White 2000).
Moreover, participation can also face the issue of fluctuation during different stages of PM&E and project cycles. For instance, it is observed that the stakeholders, particularly local people, take little part in initial phases of planning and designing and later stages of analyzing and sharing results (Estrella and Gaventa 1998). Hence, the involvement of local people is usually established to be made limited to data collection. This restrains the local people from attaining the key feature of empowerment and makes PM&E a mere “extractive process” (Guijt 1999).
Another challenge associated with the process is the politics of participation. Since multiple stakeholders are involved in PM&E, it usually becomes problematic to decide who should take part in monitoring and evaluation, such as, mainly local people, a mix of beneficiaries and external professionals, or a wider group of those directly or indirectly affected. At this stage, power relations of major actors come into play, which can influence who ultimately is permitted to engage and under what specific circumstances (Campilan 2000).
Even, deciding among local people is difficult, considering that they are not “homogenous”. Therefore, special mechanisms are required to ensure the participation of comparatively deprived groups (White 2000).
Another challenge is associated with “by whom” the PM&E is started and carried out. For example, the participation of local people is a main principle of PM&E, but deciding their roles is another issue to be addressed. Considering the fact that there is a lack of consensus on the roles of local people in the process, three varied forms of PM&E is observed. For instance, the process can be initiated or assisted by external players or professionals hired by donor agencies but having no link with the project. On the contrary, the process can be started and largely managed by insiders, such as field members, community-based groups, or villagers. Or else, the process can be a team work involving insiders and outsiders (Estrella and Gaventa 1998; Campilan 2000). According to Campilan (2000), a joint collaboration has the potential to strike a balance between diverse views in assessing and figuring out changes.
Power Relations
Power relations are regarded as a main challenge in translating true objectives of PM&E into reality. Since, PM&E is constantly acknowledged as a “social process” of negotiation, the issue of power relations unavoidably surface when multiple stakeholders with varied information requirements, priorities and expectations are engaged in the exercise (Guijt 1999).
A major question to be addressed in examining the issue of power is who commands and shapes the PM&E process. For example, in the case of Uganda’s national agricultural extension programme, disagreement surfaced among the project staff and farmers, as well as among the farmers, as the PM&E design was chiefly shaped by donors, project team, politicians and certain farmer leaders. A number of farmers also observed that the project forms caused apparent division between educated and illiterate farmers, both between and inside farmer groups (Parkinson 2009).
Similarly, power disparities can function within the same gender group (De Koning and Martin 1996). The issue is reflected in the case of a project initiated in Kenya by Moving the Goalposts Kilifi (MTGK) to empower girls and young women by means of football. During the planning stage of the project, it was established that uneven power relations functioned among the beneficiaries, as the participants were a combination of female groups having different ages, level of education and status. Well educated girls, senior players and girls representing committees were found to be energetic in debating the indicators for evaluation of change. While, young and less educated girls remained inactive during the process (Holte-McKenzie, Forde, and Theobald 2006).
Methodological issues
Concession between scientific rigor and participation is one of the major issues surrounding PM&E. For example, Estrella and Gaventa (1998) argue that “‘methods, indicators, data collection and interpretation” in PM&E are regarded as devoid of scientific standards. Similarly, the involvement of untrained participants in the process is supposed to decrease the quality and integrity of data. Whereas, quantitative data is usually believed to be rigorous for being factual and representative (Estrella and Gaventa 1998).
However, Guijt (2000) contends that participatory methods are pertinent to knowledge and requirements of local people, and might make possible the evaluation of true changes and empowerment. He argues that to be applicable locally, the approaches are required to be simple and regularly modified than being scientifically complicated.
In order to understand whether a project should involve rigor or participation, it is important to clarify the objectives. For example, qualitative and participatory approaches are needed when the emphasis is on learning. However, if the emphasis is on scientific evidence and policy purposes, traditional approaches might be required (Guijt 2000).
On the other hand, there can also be a combination of rigor and participation, which could be helpful in explaining quantitative data. This approach is necessary when the purpose is to promote policy change (Abbot and Guijt 1997).
Another methodological issue is associated with the development of indicators. For example, differences among the stakeholders may surface when deciding the criteria for and nature of indicators i.e. qualitative or quantitative. The issue necessitates a process of negotiation to select indicators that are related to the context. Moreover, discovering other kinds of indicators to assess non-conventional approaches in development is yet another challenge (Estrella and Gaventa 1998; Guijt 2000).
Conclusion
The above discussion suggests that there are certain issues and challenges which make realization of the core principles of PM&E i.e. participation, learning, negotiation and flexibility, a difficult task. In spite of these hindrances, participatory approaches are widely acknowledged and applied in development programmes for having the capacity to integrate local knowledge and develop the analytical ability of the stakeholders.
Although, PM&E has received a great recognition, the approach is particularly criticized due to ambiguities and complications related to the participation of local people, politics of participation and methodological issues, such as, absence of scientific standards and low quality and integrity of data due to involvement of unskilled participants.
Moreover, PM&E may end up being a simple extractive process if the participation of local people is restricted to data collection exercise only and are left out of the key stages of planning, designing and analyzing. Similarly, difference in the age, education and status of local people is also important factor to be considered to ensure their active participation in the process.
Likewise, objectives of a project must be clear which will be helpful in deciding whether a project should involve scientific rigor or participation or whether it should be a combination of conventional and participatory approaches.