Monitoring and evaluation for learning 2, 2016 by Helena Wiklund https://padlet.com/helena_wiklund1/ujf6mn1q07y1 What are the main challenges to M&E in the context of capacity development projects? Add your thoughts! en-us 2016-09-16 11:20:43 UTC 2026-01-26 03:18:53 UTC hello@padlet.com https://padlet-uploads.storage.googleapis.com/53247602/f080f772cb156c9b8b35130e606b5612a080a23b/2436f2154b5a4652e88412443172c4ec.png Challenges to M & E for learning in CD projects (Peter Medway) https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/132608861 In general the measurement of soft capacity development, for example the effectiveness of leadership, or the strengthening of collaboration in a network, is more difficult than measuring quantitative outputs. However, the course has made the case for the application of non-linear learning processes that can cope with measurement in an unstable and changing context. Methodological pluralism is encouraged. In many project settings the skills and experience to manage such an approach may be lacking and, given the commitment of time and resources needed to be continually reflective, maintaining all stakeholders commitment  may be difficult. One major challenge for M & E for learning is that findings cannot be easily generalised - each context is unique - so it can be difficult to apply our insights more broadly to policy and practice, creating a positive impact on the enabling environment. Further, the accountability vs learning paradox can create challenges. Most development workers today are well-versed in and accept the requirements of (mainly upward) accountability, However, an unbalanced bias towards accountability can make projects more risk averse and less transparent, in direct opposition to the values required for a successful learning approach, where failure can represent a great learning opportunity. Skills, culture and value changes are needed for all stakeholders in a CD project in order to really benefit from the M & E for learning approach.

]]> 2016-10-24 06:24:25 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/132608861 How leifandren https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211360 What kind of tools are you going to use?]]> 2016-11-13 18:16:41 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211360 When leifandren https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211447 Determine a timeschedule for M&E]]> 2016-11-13 18:17:30 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211447 Who leifandren https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211548 Agree with all stakeholders who is going to perform M&E]]> 2016-11-13 18:18:41 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211548 What leifandren https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211643 Decide, toghether, what to M&E]]> 2016-11-13 18:19:53 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211643 Agree leifandren https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211763 What are we going to do with the results from the M&E]]> 2016-11-13 18:21:00 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211763 Disagreements leifandren https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211894 How do we handle disagreements]]> 2016-11-13 18:22:50 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/137211894 Module 12 – Monitoring & Evaluation for Learning Thoughts Author: Thomas McKnight 17 November 2016Monitoring and Evaluation’s emergence and entry into the Program management cycle scheme beginning in the late 1980s and early 1990s and gradual recognition and incorporation as a professional discipline into the humanitarian agency system though having evolved as a specialized skill set, tasked to meet donors’ demand for quantifiable humanitarian return on investment (HROI) reporting, represents a continued management science evolutionary track progression impacted by ongoing framework, policy, and systems lessons learned adjustments. I raise the matter of current M&E staffing within our aid delivery institutions – some dedicated to development with access to relatively stable enabling environments and functioning at individual, organizational and institutional levels where peace, as a prerequisite are conducive to capacity development vs. complex emergency environments where one plants an opportunistic capacity development seed of hope where possible; where peripherally based clients and partners lack knowledge or training opportunities; fail to collect and report disaggregated age and gender data for numerous reasons. In many institutions M&E representation is either limited, non-existent or overstretched to support the quantitative cluster system and other stakeholders’ needs. South Sudan, divided by political, ethnic and tribal conflict is an example. As we explore CD frameworks with expectation to develop capacity in this latter setting I question as critical as quantitative knowledge, skills and capacity building is it will be awhile, if ever before implementing M&E for Learning becomes cross operationalized. The main challenges in this context is having an environment peace and stability. On the other hand Somaliland (Hargeisa) having seceded from greater Somalia since 1991; formed their own government; adopted a relief-recovery-development continuum tract, possesses a National Development Plan and vision that “by the year 2030, Somaliland will be contributors of aid instead of being recipients of aid’ created a secure and stable enabling environment. Of the 26 government ministries and a micro view of the MoNP&D’s five departments (Admin), (Planning), (Statistics), (Development) and (Coordination) under its jurisdiction, comprising 21 offices, the following offices highlight part of the enabling environment infrastructure that guided the government’s transformation from relief to recovery and placed the country on a development track. Notwithstanding ongoing legislative, policy and systems reform and development activity summarize Hargeisa’s current investment and development climate. • Procurement & Logistics• Accounting & Finance• Personnel ***• National Planning & Policy ***• National Planning Commission Secretariat ***• Institute for Statistics and Public Policy Research• Census and Surveys• Social Statistics• Management Information Center• GIS• Regional Planning and Development• Regional Planning & Development• Regional NGO Coordination & Development• Regional Statistics • Monitoring & Evaluation Though unrecognized as a sovereign nation by the international community this is a unique ‘M&E for Learning’ opportunity, having case study – adaptive learning and replication potential for Greater Somalia (Mogadishu), Puntland (Garowe) and other East African, Horn of Africa neighbors. With traditional M&E quantitative systems having been implemented and overseen by the Ministry of National Planning and Development in collaboration with UNDP Somalia, bench marks, lessons learned experiences, inventories of public, private stakeholders and institutional memory to draw upon only strengthen future capacity development initiatives. This is however dependent upon this location remaining stable and its government administration team remaining in office and/or processes handover with minimal alteration. The UNDP manual on measuring capacity that provided the framework for M&E’s quantitative blueprint would equally contribute to the M&E for Learning capacity building initiative as a continued policy, practices, and systems evolutionary process. https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138593331 2016-11-18 00:46:21 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138593331 https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138595306 2016-11-18 01:03:40 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138595306 Key challenges of M&E in developing countries in Asia are as follows (Shesh Kafle): https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138911932 - Less emphasis on M&E while designing and implementing CD programs
- No enough budget, staff and time allocated for M&E
- No follow up with the findings received from M&E processes
- No adoption or follow of M&E findings in new project design. ]]> 2016-11-20 06:57:14 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138911932 Challenges of M&E https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138913087 2016-11-20 07:39:43 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138913087 Chris Jung https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138945692 The main challenges to M&E are more about purpose and politics than technical concerns. I took two key points from the module presentation and readings, and both are related to why M&E is important and how to choose the right methodology based more on context than technical paradigm.

First, there has been an important shift in priorities for M&E away from a quantitative tool to satisfy donor accountability. Expanding the purpose of M&E to include learning for improvement is a key difference in philosophy, but I have not yet seen this well executed in CD projects. Too often, M&E is still used to validate a project and justify future funding, rather than as a central way to dissect failures and create lessons learned.

The second, and perhaps more difficult, challenge is that the long timeframes necessary to see CD impact mean that methodological pluralism is crucial to examining both short and long-term impact. The ODI Background Note brilliantly sets out the need for a varied range of M&E methodologies and approaches. It also explains why the CD process is heavily political, therefore the impact will sometimes require a change in political climate before being realized. If a CD project shifts the enabling environment so that stakeholders can take advantage of a window of opportunity several years in the future, this is an important impact to capture. But it's not one that traditional or even modern M&E tools are designed to measure.]]> 2016-11-20 17:14:19 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138945692 Soso Bagashvili: The challenge is that less emphasis are made on learnings from any applied important processes in capacity development, that will translate in improved individual, organizational, institutional behaviors in a longer-term. Measuring processes can be overlooked in M&E frameworks and plans, or simply there may not be sufficient resources available to monitor or evaluate the efficiency and effectiveness of applied processes. Therefore, that should be given sufficient attentions thus resources and reflected in M&E frameworks and plans. That can improve the overall effectiveness of similar undertakings in the future in similar contexts. Although contextualization of the processes remains important. In addition, measuring positive impacts made on the institutional capacities and enabling environment may be another challenge for those organisations involved in capacity development, that may not necessarily remain in the operational area or country for sufficient period of time for such understakings. Therefore, right mechanisms and counterparts need to found and capacitated in a timely manner in order to enable those to reflect and measure such impacts. Such measurements may be done remotely through partners on the ground, thus help applying the learnings in practice in the future. https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138966336 2016-11-20 21:42:20 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/138966336 Contribution to knowledge building-Ghada https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139074547 how to use the outcomes of the M&E to contribution to knowledge building ]]> 2016-11-21 13:24:25 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139074547 Faisal Ridwan https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139111474 The main challenge to M&E in the context of capacity development projects that I faced on my disaster-recovery missions was that the top management (which was related to accountability to donor) was pushing to focus on the quantitative result which created a risk-averse culture. Problems are considered as failure and need to be hidden, instead of using it as a valuable lesson for capacity development.]]> 2016-11-21 14:59:54 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139111474 Main challenges (Thorsten) https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139162518 From my perspective, and having worked together with CD experts in quite a number of projects, I see challenges and difficulties on various levels:

1. What is to be monitored? In many cases an "M&E Expert" was sent by the country office to the field office, usually when the project was already going on (which in itself is an indicator for the importance of M&E). In one case, this expert had his own conception about the indicators, which was probably derived from the logframe in the funding application but during time had changed.

2. When things do not go as planned, especially in the United Nations this is often seen as a problem, and as Faisal writes, as failure. Which may impact the UN person's promotion scheme and thus is not very popular.

3. The attitude of the CD experts deployed plays a big role. There is a definite partition between "death by Powerpoint" and CD. A (not so) fictitious dialogue in a project a couple of years ago: 
CD Expert A: "Yay! 200 community leaders trained in the use of disaster risk maps today!" (gives CD expert B a high-five).
Disaster Waste Expert: "200, in a day. Wow. How do you know all of them understood what you explained?" 
CD Expert A: "Err, sorry?"
Disaster Waste Expert: "Are you sure all of them are able to read a map?"
CD Expert A: walks away.

4. Concentration on figures instead of impacts by M&E experts deployed to the teams on the ground, based on a logical framework analysis (LFA) or results-based management (RBM) approach. Outcome mapping looks at changes in behaviour and long-term impacts beyond the duration of the project, but at times may be incompatible with an M&E system established or required by a donor.]]> 2016-11-21 17:26:57 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139162518 Tova https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139225832 As I see it a challenge is the knowledge limitation. You can use a lot of resources and time to get an comprehensive report but you will need skilled individuals and like I said resources. On the other hand you can do an overview which gives you the scope but not the full picture. It would cost less and perhaps not be as complicated.
But another big hurdle is how to actually use and communicate the gathered information. As I see it there is an idea in some organization not to admit to mistakes and make everything to a success story. Thus hoping to get more funding from donors in the future. However by not admitting to ones mistakes how can you learn?

]]> 2016-11-21 21:57:00 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139225832 What https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139730769 What is needed?]]> 2016-11-24 10:23:03 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139730769 Who https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139730884 Who is the recipient?]]> 2016-11-24 10:23:34 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139730884 When https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139730996 When is it needed?]]> 2016-11-24 10:24:04 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139730996 Before https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731153 How is the situation before?]]> 2016-11-24 10:24:46 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731153 During https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731291 Can you see any impact, contribution, obstacles??]]> 2016-11-24 10:25:24 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731291 After https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731431 Did the project make any change?]]> 2016-11-24 10:26:00 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731431 How https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731743 How do I get all stakeholders involved?]]> 2016-11-24 10:27:29 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731743 Misunderstandings https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731953 How do I handle misunderstandings?]]> 2016-11-24 10:28:32 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139731953 Fundings https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139732196 How do I handle the situation if there is a risk that the budget will be overdrawn?]]> 2016-11-24 10:29:42 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/139732196 Module 12 sid4 https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141625260 2016-12-04 21:31:25 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141625260 Sidney Tupper- Effective M&E effort in a project will consume a significant portion of its budget so planning for it must be specifically and adequately included at the project design stage. In an environment of scarce resources, the temptation will be to allocate more to activities and less to M&E.- M&E requires skills built from specialized training, experience and domain knowledge, which project implementers also need, so availability of M&E practitioners may suffer. On the opportunity side, the project independence that M&E should have can be met by older practitioners with long experience in project implementation.- The LFA approach which designs M&E into a project by defining performance measures and expectations may make project execution less flexible in situations where scope, context and/or conditions change unpredictably. The challenge is to express M&E criteria to accommodate change. Theory-based evaluation may be more appropriate but has its challenges too, e.g., difficult-to-manage complexity.- Results-based evaluation ultimately should include impact assessment but impacts often don't manifest for years after the activity outputs are complete, when no resources remain to do the assessment. sid4 https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141625899 2016-12-04 21:39:52 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141625899 Sidney Tupper sid4 https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141626269 S
  • Effective M&E effort in a project will consume a significant portion of its budget so planning for it must be specifically and adequately included at the project design stage.  In an environment of scarce resources, the temptation will be to allocate more to activities and less to M&E.
  • M&E requires skills built from specialized training, experience and domain knowledge, which project implementers also need, so availability of M&E practitioners may suffer. On the opportunity side, the project independence that M&E should have can be met by older practitioners with long experience in project implementation.
  • The LFA approach which designs M&E into a project by defining performance measures and expectations may make project execution less flexible in situations where scope, context and/or conditions change unpredictably.  The challenge is to express M&E criteria to accommodate change. Theory-based evaluation may be more appropriate but has its challenges too, e.g., difficult-to-manage complexity.
  • Results-based evaluation ultimately should include impact assessment but impacts often don't manifest for years after the activity outputs are complete, when no resources remain to do the assessment.
]]> 2016-12-04 21:45:07 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141626269 - Effective M&E effort in a project will consume a significant portion of its budget so planning for it must be specifically and adequately included at the project design stage. In an environment of scarce resources, the temptation will be to allocate more to activities and less to M&E.- M&E requires skills built from specialized training, experience and domain knowledge, which project implementers also need, so availability of M&E practitioners may suffer. On the opportunity side, the project independence that M&E should have can be met by older practitioners with long experience in project implementation.- The LFA approach which designs M&E into a project by defining performance measures and expectations may make project execution less flexible in situations where scope, context and/or conditions change unpredictably. The challenge is to express M&E criteria to accommodate change. Theory-based evaluation may be more appropriate but has its challenges too, e.g., difficult-to-manage complexity.- Results-based evaluation ultimately should include impact assessment but impacts often don't manifest for years after the activity outputs are complete, when no resources remain to do the assessment. sid4 https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141626934 2016-12-04 21:56:32 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141626934 M&E challenges sid4 https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141627528
  • Effective M&E effort in a project will consume a significant portion of its budget so planning for it must be specifically and adequately included at the project design stage.  In an environment of scarce resources, the temptation will be to allocate more to activities and less to M&E.
  • M&E requires skills built from specialized training, experience and domain knowledge, which project implementers also need, so availability of M&E practitioners may suffer. On the opportunity side, the project independence that M&E should have can be met by older practitioners with long experience in project implementation.
  • The LFA approach which designs M&E into a project by defining performance measures and expectations may make project execution less flexible in situations where scope, context and/or conditions change unpredictably.  The challenge is to express M&E criteria to accommodate change. Theory-based evaluation may be more appropriate but has its challenges too, e.g., difficult-to-manage complexity.
  • Results-based evaluation ultimately should include impact assessment but impacts often don't manifest for years after the activity outputs are complete, when no resources remain to do the assessment.

    • ]]>     2016-12-04 22:05:59 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141627528     Challenges I have faced - Per Velandia https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141858695 Based on my experience I have faced a few challenges that I will try to summaries below


    Lack of ownership by main stakeholders - Several times I have faced an overall acceptance for evaluations, but when push comes to show, the actual buy in for learning from the results of the evaluations and willingness to implement the recommendation has overall been low. This may be a lack of inclusion of stakeholders which has lead to a low true buy-in. Also, M&E has been seen in isolation and an add-on, rather than an inclusive part of a project implementation.

    Unclear project objectives and lack of indicators - The M&E processes that I have been part of myself has often faced that the projects that are supposed to be monitored have had a lack of clear objectives and non-existing indicators, and lack of baselines. This has lead to difficulties to actually prove that change has happened and whether the project has been really successful or not. Furthermore, CD elements in humanitarian projects have most often been limited to training of individuals. This has often resulted in anecdotal discussions on what was good and was less successful, without clear link to actual results achievement (e.g. focus has tended to be put on inputs and outputs and not much focus on outcomes and impact). 

    Sensitivities - At times I have also faced (as the presentation and also Chris Jung mentions) that M&E is often a way to please donors rather than activities with the aim to promote active learning environments. Problem identification can also often lead to sensitivities and I have more and more started to use the approach focusing on what we did good. An aspect of this, is that there is a risk that MSB partners may be honest in their reflections as they may believe that it may affect the possibility for further engagement from MSB.  

    Methodology - I have most often face very limited knowledge and capacity in terms of methodology for M&E. E.g. several monitoring activities and evaluations have major flaws in the methodology which hampers the usefulness of these processes and products. This can also be explained by the fact that several monitoring systems/approaches/tools have been used for similar projects, which has lead to low possibility to compare results between projects.  


    ]]>     2016-12-05 19:26:13 UTC https://padlet.com/helena_wiklund1/ujf6mn1q07y1/wish/141858695