# data[].attributes.body
Understand stakeholders to understand your context
Be inclusive
Keep the focus on opportunities for adaptation
Practical and applied
Be satisfied with plausible evidence instead of PROVEN
Those feeding into the M as E system should be the first to benefit
Proactive instead of responsive / reactive
Usefulness and ownership
Distributed utility
Action oriented
Should monitor for blind spots and unintended consequences

# data[].attributes.body
Very interesting workshop! I learned a lot and hope to apply some of these ideas.

Here is my contact information:
Erik Bloom
Principal Evaluation Specialist
Asian Development Bank
ebloom@adb.org
+63-999-999-6667

# data[].attributes.body
https://smcugandaeval.wordpress.com/

# data[].attributes.attachment_link.readable_text
Disclaimer

This is the working website of the SMC North Uganda evaluation. It contains background materials, field notes, evidence and references.
Notes are provided to give a sense of the conversations that took place, and to put evidence in context. They are not been edited to be a final product. They might contain inaccuracies and glitches, and views that will change. Information, findings will be further revised crosschecking evidence and project reports. Readers are encouraged to comment: to correct inaccuracies and provide clarifications. And to provide more ideas and food for thought!

This is not an official SMC, ADRA or IAS site.

# data[].attributes.headline
Article on addressing business as usual reluctance to use innovative methods such as MasE. 

# data[].attributes.subject
Article on addressing business as usual reluctance to use innovative methods such as MasE. 

# data[].attributes.attachment_link.readable_text
Tanya Beer, Center for Evaluation Innovation
Developmental evaluation requires changing the way evaluation commissioners and evaluators work together. If it feels like business as usual, you're probably not doing it.
On May 31, 2019, Community Science sponsored the webinar "Developmental Evaluation: Rewards, Perils, and Anxieties." A foundation officer (Kristi Ward, Bush Foundation) and two evaluators (Kien Lee, Community Science and Tanya Beer, Center for Evaluation Innovation) discussed what developmental evaluation is intended to be and the realities of practicing it. This article is based on Tanya Beer's introductory comments.

First, let me clarify that this is not an introduction to developmental evaluation (DE). I am assuming you've done some thinking and reading, and perhaps even working, on developmental evaluation. [If not, start here and then come back and read this before you commission or conduct a developmental evaluation.]

I work for a nonprofit that focuses on improving the value and practice of evaluation in philanthropy, where evaluation commissioners have a high degree of freedom to experiment with both their social change strategies and their evaluation approaches. [I suspect, however, that many of the same tensions and lessons below also will apply to public sector DE work, where less flexibility for strategic and evaluative experimentation can make routines and practices even harder to budge.]

These reflections come from my experiences as a developmental evaluator for foundations, and from the Evaluation Roundtable, a network of evaluation leaders from over 100 foundations in the US and Canada. We regularly interview these leaders about their experiences with evaluation, so some of our observations about what gets in the way of meaningful DE engagements come from them.

I'm drawing on a definition offered by Michael Quinn Patton, who literally wrote the book on developmental evaluation. Patton says:

"Developmental evaluation supports innovation development to guide adaptation to emergent and dynamic realities in complex environments by collecting and analyzing real-time data in ways that lead to informed and ongoing decision making as part of the design, development, and implementation process."

It's a meaty definition. Yet we've observed that significant aspects of it are either routinely ignored or swamped by competing evaluation demands and mindsets. As a result, we have a hunch that DE has entered the zone of jargon that - just like "learning" or "collaboration" or "systems change" - has come to represent a wide and muddy range of concepts and behaviors. So it's worth giving the definition some real attention.

First, as Patton regularly reminds folks in his writing and workshops on DE, the primary distinction between DE and other evaluation approaches is a distinction in purpose.

What does this really mean? Is "supporting innovation development" something distinct from applying lessons learned via evaluation to one's program or strategy? An evaluator colleague and I have had long debates about whether DE is a distinct approach or just good utilization-focused practice. After all, you can learn from and use any kind of evaluation.

We've heard many funders say they're looking for a developmental evaluation, but when we probe, we discover they're using the term "developmental evaluation" as a stand-in for "learning." What they really want is an evaluation they find useful for learning, regardless of whether they are working on innovative strategies that are: 1) under development (rather than fully-baked) and 2) expected to adapt in response to an emergent and dynamic context.

We've also seen many evaluators pitch DE by peppering proposals with phrases like "real-time learning" or "complexity," but then they do an evaluation that looks and feels no different from a conventional formative evaluation. They lay their questions and methods out at the start and schedule a predictable supply of "lessons learned" deliverables.

So is there a difference between evaluation that supports learning in general and evaluation that supports innovation development? I think there is.

Our field's hazy distinction around this is problematic because it makes it difficult for commissioners who really do want DE to find evaluators who operate with a clear understanding of how DE is distinct from formative evaluation or just learning in general - and vice versa. Getting clarity on this distinction requires a quick look at the origins of evaluation practice.

The discipline of evaluation was born in a technocratic era of models, projects, and programs, where the nature of change work was about developing a set of components that, if implemented well, would reliably produce the same outcomes across time, locations, and contexts. (I recognize this is reductive and covered elsewhere more thoroughly, but bear with me...)

Now let's differentiate this program-based mindset from innovation, using the definition offered by Michael Quinn Patton, Kate McKegg, and Nan Wehipeihana in their book Developmental Evaluation Exemplars:

"Innovation as used here is a broad framing that includes creating new approaches to intractable problems, adapting programs to changing conditions, applying effective principles to new contexts (scaling innovation), catalyzing systems change, and improvising rapid responses in crisis conditions."

This kind of innovation occurs in dynamic conditions where all kinds of factors interact in ways that we cannot fully predict or, of course, control. And despite our best efforts to draw increasingly complicated theories of change that predict it all, innovative strategies must by their very nature emerge and be adapted along the way. Both the pathway to change and even the outcomes themselves are likely to shift as actors negotiate their way through sticky systems dynamics.

This makes the second important part of Michael's developmental evaluation definition worth pulling out:

"...to guide adaptation to emergent and dynamic realities in complex environments."

Supporting innovation development requires that evaluators bring data and facilitate sensemaking that offer innovators insight about existing and emergent dynamics and how they might shift their innovations in response.
The language many use to *talk* about their work, which aligns with the definition of DE ("Change is complex! Systems are dynamic! We have to learn our way into solutions!"), is often at odds with the way the work actually is designed ("We must have a detailed theory of change before we can design an evaluation. We must plan out the indicators and outcomes and methods and report at regular intervals over the next five years. We must deliver lessons about progress or impact at predetermined moments as determined by our contractual agreement.")

So even though evaluators and strategists know the work is complex and uncertain, the way it is carried out does not always reflect this understanding. This can leave strategists feeling that DE failed to help them innovate and adapt in powerful ways, and the evaluator feeling like the learning they offered didn't matter to strategists' decisions.

The funder isn't really innovating and the evaluator isn't really helping to support it. Everyone is dissatisfied. So what's the problem?

These practices are codified into our evaluation commissioning, designing, contracting, and budgeting routines, and even into evaluation firms' basic business models. They create tough-to-negotiate tensions with DE.

Tension 1: Our routines around setting pre-determined evaluation questions, methods, timelines, and deliverables do not fit with the uncertainty and unpredictability of innovation in systems.

I have seen many evaluation commissioners put out a DE request-for-proposals with all of the evaluation questions already listed, along with a request for applicants to detail their full design, methods, and deliverables.

Additionally, evaluators and evaluation commissioners can find themselves tied in knots at the outset of a DE when evaluators, who likely were trained in a program-based evaluation mindset, ask "What's your theory of change and how clear is it?"

For innovations in complex systems, the theory of change cannot be (or should not be) fully fleshed out at the start. If it's actually innovation, the best a DE evaluation commissioner can say is, "We have some initial hypotheses about where to start and what we want our end game to be. But we won't know how this will unfold until we get into the field and see how people respond to it." This creates a whole different situation for evaluators who are accustomed to crafting full evaluation designs at the outset.

If you start your DE with pre-set questions and a fully developed theory of change, you already have missed the point. You have focused your lens too narrowly and assumed predictability when what we need is the space for questions to emerge and for deploying methods that are responsive to unanticipated questions and decisions.

Just as developmental evaluators need design flexibility, they need budget and scheduling flexibility since they can't chart out entirely how the work will look (if they have, they're not really offering DE). Strategists will need the support of a developmental evaluator with a lot of intensity in some phases of the work and not others. When new roadblocks or challenges arise, evaluators have to collect data rapidly that they were not anticipating. Our traditional contracts with fixed deliverables and timelines generally don't allow for this kind of flexibility. And evaluation consultants' staffing models struggle to accommodate it.

Tension 2: With DE, the need for real-time data is in tension with conventional conceptions of methodological rigor, requiring a constant negotiation of trade-offs between level of certainty and speed.

If strategists actually are approaching their work as innovation in complex systems (again, often they're not when they say they are), DE questions will emerge that we can't anticipate in advance. Let's say we tried something on the ground and encountered unanticipated resistance and we don't understand why. We have to identify quickly what fast-paced methods we can use to get the data that helps us see our way through the dilemma.

Evaluation commissioners and contractors really struggle with DE when they are uncomfortable with small samples and other challenges that limit confidence in the conclusions we're drawing. The amount of time it takes to collect data and do robust analysis means evaluators often miss decision-making opportunities. For DE to work, evaluation commissioners and consultants have to decide together how much certainty is needed versus how quickly they need the information and what trade-offs they're willing to make, all the while being honest with themselves about the potential biases and blind spots that information may have.

Tension 3: Conventional contract accountability mechanisms, such as payments triggered when pre-set deliverables are provided, are in tension with the naturally uneven flow of innovation and DE.

Our current evaluation contracting models are transactional. They purchase products or knowledge at predetermined points in time. We hold evaluators accountable for the deliverables they produce. This hamstrings the evaluator's ability to say, "I don't think what we thought would be useful six months ago is what we need right now."

With developmental evaluation, the work is far more consultative, meaning we're in it together, deciding together what's needed at what point in time. A lot of time is spent doing sensemaking - not in the production of data and deliverables, but in the actual exchange between strategists and evaluators to say, "What does this mean? What are we seeing? What are the patterns? Where should we head from here?"

The accountability mechanism we build into DE contracts needs to look different so that we're not locking evaluators into a specific set of actions and timelines. Instead, we need to think about accountability more like we would for a capacity-building technical assistance provider or a strategy consultant, where contractual accountability is based on the expectation that you're "buying" time and quality thought partnership over the long run.

Tension 4: The inevitable moment when external evaluation users such as foundation boards and leaders ask "what's our impact" is in tension with the focus and fundamental purpose of DE.

Given DE's purpose, the primary users are strategists themselves. I have been in many situations where evaluation commissioners say they want DE, and then halfway through, their board asks for conclusive information about outcomes or impact. We are charged with responding to that need even though our DE lens hasn't been focused on drawing conclusions about impact. We're caught in the middle of competing demands without a clear sense of how to proceed.

We'll always have to help boards and other evaluation users think differently about their accountability roles with innovation and emergent strategies. We have to find ways of setting expectations about what a developmental evaluation is and isn't, and to manage the inevitable questions that will come up about outcomes and impact.

Everyone has that one friend whose house you would NEVER show up at unexpectedly. The one who breaks into the sweats when you add a fifth person to the four-person dinner reservation at the last minute. Sometimes I suspect there are a disproportionate number of those folks in the evaluation field. (And thank goodness that's the case.) But this also means that not every evaluator -nor every funder - is cut out for DE, regardless of their historical experience with other kinds of evaluation or even their commitment to "learning." DE seems to take a certain disposition and orientation towards the unknown.

If you are in an organization that is uncomfortable with ambiguity, or that does not allow for significant shifts and adaptations along the way, you may not be able to use DE in a way that makes it worth the effort. You might get great feedback from DE, but you're too constrained to use it because strategy changes require a long sequence of approvals.

You also need to truly value continuous learning in ways that you can point to operationally in your everyday work. Do you have sufficient resources and staff bandwidth to engage in ongoing inquiry? Are program strategists hungry for information and open to the idea that their assumptions not be on the mark? Will evaluators have a hard time getting on program staff calendars? (In our experience, this is a chronic problem at foundations.) Are you ready to have an evaluator with you regularly at the strategy table listening for questions, dilemmas, and issues that are emerging?

A general interest in learning does not mean that the organization is a good fit for developmental evaluation. We can learn from any kind of evaluation, but the degree to which we're actually innovating, adapting, and being responsive to push-back from the system around us is the thing that makes our work ripe for DE.

Evaluation commissioners have to be transparent and candid with their consultants about the degree to which these conditions are in place. Commissioners and consultants have to talk through how dynamics and mindsets within the foundation might affect the work, and foundation evaluation staff have to serve as internal advocates for getting the developmental evaluator the time and flexibility they need.

Developmental evaluators, too, have to be comfortable with uncertainty and ambiguity. I like being analytical. I like laying things out in sequence and thinking thoroughly and deeply into the distance to clarify assumptions and theories of change. (In fact, I'm not a big fan of people showing up at my house unexpectedly!) Our first instinct is to force our clients to clarify their theory of change so that we can evaluate against it. But with innovation, sometimes strategists only have a big vision, a general sense of what new ways of working or new approaches are needed, and a set of starting hypotheses for how to make it happen. We evaluators need to be able to sit with that uncertainty, to sense when it's time to push for clarity and when (and about what) it's okay for innovators to be unsure.

Developmental evaluators also need tolerance for conflict and disagreement. The evaluator needs to be comfortable pushing back on assumptions and surfacing conflicts in thinking. Evaluators often aren't accustomed to saying, "I think you're thinking about this wrong. Let's go test it." On the other side, for those commissioning developmental evaluations, you have to create and protect that space for the evaluator. Foundations tell us they want evaluators to push back on their thinking, but evaluators tell us that foundations are not often open to that in practice.

Developmental evaluators need to be comfortable participating in strategy discussions. We've all heard that the developmental evaluator functions like a "critical friend." We underestimate how uncomfortable that role can be for both the commissioner and the evaluator.

This role asks us to move far away from our traditional outsider stance, distanced from strategic discussions and decisions. Developmental evaluators need to hear strategy discussions and bring in evaluative information to help people consider in the moment, "How are you thinking about this? What assumptions are you making? How is it unfolding? Given what we know, what might we anticipate about how this will be received?" This role can be uncomfortable for evaluators who feel like they're walking a fine line between being a strategist and being an evaluator.

DE requires different ways of doing evaluation business on both sides of the evaluation contract. Applying conventional ways of evaluating to unconventional ways of strategizing will lead to anxieties, disappointments, and ultimately wasted evaluation potential.

We've learned - often painfully - a few basic tips about how to get out of your own way from the very beginning of a DE:

1. Design Requests for Proposals differently. When creating an RFP for developmental evaluation, don't lay out all of your evaluation questions and ask applicants for a full description of the evaluation design, methods and deliverables. Instead, identify a few early stage questions that are currently puzzling you, and ask for applicants to suggest some ideas for how they might bring relevant data to the table. Focus the bulk of the RFP on understanding how the applicant envisions working with the team over time to identify emerging questions, decide when data collection is necessary, and feed critical information back to the team for join sensemaking in a timely way. How does she/he understand innovation and systems? What has she/he learned about what it takes to navigate organizational dynamics to really integrate DE into the strategic decision making process?

2. Allow for flexible budgeting. Anticipate higher budgets than other types of evaluations, as evaluators need to be much more present at the strategy table in an ongoing way. Don't ask for budgeting detail beyond a rough initial outline (Evaluators can only give line-item detail when they know exactly what methods will be used when). Instead, consider setting aside an amount to be expended quarterly or semi-annually, with the evaluator and you together deciding how those resources should be allocated to particular evaluative approaches or methods as the work progresses and questions emerge. Whatever you do, don't cut the budget for time together in meetings (usually the first thing to go when budgets are being trimmed). Time for the evaluator to listen for hidden or conflicting assumptions and emerging questions, as well as to help the team make sense of what they're seeing and grapple with what the data might mean for next steps, is an indispensable aspect of DE.

3. Increase time and access. Plan for your evaluator to be present at the strategy table far more frequently than for other kinds of evaluation, and an extra amount for the first several months of the engagement as they build a deep understanding of the strategy team's aims and thinking. Invite the evaluator to operate as a core member of the strategy team, albeit with a distinct role. Do not reserve separate end-of-meeting time slots for presentations on evaluation, but rather invite evaluators to bring data and tough questions to the table throughout strategy discussions, grantee meetings, etc., so that strategists don't fall into old patterns of thinking about evaluation as reflecting on the work rather than informing it. Eliminate gatekeeping between evaluators and strategists, grantees, participants/beneficiaries, and other stakeholders. If your evaluation staff person is the main contact for the DE team, you're already limiting the potential for DE to serve the real information needs of strategists.

4. Redesign contract terms. Rather than pinning contract accountability and payment to specific predetermined deliverables, consider designing a contract that triggers payment based on a regular performance review where the program team, the evaluator, and the commissioner's contract manager review the array of evaluative work that has been conducted over the past 6-12 months and have a candid conversation about 1) whether expectations and agreements are being upheld by both sides, and 2) what developments in the innovation the evaluation has supported (see #5 in the consultants advice section below). Note: This is a significant and risky shift for evaluation firms, since the sudden loss of a contract that was expected to last much longer can be devastating to a business, particularly a small firm. Consider building in a "probationary" or ramp-down period if the parties decide the work isn't adding sufficient value. Otherwise it's too risky for the evaluator's ability to maintain necessary cash flow to pay their staff.

5. Plan for the impact question. Anticipate that your board will ask questions about the impact of the systems innovation (probably prematurely) and protect the developmental evaluator from this kind of mid-way switch-up in purpose, questions, user, and design. Remind the board as often as possible about the nature of complex systems innovation and what to expect from developmental evaluation. Consider asking the evaluator to commit some of their resources to track emerging outcomes in preparation for the board, while keeping a significant portion of the budget open for developmental questions (what Patton calls "patch evaluation design"). Alternately, consider engaging a separate evaluator at a different point in time to do a retrospective evaluation that will determine the impact of the innovation on any outcomes that begin to appear.

1. Push back on commissioners. Look for - and push for - all of the above with evaluation commissioners. Don't be afraid to say "I don't think you really want DE" when commissioners are asking for fully designed approaches, line-item budgets, conclusions about impact, and/or are signaling that your access to program strategists will be limited.

2. Experiment with flexible staffing. Consider how your staffing models could be restructured to allow for rapid deployment of unexpected methods, e.g., a floating pool of "rapid response" staff with good methods chops and cross-training, a bank of outside subcontractors at the ready, etc. (though it will be critical to maintain the core team at the strategy table throughout the engagement). If your relationship with the commissioner is solid, consider asking for a consistent payment each quarter, even if the work is more intense in some and less intense in others, tracking so that it all comes out even at, say, the end of the year. (Frankly, we haven't quite solved this challenge either, so let us know what you come up with!)

3. Allow for unknowns and "I have no ideas". Check your habit of asking for fully fleshed-out theories of change at the outset. Instead of getting frustrated with lack of clearly articulated theory or anticipated outcomes on the part of the strategists, consider how data collection and other evaluative work can help people explore what options are possible moving forward in shorter, testable increments. In other words, how can your evaluative support help them work their way towards a TOC by more rapid-cycle testing and feedback. For example, if innovators don't know what will come of action X but they have a hunch it's important, design data collection to help capture the range of outcomes that emerge from action X to determine whether that pathway shows promise.

4. Stop doing midterm and final reports. Instead of packaging and delivering evaluation findings at regular pre-planned intervals, get accustomed to lighter-burden, rapid delivery of insights when the strategy question is actually on the table. This may take the form of a short learning brief focused on an individual question, a quick deck or visualization, a real-time participatory data collection and sensemaking meeting, or even just a phone call to say to strategists, "Here's what we're seeing in the data. What do you think its implications are?" Be liberated from the final report.

5. Keep track of developments. In our experience, it's difficult for people to see how their own thinking and strategy changes over time. As we come to understand more about the system we're working to change, we forget what we used to believe and assume. It can be useful for developmental evaluators to keep a log of key developments in the innovation, as well as the team's rationale for its changes in strategy. This makes visible the strategists deepening understanding of how to make the changes they want to see, and can help you illustrate where and how DE contributed to that growth.

Above all, just like innovative strategists, developmental evaluators and commissioners need to approach their work together as an innovative enterprise...testing, learning, and adapting the evaluative approach as we go.
Tanya Beer is associate director of the Center for Evaluation Innovation and co-director of the Evaluation Roundtable. Email: tbeer@evaluationinnovation.org

# data[].attributes.attachment_link.title
Are you really ready for developmental evaluation? You may have to get out of your own way.

# data[].attributes.attachment_link.description
Tanya Beer, Center for Evaluation Innovation Developmental evaluation requires changing the way evaluation commissioners and evaluators work together. If it feels like business as usual, you're probably not doing it. On May 31, 2019, Community Science sponsored the webinar "Developmental Evaluation: Rewards, Perils, and Anxieties."

# data[].attributes.headline
Question of "do no harm" - how to make sure that the open information shared in the blog is / can not be used against the informants or stakeholders?

# data[].attributes.subject
Question of "do no harm" - how to make sure that the open information shared in the blog is / can not be used against the informants or stakeholders?

# data[].attributes.body
Especially within the context of more repressive, autocratic or hybrid regimes; for non-progressive policy making etc.

# data[].attributes.body
Eric Below: https://youtu.be/ThV4pnPbI8E

Cynefin  framework https://youtu.be/epXqgrm2hs4

David Snowden: How to organize a kids birthday party, according to Complexity Theory.

https://www.linkedin.com/pulse/how-organize-kids-birthday-party-according-complexity-maria-campillo

# data[].attributes.attachment_link.title
Eric Berlow: How complexity leads to simplicity TED talk

# data[].attributes.headline
Disseminating evaluations guidebooks (EC DG INTPA-ESS)

# data[].attributes.subject
Disseminating evaluations guidebooks (EC DG INTPA-ESS)

# data[].attributes.body
https://europa.eu/capacity4dev/evaluation_guidelines/wiki/disseminating-evaluations

# data[].attributes.headline
INTPA (Evaluation Support Services) resources

# data[].attributes.subject
INTPA (Evaluation Support Services) resources

# data[].attributes.body
https://europa.eu/capacity4dev/devco-ess

# data[].attributes.headline
Participatory Process Review: Replication & Consensus Workshop

# data[].attributes.subject
Participatory Process Review: Replication & Consensus Workshop

# data[].attributes.body
What happened? -- Facts

What difference has it made? --Opinions

What should be done in the future -- Recommendations

Get a group together. Have them meet, and discuss. Then post opinions. Then work on recommendations. Vote on the opinions.

Can be done in a week.

But who are the members of the community? Who participates?

Convenient sample. Not a statistical sample.

Try to get a stratified set of people. Maybe gender, disability, age, education, etc. But it depends on the context.

This could be a mixed group.

It could be a "specialized" group; i.e. only older women.

# data[].attributes.body
RTE = the added value of learning before the report comes out

# data[].attributes.body
RTE will give you real time assessment it will show gaps and trends in implementation in which you can act. It will not give you a baseline, not experimental, not counter factual information.

# data[].attributes.headline
Would be great to think / learn about RTEs beyond emergency / humanitarian contexts. 

# data[].attributes.subject
Would be great to think / learn about RTEs beyond emergency / humanitarian contexts. 

# data[].attributes.body
For example within the context of democracy & CS support, I wonder what it would look like. Probably the time frame would be spread over a longer period as opposed to an emergency context. Or is this not RTE and something else?

# data[].attributes.headline
Question: How do you report back? To which clients? 

# data[].attributes.subject
Question: How do you report back? To which clients? 

# data[].attributes.body
And how do you decide on the process of reporting back (w. your internal, external clients)?

# data[].attributes.body
http://serresconseil.com/WASH/Watsanmissionassistant/mainSpace/files/IFRC%20RTE%20Guide%20and%20Procedures%20Working%20Draft%20March-2017.pdf

# data[].attributes.body
https://thecynefin.co/about-us/about-cynefin-framework/

# data[].attributes.attachment_link.readable_text
Cynefin, pronounced kuh-nev-in, is a Welsh word that signifies the multiple, intertwined factors in our environment and our experience that influence us (how we think, interpret and act) in ways we can never fully understand.

The Cynefin Framework was developed to help leaders understand their challenges and to make decisions in context. By distinguishing different domains (the subsystems in which we operate), it recognises that our actions need to match the reality we find ourselves in through a process of sense-making. This helps leaders cultivate an awareness of what is really complex and what is not and respond accordingly so that no energy is wasted in overthinking the routine but they also never try to make the complex fit into standard solutions.

The Cynefin Framework is and has been used in almost every conceivable sector: whether strategy, police work, international development, public policy, military, counterterrorism, safety, energy, healthcare, sales or education it has had something to offer. In this video, Dave Snowden introduces the Cynefin framework with a brief explanation of its origin and evolution and a detailed discussion of its architecture and function.

Cynefin® Beginner Courses - Start Your Journey Today

Embark on a transformative journey with Cynefin® Beginner Self-Study Online Courses. Explore complex systems and discover practical solutions for everyday challenges. Enrol now!

Join Today Here

About the Cynefin Company

The Cynefin Company (formerly known as Cognitive Edge) was founded in 2005 by Dave Snowden. We believe in praxis and focus on building methods, tools and capability that apply the wisdom from Complex Adaptive Systems theory and other scientific disciplines in social systems. We are the world leader in developing management approaches (in society, government and industry) that empower organisations to absorb uncertainty, detect weak signals to enable sense-making in complex systems, act on the rich data, create resilience and, ultimately, thrive in a complex world.

ABOUT US

Cognitive Edge Ltd. & Cognitive Edge Pte. trading as The Cynefin Company and The Cynefin Centre.

© COPYRIGHT 2023

Social Links: The Cynefin Company

Social Links: The Cynefin Centre

# data[].attributes.attachment_link.description
The idea of the Cynefin framework is that it offers decision-makers a "sense of place" from which to view their perceptions.

# data[].attributes.body
https://www.alnap.org/system/files/content/resource/files/main/alnap-from-real-time-evaluation-to-real-time-learning.pdf

# data[].attributes.body
https://www.adb.org/documents/real-time-evaluation-adb-s-response-covid-19-pandemic

# data[].attributes.attachment_link.readable_text
Evaluation Document |

IED is conducting an assessment of ADB's response to the coronavirus disease (COVID-19) pandemic. The evaluation will be formative, with the goal to help shape ongoing and future support to developing member countries (DMCs). Because the evaluation will be undertaken in real-time, while the pandemic is ongoing and during the implementation of ADB operations, IED is mindful of the pressures that ADB teams and DMC governments are facing. The evaluation will focus on assessing ADB's support at the implementation stage and identifying early lessons.

The purpose of the evaluation is to provide ADB's Board of Directors and Management with timely evidence on the implementation of ADB's support to address the health, social, and economic crises brought about by COVID-19. The evaluation will provide emerging lessons on the relevance of ADB's support, the design and implementation of its operations, and the organization's preparedness, to inform possible future COVID-19 support. The evaluation will analyze a range of evidence gathered from interviews, data analysis, reports from DMCs, literature and project document review, data mining, and country-level assessments.

# data[].attributes.attachment_link.title
Real-Time Evaluation: ADB's Response to the COVID-19 Pandemic

# data[].attributes.attachment_link.description
IED is conducting an assessment of ADB's response to the coronavirus disease (COVID-19) pandemic. The evaluation will be formative, with the goal to help shape ongoing and future support to developing member countries (DMCs).

# data[].attributes.body
Reality differs from our model.

Linear: "If, then." Push leads to movement. "You can put it back together"

Complexity: Reaction, action. Pushing, pushing back. "If you move it, you break it."

"Three-body problem"

# data[].attributes.body
https://socialinnovationsjournal.com/index.php/sij/article/view/704

# data[].attributes.attachment_link.readable_text
Authors

Scott Chaplowe Evaluation and Strategy Specialist, International Evaluation Academy (IEA), Transformational Evaluation Working Group Coordinator, www.scottchaplowe.com

Adam Hejnowicz Research Associate, School of Engineering, Newcastle University; Visiting Research Associate, Department of Biology, University of York

Keywords:
evaluation, transformational change, project fixation, quantitative fixation, accountability fixation

Abstract

The call for transformation is a response to the dire global emergency; it is a call for radical innovation at multiple levels if humanity is to survive into the next Century. How can evaluation, a profession in the business of assessment and advising, inform and hasten transformation? As a field that straddles both theory and practice, evaluation is uniquely positioned to support transformational learning and change, but this potential depends on its ability to transform from within. This article identifies four interrelated "boxes" that confine evaluation's transformational potential: a project fixation, a short-term temporal fixation, a quantitative fixation, and an accountability fixation. It also examines the uptake and influence of complex systems analysis in the field of evaluation as a means to "breakout" of these boxes and nudge evaluation towards the inner transformation required for it to contribute to transformational change.

Author Biographies

Scott Chaplowe, Evaluation and Strategy Specialist, International Evaluation Academy (IEA), Transformational Evaluation Working Group Coordinator, www.scottchaplowe.com

Scott Chaplowe is an evaluation and strategy specialist with over 22 years' experience working with civic, public, and private organizations promoting socially and environmentally responsible solutions for sustainable development. His background in Geography, with a focus in cultural and political ecology, informs his specialization in complex systems analysis and adaptive design and evaluation. Email: Scott@ScottChaplowe.com

Adam Hejnowicz , Research Associate, School of Engineering, Newcastle University; Visiting Research Associate, Department of Biology, University of York

Adam Hejnowicz, PhD, is a postdoctoral research associate in the School of Engineering at Newcastle University and a visiting research associate in the Biology Department at the University of York. He works with the York Environmental Sustainability Institute (YESI) and is a member of the Centre for the Evaluation of Complexity Across the Nexus (CECAN). Email: adam.hejnowicz@newcastle.ac.uk

# data[].attributes.attachment_link.title
Evaluating Outside the Box: Evaluation's Transformational Potential

# data[].attributes.attachment_link.description
Scott Chaplowe Evaluation and Strategy Specialist, International Evaluation Academy (IEA), Transformational Evaluation Working Group Coordinator, www.scottchaplowe.com Adam Hejnowicz Research Associate, School of Engineering, Newcastle University; Visiting Research Associate, Department of Biology, University of York evaluation, transformational change, project fixation, quantitative fixation, accountability fixation The call for transformation is a response to the dire global emergency; it is a call for radical innovation at multiple levels if humanity is to survive into the next Century.

# data[].attributes.body
At some points we have to account for the cost of not used data.☝️

# data[].attributes.body
Monitoring-- sensing. WHAT?
Progress towards an objective. A form of "enumeration."

Evaluation-- making sense. WHY?
Evaluative judgment. More akin to "research."

# data[].attributes.body
Marry the twins that were separated!

# data[].attributes.body
Plans are important. But rigid plans don't always work.

Does that mean that we choose an objective? And then figure out how to achieve it at the end of the process. Or is that just linear thinking? Is there a role for monitoring outputs? Is it important?

TRUST is also important to let go of the "rigid control." If we don't have TRUST, then it is hard to give up rigid planning (principal-agent problem?).

Does evaluation promote change? Or is it a delaying/conservative factor?

"Fear of evaluation" may lead to more rigid planning.

# data[].attributes.body
Evaluation now of often contributing to the status quo and a road block to transformational change

# data[].attributes.body
Challenges are COMPLEX; they are often non-linear.

Our logic models tend to be simple.

The new "norm" may not exist, because we don't reach equilibrium.

# data[].attributes.body
Transformation has winners and losers. It may benefit some but not others.

It may be fast or it may be slow.

# data[].attributes.body
Transformational change. The opposite of incremental change.

Not something that happens every day.

# data[].attributes.body
Typically unplanned non lineair and NOT a project. It is the result of a combination of factors and actors. It can be positive OR negative

# data[].attributes.body
You can share in this padlet resources, ideas, Eureka moments, insights, comments.

# data[].attributes.headline
On emergence: "The planets are weirdly in sync - Steve Mould"

# data[].attributes.subject
On emergence: "The planets are weirdly in sync - Steve Mould"

# data[].attributes.body
https://youtu.be/Qyn64b4LNJ0

(On how complex systems may have organised and structure cycles)