# data[].attributes.attachment_link.description
Design Kit is IDEO.org's platform to learn human-centered design, a creative approach to solving the world's most difficult problems.

# data[].attributes.attachment_link.readable_text
Co-design is a powerful way to improve design processes in two ways: speed and quality. When your work environment allows and encourages people to work remotely in a temporary or permanent way - as we do at Optimal Workshop - we need to get creative if we want to find ways to include everyone.

After a research project, my marketing team was looking at redesigning our Enterprise landing page. We wanted to speed up our experimentation and determined that collaboration was the best approach. Ideation exercises require more than just a team of designers - we needed to involve the right people with the right knowledge. To generate more diverse, creative and insightful outputs, we invited different personalities as well as different roles. Our team was comprised of a mix of sales, customer success, marketing, developers and designers. All located in different locations around the world, including the United States, New Zealand, France and Ireland.

To make this global session happen, we used Mural, a digital workspace for visual collaboration. It's a great way to explore ideas in a collaborative space from anywhere with an internet connection and a laptop.

What is a co-design?

A co-design is a fast-paced workshop that generates abundant and diverse ideas beyond what just one individual can produce.

What does it involve?

A clear plan

Define the problem you want to solve, articulate user needs and any pain points you've identified. It's also important to explain why this problem is something worth investigating. This preparation work will help to cut back on inefficient conversations and ensure your team and stakeholders are aligned on goals, opportunities and problem statements.

HMW (How Might We)

HMW questions are a way to frame the challenge you are trying to solve using short, compelling questions to spark creative thinking and discover solutions.

Divergence

Diverging is a quantitative process where you create as many ideas as possible by answering the HMW questions. Each member then presents their ideas while the rest of the team writes their feedback on Post Its in two ways: 'I like' - what you like about the solutions and 'I wonder' - how you might improve the solutions.

Convergence

Converging is a qualitative process. Once everyone has presented their ideas and looked over the feedback, it's time to consolidate the best designs into one solution. After presenting their solutions, team members vote on their favorite ideas or the most compelling features by placing stickers on the ones they like. Commonly, each person should have 3 stickers.

What do you want at the end of the co-design?

Once you've passed through the divergent and convergent stages for your HMW questions, you'll hopefully have some key areas of impact to focus on that are worth prototyping in the form of a wireframe or a higher fidelity design. Using this process, you should be able to develop a design that incorporates input from all disciplines and departments.

Tips for running a co-design:

Plan ahead

8 people max per workshop

Prior to creating a calendar invite, ask who would be interested to participate via a message or in person. Allow teams to choose who will represent their team.

Have your low-fidelity sketching supplies ready: Post-its, markers, blank sheets etc

Setting the context and explaining the rules

Write a description within the calendar invite to give some context prior to the meeting. e.g "What we are working on?", What will we do with the output?" and "What if I am busy and can't come?".

Send reminders the day before to make sure people are registered to the software you might use, specify if they need to bring anything specific(laptop, tablet, etc).

Adding a "What's coming" section to the agenda helps people to stay on track.

Set a clear presentation to encourage creativity - inspire your team!

A timer

A mechanism for tracking time down to the minute is crucial for keeping track of the work achieved.

Schedule a good amount of time for feedback and voting.

Allow a 5 to 10 min break time.

Involve a second facilitator to help with time tracking.

How we ran a remote-friendly co-design

What worked well?

True effort and planning

The classic paper/post-it system isn't suitable for remote workers. Despite video conferencing, it is still hard to hear everyone or read a whiteboard. I wanted a tool that would allow me to include everyone the same way, and allow easy follow up on the achievements. This system needed to work even for people who couldn't attend or had to leave halfway through the meeting.
Feedback received:

"It takes real effort to get these up and running and the use of other software to make use of the time of everyone"

"I can't imagine doing workshops in any other way in the future"

Communication with remote workers

Avoid questions like "please vote if you are okay to come in early" which makes remote workers feel bad.
Feedback received:

"Remote workers feel like they're a burden in meetings, but you made us feel super included"

What could have been done differently?

Time and people management

Handling time and people is tricky, you can easily run out of time and be forced to rush through the last steps of your meeting. Don't hesitate to ask for help facilitating the co-design.
Feedback received:

"More time to give feedback to everyone's ideas would have been nice"

Remote organization

Find a way to be as inclusive as possible, like finding a remote solution for snacks and drinks.
Feedback received:

"Consider snacks for overseas people or let us know that we should get something to be included in that"

Tips for running a remote-friendly co-design:

A well-chosen space and time

Involving people from multiple countries or time zones requires preparation and consideration. Don't be afraid to ask people to arrive earlier than their standard working hours.

Collaborative tools

Use inclusive tools during meetings that allow everyone to follow the work being done. Also making sure that the tool makes it easy for people to come back and review the work later if they can't attend the meeting.

Set up aMural board that will replace your slide presentation. Everything should live in same place including Context, HMW, dedicated places to upload images etc.

Make sure everyone knows how to use all of the tools needed. It might help to start with a small exercise to help everyone get familiar with it. If there's not enough time for it, set that exercise prior to the meeting (maybe as part of your reminder).

Manage noises

Depending on the number of participants, background music can prevent remote workers from hearing and understanding conversations.

Avoid background music.

Manage people and time to avoid banter and chit chat where possible..

Be inclusive

Make sure everyone feels like they're part of a team rather than a burden for being remote. Allow time for questions, repetition and clarification.

Avoid questions that make remote workers feel bad like "please vote if you are okay to come in early".

Allow time for questions, repetition and clarification.

If you plan some snacks and drinks, either include it in your workshop budget or let the remote workers know they can bring their own food (depending on timezone).

# data[].attributes.attachment_link.title
How to run a remote-friendly co-design - Optimal Workshop

# data[].attributes.attachment_link.description
Co-design is a powerful way to improve design processes in two ways: speed and quality. When your work environment allows and encourages people to work remotely in a temporary or permanent way - as we do at Optimal Workshop - we need to get creative if we want to find ways to include everyone.

# data[].attributes.attachment_link.readable_text
Setting context and explaining rules (5-10 mins)

This will be an unfamiliar style of exercise for many. It may be important to set up the event by sharing some stories and perspectives on what's about to happen, in order to:

Establish credibility: Feel free explain where these methods come from. (Google Ventures, Adaptive Path, IDEO)

Encourage creativity: Participants may be uncomfortable being asked to contribute actively and creatively. Referencing work like David Kelley's TED talk can help.

Explaining the flow: Participants may need to see "what's coming" in the agenda so they can prepare for this participation style. Feel free to offer a peek at this agenda in summary form.

2. Forming teams (3-5 mins)
Dividing people into teams can take a lot of time, so know your method beforehand. Decide whether you'll ask people to self-organize by certain traits ("find your shoe tribe" - connect with people wearing similar shoes), or number people off. Aim for three people per group. (Often, I'll create groupings in advance: not all expert designers in one group, not all clients in one group, not all workshop-newbies in one group.)
3. Issuing the brief (3-5 mins)
It is crucial to have utmost clarity on the brief in advance. Deciding on the exact scope of what you're asking people to redesign will make sure the activity is achievable. The "brief" may put forward a specific product to improve (with screenshots of a single screen), or issue a general challenge you want people to address (the specific scenario or user journey faced by your audience). You may wish to create post-its or posters in advance for the wall, outlining the scenario at hand.
4. Discovery (10 mins)
This is a rapid version of the Interview Matrix facilitation method. The aim is to have people interacting with real users-a key practice in Design Thinking.

Pick a question from Step 3, such as, "What do you need most to accomplish this task?" or "What are the key pieces of information that will help you do this?" Groups then split up in a lightning-round of asking questions and taking notes on the insights that arise.

If you don't have access to actual users during this workshop, ask participants to imagine that they themselves are the users in this journey. Ask them to think of all the steps, pieces of information, or tools they would need in order to complete this journey and to record their responses. These notes can be unprioritized and loosely organized.
5. Synthesis and prioritization (10 mins)
Instruct teams to regroup and collate insights into a single sheet. Teams are encouraged to group similar items, and place asterisks beside the top 3-5 must-have insights they believe users can't do without.
6. Crazy Eights (6 mins x 2)
Crazy Eights is a well-documented exercise (based on one called 6-8-5), perfect for generating creative output quickly. Working as individuals...

Have each person fold an 8x11 piece of paper into eight panels. (Or four, double-sided, if you want more space.)

Set a timer for 6 minutes, marking 40 second intervals

Use those 40 seconds to scribble out a sketch or idea in a single panel. It can be a button, a feature, a user scenario, a tagline...wherever your mind goes.

You'll fill out eight panels (hence the crazy and the eight), front and back.

Your ideas will be bad, obvious, simple, messy and confusing. That's the point.

7. Team Sketches (15 mins)

Teams now come together and compare their Crazy Eights drawings.

Groups discuss and explore the ideas from the drawings, and come to an agreement on the top features or ideas they want to include in one consolidated master sketch.

Groups sketch a single layout that incorporates the top ideas. (You can draw an outline of a smartphone, tablet or laptop to serve as the frame.)

8. Flamethrower Rounds (10 mins per round)
At the end, take a look at what you drew, and start the timers again, with another set of 8 panels. Expand, explore, push, scribble, generate, play. It'll make sense later. (And if you want even more detail on Crazy Eights, here you go.)

Time to create a space for speed-critiques. A member from each group is sent to another group...

a. The host group has 3 minutes to present and pitch their sketch

b. The visitor has 3 minutes to ask any questions

c. The hosts turn their back and take notes as the visitor "flame-throws," tearing the idea apart critically, mentioning any bad ideas they notice.

While feedback is being given, host teams record feedback on a "feedback grid": a quadrant with each square labeled Plus, Minus, Ideas and Questions.
This is worth repeating:

Yes, the hosts should actually turn their backs. It sends an extraordinary signal about being on the same side, and being impervious to the criticism.

Yes, recording feedback on the grid is a valuable step.

9. Revisions (10 mins)

10. Pitches (or "Gallery") (20-30 mins)
(You can do this step twice to increase the amount of feedback.)

The teams regroup and share their insights and feedback from the Flamethrower found. Making any changes they see to be necessary, the group redraws their product idea as a master sketch.
11. Silent Critique (5-10 minutes)
Teams now pitch their product/screen/redesign to the entire group, in 3-minute pitches.
12. Debrief (20 mins)
It's important to keep to this time limit. Depending on the number of participants, people can be tempted to talk about their drawings for quite some time.

Which features or ideas stood out? Which are most intriguing and exciting?

What seems most challenging and difficult to achieve?

Where do we see areas that are clear no-go's?

What must-have's are we seeing?

What was a challenge? What conflicts emerged?

What do we want to test with real users?

How did this feel for you as a participant?

Hang all the final drawings on the wall. Participants use colour-dot stickers, post-its or sharpies to indicate the features or ideas they were most drawn to.

The entire group discusses what common themes emerged:

# data[].attributes.attachment_link.title
The Co-Design Workshop: The Facilitator's Pocket Guide

# data[].attributes.attachment_link.description
Setting context and explaining rules (5-10 mins) This will be an unfamiliar style of exercise for many. It may be important to set up the event by sharing some stories and perspectives on what's about to happen, in order to: Establish credibility: Feel free explain where these methods come from.

# data[].attributes.attachment_link.readable_text
Designing with your users and stakeholders.

Photo by You X Ventures on Unsplash

Help users and stakeholders to actively contribute to the design process and give their valuable insights and feedback.

"Participatory design integrates two radical propositions about design. The first is the moral proposition that the people whose activity and experiences will ultimately be affected most directly by a design outcome ought to have a substantive say in what that outcome is. The second is the pragmatic proposition that the people who will need to adopt, and perhaps to adapt to an artefact or other outcome of design, should be included in the design process, so that they can more offer expert perspectives and preferences regarding the activity that the design will support, and most likely transform.
Carroll and Rosson, 2007

Procedure

1. Recruit participants. These can be people with relevant experience or background in the design space, business stakeholders or anyone else playing a part in the project.

2. Conduct workshops. Plan activities to encourage participation from everyone. There can be different levels of participation from people -
they could just share their experiences and stories, brainstorm ideas and make mind maps, or create prototypes. It depends entirely on the context of your project and your participants.

3. Record sessions and take notes. As always, observe everything, record conversations with participants and take notes about the outcomes of your workshops.

Example

In ' Participatory Design and the Making of health ', Dr. Joyce Lee talks about her son's life threatening allergies and how the instructions provided by healthcare were confusing to understand in case of emergencies. She recorded a video of her son drawing and narrating the same instructions which are much easier to understand. This video went viral and inspired further prototypes. She talks about the importance of involving patients and other healthcare partners in the process of healthcare design.

Sensemaking

Parti cipatory design workshops can provide powerful insights for the project through the contribution of stakeholders. The information, ideas and artifacts from these workshops can be analysed to make design decisions, find a direction to move forward in the process, and again keep involving stakeholders as needed.

Design toolkits with specific activities and get inspired by how your participants engage with them. This method also advocates for active involvement of people in the design process. Participants are asked to do specific activities and create artifacts using the tools provided to them. These artifacts can give designers a lot of insight into what people feel and how they think.

We put a large number of components together into 'toolkits'. People select from the components in order to create 'artifacts' that express their thoughts, feelings and/or ideas. The resulting artifacts may be in the form of collages, maps, stories, plans, and/or memories.

A toolkit usually contains a background on which to work, together with a large number of simple and ambiguous components that can be arranged and juxtaposed in a variety of ways.
E.B.-N Sanders, 2000

Procedure

1. Decide on the information you are looking to get through the toolkits. There are a lot of different types of toolkits such as sensitizing toolkit, emotional toolkit, cognitive toolkit, etc. which serve different purposes.

2. Create the toolkits. Make sure they are easy to understand by the participants, you can include a set of instructions for the activity. Provide all the material they might need to complete this activity.

3. Ask questions, take notes and analyse the artifacts created through the toolkits to get insights into your design space.

Example

My team created 5 different toolkits to engage with our stakeholders about the problem of waste management in our city. Our main purpose with the toolkits was to get information about the behavior of people while managing waste, how they feel about this issue, their cognitive understanding of waste segregation and their emotional response to the impact on the environment because of this problem. Based on the outcome of these toolkits, we went on to create design concepts to tackle this issue.

Sensemaking

This completely depends on the type of information you get from each toolkit. You can analyse the emotional involvement of people into certain issues, find patterns in their behavior or get insights into their understanding of a process or a concept.

Work with members of a community to generate insights about important social issues. This group based qualitative method helps researchers address social problems by working together with members of a community. The members are engaged in an activity to use a map to indicate important areas, services and their relationships with places in a local area.

"Participants create a map that reflects their daily experiences and perceptions; with this visual reference in front of them, they can acknowledge and discuss features that are influential from their points of view."
White and Stephenson, 2014

Procedure

1. Recruit participants. Identify people you want to work with, usually this is someone with local knowledge of a community and experience in the problem space.

2. Plan the mapping activities. Would you want the participants to work on a physical map, or use digital tools? Consider their familiarity with technology, the collaborative nature of your activity and relevant factors to your project to pick the most suitable option.

3. Have facilitators in your session. Someone to support the participants and guide them through the activity.

4. Record everything. With informed consent from the participants, take extensive notes of their conversations or record the sessions to analyse later.

Example

Project Accept, a community-randomized trial of HIV counseling and testing sponsored by the National Institute of Mental Health, used participatory mapping as a formative research method.

It was used to identify community boundaries, learning about high-risk places and exploring potential counseling and testing sites to increase access to these services for the community. (Maman et al., 2009)

Sensemaking

The maps generated through this method can be analysed to find important information about the area and community, such as places visited frequently or experience of community members with certain locations. Notes from the session such as conversations of community members and their approach to the mapping activity can also reveal a lot of useful information.

Understand people's mental models about a place, process, or a system. By asking participants to draw a visual representation of how they think about a concept or navigate through a place, important relationships and patterns can be discovered.

A cognitive map helps break down complex research questions, establish priorities for follow up research, and add clarity to abstract concepts.
Sarah Gibbons, Nielsen Norman Group

Procedure

1. Prepare your participants for the activity. Let them know what to expect without giving away too many details (so they don't prepare). Inform that you will be asking them to write or draw stuff.

2. Find the right materials and place to conduct the session. You can use paper, pencils, colored pens, stickers or a white board to help them draw. Many people are uncomfortable with drawing, so materials that let them erase and redraw can help with the activity, along with a comfortable space.

3. Ask participants to draw from memory and talk through their process. This can include how they navigate in a physical or metaphorical space, their idea of a concept or process, or how they use a service.

4. Record the session and take notes. As always, inform your participants and with their permission record everything to get as many details as you can.

Example

My team used cognitive mapping in a project to explore how people feel about Bloomington, Indiana as a smart and connected community. In the sessions, we asked our participants to draw a map of their neighbourhood in Bloomington and indicate places that made them feel positive about the city.

Sensemaking

The maps collected from participants can be used to understand patterns in how people perceive something, and give an insight into their way of thinking and navigating through the problem space. Notes from conversations with participants during the activity can be analysed for more details about the same.

Observe, track and record behaviors in a specific setting or time. It can be either place based to observe the behavior of people in a particular setting, or individual based which follows the activities of selected people over different locations and time.

Behavioral maps are used to document readily observable characteristics, movements, and activities,including approximate ages and genders, whether people are alone or with others, what they are doing, time spent at fixed locations or in transit, and the details of environmental context.
Bella Martin and Bruce Hanington, Universal Methods of Design

Procedure

1. Place-centered mapping: Have a plan of the place you want to observe and note any significant areas, objects related to your problem space. You can decide beforehand which behavior you want to record, or have an open ended approach where you record everything that seems significant.

2. Individual-centered mapping: Allow the participant to get comfortable with the idea of being observed. Record their movement and behavior at different places.

3. Recording the behavior: This can be done either in real time by taking notes or by setting up cameras at specific locations.

Example

A study was conducted in Australia to examine children's free play in their neighborhood based on the access they had to safe public open spaces, their mobility and the variations by factors such as age, gender or socioeconomic status. The mapping activity was done around several primary schools, and was combined with a survey to get more input from children. (Veitch et al., 2008)

Sensemaking

The video recordings, notes or any other data gathered from behavior mapping sessions across different locations or time can be combined to identify patterns in behavior, frequency of accessing specific areas and understanding the motivation behind people's behavior.

Identify opportunities within an area to get insight for creating new services or improving existing ones, based on input from participants.

Procedure

1. Identify an area or community for conducting the mapping.

2. Recruit participants that are familiar with, or residents of that area. Keep in mind different factors such as age, gender, occupation to get diverse perspectives from participants.

3. Ask them to locate places or services they are familiar with. This can be done on a physical or digital map of the selected area. Invite them to share their experiences with different places or services, their opinions on improvements or any unaddressed needs that can lead to new opportunities. Alternatively, you can also ask them to draw their own map indicating these areas.

Example

My team used Opportunity Mapping in a research project to figure out areas for improvement within my city. We asked participants to draw a map of all the services they used within the city like shopping, gyms, health centers, etc. We then invited them to share their experiences with these services and identify any scope for improvement.

Sensemaking

The maps generated through these sessions can be analysed to get a better idea of opportunities for improvement within specific communities or areas. Getting a personal account from people who have experienced different services can give you a lot of insight into their use, help you identify pain points and needs to create new services.

References
Carroll J., Rosson M. (2007) Participatory design in community informatics.

Sanders E.B.-N (2000) Generative Tools for Co-Designing.

White D., Stephenson R. (2014) Using Community Mapping to Understand Family Planning Behavior.

Martin B., Hanington B. (2012) Universal methods of design: 100 ways to research complex problems, develop innovative ideas, and design effective solutions. Beverly, MA: Rockport Publishers.

Maman S., Lane T., Ntogwisangu J., Modiba P., Vanrooyen H., Timbe A., Fritz, K., Visrutaratna S. (2009) Using Participatory Mapping to Inform a Community-Randomized Trial of HIV Counseling and Testing. Field Methods, 21(4), 368-387.

J. Veitch, J. Salmon, K. Ball (2008) Children's active free play in local neighborhoods: a behavioral mapping study, Health Education Research, Volume 23, Issue 5, October 2008, Pages 870-879.

# data[].attributes.attachment_link.description
Help users and stakeholders to actively contribute to the design process and give their valuable insights and feedback. "Participatory design integrates two radical propositions about design. The first is the moral proposition that the people whose activity and experiences will ultimately be affected most directly by a design outcome ought to have a substantive say in what that outcome is.

# data[].attributes.attachment_link.readable_text
About a year ago, I started working with @MaxHardy building an Authentic Co-design process to assist Government, corporations, or community organisations with delivering complex projects and build trust across multiple stakeholders. Most of us have faced the complex project over time - the one with the multiple stakeholders with skin in the game, with complex or uncertain science or technical elements, and with a need to deliver it and at least maintain trust. Often there is a community involved who are deeply engaged and probably feel like they are not being heard.

Out of a series of workshops with entities who had such projects, we built the Authentic Co-design approach. It has a Framework with 6 steps, five principles and a set of conditions for success. Unsurprisingly we built it by co-design with people who had these challenges, for use in co-design to solve the challenges!

It has matured to a point now where there is a 90 page "how to" manual, a simple ready to use one page Framework, and a full online course delivered in partnership with @SusanCarter from The Community Studio. The course goes live next week - on June 9th.
We are often asked why 6 steps?
Each step has a specific role and need. Each can be done in what depth is required and at sometimes two steps even roll into one for delivery. There are moments in the steps where you ask yourself, is co-design still the approach for me to use to delivery this project? If not, we point to alternatives.
Step One - CHECKING READINESS TO COLLABORATE
Are you ready to commit to collaborate? This step is about getting prepared and understanding the level of readiness of all the potential participants. Some call it the 'readiness runway'. It is essential that it happens BEFORE you start to co-design as you will need a "stickable mandate" for your co-design.

Step Two - BUILDING A SHARED UNDERSTANDING

Step 2 - How do you build a shared understanding among a group with potentially little or no understanding in common, or with opposing positions? This step is about building a shared understanding about the problem, the co-design process, and the context for participants.
Step Three - DECIDING WHAT WE CAN DO TOGETHER
Step 3 - This step is about generating agreement that the problem to solve is one that enables all participants to share in the decision-making. This is important and at all times, refer to the Five Principle of Authentic Co-design that @MaxHardy discussed recently here.
Step Four - CO-DESIGNING WHAT WE AGREED
Step 4 - This step is about how participants co-design what they have agreed to do. Sounds simple really! There are loads of tips and lessons to learn about how to actually co-design.
Step Five - IMPLEMENTING AGREED ACTIONS
Step 5 - This step is about building something or actively delivering a decision that was co-designed. Of course, once you've planned something together, there is nothing more satisfying (and important!) than delivering it successfully.
Step Six - REVIEWING PROGRESS
Step 6 - Did we, collectively, achieve our goals? This is a process of genuine review by all who agreed to be involved. It is about adaptive learning, as well as holding each other to account that you did deliver on what was agreed. Super important to check in to see if we achieved what we aimed. If not, you might need to go back into Step 1, or perhaps to a later step depending on what the review found.

We also get asked why it is in a figure of 8 shape and not a line or circle, or other shape. It is a great question and is driven mostly by three things. First, it is a continuous cycle for improvements and hence should lead back into itself at the points it needs to, so that you can adaptively learn and deliver complexity. Second, while it is linear in a sense, there are cross-over points where you do need to be aware of how all 6 steps interlink. We felt that a circle does not allow for that. Finally, there is the practical reality that a figure of 8 just fits more neatly into a portrait image than other shapes do! While practicality and real-world delivery drive us, honestly the last reason is less than 20%, and the other two make up the majority of the drivers.

So, there we have it - six simple steps that enable you to authentically deliver co-design for complex projects that are full of multiple stakeholders and close to the heart of an engaged community.

Check out Max Hardy's blog about why it is called the Authentic Co-design Framework, and what inauthentic co-design might look like here.

We launch our online course for Authentic co-design June 9th, so if you'd like more information or to sign up to the waiting list, and to receive a FREE copy of the Authentic Co-design Framework, please visit here.

#codesign #complexity #scienceimpact #collaboration #gettingthingsdone #projects

# data[].attributes.attachment_link.title
Authentic Co-design - a framework for solving complex issues with communities.

# data[].attributes.attachment_link.description
About a year ago, I started working with @MaxHardy building an Authentic Co-design process to assist Government, corporations, or community organisations with delivering complex projects and build trust across multiple stakeholders. Most of us have faced the complex project over time - the one with

# data[].attributes.headline
Co-design: from expert- to user-driven ideas inpublic service design

# data[].attributes.subject
Co-design: from expert- to user-driven ideas inpublic service design

# data[].attributes.headline
Nesta- BY US, FOR US:THE POWER OF CO-DESIGN AND CO-DELIVERY

# data[].attributes.subject
Nesta- BY US, FOR US:THE POWER OF CO-DESIGN AND CO-DELIVERY

# data[].attributes.headline
The dialogue-labs method: Process, space and materials as structuring elements to spark dialogue in co-design events

# data[].attributes.subject
The dialogue-labs method: Process, space and materials as structuring elements to spark dialogue in co-design events

# data[].attributes.attachment_link.readable_text
Practitioners from different fields of research and design have understood the importance of involving diverse groups of users in the generation phase of novel artefacts, products and services, and thus facilitating participation has become one of the cornerstones of co-design. Underpinning this approach is the supposition that stakeholders, including users, can contribute productively through involvement in the design process since they bring privileged insights into the domain that designers are trying to address and the ways in which future products and services may fit into and affect that domain. This work addresses how stakeholders can be involved in the ideation, concept development and early prototyping phases of co-design.

Dialogue-labs and its three structuring elements (i.e., process, space and materials), provide a structured way of generating ideas through a sequence of co-design activities. The process provides a clear step-by-step procedure for a two-hour idea-generation session in which participants work in pairs. The space is carefully crafted to align content to different locations, inspire participants and encourage them to move around the room. Finally, the materials are the means for participants to build a design language of their own and to provide different entry points to the design problem.

# data[].attributes.attachment_link.description
Practitioners from different fields of research and design have understood the importance of involving diverse groups of users in the generation phase of novel artefacts, products and services, and thus facilitating participation has become one of the cornerstones of co-design.

# data[].attributes.body
How to plan effective co-design (thinknpc.org)

# data[].attributes.attachment_link.readable_text
One of the big lessons from this project is that loneliness affects all kinds of people and in different ways . At Christmastime many charities will be campaigning for action on lo neliness , so what can we learn from the Bui lding Connections Fund about what works?

W e conducted a qualitative evaluation of these projects . Here's what we learnt about what co-design achieved , and how you can maximise and evaluate the benefits of working in this way.

What does co-design mean for young people?

Co-design is when an organisation and its stakeholders work together to design or rethink a service. W e found that co-design created valuable conversations which changed how young people thought about and understood loneliness.

Our evaluation found that co-design:

Enabled young people to influence service-delivery, so charities could improve their work to tackle loneliness.

Led to changes that aren't specific to loneliness. For the young people, participating in co-design often resulted in increased confidence and empowerment. This in turn impacted other areas of their lives, such as school work , behaviour, and interpersonal skills.

Influenced how charities approach their work, by generating useful insights and ideas, and improving decision-making about service-delivery.

What are the challenges for effective co-design?

How can you evaluate co-design?

T here is a long tradition of involving young people in the design of youth work and it is clear that co-design can unlock positive outcomes for participants, organisations, and communities . What is less widely embedded are approaches for assessing the effectiveness and impact of co-design.

This is understandable as evaluating co-design can be tricky. Different participants are likely to engage with co-design in different ways. How do you evaluate a range of activities and experiences? What if your co-design is delivered as a process, rather than a distinct intervention? You may not have predefined outcomes for your co-design from the start.

1. Set clear objectives for your co-design at the start

Evaluating co-design can be challenging but all organisations can create space to reflect on experiences and emerging insights. As a first step, it's vital to be clear about the purpose of your co-design. For example, your objective may be to redesign a service to better reflect user needs, or you may want to strengthen relationships between staff and volunteers. You need to know where you are going to know if you are heading in the right direction.

2. Know who should take part in your co-design

To understand the benefits you're achieving, you need to understand if you're including the right people in co-design. You also need to find out what participants think of your co-design processes, what changes are being achieved for them in the short-term, and whether these changes lead to longer-term outcomes for participants.

It is important to consider if there are any risks or potential harm to participants. For example, a mismatch in expectations could result in frustration , distress, and loss of trust .

3. Assess the quality of your co-design

For these Building Connections Fund projects, co-design was delivered and experienced was critical to achieving outcomes. Two factors helped enable positive experiences for participants:

C ultivating a sense of ownership, shared purpose and achievement ; and

B uilding relationships and trust.

To understand how your co-design process is being experienced, you could ask questions of your participants such as:

Why did you get involved in this co-design project?

Which activities did you enjoy and not enjoy?

What were your relationships like with the facilitators and other stakeholders?

Did participation have any impact on you?

4. Gather insights into how to achieve your intended impact

Understanding how and why change occurs as a result of your organisation's work matters as much as understanding whether there was any change at all. Even from these short-term projects, co-design generated valuable insights into what participants felt were the best outcomes for them, and how they might be achieved.

To draw out these insights, try to answer questions like:

What do you know about the problem? What do your co-design participants say about what would help? Through these Building Connections Fund projects, co-design participants identified factors to make community spaces more engaging and attractive to young people , and more conducive to reducing loneliness. These p rimarily relate d to the use of technology and specific design features.

What are the "active ingredients" of your work? What are the key experiences that people need for your work to be successful? For example, people may need to trust staff or feel confident enough to engage with activities. What have you learned about key design features for your work, for example duration, intensity, targeted or open activities ? What barriers do participants identify, for example lack of transport or needing more flexibility to balance other commitments ?

What have you learned about what success looks like? Is this different for different types of projects and participants? F or some participants , success could be improving attainment at school while for others, simply engaging with an activity could be a significant outcome .

Ultimately, co-design can and should take different shapes and forms to respond to different people with differing needs. The crucial point is to set up your co-design with clear objectives in mind, and to embed opportunities throughout the process to reflect on those objectives and assess if you're meeting them.

To help with this, I've worked with colleagues on a toolkit for designing, implementing and evaluating co-design . I'd love to know what you think and hear about your experiences with co-design. You can reach me at michelle.man@thinknpc.org or on Twitter .
As we move into 2020, we're now evaluating the impact of the Building Connections Fund as a whole, and we'll be making recommendations for how charities can best tackle loneliness. Sign up to our newsletter to stay up to date with all our latest research.

# data[].attributes.attachment_link.description
One of the big lessons from this project is that loneliness affects all kinds of people and in different ways . At Christmastime many charities will be campaigning for action on lo neliness , so what can we learn from the Bui lding Connections Fund about what works?

# data[].attributes.body
Co-design - Mind

# data[].attributes.body
What is co-design? — Beyond Sticky Notes

# data[].attributes.attachment_link.readable_text
Co-design is designing with, not for. It's about making things and learning together.

Co-design is a movement, mindset (read more) and an expanding set of methods and tools.

Co-design challenges the power of individuals who make important decisions about others' lives, livelihoods and bodies. Often with little to no involvement of the people who those decisions will most impact. Co-design is about working with the people closest to the solutions, prioritising relationships, being honest, making sure people feel welcome, using creative tools, balancing idealism and realism, building and sharing skills. Co-design uses inclusive facilitation that welcomes and works with many ways of knowing, being and doing.

Co-design has a 'co' bit (e.g. community, co-operation) and a 'design' bit. Both bits (community and design) are important but neither have all the answers.

There are many histories and many practices of co-design.

What are the principles for co-design?

There are many ways to work together. Here are four principles for effective co-design.

We believe the principles for co-design also matter for other 'co' things, too: planning, delivery and evaluation
(Figure 2).

How can I tell if it's co-design?

Ask these questions to determine if something is co-design. And know that because something isn't co-design doesn't automatically make it less useful, inclusive or worthwhile.

Co-design isn't the only way or always the best way.

How is co-design different from other approaches?

There are lots of different ways to work together. Co-design and human-centred design aren't the same.
If you're not doing co-design, don't say that you are.
The table below describes different design and participation approaches. We say who gets a say and who doesn't. We can design at people, for people, with people - or be led by.

These things are transactional at best, harmful, degrading and violent at worst

Approach

Keywords

Where power is held

Designing at people

Designer, professional or policy-maker as expert, top-down decision-making

Designing at people is about what decision-makers and designers think and want.

Designing for people

Design thinking, anything 'centred' e.g. human-centred (HCD), user-centred (UCD), patient-centred, citizen-centred design etc.

Designing for people is about what designers and decision-makers want to know and achieve. Good intent usually ends up as system-centred, designer-centred, executive-centred or staff-centred by implementation.

These things are transformational at best, however they are not inherently inclusive or equitable

Designing with people and planet

Co-design, participatory design, deliberative democracy

Designing with people is about what matters to people with lived experience and decision-makers (co-decided).

Led by the people

Co-production, community-led design, citizen movements, design justice*

Led by the people is about what people, families and communities want for themselves.

Kelly Ann McKercher, 2020

*As described in the Design Justice Principles and Design Justice by Sasha Costanza-Chock.

What is the 'social movement' part of co-design?

To make co-design a reality, we need systems, organisations and communities to embrace the leadership and contributions of people with lived experience. Doing that requires different ways of thinking and being, which are missing from many teams, organisations and systems. The table below describes several social movements that are necessary to make co-design a reality and a norm.

Making decisions for people with lived experience

Making decisions with people with lived experience

Valuing professional expertise above all

Valuing professional and lived experience equally

Seeing marginalised people as a burden

Seeing marginalised people as resilient, creative and capable

Colonising, heteronormative and ableist systems

Compassionate systems that see and respond to dimensions of difference

Believing that resources are scarce to make change

Seeing an abundance of experience, ideas and energy for change

Focusing on 'consumer' councils and committees

Embedding participation in everyday practice

Rushing to solutions

Slowing down to listen, connect and learn

McKercher, K. A. (2020). Beyond Sticky Notes. Doing Co-design for real: mindsets, methods and movements.

What are the conditions for co-design?

Co-design can be derailed through a lack of sponsorship, resources or a lack of broader organisational or community commitment. Here are some conditions that can support co-design.

We need people to endorse and reinforce the approach we're taking and the outcomes we want to achieve. Sponsors and supporters help to build commitment, remove obstacles and overcome resistance as and when it arises.

To do co-design we need time and money for:
Facilitation and convening (co-design is not free)
Paying people with lived experience for their time and for any expenses
Investing in approaches (after they have been co-designed)
Supporting lived experience capability and leadership
Prototyping, testing and learning (prior to implementation)
Communicating the work throughout to build commitment

3 Culture and climate

4 Commitments

Supportive culture and climate includes:
Authorising environments from formal and informal leaders
A focus on learning not control
Connective tissue to share learning, failure, success
Support to adopt the mindsets, especially when we regress to old ways of being
Support to develop the skillsets for co-design
Accountability to the people we engage through
co-design (they can call us out)

Commitment to co-design looks like:
Focusing on outcomes (value) over outputs (busyness)
Following through into implementation
Staying committed to elevating the voice and contribution of lived experience
Practising cultural intelligence and widening inclusion
Partnering, not parenting
Sharing decision making, power and attribution
Value and reciprocity with co-designers

Kelly Ann McKercher, 2020

# data[].attributes.attachment_link.title
What is co-design? - Beyond Sticky Notes

# data[].attributes.attachment_link.description
Co-design is a movement, mindset (read more) and an expanding set of methods and tools. Co-design challenges the power of individuals who make important decisions about others' lives, livelihoods and bodies. Often with little to no involvement of the people who those decisions will most impact.

# data[].attributes.attachment_link.title
Resources - Scottish Co-production Network

# data[].attributes.attachment_link.description
East Dunbartonshire Advisory Clinic set up to address the issue of an aging population in the the area, based strongly on the principles of co-production and involving both statutory and voluntary organisations in reshaping care.

# data[].attributes.attachment_link.readable_text
The Auckland Co-design Lab (the Lab) is a public sector innovation team based in South Auckland. It is a unique collaboration between central and local government.

Set up in January 2015 the Lab was established to develop fresh ideas in response to complex social issues. Our aim is to use co-design principles and practice to work with, better understand and empower the people closest to the issues.

A key goal is to create a space for multi-agency teams to collaborate, work alongside citizens and to support and broker innovative ideas and solutions.

The Lab is funded by Auckland Council and eight central government agencies; Treasury, NZ Police, Te Puni Kōkiri, Ministry of Health, Ministry of Business Innovation and Employment, Ministry of Education, Ministry of Social Development and Department of Prime Minister & Cabinet.

The Lab is based with Auckland Council's Southern Initiative team in Manukau, South Auckland.

WHAT IS CO-DESIGN?

There are lots of different definitions and terms people use to describe co-design practice. Below is a definition that we like, but you might also hear the approach described as human-centred design, design thinking, participatory design.

# data[].attributes.attachment_link.description
The Auckland Co-design Lab (the Lab) is a public sector innovation team based in South Auckland. It is a unique collaboration between central and local government. Set up in January 2015 the Lab was established to develop fresh ideas in response to complex social issues.

# data[].attributes.attachment_link.readable_text
As a child, my mum used to tell me that I always learnt the 'hard way'. By that she meant I learnt by doing - sometimes in spite of what I might have been told. It follows that I ended up studying 3D design, using tangible objects and experiences to develop and communicate ideas. But back then I don't think I ever imagined this could be relevant to government.

Policy Lab has built a reputation for experimenting and stretching the scope of policy making across government by mixing up the three main learning styles - auditory, visual, and kinaesthetic. Where the written and spoken word is relied upon heavily for accurately conveying concepts; visual, tactile and experiential methods are proving useful approaches to help policy makers explore the more uncharted waters of their work - whilst at the same time engaging wider audiences and capturing the imagination.

Drawing on examples from the Lab's recent work with the Department for Transport's (DfT) Maritime Technology team, this blog post will reflect on how a more kinaesthetic - learning through doing - design approach might be adopted at different stages in the policy making process. Our brief was to help the policy team shape their future vision and technology strategy through a series of consultation workshops with key stakeholders.
Discovery: prototyping as user research
Typically the discovery phase of a design project is when the net is cast out to understand the current landscape and its users, gathering insights and identifying opportunities for intervention. It's the research phase: exploring, listening and being open to receive information. However, there can also be an opportunity to add in some more active doing, or 'early-stage prototyping', with the potential to accelerate and enrich the research process. It might seem premature to jump to solutions before fully completing research, but it can be an engaging way to test assumptions and elicit deeper insights from users.

We used this approach for the maritime consultations by designing a visually provocative, speculative series of utopian and dystopian visions of the future to probe stakeholders on how different aspects of the UK maritime industry could play out in the near future. Each stakeholder was given one of these visual prototype 'vision cards' to reveal in turn, prompting discussions around specific policy areas, form the impact of 'smart' ports and autonomous passenger ferries, to cyber security threats and the potential impacts of seabed mapping.

Although the visions didn't always resonate with everyone around the table, the act of revealing these visualised future scenarios certainly stimulated debate, effectively testing how the policy team's proposed vision and strategy might be received.

and Camilla Buchanan have also written about other examples of this early-stage prototyping in their previous posts, and Stephen's recent post highlights the potential for speculative design in policy making.
Communication: immersive research as a communication tool
It's not always practical to bring all stakeholders to the frontline of research for a fully immersive, 'learning by doing' experience, but Policy Lab finds that by curating their own video ethnography in collaboration with policy teams, policy makers and stakeholders can be bought much closer to citizens. Compared to written or even visual research reports, playing back personal lived experiences into the policy making process has provided rich insights for policy teams to learn from.

The Lab's collaborative work with the Maritime Technology team took a feet-first approach to researching the emerging technology landscape. Following desk research and foresight from the Government Office for Science , Sanjan headed out with the policy team to experience first-hand a small autonomous sea-going vessel called ' ' currently being used for seabed mapping in Norway. Meanwhile, Kyna, Policy Lab's in-house video ethnographer, visited ports across the UK from Grimsby to Glasgow to capture the perspectives of people currently working in the industry.

These valuable first-hand insights from people at the front-end of policy decisions were edited together and shared with stakeholders at the beginning of each consultation workshop. They provided a highly engaging introduction and helped to set the context for the activities that followed.
Ideation & testing: policy co-design through simulation
Smarter policy though simulation was one of Nesta's top ten predictions for 2018 , and it turns out they weren't wrong. Co-designing and testing things as intangible as visions and strategies is challenging, but introducing a more tactile dimension to this stage of the policy making process can help to articulate and iterate ideas in a group of stakeholders. The military have long used war-games to help anticipate and simulate non-linear dynamics. Conversely, in the 1960s, design visionary Buckminster Fuller inverted the traditional competition mindset of political gaming, with his World Peace Game to explore routes to international peace through cooperation. These concepts are now re-emerging through collaborative foresight games such as designed for the Canadian government and the 2030s SDG Game developed in Japan as a reaction the the UN's sustainable development goals.

With all this in mind, we developed a tabletop activity as a type of co-design tool for the consultations, to explore how multiple variables - technology, regulation and skills - combined with different stakeholder perspectives might influence a national vision and strategy for the maritime industry.

Dubbed the 'Shipping Forecast', the aim of the activity was for the stakeholders to deliberate their visions and play out their strategies together, using the activity board, vision and technology cards, and playing pieces we had designed. Led by a Lab facilitator, the open and dynamic format allowed the policy team to observe the activity play out and conversations, comparing notes with their proposed policy. The positive feedback from stakeholders at the end of the each consultation activity was testament to how an tactile, simulation approach to co-designing policy can add value for both policy teams and their stakeholders.

These are just a few examples of how 'doing', and taking a more kinaesthetic approach to design in policy making, through speculative prototyping, immersive research, and simulation, can be valuable for policy makers, their stakeholders and citizens. I'm excited to see what Policy Lab comes up with next!
This was a blog by Becky Miller, Service Designer at the Ministry of Housing, Communities and Local Government. Becky worked with the Policy Lab team in 2018.

# data[].attributes.attachment_link.title
Co-design in policy: learning by 'doing'

# data[].attributes.attachment_link.description
As a child, my mum used to tell me that I always learnt the 'hard way'. By that she meant I learnt by doing - sometimes in spite of what I might have been told. It follows that I ended up studying 3D design, using tangible objects and experiences to develop and communicate ideas.

# data[].attributes.headline
Tools for climate policy: 1) co-design in parallel to COP26

# data[].attributes.subject
Tools for climate policy: 1) co-design in parallel to COP26

# data[].attributes.attachment_link.readable_text
We are sharing a short blog series on different methodologies that Policy Lab has experimented with in the context of action to address climate change. An important part of our ethos is learning-through-doing. This series aims to show how we have deployed different innovative techniques in real life policy settings. Our intention is to provide inspiration for how such techniques, coupled with our broader experimental mindset, could be deployed in relation to the climate crisis. This, the first blog, describes a policy co-design process we ran last month at the Design Council's Design for Planet conference coinciding with COP26.

What is co-design?

A co-design approach allows ideas to be shared and shaped quickly, collaboratively and iteratively. The designer's role is to create and facilitate a process that cuts away the layers of mediation and bureaucracy that tend to exist between people and large organisations. The outcome is to swiftly gather an understanding of the rich perspectives of people with lived experiences of an issue, whilst inviting those people to contribute ideas for change themselves.

Through our past projects, Policy Lab has shown how co-design can add value for policymakers working across government, and with a whole range of stakeholders, citizens and users. For example, our work with the Youth Policy Team in the Department for Digital, Culture, Media & Sport (DCMS) in 2019 allowed us to co-design a new youth democracy engagement platform with young people using Instagram. In a completely different policy context, our collaboration the year before with the Department for Transport (DfT) saw us designing a workshop for maritime sector stakeholders to co-design new visions and strategies for the future of their industry, which fed into the Technology and Innovation Route Map for the Maritime 2050 Strategy.

Designing for the planet

Design for Planet was a 2-day festival that took place in November 2021 at the Victoria & Albert Museum in Dundee, held alongside COP26 and streamed online. It's aim was to support the UK and global design communities to address climate change more directly. Convened by the Design Council, the hybrid event included speakers such as economist Kate Raworth and environmentalist Anita Okunde, 120 physical participants including representatives from 20 national design bodies around the world, and 5,500 online registrations. You can watch the Design Council's two minute video on the festival , and read a summary blog by Cat Drew , Chief Design Officer at Design Council and previously in the Policy Lab team.

The theme of 'designing for the planet' aimed to explore how designers can connect and align individual human needs with our planetary system. This is an evolution from an arguably more extractive historical model of design, and builds on the insights of practitioners such as Victor Papenek, who contemplated the environmental responsibility of the design sector in his 1971 book, Design for the Real World .

How can we... unleash the design sector on climate change?

There are many existing examples of designers deploying imaginative approaches to tackle climate change, from Olafur Eliasson's to Superflux's However, given the pressing nature of climate change, combined with the UK Government's commitment to addressing it , the Design Council asked Policy Lab to explore what policy changes could further unleash the sector on this global issue.

We used a co-design approach to invite designers to express in their own terms the barriers, ideas and opportunities the design sector faces in addressing climate change. The ideas that participants developed ranged from a radical rethink of skills training for designers focusing on climate change and regenerative design, to carbon accounting in all design projects, and more responsible ways to consider resource usage with other countries.

Framing and crafting

We deployed a couple of specific techniques to facilitate this co-design session. First, we used a framing tool which we have written about before: our Styles of Government Action framework. This matrix allowed us to consider with participants both changes may facilitate designers to address climate issues, but also we might deliver those changes. This was important for the session because part of creating change in large systems involves not only designing a concept, but also the process by which that concept can come into being.

Second, we encouraged people to use their hands. The scale, urgency and entangled nature of the 'design for planet' challenge can feel overwhelming. One thing that designers do powerfully is to use visual and haptic cues in the physical environment to signal the presence of new culture and ways of working. We have previously written about using different sensory formats for policy discussions . For our workshops at Design for Planet , we were additionally inspired by 'craftivist', Sarah Corbett who uses crafted objects to contemplate global issues, provoke thought and start conversations. This approach influenced our deliberate slow repurposing of found artefacts. These included handmade policy canvases to help participants develop their ideas, and origami polar bears used for an icebreaker exercise, crafted from discarded buffet cart bags collected on our train journey to Dundee.

These simple objects created moments of mindful reflection for both our team in the crafting process, and the workshop participants through the physical interactions they afforded. Our experience at Policy Lab tells us that thoughtfully designed objects can be an incredibly effective tool for engaging conversation, creating shared understanding and in enabling people to transition into a different way of working and thinking.

What next?

Perhaps the strongest message coming from participants at the festival was the need to be intentional about the relationship between our collective practice and the planet and climate upon which we all depend. This ethos underpins our work, and we will describe more examples of how it can play out in practice in future articles. Subscribe to our blog to catch the next in our series on Policy Lab's climate-related policy work. And if your team is keen to take an innovative approach to policy challenges, climate-related or otherwise, we'd love to hear from you: Design for Planet team@policylab.gov.uk.

# data[].attributes.attachment_link.title
Tools for climate policy: 1) co-design in parallel to COP26

# data[].attributes.attachment_link.description
We are sharing a short blog series on different methodologies that Policy Lab has experimented with in the context of action to address climate change. This, the first blog, describes a policy co-design process we ran last month at the Design Council's Design for Planet conference coinciding with COP26.

# data[].attributes.headline
Co-design: A Powerful Force for Creativity and Collaboration

# data[].attributes.subject
Co-design: A Powerful Force for Creativity and Collaboration

# data[].attributes.attachment_link.readable_text
We can all agree that creativity and collaboration are good things, but not everyone is in agreement on what they look like in practice or the best way to support them.
In a world of increasingly complex problems, collaboration is king. The impetus to work together to create innovative ideas in areas such as healthcare, education and governance is more pressing, and exciting, than ever. These problems require multiple stakeholders (internal and external) to come together and actively create, in order to improve the systems and services.

Collaboration is more than just tapping into the individual knowledge that internal and external stakeholders possess. It is about discovering their unique, and collective perspectives on the systems in which they live, which makes it vital to create together.

Let's start by talking about some key definitions:

Co-creation is "the joint creation of value by the company and the customer; allowing the customer to co-construct the service experience to suit their context." This may seem like a general term, but ultimately "co-creation" helps identify a more specific type of value-based collaboration between stakeholders and users, in contrast to standard market research.

Co-design is the act of creating with stakeholders (business or customers) specifically within the design development process to ensure the results meet their needs and are usable. (Co-design may also be called participatory design- a term which is used more often within the design community.)

Many human-centered design firms design for people. At Stratos, we put a major emphasis on designing with people. There's a fairly big difference in terms of approach. As much as possible, co-design practitioners should bring the people that the outcomes will serve into the design development process. But what does this actually look like in practice?

As mentioned previously, the complex challenges our world faces are too great to be tackled by a single discipline. More than inviting designers or engineers into the process, users, customers, patients and other stakeholders, need to be brought in as active co-designers, to confront the big issues and develop actionable ways to improve their current experiences or co-create a new solution.

As champions of co-design, we have witnessed the powerful impact that the co-design mindset, approach and tools have had on the outcomes of the work we've accomplished for our clients. Here are a few examples:

* For a multinational financial services company, business leaders, the innovation team, UX teams and consumers were brought together to create the ideal experience for a new insurance rating model.

* In today's rapidly changing media environment, a media outlet engaged Stratos to help break down company silos that hindered production time and effectiveness. This was made possible using co-creation techniques to facilitate collaboration around their future working practices.

* A national industrial product brand minimized risk by co-designing customer engagement ideas, with a new market segment, before investing large amounts of money.

* For a garden products manufacturer, the time needed to develop a mobile engagement solution (app) was significantly reduced by adapting a co-design approach. Researchers, designers, consumers and business sponsors were brought together for a rapid three day test and redesign cycle.

Co-design practitioners aim to become design process guides or facilitators. We leverage our expertise of design and research, but don't let our own ambitions or ideas drive the outcomes. While guiding stakeholders through the design process of ideation and development, we help people develop and raise the fidelity of their ideas. These ideas can then be shared with an external audience not involved in the process.

Co-design can take place at any point across the design development process. However, involving people with different perspectives early (and often) can help determine the real problem space early on- leading to more advantageous outcomes. Admittedly, involving multiple stakeholders throughout the process can be challenging, but through a guided process of discovery, ideation and development, we can give a voice to both end users and the people who serve them. By doing so, informants turn into participants, contributors, and partners- and that can have a powerful and meaningful impact on the outcomes.

Talking about specific roles and responsibilities in co-design is dependent on the context in which we are working. Let's imagine that we are working with a healthcare provider to improve the lives of young people living with type 1 diabetes.

If we are to take a co-design approach to this challenge, we need to bring the different people and organizations together to get a full picture of the opportunity landscape and to co-create new solutions. For example, this group of participants might include: type 1 diabetics, their personal circle of care (parents, siblings, friends), healthcare practitioners (endocrinologists, nurse practitioners, diabetes educators), schools (teachers, school nurses, bus drivers), and payers and pharmaceutical companies (to name a few). Ultimately, the scope or scale of the project will determine how many stakeholders are represented as well as where in the process they will contribute.

Understanding co-design and its terminologies, applications, and possibilities leads to more successful collective creativity. Our collaborative, cooperative and creative potential hinges on our ability to find common ground, share ideas, use common language and work together to co-create a better world. We live in an exciting world that needs people to work together now, more than ever, to tackle the incredible problems we face. Collaboration is king!
Have you used co-design methods, before? If so, we would love to hear about your experience and any thoughts you would like to add. Feel free to drop us a line at monica@thestratosgroup.com@TheStratosGroup to learn more about co-design. As always, you can find more on our website (www.thestratosgroup.com

# data[].attributes.attachment_link.title
Co-design: A Powerful Force for Creativity and Collaboration

# data[].attributes.attachment_link.description
We can all agree that creativity and collaboration are good things, but not everyone is in agreement on what they look like in practice or the best way to support them. In a world of increasingly complex problems, collaboration is king.

# data[].attributes.attachment_link.title
Why is co-design so important? – Good Things Foundation

# data[].attributes.attachment_link.description
By definition, co-design is the approach of actively involving stakeholders in the design process. From end-users to partners, each opinion is invaluable in the creation of something new.

# data[].attributes.attachment_link.title
Design in the time of policy problems — DRS2016

# data[].attributes.attachment_link.description
This website is for the 2016 Design Research Society Conference held in Brighton, UK.

# data[].attributes.attachment_link.readable_text
A little over a week ago we published the Agricultural Transition Plan. On page 6, we say:

We will be more flexible and will co-design our policies with farmers and other experts.

"Co-design" is also at the top of our list of guiding principles, on the next page of the plan.

So co-design really matters, and it's really important to the team here at Defra who are working on the transition plan roll-out.

I want to set out what we mean by "co-design", because it's a word we're going to be using a lot in the months and years ahead.

That's not to say co-design is new to us: we've been using it for a while already, for example with our tests and trials work, and through detailed, ongoing engagement with farmers and experts on the design of Environmental Land Management.

We plan to build on that work, expanding and extend the principles and practices of co-design to our whole programme of work, in a more open way, from now on.

Our definition of co-design may change over time, but this is how we see it right now.

Co-design means involving people affected by our work

Co-design is a design approach that actively involves users and stakeholders from the beginning of a project, right through to roll-out.

It means we collaborate with everyone who has an interest to solve real problems with them. We actively seek their input and feedback, based on their lived experience, as we iterate and improve services.

Many government colleagues will be familiar with different names for similar approaches, particularly names like "user-centred design". There's a lot of overlap with those approaches, but co-design puts more emphasis on what we learn from the aggregated opinions of participants.

We respect and value the opinions of people who are living with our policies and systems every day and have experiences, stories, insights and information to share. They know more about what's worked in the past and what hasn't, and what we should learn from that.

For example: we are working with people who have deep expertise in different types of farms to make sure our schemes work for all farms.

So it's not just about observing their behaviour when shown a prototype service or work-in-progress.

It's also about asking for their views on how prototypes, works-in-progress and policy decisions will affect their lives. Co-design is a partnership, more than it is a process.

Talking to people helps us learn

One of the things I've learned from speaking to farmers is that there's no such thing as "a typical farm". You can have a wide variety of personal, financial, agricultural and environmental circumstances in just a few square miles; neighbouring farms can have very different experiences.

It's important that we listen to as many voices and views as possible.

We cannot promise to make everyone happy, all of the time. But co-design helps us understand more of those individual circumstances, and respond as best we can.

It's also a useful way to involve many different sorts of people, with different interests. Of course we'll be talking to farmers, but not just farmers. There are many others who will be affected by agricultural transition, such as landowners, inspectors, vets, advisers, and many more. My colleague Neville Cavendish, Head of Co-design for the Future Farming and Countryside Programme (Defra), will have more to say about this in another blog post soon.

Building faster and building better

Co-design helps us move faster: from the point that we identify a problem, to our ability to standup a prototype and begin to test and learn. Co-designing with people helps us understand the real-world impact of a decision. The earlier we can involve them in the process, the sooner we learn and the sooner we can work on improvements.

Co-design also helps us build better: if we don't meaningfully involve our users in our work, we will risk building things that don't work for them. We might fail to solve real problems, missing things and messing things up. Getting people involved in co-design doesn't just help us spot what's wrong, it also helps us find the right improvements.

Co-design can feel scary at first

Introducing co-design can raise expectations that we might not be able to meet. But it also helps us understand the perspectives of people affected by our work.

So there's a balance to strike. We strongly believe that co-design is the right approach, and that using it will help us avoid a lot of pitfalls along the way. But it's not a panacea, it won't solve every problem.

We've already started running co-design sessions, and we'll tell you more about some of them in another blog post soon. There are many more sessions planned in 2021. Keep an eye on this blog for more details.

If you'd like to get involved in co-design, please get in touch. Send an email to ffcpcodesign@defra.gov.uk

# data[].attributes.attachment_link.description
A little over a week ago we published the Agricultural Transition Plan. On page 6, we say: We will be more flexible and will co-design our policies with farmers and other experts. "Co-design" is also at the top of our list of guiding principles, on the next page of the plan.

# data[].attributes.headline
Benefits of Co-design in Service Design Projects

# data[].attributes.subject
Benefits of Co-design in Service Design Projects

# data[].attributes.attachment_link.readable_text
Benefits of Co-design in Service Design Projects

Marc Steen 1,*, Menno Manschot 2, and Nicole De Koning 1

1 TNO, Delft, The Netherlands
2 Delft University of Technology, Delft, The Netherlands

In many service design projects, co-design is seen as critical to success and a range of benefits are attributed to co-design. In this paper, we present an overview of benefits of co-design in service design projects, in order to help the people involved to articulate more precisely and realistically which benefits to aim for. Based on a literature review and a discussion of three service design projects, we identified three types of benefits: for the service design project; for the service's customers or users; and for the organization(s) involved. These benefits are related to improving the creative process, the service, project management, or longer-term effects. We propose that the people involved in co-design first identify the goals of the service design project and then align their co-design activities, and the associated benefits, to these goals. The paper closes with a brief discussion on the need for developing ways to monitor and evaluate whether the intended benefits are indeed realized, and the need to assess and take into account the costs and risks of co-design.

Keywords- Benefits, Co-Design, Service Design, Cases.

Relevance to Design Practice- An overview of benefits of co-design in service design projects is presented, in order to help the people involved to articulate which benefits to aim for.

Citation: Steen, M., Manschot, M., & De Koning, N. (2011). Benefits of co-design in service design projects.International Journal of Design, 5(2), 53-60.

Received October 22, 2010; Accepted April 30, 2011; Published August 15, 2011.

Copyright: © 2011 Steen, Manschot, & De Koning. Copyright for this article is retained by the authors, with first publication rights granted to the International Journal of Design. All journal content, except where otherwise noted, is licensed under a Creative Commons Attribution-NonCommercial-NoDerivs 2.5 License. By virtue of their appearance in this open-access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

*Corresponding Author: marc.steen@tno.nl

Marc Steen works as a senior scientist at research and innovation organization TNO ( http://www.tno.nl) in Behavioural and Societal Sciences. He earned MSc, MTD and PhD degrees in Industrial Design Engineering at Delft University of Technology. He worked at Philips and KPN before joining TNO. His expertise is in human-centered design, co-design, open innovation, innovation management, service design and marketing strategy. One of his ways of doing research is by critically reflecting on innovation and design projects, through participant observation and reflexive practice.

Menno Manschot works as a design consultant and researcher in service design at the faculty of Industrial Design Engineering of Delft University of Technology. His research interests are in strategic design thinking, service design and people-centred design approaches. He is currently involved in a multi-year study on the strategic values of service design approaches for businesses and their clients. In addition to his research activities, Menno runs a consultancy in service design and collaborative innovation.

Nicole de Koning works as a senior consultant at research and innovation organization TNO ( http://www.tno.nl) in Behavioural and Societal Sciences. She earned an M.A. degree in Cognitive Science at Radboud University Nijmegen. She worked at Baan and KPN before joining TNO. Her expertise is in human-centred design, co-design, innovation management, service design, customer experience and customer behaviour.

Introduction

Co-design is increasingly popular in many businesses and organizations (Binder, Brandt, & Gregory, 2008). However, co-design is sometimes used as a buzz word and it is not always entirely clear how co-design contributes to a service design project. The goal of this paper is to help the people who are involved in co-design to articulate more precisely and realistically which benefits to aim for, and to match these benefits to the goals of a service design project. We do that by identifying and discussing a range of possible benefits of co-design in service design projects.

Sanders and Stappers (2008) used the term co-creation to refer to "any act of collective creativity, i.e. creativity that is shared by two or more people", and used the term co-design in a more narrow sense to refer to the "collective creativity as it is applied across the whole span of a design process". In line with this use of these terms, we will focus on co-design in this narrower sense, that is, on creative cooperation during design processes-rather than on the co-creation, which also refers to creative cooperation during service delivery and usage, for example, to interactions between customers and service provider at service touch points. In co-design, diverse experts come together, such as researchers, designers or developers, and (potential) customers and users-who are also experts, that is, "experts of their experiences" (Sleeswijk Visser, Stappers, Van der Lugt, & Sanders, 2005)-to cooperate creatively. We will pay special attention to involving users and customers in the design process and putting their experiences central (Alam, 2002; Edvardsson, Gustafsson, Kristensson, Magnusson, & Matthing, 2006; Kujala, 2003; Muller, 2002; Sanders, 2000). Furthermore, we use the term service design to refer to the process of planning and organizing people, infrastructure, communication and material components of a service, with the goal of improving the service's quality, the interactions between a provider and its customers, and the customers' experiences (Mager, 2008).

This paper is based on the assumption that co-design is critical to service design because different perspectives, and a productive combination of different perspectives, are needed in order to understand both a service's demand side, i.e. users' and customers' needs, and its supply side, i.e. technologies and processes, in order to develop successful services. Businesses and organizations expect that co-design will deliver specific benefits and that these will help them to realize specific goals in their projects. The wide range of benefits can include improving customers' loyalty, reducing costs, increasing people's well-being, and organizing innovation processes more effectively. These examples illustrate that the benefits that are sought after by means of co-design can be very diverse.

Based on informal observation in diverse projects, we speculate that the people involved in co-design in service design sometimes fail to articulate precisely and realistically which specific benefits they aim to achieve. If there is a mismatch between the benefits of co-design and the project goals, there is a risk that the co-design activities produce fewer benefits than they could have realized-had the people involved articulated more precisely and realistically which benefits to aim for. Therefore, we propose that the people involved in organizing co-design in a service design project first identify the specific goals of the project as well as the possible benefits of one or more co-design activities, and then align these goals and these benefits.

In this paper, we will provide an overview of benefits of co-design in service design, which is intended to help the people involved with articulating more precisely and realistically which benefits they will aim for. In the next section, we review some literature on the benefits of co-design and user involvement. In the section after that, we present three cases of co-design in service design projects, in order to further explore possible benefits. After that, we present an overview of different benefits of co-design in service design. We close the paper by articulating conclusions and recommendations, and briefly discussing some ideas for future research.

Literature Review

Sanders (2002) distinguished three approaches to interacting with users and customers during a design process: "say", "do" and "make", where "make" is associated with co-design. In interviews, one can listen to what other people "say" and interpret what they express. Through observation, one can watch what other people "do" and how they use products or services. And in creative workshops, people can jointly explore and articulate their latent needs and jointly explore and "make" solutions. The key benefit of such "make" or co-design approaches is that they help to organize joint creativity.

Kujala (2003) identified the following benefits of user involvement (based on cases of ICT systems design): higher quality of system requirements, higher system quality, a better fit between the system and users' needs, and improved satisfaction of users or customers. Alam (2002) similarly identified the following benefits of user involvement (based on cases of service design): development of differentiated new services with unique benefits and better value for users, reduced development time, education of users (about the use, attributes and specifications of a new service), rapid diffusion and better market acceptance, improved public relations, and better long-term relationships between service provider and customers.

In addition, Muller (2002) discussed various participatory design methods and practices (Schuler & Namioka, 1993) and articulated a range of benefits, such as: improving mutual learning and understanding, combining and integrating different people's ideas, enhancing communication and cooperation between different people, and joint creation of new ideas. Furthermore, Kristensson, Magnusson and Matthing conducted various experiments in which they invited "ordinary users" to generate ideas for innovative mobile ICT services (Kristensson, Magnusson, & Matthing, 2002; Magnusson, 2003; Magnusson, Matthing, & Kristensson, 2003; Kristensson & Magnusson, 2010). They found that "users" can generate ideas that are useful input for service innovation; their ideas are more innovative ("originality") and better match users' needs ("user value") than the ideas generated by professional developers. However, the professional developers' ideas are more technologically feasible ("producability") than the ideas of "users".

Several authors from the UK wrote about co-creation (Cottam & Leadbeater, 2004), service design (Parker & Heapy, 2006) and transformation design (Burns, Cottam, Vanstone, & Winhall, 2006), with special attention to public services innovation and cases in the health care sector. Cottam and Leadbeater (2004), for instance, quoted from an article in the British Medical Journal that stated that "the key to successful doctor-patient partnerships is to recognise that patients are experts too"-experts of their experiences, "their social circumstances, habits and behavior, attitudes to risk, values and preferences"-and that both types of knowledge are needed in co-design. Similarly, Parker and Heapy (2006) advocated organizing cooperation between frontline professionals, who deliver the service, and the service's customers, who experience the service, because both their perspectives are needed for successful service design. Furthermore, Burns et al. (2006) discussed transformation design as a way not only to (re)design a service, but also to organize change processes and to promote creativity and innovation, so that the people involved can engage in continuous learning and innovating. Co-design approaches are critical in transformation design because they allow people to communicate and cooperate across disciplines and between organizations.

Roser and Samson (2009) identified the following benefits: access to customers' or users' experiences, which improves idea generation through shared knowledge, increased speed to market, better quality of products, higher satisfaction of customers and users, increased loyalty of customers and users, and lower costs. Furthermore, they identified several positive effects on the organizational level, for example: on "innovation practices and processes", on the "quality and speed at which decisions are made in relation to the development and filtering of ideas", and on the "creativity at individual and group level". Roser and Samson also discussed ways to use the Internet to facilitate interactions between a company and its customers, not only for the design and development phases, but also for delivery and usage, i.e. for co-creation. Using the Internet in such a manner provides additional benefits, such as being able to access and communicate with a relatively large number of people for relatively low costs.

Finally, Hoyer, Chandy, Dorotic, Krafft, and Singh (2010) recently reviewed a range of benefits of co-creation, categorizing them as improving efficiency or improving effectiveness. Efficiency can be improved, for instance, because users' input can partly substitute employees' input, and because co-creation facilitates continuous product or service improvements and reduces the risks of products' or services' failure. Effectiveness can be improved, for instance, because co-creation can help to develop products that better match customers' needs, resulting in more positive attitudes of customers towards products and services, and better relationships between the organization and its customers.

Cases of Co-design in Service Design

In order to further explore different types of benefits of co-design in service design, we will discuss three projects in which the authors were involved. The projects had different goals and different co-design approaches were followed, which enables us to discuss different types of benefits.

Case A: Co-design with Elderly People, to Develop Concepts for Health Care Services

This project, in the health care sector, aimed to develop and evaluate new service concepts, with the goal of helping people to better and more actively participate in their social networks. The main goal of following a co-design approach was to gain "insider knowledge" of the perception of elderly people regarding the development of their social networks as they grow older. Co-design was intended to counter people's tendency to overestimate their ability to understand elder people's daily lives, experiences and abilities (Hofmeester & De Charon de Saint Germain, 1999). A second goal was to improve the user value and the validity of the concepts that were developed.

Several co-design techniques were applied, aligning the benefits of co-design to the project's goals. In a first phase, a series of 17 guided interviews were conducted in two groups of people that were between 55 and 90 years old, from urban and rural communities, in order to jointly explore their ideas and perceptions of their social networks and the role these play in their daily lives. In a second phase, a study of their daily lives and routines was conducted, involving eight elderly people (from these 17 people), using diary studies over a period of four weeks. In a third phase, three elderly people (from these eight people) were invited to participate in a co-design workshop to validate the findings of the previous studies, and to identify ways in which they would like to be supported in actively participating in their social networks (See Figure 1).

The first goal, to gain insider knowledge, was achieved, because we made a number of observations that helped us to change some of our implicit assumptions about the ways in which elderly people participate in their social networks. For example, we had expected them to be more isolated than they were. This led to the identification of four new themes for further research. The second goal was to improve the value and the validity of concepts. Through co-design, we engaged in an ongoing dialogue with the elderly people that participated, which enabled us to jointly develop, verify and further develop ideas and themes, which helped us to generate valuable and validated concepts-more valuable and validated than concepts that would have been developed without interacting with users.

The process of co-design also yielded some unexpected benefits. First, the research results changed the implicit assumptions of the researchers. So, in addition to gaining new ideas or views, co-design can also help to change existing ideas or views. Other findings confirmed some initial ideas of the researchers, which helped them to feel that they were on the right track with their project. Furthermore, the validation of concepts by users, which was possible because some elderly people participated in three subsequent phases (interviews; diary studies; workshop), made the findings more convincing to people outside the project, who were not directly involved in the co-design process. Having validated the concepts enabled the researchers to more effectively present and communicate their findings and the service concepts to other people, such as project stakeholders.

The project also encountered some difficulties in the setting up and roll-out of activities. The main challenges were related to the targeted users, most of whom were of considerable age. A significant challenge lay in preparing the materials and methods in such a way that they would resonate with the participating elderly. A test sample showed that the purpose and way of working of the diaries needed some careful explanation. An additional measure was taken to do occasional checks by telephone with the participants, to verify if there was any confusion while doing the diary exercise.

Another challenge proved to be to retain a large enough user group over the envisioned series of activities. The project consisted of three stages of co-design activities, to be done with the same users, as the activities were closely linked to each other and built upon the generated materials. This proved to be a challenge, as many participants had to abandon the project prematurely for a variety of reasons. Part of this was because of the unavailability of the participants. Another factor was that the exercises were increasingly demanding for the participants in terms of time and attention, and participants were let free to opt out at any moment along the project.

Case B: Co-design with Children, to Generate Ideas for New Telecom Services

The commissioner of this project worked in a new business creation department of a large telecommunication services provider. He was looking for ways to stimulate creativity and innovation in his department and in the larger company, and developed the idea to organize co-design workshops in which children would generate innovative ideas for telecom services. He assumed that children are more capable of out-of-the-box thinking than adults (Druin, 2002), especially adults that have been working in the telecom industry for years. Furthermore, he would like to select some of the children's ideas as input for new business creation. Moreover, his idea was to invite children of the company's employees, so that these children's creativity might inspire their parents to think more creatively. In short, the intended benefits of co-design were: to generate innovative ideas as input for new business creation; and to improve some of the company's employees' creativity, via these employees' children's creativity.

We organized three workshops with a total of 50 children between seven and 10 years old. In these workshops, children worked in groups of approximately four children and one facilitator. The facilitators started with storytelling of adventures-time travelling to the future, flying to the moon, getting lost in a foreign country, or being in mediaeval times-and then invited the children to join the storytelling, and to generate ideas for "inventions" that the people in the stories would want to use or need to use. The children visualized their ideas first as drawings and then created models, using diverse materials, such as paper, cardboard, foam, cork, leather, and diverse plastic and metal parts (See Figure 2). At the end of each workshop, the children presented their "inventions", like they were in a children's TV show. We documented their ideas in colorful booklets with pictures of the drawings and models, sent these to the children as a souvenir for them, and to inform and inspire their parents. We also created compilations of the children's video presentations for the commissioner. Furthermore, we selected eight ideas for new business creation for the shorter term, such as a picture frame for video communication, a device for jointly making music while from different locations, a "hotline" between children and their parents, and a mobile buddy finder; and eight ideas for the longer term, such as wearable language translator, a device to touch things at another location, a hat that enables one to co-experience another person's experiences, and a pillow that produces pleasant dreams.

The goal of generating ideas for new business creation was realized: the children generated out-of-the-box ideas that were assessed as innovative by the commissioner. In other words, the project's goal of generating ideas for new telecom services and the benefit of mobilizing creativity by organizing co-design workshops with children were aligned. However, the commissioner, who was also its main sponsor, was unavailable to organize follow-up activities because he got another job within the company and then left the company. It was difficult for the project team members to find a new supporter for the project. As a result, the children's ideas were not actually used for new business creation. The idea for the project was very much the idea of this commissioner. This was an advantage because the project had the commissioner's full support, but this also proved to be a weakness after he left the organization and the project failed to produce concrete follow-up activities.

The other goal of the project, namely to improve the creativity of the parents, is difficult to evaluate. We had ideas for distributing the booklets and video compilations more broadly within the company, but without support from the company, we could not do that. In other words, we were unable to organize an effective process to involve the parents in the dissemination of the findings and ideas, and to monitor and evaluate whether and how the parents' creativity was influenced by their children's creativity.

Case C: Co-design with Employees, to Improve a Current Logistic Service

This project was commissioned by a company in the logistics sector. The company wanted to improve its service and customer process of ordering and first use of a post office box, as well as to improve customers' satisfaction with this service. The goal of organizing co-design was to foster commitment from the company's employees that are involved in the service design project to improve the customer processes. This was especially important given the fact that these employees are from different disciplines and different departments. Furthermore, it was important for the company that the employees of the service management department would also learn to apply co-design methods in future projects (See Figure 3).

We used a Customer Journey (De Koning, 2010) method to investigate and improve the customers' experiences. This method consists of three steps: 1) Measure; 2) Experience; and 3) Improve. The Customer Journey started with analyzing customers' experiences. Measuring the current customer experience and enabling the employees to experience their customers' experiences and emotions motivates them to improve the customer experience and to change their customer processes. In all three stages, co-design with employees is critical.

First, we cooperated with two service managers to analyze the current customer journey: What are the experiences of customers when they interact with the company? What are all the steps for ordering and first use, from a customer's perspective? What are the interaction moments, via a website, a contact center, brochures, or at a post office? This was a complex task since in large organizations there are often isolated departments ("silos"), in which employees tend to focus on only a part of the service or on internal processes ("inside-out thinking"). Only few people have an overview of the total customer experience. Together with two service management employees, we developed a way to measure customers' emotions during their customer journey. Since emotions influence behavior, an understanding of customers' emotions helps to assess which things go well and which issues need to be improved.

Based on customer experience measurements, we then designed a "teaser" for the employees that were involved in the post office box customer process. The ways in which we engaged them were intended to simulate the experiences that their customers go through. For example, we made it difficult for the employees to register for the workshop, similar to how they made it difficult for their customers to order their service. And we used formal language in our communication with them, to simulate their way of communicating with their customers. As a result, the employees felt the similar emotions as their customers, and they felt a sense of urgency to improve their service. Finally, we organized a workshop with all the employees that were involved in the process: from call center agents to product managers. In this workshop, we jointly designed an improved customer journey, which was further developed in some smaller working groups.

The improved customer journey provides customers with an online tool that allows them to easily find and apply for a nearby post office box. This improvement was evaluated and significantly improved customers' satisfaction and their intention to recommend the service to others.

The goals of using co-design were realized. The quality of the service was improved and customers were more satisfied. Moreover, the commissioner was so enthusiastic about the Customer Journey approach that they integrated it into their standard ways of working, as a periodic check-up of their service. The employees learned to improve customer processes and were able to apply the method for themselves, and the involvement of people from different disciplines improved the organization's commitment to change.

Overview of Benefits

Based on the literature review and on the three cases, we identified a range of benefits of co-design in service design (See Table 1). We propose to order these benefits into three categories, the columns in Table 1: 1) benefits for the service design project; 2) benefits for the service's customers or users; and 3) benefits for the organization(s) that are involved. Additionally, we propose to order the benefits also into four categories, the rows in Table 1: 1) improving the creative process, for example, idea generation; 2) improving the service, that is, the outcome of the project; 3) improving project management, for example, in terms of business rationale; and 4) improving longer-term effects, for example, on the market or on society.

Benefits for the Service Design Project

Many businesses and organizations expect a wide range of benefits from organizing co-design, such as: improving the creative process, developing better service definitions, organizing the project more efficiently, and improving customers' or users' loyalty. Case A provides an example of how cooperation with elderly people helped to develop a better understanding of these people's needs, and to jointly develop and validate concepts for new services. Case B provides an example of a similar benefit, that is, the development of out-of-the-box ideas by organizing co-design workshops with children. In Case C the co-design process with the service provider's employees delivered another, and rather immediate, benefit, namely: recommendations to improve the definition of an existing service.

Benefits for the Service's Customers

During a service design project, the actual service is not always already available, except in a project of re-designing an existing service. Therefore, customers are often not yet able to directly experience the benefits of co-design during the service design project. However, they are likely to experience benefits after the service is developed and provided. For example, in Case A the elderly people that cooperated during the co-design process may, in the future, experience the benefit of using a service that better matches their needs after the service is developed and becomes available. In Case C, for example, the benefit for customers and users is more immediate: they are likely to experience a higher quality of service.

Benefits for the Organization(s)

Organizing co-design processes can also yield benefits for the organization(s) that are involved, independent of the actual service design project. For example, organizing and participating in co-design can help an organization to foster creativity or to develop its capabilities to innovate. In Case B, one of the intended benefits of co-design was to promote out-of-the-box creativity within the organization, also outside the immediate context of the project. Case C provides two other examples of benefits for the organization: involving employees from different departments promoted communication and cooperation between them; and several employees learned to conduct co-design, from which they can benefit in future projects.

In addition, we would like to note that the identified benefits relate to the benefits of organizing co-design during a service design project, and not to the benefits of providing (better) services. This remark may help to appreciate the relatively small number of benefits for the service's customers or users. There would have been a larger number if we had included the general benefits of providing (better) services. The entire process of developing and providing services is (or should be) oriented towards delivering benefits for customers and users. Furthermore, we found relatively a lot of benefits for the organizations involved that go beyond the immediate benefits for one specific project.

Moreover, we would like to draw attention to the fact that co-design can offer benefits to service design both in commercial sectors, such as financial services, and in not-for-profit sectors, such as health care (Mager, 2009). The types of benefits may appear to be rather different, especially in their different wordings. For a financial service one would speak, for instance, about numbers of customers, about improving sales and profits; whereas one would, for instance, for a health care service, speak about the number of clients, improving their health and reducing the costs involved. Despite these different wordings, we think that the benefits refer to the underlying concepts that are not very different for commercial or not-for-profit sectors: improving services and people's experiences. One may want to keep such translations in mind, in order to imagine benefits in not-for-profit sectors.

Ideas for Future Research

In order to further improve co-design practices in service design, there is a need for methods or tools to monitor and evaluate whether the intended benefits are actually realized. Roser and Samson (2009), for example, suggested articulating key performance indicators (KPI's) and monitoring the realization of these. They suggested a range of KPI's, such as: the amount of new ideas for products/services, the originality, value or realizability of these ideas, time to development of new ideas, time to market for new products/services or improvements, cost reductions, revenues, profitability and market share of the new/improved product/service, time to break-even for new product/service introductions, customer loyalty, and customer satisfaction.

Other topics that would require further research are the costs and risks of co-design. Obviously, there are costs involved in organizing a co-design process, in terms of people, time and money. One may argue that these costs are "paid back" by the benefits which co-design offers. But in a business context, it can be advantageous if both costs and benefits can be articulated in financial terms. Furthermore, there are risks associated with co-design. Hoyer et al. (2010), for example, discussed two types of risks. The first type is related to diminished control over the project, because other people, other departments or other organizations are involved (see also Roser and Samson's (2009) discussion of increased dependency on outside collaborators). The second type of risks is related to increased complexity of the project, because the objectives and interests of diverse people, departments or organizations must be managed and balanced, which can require extra coordination efforts (see also Roser and Samson's (2009) discussion of extra co-ordinating costs and the need for new management skills and different management styles).

Conclusions and Recommendations

In this paper, we identified three types of benefits of co-design in service design projects (Table 1):

Benefits for the service design project itself, such as improving the creative process, developing better service definitions and organizing the project more effectively or efficiently;

Benefits for the service's customers or users, such as creating a better fit between the service offer and customers' or users' needs, a better service experience and higher satisfaction;

Benefits for the organization(s) involved, such as improving creativity, a focus on customers or users, cooperation between disciplines, and capabilities and enthusiasm for innovation.

Furthermore, we would like to recommend that the people involved in co-design-for example, researchers, designers, developers, managers and other stakeholders-first identify the desired goals of the service design project in which they will work and to which they will contribute, and also identify the intended benefits of their co-design activities, and then carefully align these goals and benefits-for example, by selecting appropriate co-design methods and applying these in ways that contribute optimally to the project. Additionally, we recommend that the people involved find ways to monitor and evaluate whether the intended benefits of co-design are actually realized in the project, and ways to take into account the costs and risks involved in co-design.

Moreover, it is interesting that we found relatively a lot of possible benefits for the organizations involved, and also positive longer-term benefits-more than we had expected. Based on this, we speculate that organizations are often not aware of these organizational and longer-term benefits of co-design. We therefore propose that the people involved in co-design-and, more specifically: the people involved in the decision making process of weighing the pros and cons of co-design, in assessing the costs and risks and benefits of co-design, and in defining the goals and scope and budget of co-design projects-draw attention to these "bonus" benefits, in addition to the immediate benefits in one project. In that sense, we believe that co-design offers many opportunities to further promote a focus on customers and users, to foster creativity and cooperation, and to improve organizations' innovation capabilities and practices.

For co-design efforts to be effective, that is, to deliver the intended benefits, it is important to select appropriate methods and ways of working, and to apply them appropriately. In the cases presented above, the methods and ways of working were tailor-made to match each service design project. This is a key challenge of co-design, because each choice regarding methods and ways of working can significantly affect the project's process and outcomes. It is critical to identify the appropriate people (for example, employees, customers, users), to involve them in appropriate stages and to give them appropriate roles in the project.

In order to better realize the potential of co-design, we propose that the people involved first identify the specific goals of the service design project and then match their co-design activities to these goals. Furthermore, we advocate documenting and disseminating the findings from co-design, within the organization(s) involved, in ways that engage relevant people, for example, those responsible for budgets and decision making, in order to improve the adoption and application of these findings. Moreover, we advocate documenting and sharing findings, also regarding methods and ways of working, within the organization(s) involved, so that these can be used in future projects. Such practices will help organizations to improve their capabilities to conduct co-design effectively, that is, to cooperate across disciplines and to cooperate with customers and users.

References

1. Alam, I. (2002). An exploratory investigation of user involvement in new service development. Journal of the Academy of Marketing Science, 30(3), 250-261.

2. Binder, T., Brandt, E., & Gregory, J. (2008). Editorial: Design participation(-s). CoDesign, 4(1), 1-3.

3. Burns, C., Cottam, H., Vanstone, C., & Winhall, J. (2006). Transformation design. London: Design Council.

4. Cottam, H., & Leadbeater, C. (2004). Health: Co-creating services. London: Design Council.

5. De Koning, N. (2010). Customer journey: Experience your customer's journey. Retrieved October 30, 2010, from http://www.tno.nl/downloads/customer_journey_226e1.pdf

6. Druin, A. (2002). The role of children in the design of new technology. Behaviour and Information Technology, 21(1), 1-25.

7. Edvardsson, B., Gustafsson, A., Kristensson, P., Magnusson, P., & Matthing, J. (Eds.). (2006). Involving customers in new service development. London: Imperial College Press.

8. Hofmeester, K., & De Charon de Saint Germain, E. (Eds.). (1999). Presence: New media for older people. Amsterdam: Netherlands Design Institute.

9. Hoyer, W. D., Chandy, R., Dorotic, M., Krafft, M., & Singh, S. S. (2010). Consumer cocreation in new product development. Journal of Service Research, 13(3), 283-296.

10. Kristensson, P., & Magnusson, P. (2010). Tuning users' innovativeness during ideation. Creativity and Innovation Management, 19(2), 147-159.

11. Kristensson, P., Magnusson, P., & Matthing, J. (2002). Users as a hidden resource for creativity: Findings from an experimental study on user involvement. Creativity and Innovation Management, 11(1), 55-61.

12. Kujala, S. (2003). User involvement: A review of the benefits and challenges. Behaviour and Information Technology, 22(1), 1-16.

13. Mager, B. (2008). Service design. In M. Erlhoff & T. Marshall (Eds.), Design dictionary: Perspectives on design terminology (pp. 354-357). Basel: Birkhäuser.

14. Mager, B. (Ed.). (2009). Health and service design [Special issue]. Touchpoint: The Journal of Service Design, 1(2).

15. Magnusson, P. (2003). Benefits of involving users in service innovation. European Journal of Innovation Management, 6(4), 228-238.

16. Magnusson, P., Matthing, J., & Kristensson, P. (2003). Managing user involvement in service innovation: Experiments with innovating end users. Journal of Service Research, 6(2), 111-124.

17. Muller, M. J. (2002). Participatory design: The third space in HCI. In J. Jacko & A. Sears (Eds.), The human-computer interaction handbook: Fundamentals, evolving technologies and emerging applications (pp. 1051-1068). Mahwah, NJ: Lawrence Erlbaum Associates.

18. Parker, S., & Heapy, J. (2006). The journey to the interface: How public service design can connect users to reform. London: Demos.

19. Roser, T., & Samson, A. (2009). Co-creation: New paths to value. London: Promise / LSE Enterprise.

20. Sanders, E. B. N. (2002). From user-centred to participatory design approaches. In J. Frascara (Ed.), Design and the social sciences: Making connections (pp. 1-8). London: Taylor & Francis.

21. Sanders, E. B. N., & Stappers, P. J. (2008). Co-creation and the new landscapes of design. CoDesign, 4(1), 5-18.

22. Schuler, D., & Namioka, A. (1993). Participatory design: Principles and practices. Hillsdale, NJ: Lawrence Erlbaum Associates.

23. Sleeswijk Visser, F., Stappers, P. J., Van der Lugt, R., & Sanders, E. B. N. (2005). Contextmapping: Experiences from practice. CoDesign, 1(2), 119-149.

# data[].attributes.attachment_link.title
Benefits of Co-design in Service Design Projects

# data[].attributes.attachment_link.description
In many service design projects, co-design is seen as critical to success and a range of benefits are attributed to co-design. In this paper, we present an overview of benefits of co-design in service design projects, in order to help the people involved to articulate more precisely and realistically which benefits to aim for.

# data[].attributes.headline
The Assessment of Meaningful Outcomes from Co-design: A Case Study from the Energy Sector

# data[].attributes.subject
The Assessment of Meaningful Outcomes from Co-design: A Case Study from the Energy Sector

# data[].attributes.attachment_link.readable_text
Highlights:

*

A theoretical framework can scaffold a co-design process, the substance of user contributions to it, and assessment of these contributions.

*

Co-design enables user needs to be uncovered and addressed.

*

Increased tailoring of design outcomes to users' needs with increased "meaning" uncovers further underlying needs.

*

The householders' needs evolved as exposure to new information helped to enhance their understanding.

*

Particular care must be exercised to ensure the internal and external validity of co-design insights.

Abstract

Even though co-design is a well-accepted approach for designing to meet user needs, what influence it has on design outcomes remains unclear. This article presents the co-design process of a prototype energy advice service. We evaluate the impact this process had on the outcome over time, demonstrate how co-design generated informative insights, and identify the benefits and challenges of employing a co-design process to design and develop meaningful content for future 'information-intensive' services. A theoretical framework, a "think aloud" approach, and systematic data coding, enabled us to uncover user perceptions of the evolving design qualities. This meaning-making co-design process enabled user needs to surface and be iteratively addressed. As the content of the reports became increasingly tailored, and the users' familiarity with the topic increased, the process highlighted further evolving and underlying information needs. This confirms the value of adopting a content first approach when designing information intensive services and foregrounding meaning making within the complex energy demand reduction context.

Recommended articles Citing articles (0)

# data[].attributes.attachment_link.title
The Assessment of Meaningful Outcomes from Co-design: A Case Study from the Energy Sector

# data[].attributes.attachment_link.description
A theoretical framework can scaffold a co-design process, the substance of user contributions to it, and assessment of these contributions. * Co-design enables user needs to be uncovered and addressed. * Increased tailoring of design outcomes to users' needs with increased "meaning" uncovers further underlying needs.

# data[].attributes.attachment_link.readable_text
UX maturity measures an organization's desire and ability to successfully deliver user-centered design. It encompasses the quality and consistency of research and design processes, resources, tools, and operations, as well as the organization's propensity to support and strengthen UX now and in the future, through its leadership, workforce, and culture.

Background

In 2006, Jakob Nielsen developed one of the earliest UX-maturity models, defining 8 phases of UX maturity. Each stage described UX presence in different organizational circumstances - from companies that engaged in absolutely no user research to ones that achieved peak focus.

Since then, much has changed in the field of UX: how we perform design and research, our knowledge about jobs, work, processes, tools, deliverables, leadership involvement, and, how enterprises view and prioritize UX into overall strategy. With these changes in mind, we have revised how UX maturity should be modeled and measured to incorporate the new norms and to describe the organizational evolutions we've witnessed in our thousands of clients and students over the past decade and a half.

This article details the NN/g UX-maturity model. As we continue to work with organizations to assess UX maturity and gather additional data about the evolution and outcomes of UX maturity, we plan to continue refining this model.

The UX-maturity model provides a framework to assess each organization's UX-related strengths and weaknesses. We can use that assessment to determine which of the 6 stages an organization currently occupies. Further, this model provides insights about how an organization can increase its UX maturity.

The 6 stages of UX maturity are:

1. Absent: UX is ignored or nonexistent.

2. Limited: UX work is rare, done haphazardly, and lacking importance.

3. Emergent: The UX work is functional and promising but done inconsistently and inefficiently.

4. Structured: The organization has semisystematic UX-related methodology that is widespread, but with varying degrees of effectiveness and efficiency.

5. Integrated: UX work is comprehensive, effective, and pervasive.

6. User-driven: Dedication to UX at all levels leads to deep insights and exceptional user-centered-design outcomes.

Factors in UX Maturity

Improving UX maturity requires growth and evolution across several different factors, including:

Strategy: UX leadership, planning, and resource prioritization

Culture: UX knowledge and cultivating UX careers and practitioners' growth

Process: the systematic use of UX research and design methods

Outcomes: intentionally defining and measuring the results produced by UX work

None of these factors stand alone; rather, they reinforce and enable each other. Knowledge of UX processes does not create a great UX team if UX work is not prioritized by the organization's leadership; likewise, belief in the value of UX only becomes actionable when there are methodologies in place to 'practice what you preach.' Organizations must progress in all these dimensions in order to reach high levels of UX maturity and realize the full value of user-centered design.

Stage 1: Absent

A company at this stage is either oblivious to UX or believes it does not need it. User-centered thinking is not at all part of how it works. (A company has a user-centered mindset if users are the driver behind its strategy, tactics, and decisions.)

At this stage, UX work is not planned, let alone incorporated into the organization's vision. The few people at the organization who think about users are likely ignored or dismissed.

Most companies at this stage fall outside the technology and software fields (which spurred the recent UX growth) and exist in industries where UX is unknown or rarely practiced. Other organizations at this level might be anything from startups to established organizations with inherited work processes that were not focused on users.

Variations Within This Stage

When UX is absent, there may complete ignorance about UX, apathy, or half-hearted intentions which favor the idea of UX but never follow through with actions. In some cases, there may even be hostility towards the entire concept of adopting UX practices.

This stage also includes organizations where developers are aware that UX is important and attempt to create good experiences, but do not have any access to methods, resources, or organizational support. (In our original model, this 'developer-centered' UX was a separate, higher stage; we now group these organizations within stage 1 because even the best intentions of hard-working developers are not enough to yield meaningful results in the absence of organizational support.)

Obstacles to Overcome When UX Maturity Is Absent

The primary obstacle is a lack of education:

What UX is

Its benefits to the organization and customers

Possible internal UX processes

How to begin doing UX work

To progress from this stage, organizations should focus on building UX awareness.

(Detailed article about Stage 1: Absent UX Maturity.)

Stage 2: Limited

An organization in the limited stage approaches UX erratically. Small UX efforts are made, usually for one of three reasons:

1. Legal necessity

2. A UX-aware individual (maybe a leader) who takes initiative

3. An experimental team that attempts UX methods

While organizations at this stage may show some UX awareness and engage in occasional UX activities, UX work is not done routinely, nor is it consistently well-executed or incorporated into strategy and planning. UX falls low among priorities. There's no official recognition of user experience as a discipline and there are no UX-dedicated roles, processes, or budget.

When a UX budget exists in limited-maturity organizations, it is not systematically allocated and used.

Variations Within This Stage

In many companies at this stage, UX-focused activities occur in silos, within one or two departments, while the majority of the organization still sits at stage 1. Limited UX work may recur occasionally within the same isolated team or may sporadically be attempted by other teams.

Obstacles to Overcome When UX Maturity Is Limited

Challenges at this phase are often related to process: learning methods for doing UX work, organizing a team of multiple people, and beginning to establish routines and resources.

To progress from this stage, organizations should focus on getting people to listen by showcasing the small UX-related wins, compiling positive case studies, and cultivating relationships with UX championsso UX can gain traction.

(Detailed article about Stage 2: Limited UX Maturity.)

Stage 3: Emergent

When UX maturity is emergent, organizations exhibit UX work in more teams, engage in some UX-related planning, and may have UX budgets. The UX efforts are, however, small, unstable, and based on individual manager initiatives rather than organizational policies. Some teams that use several research and design methods and do multiple research studies may begin to see the benefits and results of their efforts.

In emergent-stage organizations, there are people in UX roles, but not nearly enough and not necessarily with the right skills. At this stage, organizations are still working on proving the value and impact of UX. There are no widespread, systematic UX processes in place. There is some managed usability - some leaders have bought in and may advocate for it. But when tradeoffs are necessary, UX is still the first to go. UX is not yet prioritized as an essential strategy.

It is common to see large enterprises hover at stage 3, especially in traditional fields like finance and healthcare.

Variations Within This Stage

UX work could be happening but inconsistently. Or, work might be somewhat consistent, but it has no impact and people do not see its value.

Obstacles to Overcome When UX Maturity Is Emergent

It's easy to get stuck here thinking "We do UX now" and assuming that this stage is good enough when it is definitely not. To progress from this stage, organizations should focus on building a culture of support for UX at all levels, to gather momentum and move forward across the organization and between projects. Make sure that UX priorities are given due consideration when tradeoffs are made.

(Detailed article about Stage 3: Emergent UX Maturity.)

Stage 4: Structured

Structured UX means that the organization recognizes the value of UX and has established a full UX team or multiple teams. Leadership usually supports UX and sometimes incorporates it into high-level strategies. There is a centralized definition of design and a shared, iterative human-centered design process.

User research is conducted throughout the product lifecycle. Politics and miscommunication may cause a misallocation of resources and overspending on UX-related work, product areas, or products that do not need it.

This level is where most organizations which function acceptably land and is as far as many companies will ever go in their UX maturity.

Variations Within This Stage

Individuals in UX can be comfortable and confident. Other groups (product, development, marketing, QA, etc.) are involved in some parts of UX work. The operations side of UX is usually strong at this level, as is the variety of research and design methods employed.

Obstacles to Overcome When UX Is Structured

Though teams use UX-centered processes and see their benefits at this stage, they still face hidden weaknesses, which can often be traced back to strategy: unsupportive leaders, tensions with responsibilities and ownership as teams scale, success metrics (and bonuses) that have little to do with UX, development processes that don't include discovery research or iterative design, and focus on catering to a few big customers rather than adhering to a proactive UX strategy.

(Detailed article about Stage 4: Structured.)

Stage 5: Integrated

When organizations reach the stage of integrated UX, their UX work has become comprehensive, pervasive, and universal. Almost all teams within the organization usually perform UX-related activities in an efficient, effective manner. There is often innovation in UX methods and processes and even contributions to the field of UX as a whole. The organization's important success metrics - that leaders care most about - have a focus on UX or are even driven by UX-related work.

Integrated UX is the stage that most organizations should aim to reach. At this stage, UX work has become highly effective at serving business goals. (At the next level, UX work serves not business goals, but users themselves; but that prioritization is not always practical, nor sustainable.)

Variations Within This Stage

The processes and staff may be high-quality and consistent, but the organization may get too focused on process instead of outcome and effects or the leaders may be focused on metrics that are not user-centered.

Obstacles to Overcome When UX Maturity Is Integrated

Although user-centered design is respected, understood, and supported, factors other than user-centeredness are truly driving the business. Focus on establishing user-centered outcome metrics at the highest levels of the organization.

(Detailed article about Stage 5: Integrated.)

Stage 6: User-Driven

In Stage 6, UX is the norm - habitual, reproducible, and beloved across the organization. Few companies operate at this stage.

At this stage, everyone is fully enlightened about user-centered design. Understanding user needs through research is the primary driver of the organization's strategy and project prioritization. Development encompasses user-focused, iterative design. Leaders, teams, and individuals are user-centered and look to UX in day-to-day work - from the highest level of strategy to the tiniest design elements or research studies. They plan for change and innovation.

Organizations at this maturity level have invested in contributing back to industry standards and rely on user research to drive new investments and markets.

No Real Variations Within This Stage

Unlike the other stages, this stage has no variations: The business' vision is user-centered design or is highly intertwined with user-centeredness.

However, this stage may not be sustainable for long durations of time. For example, a large tech company may have been entirely user-driven 10 years ago, but gradually, through growth, acquisitions, and changes in leadership and culture, may allow the focus to shift away from users. When conversions or other business-focused metrics displace user-centered thinking, organizations slip back down to structured (stage 4) or even emergent (stage 3) UX-maturity levels.

Obstacles to Overcome in the User-Driven Maturity Stage

While this stage is the ultimate UX-maturity goal, it is difficult to achieve and tough to maintain for long durations before issues arise (resource bloat, conflicting goals, or politics) and maturity regresses. Focus on keeping momentum of the UX effort, championing UX values, and educating new team members, to prevent the organization from falling back.

(Detailed article about Stage 6: User-Driven.)

Team Maturity vs. Organization Maturity

When it comes to UX maturity, no team is an island. A single team or division within a company cannot reach the highest maturity levels, integrated and user-driven stages, while other teams lag behind. It is the consistency among teams which enables these highest maturity levels. Good information, tools, and other resources shared across teams and groups increase UX maturity. Also, UX-focused leadership and knowledge sharing must happen at the organization-wide level, not just within teams. Thus, true UX maturity should be evaluated for an entire organization rather than for a single team.

UX maturity also goes beyond one product, because any product can affect both 1) how UX work gets done and 2) how users and customers perceive other products and brands. To assess organizational maturity, the organization as a whole, including all product groups and teams, should be evaluated.

All of that said, individual product teams or functional groups can evaluate how UX-mature their siloed area is. Looking at their own operations can help them see how they affect the organization's UX maturity.

Judging Your Organization's UX Maturity

A complete evaluation of an organization's UX maturity should be based on diverse assessment methods to thoroughly understand current UX work and output, including observation of and interviews about work practices; analysis of processes, people, and tools; assessment of deliverables; and surveys of people from across the organization.

Self-Assessment Quiz

As a first step to help you begin thinking about maturity and where your organization might be, we've created a free, self-reported UX-maturity quiz, which you can use to estimate maturity based on your current knowledge. This quiz takes most people fewer than 10 minutes to answer.

Conclusion

Understanding your organization's UX maturity is important so you can identify your strengths and weaknesses, uphold and reward what's done well, and acknowledge and improve what is not. Even if UX work seems fine at your organization, know that there are probably things you could be doing better that would improve the product- or service-creation process, people's work, your organization and its credibility, and your customers' and users' experiences.

Whatever your current level, a realistic goal is to progress to the next-higher level. It is unrealistic to expect to leapfrog levels and go directly from level N to level N+2. The organization must spend time in level N+1 while the practices that characterize that level permeate it and create the necessary readiness to advance to the subsequent level. (Of course, once at level N+1, you might set the goal to reach level N+2 a few years later.) It can take several years to uplift an organization by a single maturity level, so don't give up!

Regardless of your organization's current UX activities, increasing understanding and self-awareness about your UX maturity can only improve and sustain the quality of your UX work and, ultimately, the experiences you create for your users. Even organizations performing at high UX-maturity levels benefit from assessing what works and why. This knowledge allows systemizing UX work to make it truly part of the organization's DNA, so that good UX practices survive through big organizational changes.

We thank Jakob Nielsen, Kate Kaplan, Rachel Krause, Kim Salazar, Feifei Liu, Raluca Budiu, Kelley Gordon and the rest of our colleagues at NN/g for their work and contributions to this model.

We offer a variety of consulting projects to help you assess and improve your UX Maturity. Please contact us if you are interested: consulting@nngroup.com.

# data[].attributes.attachment_link.description
Our UX-maturity model has 6 stages that cover processes, design, research, leadership support, and longevity of UX. Use our quiz to get an idea of your organization's UX maturity.

# data[].attributes.headline
Co-design, evaluation and the Northern Ireland Innovation Lab

# data[].attributes.subject
Co-design, evaluation and the Northern Ireland Innovation Lab

# data[].attributes.attachment_link.readable_text
ABSTRACT

ABSTRACT
Around the world there are more than 100 policy labs-multi-disciplinary government teams developing public services and policies using innovation methods to engage citizens and stakeholders. These policy labs use a range of innovation methods and approaches, including co-production, co-creation, co-design, behavioural insights, systems thinking, ethnography, data science, nudge theory and lean processes. Although the methods may vary, one element is consistent: policy labs actively, creatively and collaboratively engage the public and a wide range of stakeholders in jointly developing solutions. The Northern Ireland Public Sector Innovation Lab (iLab) is part of a growing UK and international community of policy labs using co-design to engage with users for value co-creation, aiming to improve public governance by creating a safe space to generate ideas, test prototypes and refine concepts with beneficiaries. Drawing on iLab's experience, this paper explores three questions: What are the main determinants of effective co-design? What are the unintended consequences of co-design? And what lessons can be learned from iLab and shared with other policy labs?

IMPACT
There is a need to reinstate the legitimacy of public policy-making and public service development through more effective citizen engagement. To experiment with more creative and user-centred approaches, governments are establishing Policy Labs to engage citizens at multiple stages of the development process. The Northern Ireland Public Sector Innovation Lab (iLab) is part of a growing UK and international community of Policy Labs using co-design to engage with users for value co-creation, aiming to improve public governance by creating a safe space to generate ideas, test prototypes and refine concepts with beneficiaries.

Introduction

The Northern Ireland Public Sector Innovation Lab (iLab) aims to improve public services and public policies by creating a safe space to co-create ideas, test prototypes and refine concepts with citizens, civil servants and stakeholders (see www.finance-ni.gov.uk/articles/introduction-innovation-lab). It was established in April 2014 by the then minister for finance in the Public Sector Reform Division of Northern Ireland's Department of Finance. In its first two years, iLab led 18 projects focused on a wide range of service and policy challenges. The challenges ranged from improving the use of data analytics within the government, to encouraging people to pay court fines, and optimizing how patients manage their medication. In autumn 2016, iLab commissioned an evaluation of both its activities and governance based on 30 interviews with lab staff, the wider Northern Ireland Civil Service (NICS) and external stakeholders. The outputs included four impact case studies, an analysis of the strengths and weaknesses of the Lab's leadership, operating model, methods and capacity, as well as a series of recommendations for enhancing impact. Policy labs tend to position themselves as a 'safe space to innovate', so evaluations of their activities and governance are not common. There has been a policy drive for increased innovation-with citizens, for citizens-and iLab has adopted co-design as one of its main methods. The iLab is uniquely positioned to drive more transparent and inclusive public decision-making.

Based on the evaluation, this paper focuses on the determinants of effective co-design, as well as its unintended consequences, in order to advance knowledge and share lessons with other labs. Based on the iLab, the main determinants of effective co-design are:

Clarity of language and process.

Clear selection criteria for projects.

Building in evaluation from the outset.

Promoting good practices to the wider community.

Support from senior decision-makers.

Conversely, from iLab's experience, there were two significant unintended consequences of co-design-raising stakeholder expectations and a lack of prototyping prior to implementation, which are explored in more detail in this paper. This is also framed by the nature of 'value'-what sort of value are these labs attempting to create, how is value understood, and how it is enacted? This evaluation and analysis provides the framework for answering our three main questions:

What are the main determinants of effective co-design?

What are the unintended consequences of co-design?

What lessons can be learned from iLab's experience in Northern Ireland and shared with other policy labs?

Theory versus practice

To what extent can academic theory on co-design drive government practice in policy labs? There are three parts to this question: What is co-design? What are policy labs? What is the role of academic theory in government practice in this context?

What is co-design?

Co-design has entered the lexicon of government, politics and policy-making (Bason, 2014, p. 4; Christiansen & Bunt, 2014, p. 47; Williamson, 2015, p. 258; Voorberg, Bekkers, & Tummers, 2015). Co-design is a well-established approach to creative practice, particularly within the public sector, with its roots in the participatory design techniques developed in Scandinavia in the 1970s (Puttick, Baeck, & Philip Colligan, 2014, p. 13), but having taken distinctly different paths in the US and in Europe (Sanders & Stappers, 2008, p. 5). Participatory design is an approach which is focused on processes and procedures of design and is not a design style; co-design is often used as an umbrella term for participatory, co-creation and open design processes. Thus, it can be a difficult concept for civil servants and policy-makers to grasp and ground in their domain. Currently, policy labs adopt a broad range of design-related terminology to refer to designing with citizens, including co-design, service design, user-centred design, policy design, strategic design, participatory design and design thinking, among others. For those not immersed in the academic discipline-or perhaps in practice-the distinction between these various terms is certainly not clear, particularly for those groups directly engaging with the labs-civil servants, representation groups and the general public. This evolution in design research from a user-centred approach to co-designing has changed the roles of the designer, the researcher and the person formerly known as the 'user' (Sanders & Stappers, 2008, p. 5). Design is interpreted differently by its many stakeholders depending on the context and agenda; furthermore, theory and practice on design is expanding and evolving (Borja de Mozota, 2002, p. 94; Hobday, Boddington, & Grantham, 2012, p. 272; Utterback et al., 2006, p. 1). Although design has many attributes, and the literature on design is by no means homogenous, increasingly, design terminology is converging around the notion of creative, user-focused problem-solving (Brown, 2009, p. 236; Verganti, 2009, p. 12; Bason & Schneider, 2014, p. 38; Whicher, 2017, p. 81).

Sanders and Stappers ( 2008, p. 5) mapped a design research landscape in which more than 20 distinct design research areas can be identified (see Figure 1). The design research landscape is helpful for understanding co-design in the context of other design research domains. Sanders and Stappers ( 2008, p. 5) contend that, since the 1970s, there has been a gradual paradigm shift from the 'user as subject' with user-centred design approaches to the 'user as partner' with co-design. The design research landscape positions a number of design research areas across an x-axis from 'user as subject' to 'user as partner' and on the y-axis from 'led by research' to 'led by design'. The implication being that co-design occupies a position that is both research-led and design-led but firmly positioned at the 'user as partner' end of the spectrum. The plotting of the various design research areas along the x- and y-axis should be interpreted broadly as the landscape is 'continually changing' (Sanders & Stappers, 2008, p. 17). As such, new design domains have begun to attract attention in government, especially service design in the early 2000s (Morelli, 2002, p. 3) and policy design in the late 2000s (Bason, 2010, p. 5), among others, and have thus been added to the design research landscape. This illustrates that the field of design research is both expanding but arguably fragmenting.

There is a trend to prefix 'co' onto words to imply jointly developing; such as co-design, co-creation and co-production. Again, to a specialist in the field these concepts are distinct, but the nuances between them are confusing to generalists within the civil service (Osborne, Radnor, & Strokosch, 2016). It raises the question of whether they only require a higher-level understanding of why and how to involve citizen, leaving the finer details to domain experts. For Bason ( 2014, p. 4) 'design is shifting to the concept of "co": to co-llaboration, co-creation and co-design as a central feature, emphasising the explicit involvement of users, partners, suppliers and other stakeholders in the design process'. Williamson ( 2015, p. 258) specifically connects co-design to policy and public service development as the shared elucidation of 'options between actors in the governance system and its end-uses'. According to Bason and Schneider ( 2014, p. 37): 'Co-design and other participatory design methods appear as valuable tools for fostering citizen engagement and supporting shared models for decision-making'. The transition from user-centred design to co-design implies the active involvement of users at multiple stages of the development process from analysing user needs, defining the challenge with users, involving users in jointly developing concepts, testing prototypes with users and refining solutions with users. This shift from user as subject to partner has fundamentally altered the role of the citizen from passive consumers to public services and public policies to active collaborators in their formulation. Again, it is framed by the nature (and multiple dimensions) of 'value'-what sort of value are these processes attempting to create, how is value understood, and how it is enacted? While the concept of co-creation has been considered in public management theory in recent years, the discourse has suffered from conceptual limitations; in some circumstances it has been offered as inter-changeable with co-production. We thus have to consider how the public service organization is regarded as a co-creator of value, not just the service user (Osborne, 2018). The following definition was adopted for the evaluation: co-design is an approach to problem-solving that starts from an analysis of user needs and involves users in jointly developing and testing solutions at multiple stages of the process.

What are policy labs?

According to a report commissioned for the EU policy lab (European Joint Research Centre, 2016, p. 1):

... policy labs are dedicated teams, structures, or entities focused on designing public policy through innovative methods that involve all stakeholders in the design process. Practitioners describe these efforts as design or evidence-based approaches, which places the end users at the centre of each stage of the policy-making process. After proposals are formulated, they are tested and validated through various forms of experimentation. In addition to co-creating and re-imagining policies and public programs, policy labs also undertake a wide range of activities such as preparing prospective studies, organizing creativity workshops, or instilling a sense of empowerment in civil servants through training and other learning activities.

According to Nesta ( 2005), there are over 100 public sector innovation labs (policy labs) worldwide, established at all levels of government, from municipalities to national ministries. A study commissioned by the EU policy lab mapped approximately 65 policy labs across Europe, with 20 of those in the UK (European Joint Research Centre, 2016, pp. 4-5). According to Nesta's guidebook to innovation labs (Puttick, 2014, p. 6) what unites and differentiates policy labs 'is that they are all adopting experimental methods to tackle both social and public issues'. Furthermore, the methods include 'design, data or behavioural economics'. While the reports on policy labs have been informative in nature, there has been little research on public sector policy labs beyond descriptive-and at times normative-overviews. Policy labs have been largely described as versions of existing organizations: as hybrids of think-tanks, digital research and development labs, social enterprises, and charitable organizations (Tõnurist, Kattel, & Lember, 2017). Thus, the nature, organizational structure and need for such units within the public sector are largely unexamined across a variety of social and economic contexts. For Vincent ( 2016), speaking at the European Commission Lab Connections Conference, 'Let's forget the McDonald's vision for Labs. They are all different depending on the local culture. There is no blueprint'. The following operational definition has been proposed for the purposes of this project: Policy labs are multi-disciplinary government teams developing public services and public policies using innovation methods to engage citizens and stakeholders at multiple stages of the development process (Whicher, 2017, p. 81).

What is the role of academic theory in government practice?

Adapting the taxonomy from Bryman and Bell ( 2003, p. 5), we might consider that academics and policy-makers are groups of knowledge workers placing different emphasis on theory and practice. Supported by elements of theory, policy-makers contribute to practice, whereas academics develop theory based on fragments of practice. Both activities, theory generation and practice implementation, are mutually reinforcing. While the practice of 'designing' has a long history stretching back before the industrial revolution, applying design methods to jointly developing public services and policies with citizens is a comparatively recent phenomenon. Indeed, design as a line of academic inquiry for generating theory-driven knowledge is even less developed, but one which is nevertheless rapidly expanding. Arguably, in the domain of policy labs, government practice is developing more quickly than academic theory can be consolidated. The first public sector innovation lab was started in Sitra, the Finnish Innovation Agency in 1967, followed by PS21 in Singapore in 1995, the Centre for Public Service Innovation in South Africa and the VINNOVA in Sweden in 2001, and the the Danish government's MindLab in 2002 (Puttick et al., 2014, p. 13). Since then there has been an exponential growth in policy labs from around five in 2002 to over 100 by the time of Nesta's fourth global labs conference-'Lab works'-in 2015 (Puttick et al., 2014; Nesta, 2015).

Despite the rapid uptake of co-design in policy labs, there is a lack of academic evidence on the impact on public services and public policies (Bason, 2014, p. 4; Junginger, 2014, p. 57). Due to the experimental nature of the activities and projects in policy labs, they tend to operate behind closed doors. In 2015, the Northern Ireland Innovation Lab commissioned an independent evaluation both of its governance and activities, the first publicly released evaluation of a policy lab in the UK (Whicher, 2017). In an attempt to share best practices with other labs, iLab made the results of the evaluation public. According to the iLab's director: 'We hope that the evaluation will provide impetus for other Labs to share good practices and lessons to create a community of practice from which we can all benefit' (Whicher, 2017).

Co-design for public service and public policy development is an emerging domain fraught with conceptual and empirical challenges, but it offers real opportunities to enhance citizen involvement in the public service and public policy development process.

Case method

The iLab was established in April 2014 in the Public Sector Reform Division of the Department of Finance. After just over two years of activity, an evaluation of the lab's activities and governance was initiated. The 2016 invitation to tender for the research stated that iLab:

... was envisioned as a vehicle to develop solutions with an emphasis on reforming and modernizing public services, and with a focus on ensuring high quality, fit for purpose services which utilize modern technologies. The time has now been reached when it is appropriate to carry out an independent evaluation to assess the lab's effectiveness and to surface recommendations to help shape the operating model to address the challenges arising from the new Programme for Government, further decreases in budgets and Brexit.

As a result of submitting the highest-scoring proposal, one of the authors was commissioned to perform the evaluation. The evaluation was based on 30 interviews with lab staff, the wider NICS and key external stakeholders. Of the 30 interviews, 13 were conducted face-to-face with participants on 13 and 14 September 2015 at the Department of Finance in Belfast and 17 interviews were conducted by telephone between 31 August and 21 September. The semi-structured interviews lasted between 50 and 90 minutes. The interviews were recorded with permission, partially transcribed and coded thematically. There were nine interviews with iLab staff, two interviews with senior civil servants (director level), 14 interviews with 'project sponsors' within the wider NICS, and five with external stakeholders such as Belfast City Council and experts (see Table 1). Responsibility for participant selection and recruitment was managed by iLab. The interview questions for staff members and senior civil servants focused on iLab's activities and governance, including iLab's leadership, operating model, methods and capacity. The interview questions for the project sponsors (senior civil servants commissioning iLab to deliver projects) focused predominantly on the iLab project with additional questions on iLab's governance. The project sponsors were from six different departments, including the departments for economy, finance, communities, agriculture, justice and health. In the two years, some departments commissioned more than one project, such as the departments for finance, health and communities.

Co-design, evaluation and the Northern Ireland Innovation Lab

11 April 2019

Based on the interim findings, the head of iLab and director of the Department of Finance selected four projects to be developed into impact case studies-these were 'Medicines optimization' commissioned by the Department of Health; 'Waste management' commissioned by Department of Agriculture, Environment and Rural Affairs; 'Benefits uptake' commissioned by the Department for Communities; and 'Dementia' commissioned by the Department of Health. It should be noted that these projects were selected for a number of different reasons-either they were particularly successful or, as in the case of the 'dementia' project, the first phase of the project was unsuccessful and the second phase was successful for reasons outside the scope of the original project. Thus, this case offered potential learning opportunities for iLab. The other outcomes from the research included an analysis of the strengths and weaknesses of iLab's governance, including leadership, operating model, methods and capacity, as well as a series of 17 recommendations for enhancing iLab's impact and making the operating model more sustainable.

Findings

The outputs from the research included four impact case studies, an analysis of the strengths and weaknesses of iLab's leadership, operating model, methods and capacity, as well as a series of recommendations for enhancing impact. From 2014 to 2016, iLab led 18 projects focused on a range of service and policy challenges across different departments of the NICS, with a budget of approximately £540,000. At the outset, the main method adopted by the lab was co-design, but it gradually adopted other methods, including behavioural insights and systems dynamic modelling. Of the 18 projects, 15 applied co-design approaches, two used behavioural insights and one used system dynamics modelling (see Table 2). The four case studies focused on the medicines optimization, waste management, benefits uptake and dementia projects. The medicines optimization and waste management projects took a design approach, while the benefits uptake project applied behavioural insights and the dementia project was a two-phase project involving design and system dynamics modelling.

Co-design, evaluation and the Northern Ireland Innovation Lab

11 April 2019

Although there were multiple factors at work, an investment of £60,000 in iLab's medicines optimization project could result in cost savings of over £20 million per annum. The analysis of the strengths and weaknesses of iLab's leadership, operating model, methods and capacity informed a series of 17 recommendations to capitalize on iLab's expertise, achieve further impact and secure additional support from stakeholders. These ranged from short-term to long-term proposals, as well as low to high priority.

What are the main determinants of effective co-design?

Based on our insight from the 30 interviews, the main determinants of effective co-design are:

Clarity of language and process.

Selection criteria for projects.

Building in evaluation.

Promoting good practices.

Senior management support.

Clarity of language and process

In the words of one civil servant: 'stick a "co" on the front of a word and it generically means "jointly" so co-creation is jointly created and co-design is jointly designed'. However, according to the same individual: 'Terms like co-production, co-creation and co-design can feel like buzzword bingo' to government officials. For those immersed in the discipline, there are nuances between them, but for a generalist civil servant these nuances are not necessarily clear or directly relevant. Convincing them to take any of these approaches requires increased clarity on definitions, methods and processes. The focus was on three main innovation methods: co-design, behavioural insights and systems dynamic modelling. However, as previously asserted, co-design is a difficult concept for civil servants to grasp because it is intangible. Perhaps the best way to communicate the value of co-design is by experiencing a co-design process; more so with civil servants not being prepared to commission or participate in a co-design project without understanding the benefits of the approach and what it entails. At the time of the evaluation, iLab used a raft of design terminology interchangeably, including 'co-design', 'service design', 'user-centred design', 'human-centred design', 'design thinking'. Even within the field of design there is a considerable amount of jargon (as stated by one interviewee) and subject-specific terms such as 'ideation', 'double diamond' and 'personas', among others. Although co-design is entering the lexicon of government, initially, a concerted effort should be made to streamline terminology and promote a smaller number of concepts. It might be advisable for a policy lab to review the scope of terminology and adopt a small number of terms as appropriate. One of the recommendations from the evaluation was to develop a glossary-a first iteration of which was proposed through the research. The iLab has subsequently adopted a glossary of terms for projects going forward.

Most civil servants are not familiar with the methods or processes used by innovation labs. iLab adopted the double diamond as a framework for its design processes. The double diamond (see Figure 2), attributed to the UK's Design Council ( 2007, p. 9), is a four-stage process of divergent and convergent thinking: Discover, Define, Develop and Deliver.

This framework is commonly adopted by public sector innovation labs using design methods (for example Drew, 2016). Essentially, the process involves discovering user needs, defining the challenge, developing and testing prototypes and delivering a solution validated by users. It is an approach that encompasses elements of top-down and bottom-up public governance. Not only are civil servants not familiar with co-design as an approach to problem-solving, they are also not familiar with what design methods and processes encompass. According to one interviewee, 'I was not entirely sure from the outset what the project processes, milestones and outputs would be or how long a typical design project takes'. According to one member of the iLab, the team created the 'design offer and structure for design projects iteratively'. At the beginning, the discover, define and develop phases were conducted as part of an intensive consecutive five-day workshop. However, this approach was adapted and improved based on feedback from project participants to a more strategic and longer term engagement. Based on the experiences of the interviewees, a 'typical' journey for a design project was developed (see Table 3) and visualized so that it could be communicated to potential iLab clients in other departments.

Co-design, evaluation and the Northern Ireland Innovation Lab

Selection criteria for projects

The current project set-up (see table 3) involves a scoping phase with the main point of contact or sponsor, an elongated discovery phase involving user insights gathering, a two-day workshop to define the challenge and a subsequent two-day workshop to develop concepts. At this point the project is, according to iLab staff, 'passed back to the sponsor for implementation'. Of the 18 projects evaluated, nine had progressed to implementation (eight design projects and one behavioural insights project, with four projects, including one systems dynamic modelling still live). Although a 50% progression rate to implementation may not sound successful, in fact, in the context of labs applying experimental methods to challenging real-world problems, this should be considered highly successful. iLab's core ethos was to be a safe space to formulate and test ideas. Furthermore, according to one interviewee: 'knowing what not to do is as important as knowing what to do'; in essence, co-designing enables government teams to design out the risk at early stages. Based on the experiences of the interviewees, there are a number of preconditions for ensuring a co-design project progressing to implementation. The most significant preconditions were active engagement and participation in the projects by the sponsor. In the early days of iLab's activities, the sponsor was required to 'set the brief' and then might 'dip in and out of the various stages of the project'. However, this meant that the project sponsors did not experience a co-design process and therefore appreciate how it is 'different from business as usual'. Based on the evaluation, it was clear that iLab needed to develop a set of selection criteria for co-design projects with an emphasis on implementation where appropriate. At the outset, greater emphasis was placed on the discover, define and develop stages of the double diamond and not on the deliver phase becausethis was considered 'beyond the mandate of iLab'. Co-design is effective for gathering user insight, generating ideas with users, selecting options with users, testing concepts with users and refining concepts with users. According to one member of iLab's team: 'where design falls down is that a great deal of energy is invested into the early parts of the process but insufficient resources are allocated to see them through to implementation'. If a sponsor was not actively participating in workshops and insights gathering, that project was less likely to progress to implementation. For one research participant, 'it all hinges on the sponsor being part of the project journey'. Some examples of selection criteria for co-design projects was developed as part of the evaluation, including ensuring that the sponsor has the necessary personal commitment and authority to implement outcomes and ensuring that the sponsor has resources (financial or otherwise) for prototyping, testing and upscaling.

Building in evaluation

Closely linked to implementation is evaluation. First, it should be acknowledged that there is a significant lack of research on evaluating the impact of co-design in public service and particularly public policy development (Bason, 2014, p. 3; Christiansen & Bunt, 2014, p. 47; Junginger, 2014, p. 57). By commissioning the evaluation of both the activities and the governance of iLab, the Department of Finance was, according to one interviewee, 'making a bold move'. It was not possible to perform an empirical evaluation of iLab's projects within the scope of the research at the time because baseline data had not been collected from the projects. Evaluation should be built into the process from the outset. The evaluation should not be purely quantitative because testimonials and impact case studies can be very valuable. However, as part of the problem definition phase, resources should be allocated to identifying metrics against which an ex-post evaluation can be performed, as well as ongoing benchmarking and monitoring. Identifying what success looks like is a fundamental part of shaping a co-design project and will ensure that the outcomes 'meet sponsors' expectations'. It is important for a project to be translated into a case study, even if it did not progress to implementation and was not necessarily 'classed as successful because there are learning opportunities from failure' as observed by one interviewee. It should also be part of the evaluation process for iLab staff to reflect on the lessons from each project identifying what could be improved in order to continuously improve and iterate iLab's offer and operations. A number of the co-design projects were also policy-related. The lead time for implementation in a policy cycle can be over a year, which adds a further dimension of complexity for evaluation. It is important to evaluate output and outcomes. For Bentley ( 2014, p. 15), this traditional policy cycle 'is deeply embedded in the cultures of legislatures and bureaucracies around the world, is one of the main reasons why policy processes are primarily focused on the production of documents, rather than the production of outcomes'. Evaluating the impact of co-design in a policy context is not well understood in current research. What can co-design offer the policy process and how can we evaluate the impact and outcomes?

Promoting good practices

In turn, the issues of evaluation and promotion are mutually reinforcing. Part of the process of securing additional support and interest is to publish results and communicate to the wider civil service. There is a clear need to build legitimacy for and awareness of iLab in the wider NICS. The purpose of the projects, evaluation, case studies and testimonials should be to create a community of champions and advocates for the approach. According to one senior civil servant: 'the lab can move beyond the traditional skills set of the civil service'. iLab should develop a formalized approach to knowledge exchange to build capacity for innovation methods such as co-design within the NICS. At present, iLab does not have a target to engage a certain number of civil servants. It is important for iLab not to be subject to the same key performance indicators allocated to other government processes as too stringent targets will 'strangle the blue-sky thinking that is the lab's main asset'. However, it should be seeking to engage with all government departments, as well as other stakeholders such as local councils, to disseminate the methods and good practices. Developing a formalized approach to knowledge exchange on co-design among the wider NICS should be complemented by communication and awareness-raising activities. For one project sponsor: 'No one knew about the innovation lab. I was the first person in our directorate to get involved. I was approached directly by the head of the lab'. Promotion and marketing are important to create an appetite in other departments to work with iLab.

Senior management support

Co-design is an experimental innovation method that lacks legitimacy in government. Policy labs are not immune to the ebbs and flows of the political tides or changing policy focus. Furthermore, Northern Ireland also has a unique political context within the UK. For example, between April 2014 and October 2016, the country had four ministers of finance due to changes in portfolios and priorities; and devolved government has been on hold in Northern Ireland since the collapse of the Northern Ireland Assembly in January 2017, with the increasing threat of direct rule from Westminster. As such, a determinant of success for co-design projects is organizational stability, in this context, within iLab. The lab should be protected, championed and valued by the various hierarchical layers of the civil service. This involves capturing success, promoting iLab, disseminating good practices, building capacity for the methods among the wider civil service and securing the support of senior decision-makers. The lab has been given a relatively high degree of autonomy to experiment and service other departments, albeit by seeking the input of the senior civil service. It is testimony to the iLab's success that it has been able to embark on such a diverse and ambitious portfolio of projects and gained endorsement from a range of departments. iLab is currently entirely dependent on the department of finance as its sponsor; should the department withdraw its support then iLab might be 'spun out of government like other innovation teams elsewhere'. By consolidating iLab's offer and unique selling points, its leadership team could engage with permanent secretaries across other departments to enhance its income base and spread the risk. Eventually, the lab could move more towards a new hybrid funding model, which combined sponsorship and collaborative funding. Securing the support of the senior civil service, including permanent secretaries, could shelter the lab from the 'ongoing political storm'.

What are the unintended consequences of co-design?

In iLab's case, there were two reoccurring unintended consequences of co-design: raising stakeholder expectations and a lack of prototyping. The lab was already aware of the latter because it is a central co-design attribute, but the former emerged as part of the evaluation. Many research participants (particularly sponsors and external stakeholders) commended the energy and motivation generated during the workshops, which in two projects gave way to disillusion when recommendations were not followed through. This was highlighted in one health project by an external stakeholder: 'I feel we've let the stakeholder groups down as we've raised expectations and we haven't followed through to implementation due to politics and professional boundaries'. This is a common 'fallout' of co-design projects. The discover, define and develop stages of the process galvanize and inspire people but, if stakeholders are not provided with sufficient parameters, they will generate ideas that are not bounded in reality. For one sponsor: 'The lab has real value and I think it would be a shame to abandon it because it does not work every time. Nothing does'. Shifting iLab's emphasis to implementation will most likely entail a further up-skilling step for staff to ensure that they are equipped with the knowledge to prototype services and prototype policies. Through prototyping, testing and iteration, iLab is more likely to progress projects to upscale and implement solutions. Again, the role of the project sponsor is vital, enabling a senior civil servant to participate in prototyping a service or policy will enable iLab to design out risk and secure support. For one sponsor not involved in prototyping 'when the recommendations were presented back to me I found it difficult to identify the thought process behind the concepts. I didn't feel I got the benefit of the whole process'. For a different project sponsor:

I now think that the model can work very well with the right challenge, with the right participants, with the right buy-in, and with a very big 'but'. The 'but' is prototyping and implementation. They have to be built into lab projects.

It should be acknowledged that the approaches used by iLab are experimental and rarely employed by the NICS and that iLab has progressed through a learning curve. As such, the team is cognisant of these unintended consequences and 'managing stakeholder expectations has primacy'. Furthermore, by developing selection criteria for projects and ensuring that sponsors commit to prototyping and implementation where appropriate some of the risks of raising stakeholder expectations will be mitigated. It is important for any lab to develop a clear operating model and identify its unique selling points when bringing new clients on board.

Discussion

Northern Ireland's iLab highlights certain lessons for other policy labs. With the emergence-and expansion-of policy labs across the world, further detailed independent evaluation of their operations, activities, funding and governance models can provide deeper insight into their success or failure, as well as their policy traction and embeddedness into the wider public sector. The success of the iLab could stimulate the creation of a network of policy labs across the UK, potentially stimulating collaborative activities, knowledge exchange and project funding, especially in the context of citizen-centered smart city design (Tryfonas & Crick, 2018). In particular, this could be an attractive and tractable model between the devolved nations in the UK, providing a structure and framework for collaboration between Scotland, Wales and Northern Ireland on key public service innovation priorities, perhaps exploiting the devolved city deal and regional growth deal mechanism (UK Government, 2019). From a Welsh context, where the concept of public service innovation labs is beginning to take hold, there are valuable legislative levers such as the Well-being of Future Generations (Wales) Act (Welsh Government, 2015), in which is embedded the importance of sustainable development and which places a legal responsibility of certain named public bodies to improve the economic, social, environmental and cultural well-being of Wales through five ways of working to achieve seven well-being goals. The outcomes from the iLab evaluation can provide valuable insight into translating and transferring success into new domain-specific contexts, especially via sponsor engagement and through case studies and examples of successful interventions.

In 2016, the Northern Ireland government also commissioned the Organisation for Economic Co-operation and Development (OECD) to review public sector reform in Northern Ireland, involving more than 300 research participants and resulting in 30 recommendations. One recommendation specifically related to iLab. It stated that iLab is an 'impressive example of how the government is nurturing innovation in the public sector' and that there was a need to 'develop its full potential through departmental ownership, skills development, active user and sponsor-department participation in lab sessions and impact measurement' (OECD, 2016, p. 43). That, in two years, half of the 18 projects which iLab has led have progressed to implementation should be considered a high success rate. Its use of relevant design processes, especially in articulating how they can be used by sponsors and key stakeholders as adaptable problem-solving and risk reduction strategies in a range of real-world contexts, has been valuable to explore and test the usability, desirability and viability of concepts, looking at both quantitative and qualitative impact. In particular, building tolerance and acceptance from policy lab customers that the ability to stop a project before potentially costly political and financial implementation is a significant positive step forward. It has been over a year since the evaluation was performed and PDR continues to work with iLab and ideally a review of the evaluation would take place to see what proportion of the recommendations have been implemented.

Perspective and future work

The shift from user-as-subject to user-as-partner has fundamentally altered the role of the citizen from passive consumers of public services and public policies to active collaborators in their formulation. This is a step change from transactional to collaborative public governance. While there is currency and a significant policy focus on these popular 'co'-processes-with varying levels of engagement, embedding and application-this shift should be further encouraged and promoted as a wider democratic engagement process at all levels: from local and regional to national. iLab has the opportunity to position itself as a 'bastion of innovation' within the NICS for incubating ideas and engaging citizens and stakeholders at multiple stages of the service and policy development process. In a very tangible way, it is visibly contributing to the ambitions of the Programme for Government, the overarching policy directive for Northern Ireland. This tangible policy delivery contribution is key for translating the potential and ambitions for a policy lab in other environments.

Evidencing the effectiveness of the various financial and governance models is also important for adoption in other policy domains and environments, particularly in the wider context of effective public value co-creation for public service organizations (Meynhardt, 2009; Osborne, 2018). From a governance perspective, an arm's-length approach is important to provide the freedom and openness in which to operate. While there are challenges associated with being regarded as a 'special entity', receiving different treatment and privileges in a political and/or policy-making context, this freedom is crucial during the bootstrapping phase of a lab. Finally, a developing imperative for openness and transparency, especially in the context of open government, governance and democracy, provides a valuable lever for how policy labs can and should operate from their inception and initial implementation. Co-design is a transforming the nature of public governance from transactions with the 'citizen as user' to collaboration with the 'citizen as partner'.

Category of research participant

iLab staff

Senior civil servants

Project sponsors in NICS

External stakeholders

Total

No.

Title

Method

Department of

Progressed to implementation as of September 2016

Cost*

Total

£539,200

Project stage

Description

Timescale

Disclosure statement

No potential conflict of interest was reported by the author(s).

References

Bason, C. , & Schneider, A. (2014). Public design in global perspective: Empirical trends. In C. Bason (Ed.), Design for policy (pp. 23-40). Gower Publishing.Surrey, UK:

Bentley, T. (2014). Design in policy: Challenges and sources of hope for policymakers. In C. Bason (Ed.), Design for policy (pp. 11-22). Gower Publishing.Surrey, UK:

Borja de Mozota, B. (2002). Design and competitive edge: A model for design management excellence in European SMEs. Design Management Journal, 2(1), 88-104.

Bryman, A. , & Bell, E. (2003). Business research methods. Oxford University Press.UK:

Christiansen, J. , & Bunt, L. (2014). Innovating public policy: Allowing for social complexity and uncertainty in the design of public outcomes. In C. Bason (Ed.), Design for policy (pp. 41-57). Gower Publishing.Surrey, UK:

Design Council. (2007). Eleven lessons: Managing design in eleven global companies. Desk research report. London, UK: Design Council.

Drew, C. (2016). Lab connections conference, EU policy lab. European Joint Research Centre, Brussels, 18.10.16. Retrieved from How to implement experimental, creative and citizen-centred approaches in the public sector?, http://blogs.ec.europa.eu/eupolicylab/lab-connections/ [Accessed: January 2019]

European Joint Research Centre. (2016). Public policy labs in european member states. EU policy lab report, European Union.

Junginger, S. (2014). Towards policymaking as designing: Policymaking beyond problem-solving and decision-making. In C. Bason (Ed.), Design for policy (pp. 57-69). Gower Publishing.Surrey, UK:

Meynhardt, T. (2009). Public value inside: What is public value creation? , International Journal of Public Administration, 32(3-4), 192-219. doi: 10.1080/01900690902732632 [Taylor & Francis Online] [Google Scholar]

Morelli, N. (2002). Designing product/service systems. A methodological exploration. Design Issues, 18(3), 3-17. doi: 10.1162/074793602320223253

Nesta. (2005). World of labs, 19 May 2015. Retrieved from http://www.nesta.org.uk/blog/world-labs [Accessed: January 2019]

OECD. (2016). Northern ireland (United Kingdom): Implementing joined-up governance for a common purpose. , OEDC public governance reviews, OECD Publishing, Paris.[Crossref] [Google Scholar]

Osborne, S. P. (2018). From public service-dominant logic to public service logic: Are public service organizations capable of co-production and value co-creation? Public Management Review, 20(2), 225-231. doi: 10.1080/14719037.2017.1350461

Puttick, R. , Baeck, P. , & Philip Colligan, P. (2014). i-teams. The teams and funds making innovation happen in governments around the world. Nesta.London, UK:

Sanders, E. , & Stappers, P. J. (2008). Co-creation and the new landscapes of design. , Co-Design, 4(1), 5-18.[Taylor & Francis Online] [Google Scholar]

Siodmok, A. (2014). Tools for insight: Design research for policymaking. In C. Bason (Ed.), Design for policy (pp. 191-200). Gower Publishing.Surrey, UK:

Tryfonas, T. , & Crick, T. (2018). Public policy and skills for smart cities: The UK outlook. Proceedings of 11th international conference on PErvasive technologies related to assistive environments (PETRA'18), pp. 116-117.

UK Government. (2019). City deals and growth deals. Retrieved from https://www.gov.uk/business-and-industry/city-deals-and-growth-deals [Accessed: January 2019]

Utterback, J. , Verdin, B.-A. , Alvarez, E. , Ekman, S. , Walsh Sanderson, S. , Tether, B. , & Verganti, R. (2006). , Design-inspired innovation. World Scientific Publishing Co.Singapore: [Crossref] [Google Scholar]

Vincent, S. (2016). Lab connections conference, EU policy lab, European joint research centre, Brussels, 18.10.16. Retrieved from How to implement experimental, creative and citizen-centred approaches in the public sector?, http://blogs.ec.europa.eu/eupolicylab/lab-connections/ [Accessed: January 2019]

Welsh Government. (2015). Wellbeing of future generations (Wales) act. Retrieved from http://www.legislation.gov.uk/anaw/2015/2/contents/enacted

Whicher, A. (2017). Evaluation of the northern ireland innovation lab. Independent report commissioned by the northern ireland department of finance. Belfast, UK. Retrieved from http://pdronline.co.uk/Portfolio/evaluation-of-niil [Access January 2017]

Williamson, B. (2015). Governing methods: Policy innovation Labs, design and data science in the digital governance of education. Journal of Educational Administration and History, 47(3), 251-271. doi: 10.1080/00220620.2015.1038693

# data[].attributes.attachment_link.title
Co-design, evaluation and the Northern Ireland Innovation Lab

# data[].attributes.attachment_link.description
ABSTRACT ABSTRACT Around the world there are more than 100 policy labs-multi-disciplinary government teams developing public services and policies using innovation methods to engage citizens and stakeholders. These policy labs use a range of innovation methods and approaches, including co-production, co-creation, co-design, behavioural insights, systems thinking, ethnography, data science, nudge theory and lean processes.

# data[].attributes.headline
Evaluating Design: Understanding the Return on Investment

# data[].attributes.subject
Evaluating Design: Understanding the Return on Investment

# data[].attributes.headline
Evolution of policy labs and use of design for policy in UK government

# data[].attributes.subject
Evolution of policy labs and use of design for policy in UK government

# data[].attributes.attachment_link.readable_text
Abstract

Abstract
At the intersection between theory and practice on "design" and "policy," there is small but expanding knowledge base on the concept of "design for policy." Government interest in design methods for policy-making has grown significantly since the late 1990s, particularly within policy labs. Policy labs are multidisciplinary government teams experimenting with a range of innovation methods, including design, to involve citizens in public policy development. There are more than 100 labs across the globe and around 14 at national and regional levels in the UK. While new policy labs continue to pop up, some are changing and some are closing their doors. How have the operating models of policy labs evolved? How might we enhance the resilience of labs? These are some of the questions explored in a 2-year Arts and Humanities Research Council Fellowship called People Powering Policy. The research draws on insight from established Labs including Policy Lab in the Cabinet Office, HMRC Policy Lab and the Northern Ireland Innovation Lab as well as emerging labs to address the research question: how might policy labs be developed, reviewed and evaluated? Based on interviews, workshops and immersive residencies the main outputs were a Typology of Policy Lab Financing Models and a Lab Proposition Framework for establishing, reviewing and evaluating policy labs. The typology outlines four models for UK labs financing models - Sponsorship (funding from one or multiple departments), Contribution (labs recover part of the costs of projects), Cost Recovery (labs charge for projects on a not-for-profit basis), Hybrid (labs benefit from multiple income sources such as Sponsorship, charging and knowledge exchange funding) and Consultancy (labs charge a consultancy rate with a profit margin to expand operations). The Lab Proposition Framework comprises of four components (1) Proposition - the vision, governance and finance models; (2) Product - the offering, user needs and tools; (3) People - the people skills, knowledge diffusion and wider capacity building; (4) Process - the routes to engagement, user journey and promotion mechanism.

1. Introduction

A substantial bank of knowledge exists on "policy" and "design" as separate concepts but a limited body of academic theory and scholarly knowledge exists at the intersection between the two concepts that is "design for policy" (Bason 2014, 3; Amatullo 2014, 152; Junginger 2014, 57; Staszowski, Brown, and Winter 2014, 155; Jegou, Thevenet, and Vincent 2014, 156; Halse 2014, 210). While designers and governments have been applying design principles to public sector services since the 1960s (Sanders and Stappers 2008, 5; Puttick, Baeck, and Colligan 2014, 13), the application of design to public policy has only gained traction since the late 1990s (Bason 2014, 3; Howlett 2014, 199), particularly in policy labs. Policy labs are multi-disciplinary government units using a range of innovation methods, including design, to collaboratively engage users and stakeholders in service and policy development (Whicher 2020, 4). In the UK, there are policy labs and user-centered policy design teams operating at national level (for example, Policy Lab which moved from the Cabinet Office to the Department of Education in 2020, HMRC Policy Lab and Ministry of Justice User-centered Policy Design Team) and regional levels (for example, the Northern Ireland Innovation Lab and Welsh Government Innovation Lab) and many used to operate at local levels. Staszowski, Brown, and Winter ( 2014, 163), succinctly acknowledge that although the overall design agenda is growing within government (albeit with a prevalence for digital public services), design for policy is still an emerging practice. As such, in the case of "design for policy," it could be argued that practice is well in advance of theory. As these labs mature and gain credibility there is an opportunity to review their progress. There is a need to evaluate the impact of policy labs in order for them to gain greater traction across government. Consequently, the research question guiding this article is how might policy labs and user-centered policy design teams be developed, sustained and evaluated? This question was addressed as part of a 2-year Arts and Humanities Research Council project called People Powering Policy drawing on immersive residencies, interviews and workshops with established and emerging labs. Consequently, a number of iteratively tested and validated outputs have been created including a Lab Proposition Framework for developing, reviewing and evaluating policy labs and user-centered policy design teams and a Typology of Policy Labs' Financing Models.

2. Design for policy and policy labs in a UK context

As observed by Williamson ( 2015, 251), Bason and Schneider ( 2014, 34) and Junginger ( 2012, 117), the are a number of denominations or classifications for specialist government units applying design approaches to policy-making. They operate under a number of guises with slightly different nuances including "policy labs," "policy innovation labs," "social labs," "design labs," "co-design labs," "public sector innovation labs," "government innovation labs" (or "i-teams"), "public and social innovation labs" (or "psilabs"), "public innovation spaces" and "user-centered policy design teams," among other terms (Williamson 2015, 251; Bason and Schneider 2014, 34; Junginger 2012, 117). Fundamentally, the implication is that these labs deal with innovation in public services and public policies. Furthermore, when also considering the wider non-governmental space there are more examples such as Living Labs. The European Network of Living Labs (ENoLL), with more than 150 active labs, defines Living Labs as "user-centered, open innovation ecosystems based on a systematic user co-creation approach integrating research and innovation processes in real life communities and settings."¹ There are a growing number of non-governmental players within the policy lab space, including academics, designers, management consultants, community groups and think tanks, but this article refers to design for policy being conducted within government-owned teams. The terms policy lab and user-centered policy design team (UCPD) have been adopted as the operational terms for this research to narrow the focus on design for policy (rather than the wider topic of design for public services) and due to the preeminence of their use in the UK - the dominant context for this research. For Williamson ( 2015, 252), "labification" means apply the scientific approaches of experimentation, testing and measurement to the policy process.

Nesta is a UK innovation foundation and has taken a leading role in creating a global community of government policy labs through their "States of Change" work and research.² According to Nesta, in the decade 1991 to 2000 there were only two policy labs in operation - Helsinki Design Lab in Finland (now closed) and PS21 in Singapore. In the period 2001 to 2010, a further 14 Labs were established around the world (Puttick, Baeck, and Colligan 2014, 13) and by the subsequent decade, the number of labs had grown exponentially with Nesta estimating more than 100 in operation (see Figure 1). There is no data on what proportion of these Labs use design methods or what proportion use design methods for policy-making. A study commissioned by the EU Policy Lab has identified 65 in Europe (Fuller and Lochard 2016, 4-5) and in the UK, in 2020, there were around ten national labs and three regional labs in operation (Whicher 2020, 15).

In the UK, there appear to be two overarching policy agendas under which design for policy has gained traction - Open Policy-making and Devolution (Taylor 2014, 14; Siodmok 2014, 25; Williamson 2015, 258). Open Policy-making has become a dominant paradigm for experimentation in government focused on developing and delivering policy in a fast-paced and digital world using collaborative and iterative approaches while devolution is the transfer of power from national, centralized government to devolved, decentralized government. Specifically, in a UK context, growing interest in design for policy can be condensed into six main factors - the changing nature of evidence, growing interest in user-centered approaches, a focus on end-to-end policy-making, a drive for more meaningful public consultation, the need for rapid policy prototyping (particularly in the context of COVID-19 response) and the rise of futures thinking (such as speculative design) (Whicher 2020, 15).

Over the course of the last decade there has been a drive in government to take decision-making closer to the citizen and design research has been embraced as one of a number of approaches for understanding the lived-experiences of the citizen to inform more "open policy-making" (Siodmok 2020). There has been a gradual shift in emphasis away from the idea of "doing to" the citizen toward "doing with" the citizen. As articulated by one interviewee, "The ultimate goal is to find the Holy Grail of user-centred policy-making." it is important to articulate that a design approach to policy is not about "supplanting or usurping empirical approaches but complementing and enhancing them." This is the notion of "big data plus thick data" (Siodmok 2020). Economists tend to be the gatekeepers of policy, making generalization from large datasets (big data) while design researchers conduct deep research into smaller samples (thick data) (Maffei, Leoni, and Villari 2020;124). Policy Lab uses "big data to see the big picture before then using thick data to zoom into the detail of people's lived experiences" (Siodmok 2020). As such, design research is about humanizing the numbers. Design research can contribute to evidence-based policy-making by supplementing traditional quantitative-based evidence with in-depth qualitative-based evidence.

Traditionally, the policy process has been very siloed and there has been a need to "bring policy development and service delivery together with the public" according to a policy lab interviewee. Design research can take a "holistic view of the policy cycle or "journey" balancing the demands of people on all sides" - ministers, policy-makers, intermediaries and policy "users" according to another. Sometimes, policy users or beneficiaries are not actively involved in public engagement exercises prior to very formalized public consultation, which can often isolate those best placed to provide input to the policy process. Once a policy reaches public consultation it is unlikely to significantly change its trajectory and sometimes public consultation has been seen as a "tick box exercise." However, there is a drive within government to transform public engagement and public consultation to ensure that it is more meaningful and actively engages those who would not normally participate in formalized engagement and consultation processes.

COVID-19 has accelerated progression on many socio-economic issues such as cashless society, remote workforces and low carbon economy but perhaps most significantly it has accelerated the mode of policy-making. COVID-19 necessitated rapid, iterative policy-making where ministers and senior civil servants were required to adopt even shorter decision-making times. According to government interviewees, there is a perception that the "timelines of traditional academic research do not correspond to the pace of policy-making." One of the reasons why design research and practice has gained traction in government is because it has resulted in "shorter cycles of decision-making particularly through policy prototyping" and getting something "live into the field for iterating and testing." By introducing rapid policy prototyping at the early stages of the policy cycle, design approaches can de-risk delivery further down the line by ensuring that policy concepts are desirable, feasible and viable. Prototyping is central to all design processes and prototyping policy is also very much an emerging concept but one which unprecedented times is pushing governments to explore (Kimbell and Bailey 2017, 214). There is also a need to adopt a longer-term outlook in policy-making. Governments are also experimenting with more niche design research areas like speculative design and design futures - using alternative possible futures to provoke, inspire and provide a critical commentary to inform policy-making.

3. Method

This research has been conducted as part of an Arts and Humanities Research Council (AHRC) Fellowship project called "People Powering Policy."³ The aim of the project was to understand if, where and how design might add value to the policy process through greater citizen participation, particularly within policy labs. The Fellowship took place between September 2017 and August 2019 and involved four live policy projects with HMRC Digital (the UK tax office), the Northern Ireland Department of Health, the Financial Conduct Authority's Behavioral Insights Team and the Welsh Government Permanent Secretary's Group. Through the four immersive policy projects, interviews and 21 co-design workshops a number of outputs were created: a Typology of Policy Labs' Financing Models, a Lab Proposition Framework for establishing, reviewing and evaluating policy labs and UCPD teams, a Design for Policy Model demonstrating how design can add value to the policy cycle and PROMPT - a Design for Policy Toolkit. This article focuses on the Typology of Policy Labs' Financing Models and the Lab Proposition Framework. This enquiry focuses specifically on the research question: how might policy labs and user-centered policy design teams be developed, reviewed and evaluated?

This explorative research has adopted a design research framework in order to iteratively construct a framework to develop, review and evaluate policy labs and user-centered policy design teams. A Design research framework advocates the concept of research "for design, by design" through iterative and collaborative approaches combining "divergent and convergent thinking" (Blessing and Chakrabarti 2009, 9; https://www.designcouncil.org.uk/news-opinion/what-framework-innovation-design-councils-evolved-double-diamond). A design research framework prescribes explorative, inductive approaches where broad insight is gathered from research participants and iteratively refined, tested and validated. For the purposes of this intervention, there were five evolving stages to the design research approach:

Stage 1: Investigation into the changing financing models of Policy Lab, iLab and EU Policy Lab (interviews with lab heads and teams - n10 in September 2016 and April 2020).

Stage 2: Investigation into the governance and impact of iLab (interviews with senior civil servants, lab head, lab staff and clients - n30 September to December 2016).

Stage 3: Explorative investigation into the drivers and barriers facing HMRC Policy Lab and iLab (two immersive, one-month residencies with labs in November 2017 and September 2018).

Stage 4: Explorative workshops to define the user needs and feasibility of HMRC Policy Lab and Welsh Government Innovation Lab (London workshop 35 participants in November 2017, Cardiff workshop 18 participants in December 2018).

Stage 5: Explorative workshop to consolidate the Labs Proposition Framework (n19 participants from seven labs in November 2018).

Stage 1: Through on-going research as well as commercial interventions, including design for policy training, the author has established a long-standing relationship with Policy Lab (in the Cabinet Office from 2014 to 2020 and subsequently in the Department for Education from mid-2020) and iLab in the Northern Ireland Department of Finance. Over time, it was observed how the financing models for the two labs were changing. Face-to-face interviews were conducted with the heads and policy officers of Policy Lab, iLab and EU Policy Lab (n6) in September 2016 to collect data for the years 2014 and 2016. Telephone interviews were conducted with the heads and policy officers of Policy Lab and iLab (n4) in April 2020 to collect data for 2020 - the 2020 data for EU Policy Lab was taken from their website due to the unavailability of participants. The questions centered on the number of staff members, dominant innovation methods, annual budgets, how they charge clients for projects as well as other questions not included in this study. The outcome of this examination was a typology of lab financing models (see section 4). The main insights from this were that the financing models of policy labs have evolved from when they were established as they gain credibility and legitimacy contributing to the overall Lab Proposition Framework to develop, review and evaluate labs.

Stage 2: From September to December 2016, the author was commissioned to conduct an evaluation of the governance structures and projects of iLab. The evaluation was based on 30 interviews with Lab staff, the wider Northern Ireland Civil Service (including senior civil servants) and external stakeholders. Details of this evaluation are presented elsewhere (Whicher 2017; Whicher and Crick 2019); however, it is part of the journey toward inductively and iteratively creating the framework to develop, review and evaluate policy labs. This stage contributed to the research by demonstrating the need to build impact into lab operations from the outset in order to demonstrate value and enhance resilience. The evaluation included four impact case studies of project interventions, SWOT analysis of the operating model, leadership, processes, methods and people skills as well as a series of recommendations for enhancing iLab's operations. A number of the recommendations that were made directly relate to the Lab Proposition Framework including having a clear offer for clients, developing selection criteria for projects, conducting a skills audit of lab staff, building capacity for the innovation methods among the wider department and civil service, adapting financing models to ensure resilience and engaging externally to create a community of practice.

Stage 3: Through capacity building initiatives with HMRC Digital and iLab opportunities emerged to engage them more closely in the AHRC Fellowship and to conduct two one-month immersive residencies to understand the drivers and barriers facing labs. The collaboration with HMRC Policy Lab took place in November 2017 in London while the one-month residency on a policy project brokered through iLab took place in September 2018 in Belfast. Through immersive experiences with the two labs a series of thematic drivers and barriers to the emergence, monitoring and on-going assessment of labs were developed. This stage was an important part of the design research approach iteratively constructing, deconstructing and testing assumptions and ultimately refining and validating emerging ideas. With unparalleled access to lab heads it was an opportunity to sense-check the emerging typology and framework to ensure they were fit for purpose and corresponded to lab needs. Identifying the different and evolving finance models and their associated strengths and weaknesses led to a more in-depth examination of how the operating models of labs have emerged and changed. The investigations from Policy Lab, HMRC Policy Lab and iLab informed a set of questions that required framing as part of work to establish HMRC Policy Lab. From the experience with labs a series of pertinent questions emerged, which were relevant for developing HMRC Policy Lab including: What is the vision for the lab? What is the role of the lab in the wider organizational transformation? How can the lab demonstrate impact and value? Is there a clear "offer" for lab clients? Does the lab offer training as well as projects? What are the main innovation methods adopted by the lab? Do lab staff have the right skills or are there gaps? How can the lab build capacity for the approaches across the wider department? How do potential clients hear about the lab? What is the user journey? What is the selection criteria for projects? How does the lab engage externally with the public, if at all?

Stage 4: The aforementioned questions were used to inform the structure for collaboratively developing HMRC Policy Lab and exploring the feasibility of a Welsh Government Innovation Lab. The HMRC Policy Lab workshop took place in December 2017 and the participation was managed by the head of the lab and included representatives from HMRC, Her Majesty's Treasury, the Home Office and the Department for Work and Pensions (n35). This was the first time that the emerging framework was put to the test to establish a lab. The workshop involved three hands-on exercises (1) Mapping the HMRC Policy Lifecycle; (2) Exploring the opportunities and obstacles to HMRC Policy Lab and (3) Developing the HMRC Policy Lab proposition. The first exercise involved visually mapping the policy cycle and identifying where the lab could add value to the policy process. Based on the HMRC Policy Lifecycle, in small groups, participants explored the opportunities and obstacles to expanding HMRC Policy Lab. The exercise enabled an identification of the potential pitfalls as well as where the Lab could meaningfully enhance policy impact. The third exercise of the day focused on collectively defining the operating conditions of HMRC Policy Lab by brainstorming ideas around the Lab's proposition statement, the skills people would need, the Lab's offering as well as the selection criteria for projects. The workshop acted as a springboard for consolidating the concept of HMRC Policy Lab and generating wide engagement and support for its activities. Based on the insight gathered by participants, a series of recommendations were made for enhancing the impact of HMRC Policy Lab, which were provided to the lab head. This workshop was subsequently refined and replicated with the Permanent Secretary's Group to explore the feasibility of a Welsh Government Innovation Lab. This workshop took place in December 2018 in Cardiff with 18 participants. The Welsh Government workshop was an opportunity to further test and refine the framework and questions for defining a proposition or offer for labs.

Stage 5: Having iteratively developed and tested the typology and framework with a number of different labs - national and regional - there was a need to gain broader input from other labs in order to validate the outputs. As such, the author organized a workshop called "Lab Evolutions" in November 2018 co-hosted by Policy Lab in the Cabinet Office Sky Room. The workshop involved 19 participants from seven policy labs or UCPD teams (Policy Lab, iLab, HMRC Policy Lab, Ministry of Justice User-centered Policy Design Team, Department for Work and Pensions Policy Exploration Team, Department for Education Teachers Policy and Service Design Team as well as Welsh Government Innovation Lab). The session began with a labs showcase with each lab presenting a design for policy intervention, sharing good practices and providing insights on how the lab may have evolved. The main workshop was composed of three exercises (1) exploring the drivers and barriers to the expansion and consolidation of labs; (2) mapping the policy process and where design can add value; and (3) deconstructing and reconstructing the Lab Proposition Framework. The first exercise involved sharing baseline data from labs around the rationale for establishing the lab, their dominant innovation methods, how practices have evolved and other comparators such as number of staff. This exercise was conducted by each lab team then the labs were mixed up into different groups to enable more fruitful exchange and collaboration. The second exercise involved mapping and validate the policy process, where design can add value and specifically what design tools the labs use at different stages of the policy cycle for their interventions. The third exercise involved a question sorting exercise where a set of questions were provided to labs and they had to sort them thematically and allocate a thematic heading. The questions included those questions mentioned early around establishing, reviewing and evaluating labs. The purpose of this exercise was to validate the Labs Proposition Framework.

The outcome from the above five-stage process was the Typology and Framework involving iLab, Policy Lab, HMRC Policy Lab and Welsh Government Innovation Lab in more in-depth ways as well as validating the approaches in a more light-touch way with MoJ UCPD Team, DWP Policy Exploration Team, DfE Teachers Policy and Service Design Team.

4. A typology of policy lab financing models

In the UK, policy labs operate at multiple levels of governance - local and city (for example, Bexley Innovation Lab), regional (Northern Ireland Innovation Lab and Welsh Government Innovation Lab), national (Policy Lab and HMRC Policy Lab) and there is even a supranational lab (EU Policy Lab). The distinction is useful for "understanding the degree of influence and scale" (Bason and Schneider 2014, 35). Speaking at the Lab Connections conference hosted by the EU Policy Lab, Vincent ( 2016) asserted that there is no blueprint for Labs: "Let's forget the McDonald's vision for Labs. They are all different depending on the local culture. There is no blueprint." Many of the UK Labs are in their fourth, fifth or sixth year of operation, how have the operating models of policy labs evolved?

While MindLab (2001-2018) in Denmark was seen as the early trail blazer or archetype for labs across Europe, Policy Lab in the Cabinet Office was the pioneer in the UK. Established under the Open Policy-making agenda, Policy Lab brings "people-centered design approaches to policy-making" through the triumvirate of design, data and digital. Between 2014 and 2016 Policy Lab led 12 large projects and involved 4,000 civil servants in projects, workshops and training with a budget of approximately £800,000 (Whicher 2017, 30). Also in 2014, the Northern Ireland Department of Finance established the Northern Ireland Public Sector Innovation (iLab). iLab aims "to improve public services and policy by creating a safe space to co-create ideas, test prototypes and refine concepts with citizens, civil servants and stakeholders" (Whicher 2017, 6). Similarly, in its first 2 years, iLab led 18 projects with a budget of approximately £700,000 focused on a wide range of service and policy challenges. The challenges ranged from improving the use of data analytics within the government and reviewing business rates to encouraging people to pay court fines and optimizing how patients manage their medication. In September 2016, the author was commissioned to perform an evaluation of iLab's activities as well as its governance including its leadership, operating model, methods and capacity (Whicher 2017; Whicher and Crick 2019). Over the years, demand for Policy Lab and iLab services has grown and the budgets and operating models have evolved. Based on interviews conducted with Policy Lab and iLab on how they have evolved, broadly, there are four main financing models for UK labs with an emerging fifth model:

Sponsorship model - Lab receives a top slice of funding from one or multiple government departments.

Contribution model - Lab receives sponsorship but also recovers a proportion of implementation costs from clients.

Cost Recovery model - Lab recovers all costs from projects on a not-for-profit basis or may charge a small administration fee.

Hybrid model - Lab benefits from multiple sources of funding such as sponsorship, charging for projects as well as collaborative, research or knowledge exchange funding.

Consulting model - Lab operates like an internal consulting function charging for projects with a commercial margin in order to grow the labs operations (hypothetical model).

A lab's journey through the financing models might start with sponsorship, where the team is 100% funded by one or multiple government departments. After a year or two when the lab has begun to build credibility, it may start to charge clients a proportion of the costs of projects (such as 50%) in a Contribution model while still receiving sponsorship from the host department(s). Once it has established legitimacy after 2-3 years, a lab might be able to charge the full cost of projects on a not-for-profit basis in Cost Recovery model. At this point it might start to look for additional collaborative funding sources in a Hybrid model. The Hybrid model combines different models including sponsorship, commercial income and collaborative funding such as public-private partnerships, knowledge exchange initiatives and European funding. Some UK labs are now starting to transition to a Consulting model where they still benefit from mix income sources like collaborative funding but they have also build in a profit margin to reinvest in expanding the operations of the lab (Table 1).

Evolution of policy labs and use of design for policy in UK government

The resilience of the two UK labs lies in the experimental way that the operating models have developed organically through a continuous learning and credibility building process. Policy Lab was initially sponsored by 17 central government departments, who contributed in cash or kind to the lab's first year budget of £350,000. This budget covered staff salaries, project costs and consultancy. In the second year, the Lab asked for a smaller contribution from departments, but began charging for some projects. The total budget in the second year was just under £450,000. By its third year it had transitioned to a Cost Recovery model servicing central government departments on a not-for-profit basis. Emerging from its experimental phase, Policy Lab was able to consolidate and enter a new stage of governance. Although in early 2020 Policy Lab operated a Cost Recovery model, it is conceivable in the future that they could transition to a Consulting model building in a commissioning fee of 10-30% to enhance lab operations. Dr Andrea Siodmok leaving Policy Lab in early 2020 may expose Policy Lab without the stability of her longstanding leadership. In her words (Siodmok 2020):

[Policy Lab] has grown from simply being an idea - a single line in the Civil Service Reform plan - to becoming a vibrant, multi-disciplinary innovation team of fifteen experts applying deeply human insights and innovative tools to support policy-makers across the system. At last week's count the Lab had delivered 104 projects supporting the Prime Minister, government departments and external partners.⁴

A further change also took place when Policy Lab moved from the Cabinet Office to the Department of Education to be closer to the Policy Profession but still retaining its cross-government brief. It is not uncommon for successful initiatives within the Cabinet Office to be moved or even spun out of government such as the Behavioral Insights Team.⁵

iLab also began life with a Sponsorship model funded entirely by its host the Department of Finance with a set up fund of £350,000 but by 2018, it had transitioned to a Contribution model with the majority of its funding still from its host but it also recovered around 50% of the cost of projects from clients. The intervention rate also increased over the years so that by 2020, iLab was recovering 100% of the costs of projects. However, in 2018, iLab also began to collaborate on a European funded project called User Factor,⁶ which constitutes a transition to a Hybrid model as funding is still derived from its host department, it charges for projects and it also benefits from collaborative funding. For iLab, it was important for these transitions not to be undertaken too quickly so as to jeopardize the potentially fragile modus operandi of the Lab. Now, for the Head of iLab, one of the challenges will be ensuring that the lab is "not a casualty of COVID cost savings." Advocates of policy labs contend that design for policy is as relevant, if not more so, in the context of COVID-19 because of the need for prototyping policy at pace. However, there is also a need to demonstrate the impact of policy labs and design for policy in this brave new world and gain wider credibility across the UK Policy Profession if the approaches are to become more mainstream in the UK.

It should also be acknowledged that the distinctions between these models are blurring and that a lab might not fully fit within a single model. For example, before it closed, MindLab in Denmark could be considered a Hybrid model operating predominantly a Sponsorship model with elements of a Consulting model as it was 80% financed by three central government departments and one local municipality but also with 20% commercial income (Table 2).

Evolution of policy labs and use of design for policy in UK government

Each model has its associated strengths and weaknesses. For example, the Sponsorship Model hinges on the regular renewal of a contract across multiple government departments (often annually). With the Sponsorship model there is a risk that if key advocates (like the Permanent Secretary's Group) withdraw their support that the Lab will close its doors or be spun out of government. The Contribution and Cost Recovery models hinge on a highly effective and efficient work programme and continuous pipeline of projects as well as crucially a strong reputation. These models provide flexibility to service a range of clients enabling labs to develop selection criteria to ensure "client readiness" for user-centered design approaches and affords relative freedom to pursue higher value, strategic projects. The Hybrid model spreads risk through a portfolio approach across multiple sources of income including sponsorship, commercial and collaborative funding. It seeks to emulate the Innovate UK "Catapult" model where one third of income is underpinned from core public sector funding (Sponsorship), one third from commercial services (Consulting) and one third from collaborative knowledge exchange or public-private partnerships. Income generated from multiple sources protects the lab from cost savings exercises however, again it requires effective leadership and an efficient project pipeline to manage peaks and troughs. The Consulting model enhances resilience through income generation but relies on an established track record, strong leadership and senior civil servant endorsement. Although ultimately nothing can fully shield a policy lab from political decision-making, operating a Consulting model can enhance the resilience in light of constricting public sector budgets. It is important for labs to practice what they preach and apply design methods to continuous reflection on their own operation. In the aftermath of COVID-19, it will be timely for labs to once again reflect on their journeys moving forward.

5. Lab Proposition framework for establishing, reviewing and evaluating policy labs

The number of UK policy labs and user-centered policy design teams has grown significantly since 2014 when Policy Lab in the Cabinet Office and the Northern Ireland Innovation Lab (iLab) in the Department for Finance were established. There are now about 4000 designers working in central government - mostly interaction and service designers but a growing number of "policy designers" and even government's first "speculative designer." By 2020, there were around 10 policy labs or user-centered policy design teams in central government (see Figure 2: Map of Policy Labs and UCDP teams 2020) and three at devolved levels in Northern Ireland, Scotland and Wales. However, a number of labs at national and particularly local level have closed such as the Department for Trade and Investment's UKTI Ideas Lab (2014-2017) and local initiatives in Cornwall, Kent, Leeds, Monmouthshire, Shropshire, Surrey, Wakefield and Wolverhampton, among others (Fuller and Lochard 2016, 4-5). Why have policy labs and UCPD teams been more successful at national level than local level? Funding cuts no doubt play a significant role but this question requires further investigation. What is the lifecycle of a policy lab? How might we enhance the resilience of labs?

In 2017, the author was asked to support HMRC Digital in establishing HMRC Policy Lab and undertook a capacity building exercise in design for policy with a cohort of 25 civil servants. Although it is not possible to share details specifically of the HMRC Policy Lab case based on this intervention as well as a subsequent intervention with the Welsh Government, a framework for establishing, reviewing and evaluating policy labs and UCDP teams was developed. It is important for labs to practice what they preach and apply design approaches to continuously reflecting on their work. Policy labs can be vulnerable to political ebbs and flows and changing budgets, leadership and contexts and thus it is important to establish credibility, foster resilience, create a foundation for legitimacy and evaluate impact. At its core the framework for establishing, reviewing and evaluating policy labs has four components - Proposition, Product, People and Process (see Figure 3). This is exploratory research was developed inductively through interviews, workshops and immersive work with policy labs. Taking a design approach, this model has been iteratively developed and tested but still remains at its early stages of development and is intended as a provocation to encourage more researchers to develop models to develop, monitor and evaluate policy labs.

Proposition refers to the vision, governance, finance models and metrics of success for the lab. Policy labs tend to transition through a number of financing models (see Typology of Policy Lab Financing Models) from when they are first established tending to be entirely sponsored by one or more government departments to when they begin to charge for projects perhaps first asking clients to make a contribution and subsequently recovering the full cost to when they might benefit from multiple income sources. It is important for labs to develop a longer-term vision in order to foster resilience and legitimacy. Very often, labs are established as a one year experiment and thus after their first year of inception should seek to develop a forward-thinking vision that is continuously revisited each year in light of changing priorities and environments. It is also important for labs to understand their role in the wider departmental transformation - such as under the digital agenda. It can also be challenging to set concrete key performance indicators (KPIs) for Labs. On the one hand, setting stringent and quantitative KPIs reinforces old bureaucratic practices but on the other hand, without KPIs labs can drift from their core raison d'être. Further research is required to understand whether indicators such as "enhanced user satisfaction" are sufficient for protecting labs against the changing political tides. For this stage of the Proposition Framework, labs could explore the following questions: What is the vision for the lab now and moving forward? What is the governance structure and financing models? How does the lab contribute to the wider transformation agenda for the department? What does success look like? How can the lab demonstrate impact and value?

Product refers to the service and training offer and how that corresponds to user needs. As the lab begins to develop a "market" for their services they may transition from bespoke projects for each client to a set of "off-the-shelf package products." Examples of off-the-shelf products might be: (1) supporting teams to conduct user research and redefine the policy challenge, (2) engaging the public in co-designing policy concept, (3) prototyping and iterating policy options with users, (4) implementing more meaningful public consultation, (5) piloting, upscaling, monitoring and evaluating implementation. It is common for "clients" engaging with labs to not commission an end-to-end policy process from the outset rather to take projects step by step. A clear "menu" of services offered enables potential clients to see where design for policy approaches can add value at different stages of the policy cycle. It is also crucial for labs to embed the various innovation and design methods beyond the walls of the lab. Change management of any sort on an institution scale is challenging. It is important to move beyond ad hoc training and design sprints to fostering an agile, iterative and co-creative mindset within government. This can be done through integrating design methods into accredited civil service training modules and leadership initiatives as well as identifying champions. Questions for labs to explore include: What are the user needs? Is there a clear "offer" for potential clients? Will labs offer capacity building and mentoring?

People refers to the methods, skillsets, expertise and leadership required. Policy labs use a range of innovation methods including, but not limited to, design, big data, behavioral insights, co-production, ethnography, system dynamics modeling, lean and randomized control trials. At least in the UK, the dominant approaches are the triumvirate of design, data and digital. In light of further austerity measures in the aftermath of COVID-19 it is likely that there will be a recruitment freeze within the civil service. Sometimes lab staff are redeployed staff from other areas of the business who then procure external design expertise from agencies or academia. Very often labs contract out the design expertise to agencies, consultants or academics in the early days of the lab and seek to internalize that expertise as the lab gains notoriety and demand grows for lab services. At the point where external consultants leave and a lab seeks to build its internal capacity it is vital for a structured knowledge transfer to take place between the consultants handing over to the internal staff. As part of the process, labs could conduct a skills audit of their teams to ascertain if they have the appropriate policy, design and digital skills or whether there are gaps. It is important to have a multidisciplinary team with experts in policy and service delivery processes but also experts in the innovation or design methods such as service designers, policy designers, user researchers and interaction designers. Questions that labs might like to explore include: What are the key innovation methods offered by the lab? Do lab staff have the right skills or are there gaps? How might knowledge transfer be managed from external consultants to internal lab members? How can the lab build capacity for the approaches across the wider department?

Process refers to the user journey of the policy-maker engaging with the lab, the selection criteria for projects and how good practices will be shared. Design can be a difficult concept for policy-makers to grasp so there should be a clearly articulated user journey to enable policy-makers that are new to the approaches to understand the process. A key insight that emerged from the enquiry is the need for labs to develop selection criteria for projects. There is a temptation for labs to take on any project that comes their way. However, with limited capacity labs should be more selective about the projects they implement so as not to limit their potential by only dealing with the low hanging fruits. Some policy teams might see labs as a spare pair of hands, particularly in the context of restricted budgets, rather than an opportunity to develop new best practice and showcase results. As such, labs might consider exploring the following questions when defining selection criteria: Is the project consistent with the wider transformation agenda? Is the project of strategic importance to the department? Does the Sponsor have the necessary personal commitment, position and authority to implement outcomes from the lab? Does the Sponsor have resources for prototyping, testing and upscaling? Is there a reputational risk for the lab? What is the likelihood of achieving a consensus in the project that will result in realizable benefits? Is the Sponsor potentially prepared to share the results with the wider department or even make the results public? It is also vital to create a community of advocates in order to enhance credibility and embed the approaches beyond the lab. Labs tend to be a "safe space to innovate" and therefore operate behind closed doors; however, in order to share the approaches more widely labs should evaluate their interventions and share case studies and good practices. There is a need to "demystify" labs. A number of UK labs have active blogs sharing lessons from projects and contributing to a theoretical grounding - like Policy Lab - but a number of labs also work with very sensitive issues and do not share project examples with the wider community of practitioners - like HMRC Policy Lab. Questions for examination include: How do potential clients hear about the lab? What is the user journey? What is the selection criteria for projects? How does the lab engage externally with the public, if at all?

6. Conclusion

Policy labs and user-centered policy design team (UCPD) are multi-disciplinary government units experimenting with a range of innovation methods, including design, to collaboratively engage citizens in service and policy development. Design for policy is a creative, user-centered approach to problem-solving engaging users, stakeholders and delivery teams at multiple stages of the policy process. However, it remains an emerging domain where government practice is well in advance of academic theory. Since 2014, there has been a proliferation of policy labs at local, regional and national levels in the UK. However, it should also be noted that there has been a waxing and waning of labs in the UK local and city government over the years. In recent years, UK labs have expanded the repertoire of methods and tools in their toolboxes but the dominant approaches are "anchored in the trio of digital, data and design" (Williamson 2015, 258; Siodmok 2014, 25). Through interviews, workshops and immersive residencies as part of an AHRC Fellowship called People Powering Policy, a framework has been iteratively created to develop, monitor and evaluate policy labs and user-centered design teams exploring Proposition, Product, People and Process. Proposition refers to the vision, governance, finance models and metrics of success for the lab. Product refers to the service and training offer and how that corresponds to user needs. People refers to the methods, skillsets, expertise and leadership required. Process refers to the user journey of the policy-maker engaging with the lab, the selection criteria for projects and how good practices will be shared. A Typology of Policy Lab Financing Models was also developed outlining four main models for UK labs operations as well as an emerging fifth model - Sponsorship (funding from one or multiple departments), Contribution (labs recover part of the costs of projects), Cost Recovery (labs charge for projects on a not-for-profit basis), Hybrid (labs benefit from multiple income sources such as Sponsorship, charging and knowledge exchange funding) and Consultancy (labs charge a consultancy rate with a profit margin to expand operations). Labs and UCPD teams should practice what they preach and use design approaches to continuously review and innovate their models. This is particularly pertinent in the context of COVID-19, which has necessitated rapid, iterative policy-making with even shorter decision-making times. To mitigate against the risks of labs falling "casualty" to cost-saving exercises, labs should review how they add value to policy-making using the Labs Proposition Framework and how to enhance their financial resilence through adapting their financing models as per the typology.

Regional lab

National lab

Supranational lab

Example

Strengths

Weaknesses

Disclosure statement

No potential conflict of interest was reported by the author(s).

Correction Statement

This article has been republished with minor changes. These changes do not impact the academic content of the article.

Additional information

Funding

This work was supported by the Arts and Humanities Research Council.

References

Bason, C., & Schneider, A. (2014) 'Public Design in Global Perspective: Empirical Trends', In Design for Policy, edited by C. Bason. Farnham, UK: Gower Publishing.

Blessing, L., and Chakrabarti, A. (2009). 'DRM, a Design Research Methodology'. Springer-Verlag London, UK.

Fuller, M. , and A. Lochard. 2016. "Public Policy Labs in European Union Member States." Report prepared for EU Policy Lab, European Joint Research Centre, EUR28044. doi:https://doi.org/10.2788/799175.

Her Majesty's Treasury. 2018. The Green Book. Central Government Guidance on Appraisal and Evaluation. London, UK: HM Treasury.

Howlett, M. 2014. "From the 'Old' to the 'New' Policy Design: design Thinking beyond Markets and Collaborative Governance." Policy Sciences 47 (3): 187-207. doi:https://doi.org/10.1007/s11077-014-9199-0.

Jegou, F. , R. Thevenet, and S. Vincent. 2014. "Friendly Hacking into the Public Sector: (Re)Designing Public Policies within Regional Governments." In Design for Policy, edited by C. Bason. Farnham, UK: Gower Publishing.

Junginger, S. 2012. "Public Innovation Labs - A Byway to Public Sector Innovation?" In Highways and Byways to Radical Innovation, edited by J. Christensen and S. Junginger, 1st ed. Kolding: University of Southern Denmark & Design School.

Kimbell, L. , and J. Bailey. 2017. "Prototyping and the New Spirit of Policymaking." CoDesign International Journal of CoCreation in Design and the Arts 13 (3): 214-226.

Maffei, S. , F. Leoni, and B. Villari. 2020. "Data-Driven Anticipatory Governance. Emerging Scenarios in Data-for-Policy Practices." , Policy Design and Practice 3 (2): 123-134. doi:https://doi.org/10.1080/25741292.2020.1763896.[Taylor & Francis Online] [Google Scholar]

Nesta. 2015. 'World of Labs.' Accessed 04 January 2018 https://www.nesta.org.uk/blog/world-labs

Puttick, R. , P. Baeck, and P. Colligan. 2014. "i-Teams. The Teams and Funds Making Innovation Happen in Governments around the World." Nesta and Bloomberg Philanthropies Report, London, UK.

Sanders, E. , and P. J. Stappers. 2008. "Co-Creation and the New Landscapes of Design." , Codesign 4 (1): 5-18. doi:https://doi.org/10.1080/15710880701875068.[Taylor & Francis Online] [Google Scholar]

Siodmok, A. 2014. "Designer Policies." RSA Journal 4: 24-29.

Siodmok, A. 2020. "Lab Long Read: Human-Centred Policy? Blending 'Big Data' and 'Thick Data' in National Policy." Open Policy Blog, 17 January 2010. https://openpolicy.blog.gov.uk/2020/01/17/lab-long-read-human-centred-policy-blending-big-data-and-thick-data-in-national-policy/

Staszowski, E. , S. Brown, and B. Winter. 2014. "Reflections on Designing for Social Innovation in a Public Sector: A Case Study in New York City." In Design for Policy, edited by C. Bason. Farnham, UK: Gower Publishing.

Taylor, M. 2014. "The Policy Presumption." RSA Journal 4: 10-15.

Vincent, S. (2016) 'La 27e Region', Labs Connection conference hosted by EU Policy Lab, 17 October 2016. , [Crossref] [Google Scholar]

Whicher, A. 2017. "Evaluation of the Northern Ireland Public Sector Innovation Lab." Independent Report Commissioned by the Northern Ireland Department of Finance, Produced by PDR - International Design and Research Centre, Cardiff Metropolitan University, UK.

Whicher, A. 2020. "Challenges of the Future Design & Public Policy." AHRC Design Fellowship Report, Cardiff, June 2020.

Whicher, A. , and T. Crick. 2019. "Co-Design, Evaluation and the Northern Ireland Innovation Lab." Public Money & Management 39 (4): 290-299. doi:https://doi.org/10.1080/09540962.2019.1592920.

Williamson, B. 2015. "Governing Methods: Policy Innovation Labs, Design and Data Science in the Digital Governance of Education." Journal of Educational Administration and History 47 (3): 251-271. doi:https://doi.org/10.1080/00220620.2015.1038693.

# data[].attributes.attachment_link.title
Evolution of policy labs and use of design for policy in UK government

# data[].attributes.attachment_link.description
(2021). Evolution of policy labs and use of design for policy in UK government. Policy Design and Practice: Vol. 4, Policy Innovation Lab Scholarship: Past, Present, and the Future. Guest Editors: Adam M. Wellstead, Anat Gofen, and Angie Carter, pp. 252-270.

# data[].attributes.attachment_link.readable_text
There's lots of interest in co-design at the moment, coming from all sorts of different places. Since joining Paper Giant a few months ago, I've had the privilege of facilitating introductory workshops on co-design to service designers in Melbourne, policy workers in Auckland, public servants in Canberra, and most recently to health informaticians in Sydney.

Workshop participants at SDNow 2018

This has given me the chance to reflect on and iterate how I convey my enthusiasm (sometimes with a dose of healthy skepticism) and some practical tips on the transformative potential of co-design to professionals from a diverse range of backgrounds. The common question these workshops aims to address is:

How do you get started with co-design?

There are three key ways you can think about co-design if you want to begin to understand what makes it unique. It is a process, set of practical tools, and a set of principles.

The UK Design Council's 'Double Diamond'

One way to start is by learning about the process of design (thinking). Visual models like the double diamond (pictured above) are useful for illustrating the common practice of divergence (going wide) and convergence (getting focused) throughout a design or innovation process. These stage-based models also usefully reinforce the importance of starting by exploring a problem before you get too fixated on a solution.

But, as many of us know, in practice, the design process usually looks less like a diamond and more like a squiggle. And focusing on process models can make co-design seem completely unattainable for people who are not working in contexts where they can follow, let alone lead, a full design process from start to finish.

A 'big daddy' design squiggle by Stefanie Di Russo

Another way to introduce co-design practice is through sharing the practical tools that can be used to enable participation, collaboration and creative thinking. An empathy map, for instance, is a tangible way to show how to distill data from a user/customer/stakeholder interview or observation in a way that focuses on what the research participant might be thinking or feeling.

Empathy map by dgray_xplane.

However, as the folks over at Social Labs say: "much like frying or baking doesn't necessarily make a great meal", methods don't make a lab - or any other approach to innovation. Similarly, the OECD team dedicated to promoting public sector innovation has recently reflected that the world probably doesn't need more design toolkits. Many other practitioners and researchers have noticed that we may need to focus on mindsets rather than toolkits.

Toolkits have definitely been important - but they have not been sufficient.

This brings us to the last of the three key elements of co-design for public or social innovation: principles.

The principles of co-design canvas printed on A3, on a table at a workshop

We have identified a set of fundamental principles for co-design, inspired by the principles developed by the NSW Council of Social Services and informed by our own experience researching and leading co-design approaches.

Co-design is about achieving change and improving results or having a positive impact. While the outputs matter, the outcomes are more important.
2. Inclusive
Different types of participants with different kinds of knowledge (lived experience, professional and specialist expertise) are involved in the process.
3. Participative
People are involved as active participants with meaningful input throughout the process. Co-design is not about tokenistic consultation.
4. Respectful
All participants are seen as experts and their input (time, knowledge and other contributions) is valued and has equal standing.

Co-design is an experimental process aiming at innovation. It should be full of feedback loops, learning, iteration, and trial and error.

We have developed a canvas for people to consider these five key principles of co-design. The resource linked to here (and pictured above) also includes a case study showing how these principles have been applied in practice.

We have found that sharing the principles of co-design is a great way to introduce the fundamentals of this practice. By focusing on the principles of co-design, we can draw on the philosophical underpinnings of participatory design without preaching an overly prescriptive or rigid approach that is difficult to achieve in practice.

Intentions to apply the principles of co-design from participants at the HIC2018 UX workshop

By introducing the principles of co-design in a workshop setting, we have encouraged participants to think about how these principles can be applied in a context-specific way that is relevant to them and their work. The photo above captures some examples of how participants in one of our recent workshops intend to apply the principles of co-design in their work.

There's a lot more work to do to create conditions and build capabilities for co-design. We hope that our canvas and workshops at least offer an easy way for people to start asking the right questions as they endeavour to improve outcomes in their field and more meaningfully involve a diverse range of participants and stakeholders in their work.

# data[].attributes.attachment_link.description
There's lots of interest in co-design at the moment, coming from all sorts of different places. Since joining Paper Giant a few months ago, I've had the privilege of facilitating introductory workshops on co-design to service designers in Melbourne, policy workers in Auckland, public servants in Canberra, and most recently to health informaticians in Sydney.

# data[].attributes.attachment_link.readable_text
More than 4,000 people from across the farming community in England are now working with us to design, pilot, test and trial new farming policies and schemes.

In this post, we'll explain how we're working with the farming community. We'll also share the 7 co-design principles we're encouraging groups to follow.

Our work in 2021

Last year's work has given us a much better understanding of what people expect from us.

In September 2021, we published a blog post about what we learned when we undertook co-design at agricultural shows .

Since then, we have been working to address the feedback from the farming community on:

The feedback and lessons we've learned have not only helped us to shape policy, but 7 new co-design principles.

our commitment to working in the open

we listen and act on what people say is important to them

taking part in co-design truly shapes our policies and schemes.

Our co-design principles

1.Understand the scope

Clearly set out the problem that needs to be addressed through a policy or scheme change. Agree with those participating in co-design what we can change, taking government commitments into account.

2. Have a wide view of the problem space

3. Share power

People need to feel confident their voice is being heard. They need know they are helping to influence our policy and service design decisions .

5. Communicate clearly and effectively

Decisions should be taken after considering user needs, data, statistics, and people's lived experience.

Explain clearly and honestly what will happen and what won't happen following co-design. To improve, we must continually encourage feedback.

Coming in 2022

Get involved in co-design

If you work in farming, why not get involved in informing and shaping future Defra policy ? We hope to have more in-person events this year as restrictions ease . You can choose how much time you commit to the group, with most activities taking place online.

Sign up to take part in co-design. We'll send email updates on the events taking place for your subjects of interest .

# data[].attributes.attachment_link.title
Co-design in 2022: reflections, plans and principles

# data[].attributes.attachment_link.description
More than 4,000 people from across the farming community in England are now working with us to design, pilot, test and trial new farming policies and schemes. In this post, we'll explain how we're working with the farming community. We'll also share the 7 co-design principles we're encouraging groups to follow.

# data[].attributes.attachment_link.readable_text
There has been a plethora of claims in recent years about what co-design can do and why we should be using it, especially in the public sector.

Co-design, we are told, can help us to generate more innovative ideas, ensure policies and services match the needs of citizens, achieve economic efficiencies by improving responsiveness, foster cooperation and trust between different groups, meaningfully engage the 'hard to reach', and achieve support for change.

There is a risk that we are treating co-design as a "magic concept"[i] or a "policy unicorn"[ii], a mythical thing of beauty that has never been seen in real-life but whose existence we fantasise about. Indeed, while there are lots of good examples of co-design for human services, it's hard to find concrete and comprehensive examples of co-design applied in a policy-making process.

Sure, there is lots of talk about co-design in the public service, but the term is sometimes used loosely to refer to any collective approach to decision-making. When it's used in this way, everyone seems to be doing it, and co-design no longer has much meaning.

In a recent paper, I have argued that we need a more specific, shared definition of co-design for policy, especially if we want to advance research and practice in this field.
What is co-design for policy?
When I talk about (public) policy, I am thinking of it as a problem-solving activity that involves arranging the resources of government in a certain way to achieve a specific aim or set of objectives ( Howlett 2010). In other words - the process by which government chooses how to respond to a particular problem is the process of making policy.

A simple way to understand the term co-design is to break it down into its constitutive parts. The 'co' is typically considered an abbreviation for 'cooperative' or 'collaborative' design, which draws on the Scandinavian tradition of participatory design. 'Design' here refers to the discipline of industrial design, and is connected to the concept of ' design thinking ', which puts the people affected by an issue (or 'end users' of what is being designed) at the heart of an iterative process to find solutions to shared problems.

A more comprehensive way to understand co-design is as a process, set of principles, and application of practical tools. These are shown in the table below and elaborated on in my article in the Australian Journal of Public Administration [iii].

Co-design for policy: a process, principles and practical tools (Blomkamp 2018)

Defining co-design in this way helps us to compare and contrast co-design with approaches it is sometimes confused with. Co-design is similar to, but not the same as, participatory and deliberative models of democracy. Both these democratic models are concerned with issues of citizen participation and decision-making, but neither necessarily involve any creative methods or innovative outcomes. Co-design can go hand-in-hand with co-production, but the latter may only describe the implementation phase of policy, whereas co-design is about the full design process.
Finally, a key difference from user- or human-centred design is that, in co-design for policy, people with lived experience of the issue must be active participants throughout the policy design process.
A precise definition like this makes it harder to find and to achieve co-design for policy, but hopefully it can help us to move beyond the buzzword. Just as clearly describing a unicorn (as a creature with a single, pointed horn projecting from its forehead) helps us to tell the difference between a horse and a unicorn, this definition should help us to identify what is and what isn't co-design in practice.

# data[].attributes.attachment_link.description
There has been a plethora of claims in recent years about what co-design can do and why we should be using it, especially in the public sector. Co-design, we are told, can help us to generate more innovative ideas, ensure policies and services match the needs of citizens, achieve economic efficiencies by improving responsiveness, foster cooperation and trust between different groups, meaningfully engage the 'hard to reach', and achieve support for change.

# data[].attributes.attachment_link.title
Case Studies - Civic Service Design Tools + Tactics

# data[].attributes.attachment_link.description
A collection of case studies that illustrate the best in service design from NYC. A central resource for best practices in service design to support public servants and help spread service design methods across New York City government.

# data[].attributes.headline
Codesign: Where do you start as a policy maker?

# data[].attributes.subject
Codesign: Where do you start as a policy maker?

# data[].attributes.attachment_link.title
Codesign: Where do you start as a policy maker?

# data[].attributes.attachment_link.description
Dr Nina Terrey is a Partner at ThinkPlace and a leading expert in how design can be applied to complex public sector challenges. ThinkPlace is Australia's leading design firm working in the public sector. Nina is passionate about using design in the front end of policy making, looking for possibilities to use codesign to inform decision making.

# data[].attributes.headline
The Promise of co-design for public policy 

# data[].attributes.subject
The Promise of co-design for public policy 

# data[].attributes.attachment_link.readable_text
Co‐design holds great promise for policymakers. It can generate innovative ideas, foster cooperation and meaningfully engage the hard to reach. But does it deliver on this promise?

At a glance

In a paper for the Australian Journal of Public Administration, Emma Blomkamp (University of Melbourne) explores co‐design in public policy. As a novel method for engaging citizens to find solutions to complex problems, co‐design can potentially improve policy processes and outcomes. But there are challenges that question the feasibility of achieving these outcomes.
Related research:
Towards a definition of co-design

Co‐design is a distinct set of principles and practices for understanding problems and generating solutions. It signifies the active involvement of a diverse range of participants in exploring, developing and testing responses to shared challenges.

In the public sector, co‐design is often seen as a more effective alternative to conventional approaches in:

community engagement

public participation

service design

policy development.

When co‐design is loosely defined as any type of collaborative or participatory activity, almost everyone seems to be doing it. A clear and shared definition is lacking.

A simple way to understand co‐design is to break it down into its parts:

The 'co' is typically considered an abbreviation for 'cooperative' or 'collaborative' design.

'Design' draws on the discipline of industrial design.

Co-design and public policy

As a methodology for policy making, co‐design has three components:

1. Process: iterative stages of design thinking, oriented towards innovation.

2. Principles: people are creative; people are experts in their own lives; policy should be designed by people with relevant lived experience.

3. Practical tools: creative and tangible methods for telling, enacting and making.

A design-led process

Design thinking is an iterative, human‐centred and action‐oriented process for innovation. Defining co‐design as a design‐led process highlights its use as a methodology for innovation. It is about generating and testing new solutions to public problems, not just offering creative approaches to consultation or 'co‐production' at the delivery stage.

Principles of participatory design

Co‐design is underpinned by participatory design. Applied to policy, this means enabling or empowering people affected by a policy issue to contribute to its solution. As experts in their own experiences, citizens and stakeholders should be involved in designing services and policies that relate to those experiences.

Practical tools

Co‐design uses practical tools to generate and test ideas. There are three main techniques: telling, enacting and making. This can include diaries, collages, card sorts, model building, mapping and roleplaying. This is different from deliberative approaches to policymaking which focus on telling with very little enacting or making.

Claims about the benefits of co‐design

In the public sector, co‐design is seen to improve idea generation, service delivery, project management and longer‐term outcomes. These benefits are in relation to product and service design but are often extended to policy design. However, many of the claims have not been rigorously evaluated.

The lack of published evaluations of co‐design in public policy limits knowledge sharing and evidence building. However, there is evidence from other fields including healthcare, urban planning and the private sector. For example, applying a participatory design approach in healthcare to improve the patient experience has increased efficiency across the health system in the United Kingdom and Australia.

The challenges of co‐design for policy

There are practical challenges and risks in applying co‐design in government:

1. Diminished control over the project because other people, other departments or other organizations are involved.

2. Increased complexity because the objectives and interests of diverse people, departments or other organisations must be managed and balanced.

3. It can be time-consuming although the short‐term costs of participatory design are likely outweighed by the long‐term benefits.

4. The gap between co‐design research and conventional forms of evidence.

5. The legitimacy of co‐design activity as perceived by stakeholders and beneficiaries.

A significant challenge is the structure and culture of government is not suited to co‐design. Policy officials do not respond well to the risks of diminished control and increased complexity, while bureaucratic systems are not designed to be experimental or responsive.

What this means

The challenges of government may make it difficult to achieve the potential benefits of co‐design in practice. Yet its potential to transform the process and outcomes of policymaking warrants further exploration.

Co‐design holds great promise for policy. It may help to:

generate more innovative ideas

achieve economic efficiencies by improving responsiveness

foster cooperation between different groups

reinvigorate trust between citizens and public servants.

If co‐design can achieve some of these benefits, then public sector organisations and policymakers should be exploring ways to adopt and embed this practice. Further research and evaluation is needed to strengthen what co‐design in policy means in practice and the benefits it brings to participants, policymakers and the people they serve.

Want to read more?

The promise of co‐design for public policy - Emma Blomkamp, Australian Journal of Public Administration, March 2018, 77(4), pp. 729-743.
This Research Brief is written by Maria Katsonis as part of ANZSOG's new research translation series, The Bridge. This project is designed to bridge the gap between the research work of academics and the policy work of public managers by providing access to visible and accessible high-quality research. The Bridge is emailed fortnightly to thousands of engaged readers and centers around a Research Brief which distills academic research into an easy-to-read format. Sign up to the bridge
Recent Research Briefs include:

# data[].attributes.attachment_link.keywords[]
the australia and new zealand school of government

# data[].attributes.attachment_link.title
The promise of co-design for public policy

# data[].attributes.attachment_link.description
Co‐design holds great promise for policymakers. It can generate innovative ideas, foster cooperation and meaningfully engage the hard to reach. But does it deliver on this promise? At a glance In a paper for the Australian Journal of Public Administration, Emma Blomkamp (University of Melbourne) explores co‐design in public policy.

# data[].attributes.headline
Co-Designing the principles for defining low impact fishing 

# data[].attributes.subject
Co-Designing the principles for defining low impact fishing 

# data[].attributes.attachment_link.readable_text
The CCRI worked with the New Economics Foundation (NEF), MRAG and Cefas in 2019-20 on a project for Defra that that sought to co-design, with fisheries stakeholders, principles and criteria that can be used to define low impact fishing.

In line with Defra's support for sustainable fisheries, the project built on the commitment in the Sustainable fisheries for future generations White Paper (2018) to "consider new criteria to define low impact inshore fishing vessels to replace the current 'under 10 metre' category". What constitutes 'low impact' is not currently well defined and therefore there is a need to identify objective, transparent and workable criteria. Similarly, the Government's 25 Year Environment Plan envisions management that "...accounts for, and seeks to minimise, impacts on non-commercial species and the marine environment generally". Alongside this, the White Paper also seeks to further integrate recreational sea angling within the new fisheries framework - impacts associated with recreational sea angling were also considered as part of the project.

Through a series of consensus building deliberative workshops (in Eastbourne, Brixham and North Shields), the project explored and documented the various factors that could define low impact fisheries and identify how plans for reducing impacts of commercial and recreational fishing could be produced and updated. The final report (see link below) presents the findings from the project and makes recommendations for how principles of co-design can further help policy formulation, recognising the need to establish conditions under which developing a shared commitment to policy outcomes is possible. As Dr Julie Urquhart comments, "It is vitally important to work with fishing and angling stakeholders at the early stages of developing any potential definition for low impact fishing. Fishers, for one, know their job and the environment in which they work better than anyone, so any criteria for measuring low impact needs to be relevant, achievable, have benefits for fishers, such as improved market prices, and be agreed through consensus-building between fishing stakeholders and policy makers."

The project was led by Chris Williams from NEF.

CCRI partners include Dr Julie Urquhart, Dr Hannah Chiswell and Nick Lewis

Project Documentation
Terms of Reference - Short

Terms of Reference - Full

Literature Review - References

Presentation made at 1st workshops
Eastbourne
Notes made at 1st Eastbourne workshop

Presentation made at 2nd Eastbourne workshop

Notes made at 2nd Eastbourne workshop
Brixham
Notes made at 1st Brixham workshop
North Shields
Notes made at 1st North Shields workshop
Project update: March 2021
The following outputs from the project are now available:

DEFRA - Low Impact - Co-Design Final Report DEFRA - Low Impact - Co-Design: Literature Review

# data[].attributes.attachment_link.title
Co-Designing the principles for defining low impact fishing

# data[].attributes.attachment_link.description
The CCRI worked with the New Economics Foundation (NEF), MRAG and Cefas in 2019-20 on a project for Defra that that sought to co-design, with fisheries stakeholders, principles and criteria that can be used to define low impact fishing.

# data[].attributes.headline
Fundamental things that make a citizens’ assembly a citizens’ assembly

# data[].attributes.subject
Fundamental things that make a citizens’ assembly a citizens’ assembly

# data[].attributes.attachment_link.readable_text
Citizens' assemblies bring together people from all walks of life - selected randomly, but to be demographically representative - to consider a public issue in depth over multiple days and meetings.

Assembly members hear evidence, question witnesses and deliberate with one another, before reaching recommendations on what they think should be done. Citizens' assemblies put the trade-offs faced by decision-makers in front of members of the public and ask them to arrive at workable recommendations.

Amongst all of the enthusiasm surrounding citizens' assemblies at the moment, the term is starting to lose some of its meaning. We've been guilty of this ourselves, at times compromising the precision of definitions with the hope of demonstrating the power of the model and building towards more robust processes. While it's important that standards do not curb innovation, it's critical that methods are not watered down beyond recognition.

Not all assemblies of citizens are citizens' assemblies - and neither do they need to be

Recently we've started to see tenders for "citizens' assemblies" lasting fewer than 8 hours. While a community conversation of this nature may very well be worth doing, it does nobody any favours to pretend that it is a citizens' assembly. Misusing terms at best creates confusion and at worst builds expectations that cannot be met, and ultimately cheapens and undermines the method.

Not all assemblies of citizens are citizens' assemblies - and neither do they need to be. There is a vast toolbox of public participation methodologies that can be used to involve people in decision-making in a variety of ways. 1 This includes a range of other deliberative processes - such as citizens' juries, citizens' panels and deliberative polls - that are appropriate for different purposes and have their own sets of standards.

A citizens' assembly is a specific democratic tool to be used in specific circumstances. Their power comes from their robust process, which gives a representative group of the public time and support to engage with a topic in depth. But this process makes them time and resource intensive compared to many other methods of engagement, so citizens' assemblies should be reserved for the really knotty issues that require challenging trade-offs to be made.

Where these circumstances exist, a citizens' assembly can make a substantial contribution to helping to resolve an issue - but it must be properly resourced and well run to enable it to succeed.

Towards some standards

With this in mind, we've had a go at developing some draft standards for the citizens' assemblies that we design and run. These are based on our own practice, understanding of international practice and a range of standards that have already been developed across the globe (including by us). They are intended as a starting point for discussion with other practitioners, experts and commissioners to refine them over the coming weeks and months. We hope that they might form the basis for some collectively agreed standards among practitioners and commissioners in the UK.

There will continue to be debate around these definitions. It is true, for example, that processes that we call "citizens' assemblies" in the UK would often be termed "citizens' juries" or "reference panels" in other countries, where the term "citizens' assembly" is reserved for much larger and longer deliberative processes. Those variations across countries will need to be ironed out as standards and typologies are agreed internationally. The OECD is currently doing some great work on this through their Innovative Citizen Participation project which they will publish early next year.

At some point we will need to bring our definitions of different deliberative methods (e.g. citizens' juries, citizens' panels, citizens' assemblies, citizens' summits, etc.) in the UK into line with that emerging international consensus, but what we're most concerned with at the moment is that the term "citizens' assembly" is reserved for high-quality deliberative processes and is not used for processes with limited or no deliberation that are more akin to focus groups.

The standards below are organised into "essential" and "desirable" features of ten criteria:

1. Clear purpose

2. Sufficient time

3. Representative

4. Inclusive

5. Independent

6. Open

7. Generative learning

8. Structured deliberation

9. Collective decision-making

10. Evaluated

We consider the essential features to be the fundamental things that make a citizens' assembly a citizens' assembly. The absence of any one of these features would require detailed justification and would only be warranted in exceptional circumstances. The desirable criteria are the additional features that we consider to be current good practice.
The following draft standards are also available via an open and commentable GoogleDoc and we welcome your help in refining them!
The draft standards - for comment

1. Clear purpose

There is a clear question / set of questions for the assembly to address, which has / have a range of different possible solutions

The scope for making a difference to the policy or decision is explicitly declared at the start and things that are out of scope or cannot be changed are clearly outlined

Decision-makers make a public commitment to consider and respond in detail to the recommendations

There are a clear set of trade-offs for the assembly to address

There is support for the citizens' assembly from across key political divides

The assembly is commissioned by a public authority with responsibility for the issue in question

3. Representative

40 or more assembly members are recruited 2

A pool of potential assembly members is created through random selection, using a recognised market research recruitment methodology

Assembly members are selected from this pool using random stratified sampling based on demographic criteria to ensure that they are broadly representative of the wider population

100 or more assembly members are recruited

The pool of potential assembly members is created through a full civic lottery / sortition process

Where relevant, assembly members are selected using attitudinal sampling (as well as demographic sampling) to ensure that they are broadly representative of the wider population

4. Inclusive

Assembly members are reimbursed for all reasonable expenses

A gift of at least £50 per day is given to assembly members

The accessibility requirements of assembly members are met on request

Carers of assembly members are welcomed and provided for

There is a ratio of max 9 assembly members per group facilitator

Presentations by witnesses are accessible, avoiding jargon and not assuming prior knowledge

A gift of at least £75 per day is given to assembly members

Information / materials are provided in a range of different formats

The care costs of any assembly members are reimbursed and/or caring facilities are provided onsite (e.g. a creche)

The accessibility requirements of assembly members are anticipated and met

There is a ratio of max 7 assembly members per group facilitator

7. Generative learning

Assembly members hear balanced, accurate and comprehensive information and evidence

Assembly members hear from diverse witnesses with a range of views

Assembly members determine their own questions for witnesses and have sufficient time to question them

Witnesses are briefed so that they clearly understand that their role is to stimulate and support discussions among the assembly members, not to lead or direct them

The learning phase supports the subsequent deliberation and decision-making phases, enabling assembly members to arrive at informed and considered judgements

8. Structured deliberation

Assembly members are supported through a facilitated process to consider and weigh-up different perspectives

Assembly members are given time to discuss issues with as many of their fellow participants as possible

The assembly process is well structured, with a clear progression through learning and deliberation, to decision-making

The assembly process is designed and led by professional facilitators

The assembly process allows time for plenary feedback and summing up, so that assembly members can hear views from across the assembly

Facilitators are well briefed and provided with any necessary training ahead of the citizens' assembly

9. Collective decision-making

Visit the commentable version to suggest revisions.

# data[].attributes.attachment_link.title
When is a citizens' assembly not a citizens' assembly? - Towards some standards

# data[].attributes.attachment_link.description
Amongst all of the enthusiasm surrounding citizens' assemblies at the moment, the term is starting to lose some of its meaning. Misusing terms at best creates confusion and at worst builds expectations that cannot be met, and ultimately cheapens and undermines the method.

# data[].attributes.headline
Difference between co-design & Participatory design

# data[].attributes.subject
Difference between co-design & Participatory design

# data[].attributes.attachment_link.readable_text
* < 1 minute read

These terms are often used interchangeably but they are actually two different processes.

Co-design, design with users, not for them.
Example:
Participatory design you design for users and then get their input.

Co-design: you ask users to create the questions which are used to ask their own user group for product feedback

Participatory design: you come up with the questions to ask users for product feedback

Co-design

Co-design is about building a bridge between yourself who has the expertise in tech and design and the users who have the first-hand understanding of their own needs.

Co-design can involve coaching (I.e what technology and designs are possible within a budget) and involving them at the start of the design process so they can generate ideas to solve their own problems.

As much as possible you want to put users in the driving seat but without there being a need to teach them sketch, photoshop or programming.

Participatory design

In Participatory design, you create the product. This means doing much of the background research, interviews and data collection.

You can then get feedback and input from users as early as possible through focus groups, workshops or user-testing.

References

1. Getting an autistic mentor

2. Co-design and participatory design: a solid process primer

# data[].attributes.attachment_link.title
Difference between co-design & Participatory design - Passio

# data[].attributes.attachment_link.description
1 minute read These terms are often used interchangeably but they are actually two different processes. Co-design, design with users, not for them. Example: Participatory design you design for users and then get their input.

# data[].attributes.attachment_link.readable_text
An exploration of how design might be led by marginalized communities, dismantle structural inequality, and advance collective liberation and ecological survival.
What is the relationship between design, power, and social justice? "Design justice" is an approach to design that is led by marginalized communities and that aims expilcitly to challenge, rather than reproduce, structural inequalities. It has emerged from a growing community of designers in various fields who work closely with social movements and community-based organizations around the world.

This book explores the theory and practice of design justice, demonstrates how universalist design principles and practices erase certain groups of people-specifically, those who are intersectionally disadvantaged or multiply burdened under the matrix of domination (white supremacist heteropatriarchy, ableism, capitalism, and settler colonialism)-and invites readers to "build a better world, a world where many worlds fit; linked worlds of collective liberation and ecological sustainability." Along the way, the book documents a multitude of real-world community-led design practices, each grounded in a particular social movement. Design Justice goes beyond recent calls for design for good, user-centered design, and employment diversity in the technology and design professions; it connects design to larger struggles for collective liberation and ecological survival.
Chapters of Design Justice will be made OA on a monthly basis.

# data[].attributes.attachment_link.description
An exploration of how design might be led by marginalized communities, dismantle structural inequality, and advance collective liberation and ecological survival. What is the relationship between design, power, and social justice? "Design justice" is an approach to design that is led by marginalized communities and that aims expilcitly to challenge, rather than reproduce, structural inequalities.

# data[].attributes.body
Citizens, Assemble! | Insights | FutureGov (wearefuturegov.com)

# data[].attributes.attachment_link.description
A citizens’ assembly is a demographically-representative group of people who come together to provide recommendations on a particular issue. The decisions made should then be used to encourage change within the government.

# data[].attributes.body
IIDP-citizens-assembly.pdf (thersa.org)

# data[].attributes.body
Newham Citizens' Assembly – Citizens' Assemblies – Newham Council

# data[].attributes.attachment_link.description
Newham neighbourhood citizens’ assemblies are large open meetings where you and others from your neighbourhood decide how to improve your area.​

# data[].attributes.headline
Making it better together: a framework for improving creative engagement tools

# data[].attributes.subject
Making it better together: a framework for improving creative engagement tools

# data[].attributes.attachment_link.readable_text
ABSTRACT

ABSTRACT
This research reports on a co-design project to improve creative engagement tools with academics and public sector organisations in the northwest UK. Creative engagement (which is a staple of co-design activities but also used widely outside design) is often supported by tools and resources. However, there is a need to tailor tools for specific contexts to accommodate the skills and practices of creative engagement professionals and the contexts in which they work. While there is a literature examining tools in co-design and to a lesser extent in wider creative engagement activities, there is a lack of research on how tools can be improved. This article presents a framework that enables engagement practitioners to improve the tools they use in their practice. Following a Participatory Action Research approach, three case studies document the application and testing of the improvement framework. The paper discusses the insights and lessons learnt from this process and the impact of the new improvement activities on the practices of the creative engagement professionals. The research outcomes include building improvement capabilities in participants and understanding of how the framework works in practice and how it could be more widely applied to tool improvement within and beyond co-design.

1. Introduction

In this paper, we describe research where we work with creative engagement professionals to conceive and test a framework aimed at improving the tools and resources they use in their professional activities. In essence, any creative engagement professional (including co-designers) should be able to use this framework to help them reflect on and improve the tools and resources they use.

Creative engagement (CE) involves enabling an expressive dialogue often between communities and public bodies using creative acts (e.g. film, photography and storytelling). There is a close relationship between co-design and CE, for example, with advocacy planning (Davidoff 1965) and open philosophies (e.g. open source, design) that have been in the spotlight of design studies in the past decades (Lee 2008; Cruickshank, Coupe, and Hennessy 2013; Baibarac and Petrescu 2019). In advocacy planning, Davidoff ( 1965) asserts that an urban planner should support the development of alternative renewal approaches that could advocate for communities' interests and include the voices of those affected by a policy in public decision-making processes. Non-designers, such as community organisers and student groups, have a long history of conducting such planning processes with communities since the 1960s. Similarly, co-design processes can be initiated by anyone interested in improving their current situations and may or may not involve professional designers (Zamenopoulos and Alexiou 2018). While CE practice and co-design use similar approaches, CE, however, is also used beyond co-design, for example, where straightforward information gathering is the aim (e.g. in patient participation in the health service) or where engagement will contribute to decision-making but there is no requirement for the abductive, creative leaps common to co-design (Manzini 2015; Cramer-Petersen, Christensen, and Ahmed-Kristensen 2019).

Both co-design and CE are often supported by tools to enhance the creative abilities of those involved in engagement activities. Co-designers often use toolkits and handbooks with methods that might not address a particular problem but provide principles and instructions that guide them to design their own engagement approaches, such as the Community Planning Handbook (Wates 2000) and NESTA, National Endowment for Science Technology and the Arts ( 2014). However, many tools and methods assume anyone has the skills to easily employ them. For example, some tools are translations of designerly methods into popular versions taken out of context (e.g. IDEO tools), but the person using the tools must have the knowledge and skill to understand how to use them (Johansson-Sköldberg, Woodilla, and Mehves 2013).

Drawing on scaffolds for experiencing (Sanders 2002) and toolkit approaches that encourages democratic innovation (Von Hippel 2005), we also appropriate the concept of 'convivial tools' (Illich 1973) and good design (Norman and Draper 1986) to define tools as means of enhancing skills, giving people control when conducting tasks and enabling them to constructively apply tools in their own practice. Here we define methods as a coherent set of principles and include guidelines in terms of tools, techniques and principles for organisation (Bratteteig et al. 2013), where techniques are specific ways to perform an activity and are how tools are put into action (Sanders, Brandt, and Binder 2010) as concrete instruments that support techniques and skills, such as pencils and pens as tools for sketching, drawing and annotating. Tools could be physical, digitally downloaded and printed or entirely digital in nature. We believe tools can be fitted into a larger set of methods more or less at will (Brandt, Binder, and Sanders 2012) enabling people to freely deploy tools in their creative practice. Some tools have crossed over between practices acquiring joint identity across fields (Levina and Vaast 2005) and have been improved in the process (Sanders and Stappers 2012).

The design of tools that respond to a specific project and envision their use before use has been criticised as stakeholders and potential users will appropriate tools in unpredictable ways, requiring more flexibility in use (Ehn 2008). Each CE project needs to be designed according to an understanding of the context, where tools should accommodate multiple design languages and skills of communities to support them in the design and decision-making processes, enabling those involved to appropriate tools in their practices. For example, a co-design study required adapted approaches and tools to enable people with communication impairment to participate in the project (Wilson et al. 2015). While there is a literature examining design, adaptation, and evaluation of tools in co-design, there is a lack of research on how these can be improved. Peters, Loke, and Ahmadpour ( 2020) argue that it is ideal to evaluate tools within a rigorous context to support the improvement of tools as it often relies on observations of the tool designers themselves in a lab environment. They reviewed 76 tools and concluded that user involvement in tool design and evaluation would benefit the co-design community as tools could be context-appropriate to real-life practices.

Researchers have co-designed flexible tools with experts in urban spatial, health and social care settings for ongoing use and appropriation in other contexts in more recent tool design approaches (Baibarac and Petrescu 2019; Whitham et al. 2019). However, appropriating tools designed elsewhere to be applied in different fields requires tailoring them for local conditions such as healthcare (Donetto et al. 2015), urban planning (Iaione 2016), or social services (Cruickshank et al. 2017). Our approach here is to acknowledge the expertise of CE practitioners in their own CE practice. We worked with a wide range of CE practitioners both inside co-design and outside to develop and approach together that allows any CE practitioner to reflect on and improve the tools they use in their own engagement practice. This supports the improvement of CE practice without claiming a hierarchical position or imposing values or our practices as co-designers. This is in sympathy with Lee's 'Design Choices' research ( 2018), so, for example, Lee calls for engagement with the preconditions of co-design or a focus on co-creation events. Our concern here is to help practitioners examine and improve what they do with their own understanding. Here, the improvement framework could be brought to bear by practitioners to improve these aspects (and others) of their CE practices through an examination of the tools used and how that practitioner has appropriated them into their own practice.

Appropriation may involve adaptation of a tool or improving it to create new versions. Although there are some overlaps on what constitutes improvement and adaptation, these concepts are not the same. Adaptation processes happen in a non-deliberate manner to fit a tool better to an existing framework (De Waal and Knott 2013), whereas improvement processes involve identifying issues and proposing positive changes to a framework in a deliberative way. In this paper, improvement is an activity that consists of a cycle of critical observation, creative design inputs that lead to agreed positive changes in life and work situations as well as attempts to increase knowledge. Design knowledge is not intrinsically linked with the development of artefacts but rather tied to practice where the reflection in action is how knowledge is generated (Swann 2002). Tool design practices include stages for future iterations and expansions to gain wider relevance (Baibarac and Petrescu 2019) or continuous adaptations to suit practitioners' applications as they arise (Morris and Cruickshank 2013).

This research follows a Participatory Action Research approach to address the research question: how can CE practitioners be supported to improve their own tools? To guide this project, we draw on PD methods and principles (Bratteteig et al. 2013) and those of co-design (Zamenopoulos and Alexiou 2018) to underpin our improvement actions and on the case study framework (Yin 2018) to conduct research. The first step to address the research question was to build a proposition used for improving tools. In an approach similar to Cockton's 'meta-principles' ( 2014), we identify fundamental properties of all CEs and propose the use of three dimensions (Instruction, Functionality and Flexibility). We developed a framework that builds on the co-design practice landscape of planning, enabling and the actual doing activities in workshop-like events. The second step was to design activities around the framework to help understand how they are related to each other. These activities include questions in the form of tasks to encourage participants to work through the dimensions, where engagement practitioners worked together to learn and reflect on the propositions to understand how tool improvement occurs in practice. Lastly, we shared the research findings with experts in tools and participatory approaches to raise awareness and provide insights, discussing their implications in CE practices. In the following section, we review guidelines for designing physical and digital artefacts and activities involved in co-design to build a framework as a meta-tool to improve CE practices.

2. Building a framework for improving tools for CE: bringing tool design practice and theory together

We adapted Hopper's architectural guidelines applied in interface design ( 1986), where we consider the functionality, instruction (interface), and flexibility and adaptability to build a framework used in tool design. We describe each design aspect, drawing a parallel between architectural/interface design and tool design. Here we describe the three key dimensions of the framework.

(1) Interface (Instruction): Hooper argues that computer interfaces are like façades, which people experience primarily when they face interface designs, or like entranceways that are designed to inform the whole place in a systematic way, like European cathedrals and formal Japanese gardens. In interface design, the inside and the outside of an interface are the relationship between design concept and purpose. In general, the design specification is articulated with a briefing that guides the concept, providing essential information about the design of a building, interface or tool. A good tool can be designed with specific colours, shapes, words that enable people to familiarise with activities and things.

(2) Functionality: The function is the primary consideration in the design of digital interfaces, buildings and tools. The function of a design is to fulfil the purpose for which it was intended, i.e. a design that works. Beyond this level, the final form depends on the people's needs and the context as different forms may represent the same functionality. For example, the form of a tool used for engaging with young people should be full of colours, but the tool might not have the same form when engaging with young adults as they might feel they are being treated as children.

(3) Flexibility and adaptability: In vernacular architecture, buildings are adapted to contain larger families, or they are changed to improve on the earlier effort. Hooper highlights that mechanisms for change are critical to flexibility and adaptability. For instance, double-click timing on operational systems is the local control that enables personalisation to users' computing skills. In another example, a paper-based tool can provide different layouts or editable headings in the digital file or provide blank spaces to allow extra information. Hooper concludes that:

'flexibility in personalisation may not necessarily provide adaptable systems. One may want to rely on expert judgment of a best system as a first approximation, making changes available from this base level. One might to prevent the moving of walls, for example, but encourage the rearrangement of furniture' (Hooper 1986, 15)

These three dimensions were applied to three layers of practice in CE: a) planning (activity before events); b) facilitation of human interactions that enable the process of CE; and c) application or doing for the practical use of tools with participants.

(a) Planning activities involves considering the aims and objectives, audience and actions used for engaging with participants, where tools are often adopted to assist this practice. These elements compose a collaborative structure for a common action that enables the emergence of new designs also referred to in the literature as negotiation spaces (Pedersen 2020), design spaces (Marttila and Botero 2013), or solution space (Von Hippel 2005).

(b) Enabling activities involves implementing the plan within a collaborative space, where a facilitator uses methods, techniques, and tools to facilitate a creative exchange between participants. The facilitator's role is to make sure everyone can contribute to an activity, making the most out of the expertise and creativity of participants. Tassoul defines the job of facilitation as ' setting the right conditions for a group of people to do a good session, highly inspired and a high quality of interactions and concept generation' (Tassoul 2009, 33). In this practice known as creative facilitation, a facilitator formulates mechanisms that have specific functions (e.g. energising participants, generating ideas) and uses approaches developed in and on practice (Forester 1999) to draw participants into design processes.

(c) Doing activities involves exchanging expertise and ideas with participants through tools for making, telling and enacting activities or a combination of them (Brandt, Binder, and Sanders 2012) to collaborate in the design and decision-making processes. Visual tools assist participants in expressing their experiences in telling activities. Tools give people the ability to create things to externalise ideas and embodied knowledge in the form of artefacts in making activities. Tools can support people to imagine and act out possible futures by experiencing a design setting and exploring activities that are likely to take place in enacting activities.

Building on these overlapping practices, we will map these layers and the dimensions into a framework, called Improvement Matrix (Table 1). The proposition we tested is that this matrix of nine considerations allows CE professionals to dissect the tools they use in terms of their tool conception, introduction to participants and practical application, and use these categories to improve their tools.

Making it better together: a framework for improving creative engagement tools

26 April 2021

3. The improvement matrix: a framework for improving CE tools

section summarises the content of the framework and looks at how the improvement of tools using the design propositions predicts positive changes in each co-design practice and activity.

(a) Planning (Design)

The improvement matrix suggests three components that support the practice of planning (Table 2), known as design as shown below.

(a.1) Challenge/Briefing: the briefing formalises mutual and coherent understanding of objectives, drivers, and issues, which can be framed in a dynamic and participative process by those involved in a CE project (Murphy and Hands 2012). The briefing guides the concept, providing essential information about the design of a tool. It describes the frame in which a tool addresses a particular engagement challenge. For example, tools for engaging with people with aphasia would be framed to deal with the challenge of engaging with people who have difficulty with spoken and written language, presenting non-verbal elements that would enable them to participate in the design process of computer-based therapy tools (Wilson et al. 2015).

(a.2) Interaction models: The function is the primary consideration in the design of a space as part of an interaction model. The interaction model describes how an interface should work to enhance the use of digital products and how a system is organised and operates. The interaction model binds the intentions and engagement context which a tool is designed for. It is how a tool and inputs that are part of an activity interrelate, in ways that support real-life interactions (i.e. practical use). Interactions are inputs people have to perform when they are addressing their engagement challenges. For example, a tool that collects drawings as responses about young people's preferences enables practitioners to take decisions based on evidence, where drawing is the interaction model that satisfies the intention of collecting young people's voice in the engagement process.

(a.3) (Build) Resilience: Building tool resilience to deal with unforeseen applications involves designing tools that allow appropriation. Dix ( 2007) discusses the design for appropriation, where designers can design to allow for the unexpected by, for instance, not making systems or products with a fixed meaning. For example, in a CE project called Make it Stick (Cruickshank et al. 2017), the researchers developed a tool to enable CE without the need for participants to write, which was initially designed to be customised, downloaded, and printed. However, they noticed that the tool was not meeting the user's needs due to the limited customisation. As a result, they designed an interactive template that allowed people to customise the sticker template, enabling people to use it in unexpected ways.

The improvement of tools within this layer of practice will develop the CE practice of planning collaborative spaces, providing engagement practitioners with new ideas to address their challenges in current and future projects.

(b) Enabling (Facilitation)

The second layer of the matrix related to the practice of enabling CE, known as facilitation layer ( Table 3), suggests three components that support a creative facilitation practice with tools as shown below.

(b.1) Facilitator notes: Facilitators identify priorities and expectations of stakeholders, which can be formalised in an agreed briefing with a group of people affected by a project, similar to a typical design process as inputs for creating facilitation frameworks (Cruickshank and Evans 2012), and then establish a facilitation approach. Facilitator notes include the plans for implementing a session and the activities that participants will follow through on in a project using specific tools. These notes describe to a facilitator how a tool should be introduced to participants and include instructions about the space, duration, requirements, examples, techniques, activities, etc. This kind of information can be provided in a guideline sheet, website or handbook to support facilitators in assisting and assigning tools in their practice. A tool that instructs facilitators on how to draw out ideas from their participants can be improved to provide more appropriate guidelines for a particular context or by adding extra information for facilitators.

(b.2) Resources produced by facilitators: Once the approach is established, facilitators assign or produce resources and tools to support the facilitation of activities. The function of tools should fulfil the purposes of supporting the facilitation, enabling participants to achieve desired outcomes. These resources produced by facilitators include maps, visual materials, and inspirational exercises that support them to engage with participants, guiding their actions and collecting information needed for learning and evaluation in a planned session. For example, a tool that supports facilitators to gather collective ideas from a group of entrepreneurs can be improved to work with local residents by giving specific actions to promote better creativity and problem-solving skills that fit residents' expertise.

(b.3) (Encourage) Facilitator response: This approach can be associated with improvising sessions within a planned structure at the time of project delivery. Flexibility in facilitation is about designing a session as a type of conceptual prototype, where role-playing the planned ideas for activities and analysing the implications of these lead to a practical facilitation session. Once the facilitation approach is designed to respond to an agreed briefing and facilitator resources and notes are created to support CE, the session delivery requires flexibility and adaptability of the plan. A responsive facilitation is not about what a facilitator has to do to follow the plan, but what facilitation options are available for them to achieve an agreed objective. A way to improve facilitator response can be achieved by providing different ways to facilitate activities around a tool. Describing experiences and stories about facilitation strategies using a tool can enhance facilitators' response and their ability to improvise. For example, a tool can be improved by providing examples of uses and tips to engage participants, focusing on different situations where it might not work as expected, providing ways to change the facilitation approach and afford flexibility to new facilitators.

The improvement of tools in this layer of practice will develop the CE practice of enabling people to exchange ideas and inputs in design processes, providing facilitators ways to assist participants' understanding and their contribution to projects with their expertise.

(c) Doing (Application)

The third layer of the matrix related to the practice of doing CE, known as application ( Table 4), suggests three components that support the practical use of a tool by participants, which are described below:

(c.1) Example or use notes: Once basic requirements are met, other attributes contribute to make the tool work. The instructions (interface) are the words and examples used for guiding participants on how to complete a tool. They are textual elements that refer to wording that presents and introduces a tool and suggests its uses and are also examples or use notes that provide participants inspiration on how to fill in the blank spaces. For example, a tool for engaging with young people can be improved by changing the word 'visit' to look more informal and approachable than the word 'meeting'.

(c.2) Design of material: The form depends on a relationship between the users' needs and skills and the social context in which it is designed. This relationship and context will define the form of the design. For instance, an A6-sized tool designed to gather ideas can be improved in a bigger format to support more detailed ideas or extra notes in a lengthy activity.

(c.3) (Enable) Contrary activity: This component refers to the non-deliberative action of adaptability, where participants fit existing tools into their practice in CE activities. For example, a tool could be improved with blank text boxes instead of lines. In this way, participants would not feel restricted to complete all the lines, enabling them to draw if they wish so.

The improvement of tools in this layer of practice will develop the practice of doing CE through writing, making, and enacting activities by redesigning tools that are user-friendly to participants of a project.

3.1. Framework overview

Building on these nine elements, we tested the dimensions for improving tools through co-design workshops to develop a deep understanding of how the Improvement Matrix (Table 5) works in practice. In the following section, we present three case studies, where we delivered workshops with CE practitioners from different organisations and backgrounds and developed improved tools as the practical outputs of this study.

4. Working together to improve CE tools

We developed the Improvement Matrix through a series of three co-design workshops with engagement practitioners, where each workshop consists of testing and analysing the framework in a real-world context with little control of events. A multiple-case study as a part of the action research self-reflective spiral of cycles is defined as the methodology performed in this research project that formed part of a larger research project called Leapfrog (2015-2018). Each case constitutes a PAR cycle of planning, acting, observing and reflecting (Kemmis and Robin 2005) as illustrated in Figure 1.

The co-design workshops were documented through materials produced by participants, audio records taken from the discussion at the implementation phase, photographs, and researchers' personal accounts, such as reports and reflective blog posts. The analysis of the evidence follows a bricolage of general techniques (Yin 2018) that 'play' with data in order to search for relevant patterns, insights, or concepts, and rely on the theoretical propositions (Functionality, Instruction, and Flexibility) in order to develop a case description. In this combination of techniques, various codes were assigned to evidence, and then examined, categorised, tabulated, tested, and recombined with the assistance of the researchers' memos and diagrams to draw empirical conclusions. In this paper, we consider the four design choices' categories (Lee et al. 2018) to describe each case study.

4.1. Project preconditions

The overall objective of the workshops was to explore tools that we preselected for each engagement situation and improve tools to suit participants' ways of working. As part of a larger project, we chose tools that were co-designed with practitioners who engage with their communities on a daily basis (e.g. young people, library users) to be improved and adopted in other fields, such as health care.

Each case study considers the improvement of tools to develop an understanding of each layer of CE practice: Planning, Enabling and Doing and a cross-case analysis provide an understanding of each of the dimensions of tools: Instruction, Functionality, and Flexibility. In each case study, we aimed at testing three components of the improvement matrix across each layer to improve the tools and resources practitioners use in their engagement activities as illustrated in Figure 2.

4.2. Participants

In these workshops, groups of engagement practitioners who work with tools collaborated with us through experimenting, learning and reflecting on the improvement process as co-researchers, providing evidence to test the Improvement Matrix. These practitioners included designers and non-designers (who nevertheless work in a creative manner) and aim at involving their communities in public decision-making processes or other CE activities. They were recruited from either a pool of motivated and existing contacts from the Leapfrog project or researchers who work with CE from a design research conference.

4.2.1. Case 1

We invited design research participants, who work with groups of nondesigners or are experts in tools and participatory practices, to attend the workshop to improve their CE practices. Eight DRS2018 delegates attended the workshop during the Ph.D. By Design event prior to the main conference at Limerick School of Art and Design ( Figure 3).

4.2.2. Case 2

We worked together with Children's Champions, a team from a joint health and care system called Integrated Care Communities in Northwest England ( Figure 4). The team is a group of multidisciplinary healthcare practitioners responsible for engaging with children and Young People (YP) in their local communities to get their needs and voice listened to and heard.

4.2.3. Case 3

We worked together with the staff members of the Quality Improvement Team from Lancashire Care NHS Foundation Trust ( Figure 5). The team is composed of multidisciplinary healthcare practitioners that deal with complaints at diverse levels.

4.3. Co-design events

We tested the framework with engagement practitioners through co-design workshops, using the future workshop (Kensing and Madsen 1992) and bricolage (Büscher et al. 2001) PD techniques under the conceptual idea of toolkits for innovation (Von Hippel 2001). In these workshops, we provided participants with many copies of the chosen tools, basic materials (e.g. sticky notes, sharpies, paper) and proformas ( Figure 6) to support them to create something new and evaluate their proposals. This approach enabled participants to co-design improvements directly on tools through a cycle of trial-and-error, where they (1) critiqued the present, (2) envisioned the future and (3) implemented designs through testing and evaluating the effects of their decisions. They conducted this improvement cycle three times by looking at the instruction, functionality and flexibility of tools in each round, and then reflecting on which proposals could lead to the improvement of their CE practices at the end. We planned each of these steps to last around 10-30 minutes within a half-day workshop (1.5-3 hours).

We worked as neutral facilitators supporting participants to improve tools, minimising our inputs and working as boundary spanners (Levina and Vaast 2005) who empowered participants not only to express their views but also to perform direct changes on the tools to transform them into boundary objects-in-use, i.e. tools usefully deployed in different fields. In each workshop, we preselected tools to be improved in the workshop according to the needs and context of each group of participants described as follows.

4.3.1. Case 1

We invited DRS2018 delegates to attend the workshop and learn how to improve their CE practices, which required a scenario to provide context to participants. We presented a Leapfrog project, where we collaborated with Lancashire County Council library practitioners as a result of a massive budget cut in Lancashire libraries and museums in November/2015 that led the libraries to turn into community multi-service centres. The main challenge of this project was to create a set of tools to enable the best possible transition to Neighbourhood Centres, i.e. tools that help each centre to address challenges in their own way. We conducted a series of workshops with small groups of library practitioners, where they co-designed tools to address their challenges.

In the DRS2018 workshop, we introduced to participants the same challenges our Leapfrog partners had to face to provide the context and intentions for which the tools presented at the workshop were co-designed. We preselected three of these tools ( Figure 7) and asked participants to improve them according to their CE practices. This case study presents a challenging workshop that we conducted with one facilitator in a shorter period of time compared to the other cases due to the 1.5-hour time frame allocated for pre-conference workshops. Getting settled before the start of the workshop, developing points in more detail, and providing a clearer focus at the beginning could have enhanced the impact of the workshop. Choosing tools used for a specific context helped participants to get their head around the improvement process. Participants concluded the activities on time at the expense of better outcomes.

4.3.2. Case 2

The children's champions team were looking forward to getting better assets, engagement, and including YP's voice in their bimonthly meetings. Considering their practice, we shortlisted five tools that could be used for capturing YP's voice, translating evidence, and sharing outcomes across teams and organisations. We included a simple consensus activity to enable participants to choose three out of the five preselected tools to be improved in the workshop. This activity consisted of asking each participant to choose two tools and the five most voted ones were explored and improved by the group during the workshop.

During the workshop, participants shared their expertise on engaging with children and YP, showing how different experiences and perspectives affect the improvement of tools. One participant was excited to use tools in practice as he mentioned he employs traditional methods and techniques, such as focus group and flipcharts. His group mentioned the need to use the tool first before suggesting improvements. Some participants struggled to understand the flow customer tools ( Figures 7, Figures 9), which supports the creation of personas as a designerly technique to describe service users. Whereas a simplistic tool with a lack of instructions, such as the Storyboard contract ( Figure 8), enabled participants to generate good suggestions that could lead to the improvement of their practice.

4.3.3. Case 3

The quality improvement team believed the Leapfrog tools would benefit them and were interested in creating their own tools for their organisation. Considering their practice, we shortlisted seven tools that could help them to gather feedback from their communities, to map ideas and opportunities, to enable their communities to respond to their feedback, and to communicate improvements to their communities and wider teams prior to the workshop. Similarly to case 2, we included a simple consensus activity to enable participants to choose five of them to be improved in the workshop.

In this workshop, there was a disagreement about improving tools to specialise them to a specific activity. Although participants were from the same organisation, they worked in different teams and had distinctive engagement challenges. For instance, when the team were discussing about improving the tool Role bingo ( Figure 10), one participant suggested to redesign it into a team to-do list, whereas another participant wanted to use it as a project management tool. Many agreements and disagreements occurred when improving contrary activities, which provided a better understanding on this layer of the matrix.

4.4. Results

We report two levels of results: outputs from immediate results and further implementations and outcomes as impacts of the project. Based on participants' reflection on which suggestions led to the improvement of the tools, we redesigned new versions of the tools (visual examples below) and made them available for download on the Leapfrog website as the tangible outputs for practitioners.

4.4.1. Case 1 outputs

Participants agreed upon a course of improvement for a set of tools by providing more open and flexible design concepts that give CE practitioners more control over the process, and also flexibility in using and understanding tools. Participants' suggestions focused on extending features, providing more instructions, and new ideas to address challenges in order to give more flexibility to practitioners. We implemented these improvements in the new version of a tool to provide more flexibility to practitioners as illustrated in Figure 11.

4.4.2. Case 2 outputs

Participants agreed upon a course of improvement for a set of tools by helping facilitators to design engagement approaches and providing indications of use and practical guidance to participants on how to complete tools throughout an activity in order to enhance skills needed for the job of facilitation. We implemented most of these improvements on the Storyboard contract ( Figure 8) by adding an instruction sheet as illustrated in Figure 12.

4.4.3. Case 3 outputs

Participants agreed upon a course of improvement for a set of tools by tailoring them to suit their community needs and practices, where they focus on improving the content in the tool. Their suggestions focused on improving visual and written communication through changing the words and graphic design of tools and adding flexible features and formats in order to make them more appropriate to their organisation and the communities they work with. We implemented these improvements in the new version of a tool as illustrated in Figure 13.

4.4.4. Outcomes

Participants learnt how to improve tools through the process, enabling them to creatively deploy tools with understanding in their practice as the learning effect of this project:

After the workshop, one organisation shared with us how they adopted the improved tool in a Quality Improvement conference as illustrated in Figure 14.

5. How do the design propositions (instruction, functionality, and flexibility) improved tools in practice?

We cross-compared the evidence gathered across cases and drew insights on how each dimension works in practice to improve tools.

5.1. Instruction

To improve the instruction, practitioners highlight the lack of clarity, language issues, and restrictive aspects in the instructions, and then suggest improvements on how the tool should work, indications of use, and adding, removing, or changing the features to make the communication more appropriate for an organisation and audience. The improvement of tools involves providing clear visual design and instructions, indications of use, and friendly and clearer words for their practice.

5.2. Functionality

To improve the functionality of tools, practitioners highlight the lack of clarity, inappropriate design concepts, and restrictive aspects, and then suggest improvements by adding resources or changing the type of interactions/visual design and providing more practical guidance at the introduction and guidance during an engagement activity. The improvement of tools involves providing new ideas to address a challenge, adding or removing features to expand tool applications, clear and friendly graphic design and additional guidance and instructions to enhance the engagement of participants and practitioners in an activity.

5.3. Flexibility

To improve the flexibility of tools, practitioners highlight the restrictive aspects and suggest different uses, and then propose improvements by simplifying/removing and adding/extending features, providing editable headings, formats and instructions, and designing activities as a group. The improvement of tools involves enabling wider applications through different features, providing ideas that give practitioners more flexibility in understanding and use or generating ideas together as a group, in order to build understanding on employing tools in creative activities.

6. Summary and discussion

In this paper, we proposed a framework for improving tools by looking at co-design practices and theories and applied these to create a framework to support and enhance CE activities. We tested the frame to support potential uses in a specific project as well future projects by employing three design propositions (Instruction, Functionality, and Flexibility) within three layers of the practice landscape in CE (Planning, Enabling, and Doing) through co-design workshops.

The findings in the cross-case provided insights on the improvement process and how the propositions play out in practice to improve CE tools. The insights suggest the types of participants, and tools might affect the improvement process. Selecting tools for specific context helps participants to understand the improvement process. Simple tools can enable CE practitioners to generate good suggestions that lead to the improvement of tools, whereas complex tools can hinder the tool improvement. Participants with less tool experience provide fewer improvement suggestions, but they can still contribute with good suggestions whilst learning through the process. The workshop outcomes suggest participants have developed capabilities to develop their practices through the improvement of tools.

A peer review with experts at the EAD2019 has provided some learning points that are prompting more research. If tool flexibility is considered in the design of the methodology when working with different groups, it can enhance transferability in other research contexts. This also involves including flexibility in the Improvement Matrix as a meta-tool. We also identified a challenge in the way CE professionals explored the framework, when considering the Facilitation layer. Often, it was seen as embedded in other layers or less important in some CE practices, where a facilitator is seen more as the specialist or is absent. Other applications involved using it as an empty matrix to think through a particular co-design event as a template to populate it with participants' information, and as a visual aid to create participatory tools. Further research involves exploring the Improvement Matrix with larger groups of practitioners and in other design research areas, such as education or informatics study, to see how it would work in practice, tracking changes in the improved tools and the framework over time.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by Leapfrog Project under the Arts and Humanities Research Council grant [AH/M001296/1] and by Conselho Nacional de Desenvolvimento Científico e Tecnológico (CNPq) under grant [234824/2014-0].

References

Bratteteig, T. , K. Bødker, Y. Dittrich, P. H. Mogensen, and J. Simonsen. 2013. Organising Principles and General Guidelines for Participatory Design Projects. New York: Routledge.

Büscher, M. , S. Gill, P. Mogensen, and D. Shapiro. 2001. "Landscapes of Practice: Bricolage as a Method for Situated Design." Computer Supported Cooperative Work (CSCW) 10 (1): 1-28. doi:10.1023/A:1011293210539.

Cruickshank, L. , R. Whitham, G. Rice, and H. Alter. 2017. "Designing, Adapting and Selecting Tools for Creative Engagement: A Generative Framework." Swedish Design Research Journal 15: 42-51. doi:10.3384/svid.2000-964X.17142.

Davidoff, P . 1965. "Advocacy and Pluralism in Planning." Journal of the American Institute of Planners 31 (4): 331-338. doi:10.1080/01944366508978187

De Waal, G . A., and Knott, P. 2013. "Innovation Tool Adoption and Adaptation in Small Technology-based Firms." International Journal of Innovation Management 17 (03): 1340012. doi:10.1142/S1363919613400124

Dix, A . 2007. "Designing for Appropriation." Paper Presented at the Proceedings of the 21st British HCI Group Annual Conference on People and Computers: HCI... But Not as We Know it-Volume 2, Lancaster, September 3-7.

Donetto, S. , P. Pierri, V. Tsianakas, and G. Robert. 2015. "Experience-based Co-design and Healthcare Improvement: Realizing Participatory Design in the Public Sector." The Design Journal 18 (2): 227-248. doi:10.2752/175630615X14212498964312.

Ehn, P . 2008. "Participation in design things." In Proceedings of the Tenth Anniversary Conference on Participatory Design 2008, 92-101. Bloomington, Indiana: Indiana University

Lee, J.-J. , M. Jaatinen, A. Salmi, T. Mattelmäki, R. Smeds, and M. Holopainen. 2018. "Design Choices Framework for Co-creation Projects." International Journal of Design 12 (2): 15.

Manzini, E . 2015. Design, when everybody designs : an introduction to design for social innovation. Cambridge, Massachusetts: The MIT Press

Marttila, S ., and A. Botero. 2013. "The'Openness Turn'in co-design. From usability, sociability and designability towards openness." In Co-create 2013 : the Boundary-Crossing Conference on Co-Design in Innovation, edited by Riitta Smeds and Olivier Irrmann, 99-111. Espoo, Finland: Aalto University

NESTA, National Endowment for Science Technology and the Arts. 2014. DIY - development impact & you : practical tools to trigger & support social innovation. New York: The Rockefeller Foundation

Norman, D. A. , and S. W. Draper. 1986. User Centered System Design: New Perspectives on Human-computer Interaction. Hillsdale, NJ: L. Erlbaum Associates.

Sanders, E. , and P. Stappers. 2012. Convivial Toolbox: Generative Research for the Front End of Design, Edited by P. J. Stappers. The Netherlands: Amsterdam.

Stephen, Kemmis , and Robin McTaggart. 2005. "Participatory action research: Communicative action and the public sphere." In The Sage handbook of qualitative research, 3rd edition, edited by Norman K. Denzin and Yvonna S. Lincoln, 559-604. London: SAGE Publications Ltd

Tassoul, M . 2009. Creative Facilitation. 2rd ed. ed. Delft: VSSD

Whitham, R. , L. Cruickshank, G. Coupe, L. E. Wareing, and P. David. 2019. "Health and Wellbeing: Challenging Co-Design for Difficult Conversations, Successes and Failures of the Leapfrog Approach." The Design Journal 22 (sup1): 575-587. doi:10.1080/14606925.2019.1595439.

Wilson, S. , A. Roper, J. Marshall, J. Galliers, N. Devane, T. Booth, and C. Woolf. 2015. "Codesign for People with Aphasia through Tangible Design Languages." CoDesign 11 (1): 1-14. doi:10.1080/15710882.2014.997744.

Yin, R. K . 2018. Case study research and applications : design and methods. Sixth edition. ed. Thousand Oaks, California ; London: SAGE Publications

Zamenopoulos, T. , and K. Alexiou. 2018. Co-design as Collaborative Research, Connected Communities Foundation Series. Bristol: Bristol University/AHRC Connected Communities Programme.

# data[].attributes.attachment_link.title
Making it better together: a framework for improving creative engagement tools

# data[].attributes.attachment_link.description
ABSTRACT ABSTRACT This research reports on a co-design project to improve creative engagement tools with academics and public sector organisations in the northwest UK. Creative engagement (which is a staple of co-design activities but also used widely outside design) is often supported by tools and resources.

# data[].attributes.headline
Simple tools for bringing people together, exchanging ideas and making stronger, more active communities

# data[].attributes.subject
Simple tools for bringing people together, exchanging ideas and making stronger, more active communities

# data[].attributes.body
Leapfrog | Simple tools for bringing people together, exchanging ideas and making stronger, more active communities

# data[].attributes.attachment_link.title
Leapfrog | Simple tools for bringing people together, exchanging ideas and making stronger, more active communities

# data[].attributes.attachment_link.description
Tools to help put impact on audiences at the heart of everyday research and development

# data[].attributes.headline
Consultation vs Co-production: Why is it so vital to understand the difference?

# data[].attributes.subject
Consultation vs Co-production: Why is it so vital to understand the difference?

# data[].attributes.attachment_link.title
Consultation vs Co-production: Why is it so vital to understand the difference?

# data[].attributes.attachment_link.description
Our future is in their hands, and we will hold them to account. An article from The Guardian this past weekend revealed details of the poor and frankly risky and incomplete approach the government is taking to remove barriers for people with disabilities.

# data[].attributes.headline
Co-design in the public sector: Top tips

# data[].attributes.subject
Co-design in the public sector: Top tips

# data[].attributes.body
From user-centred design to co-design - Services in government (blog.gov.uk)

# data[].attributes.attachment_link.readable_text
Hi, I'm Janet. I used to work at the Government Digital Service (GDS) and have also worked on digital and other types of delivery in various other bits of government, including NHS Digital and the Department for Education (DfE).

In my current role, I'm Programme Director for the Future Farming and Countryside Programme at the UK Department for the Environment, Food and Rural Affairs (DEFRA). We're here to help farmers and land managers make farming and the countryside more sustainable and productive. You can read about our programme over on our blog .

This post is about how we're using a co-design approach to design our policies and services, how that's different to the user-centred approach set out in the Government Service Standard , and why that difference matters.

Start with user needs; improve digital services

The first Government Design Principle , which started life in GDS, is: start with user needs . Understanding your users and their needs is the basis of the Service Standard, and the UK government's approach to designing and delivering digital services.

This approach helps us to understand what our users need from a particular service or transaction, and make sure that we design services that meet those needs.

It helps us make forms work better for people. It helps us to make transactions simpler, clearer and faster. It can save us and our users a lot of time and money.

However, this approach has some important limitations when we look at things beyond relatively straightforward transactions:

it doesn't help us understand the relationship between policy intent and user needs - you need to understand both in order to achieve real-world outcomes

it doesn't help us with things that aren't services such as laws, policy decisions, stewarding big complex systems like health or education, or setting and enforcing rules

it can lead us to miss some of the complexity of people's lives and the context they're operating in, because we tend to focus quite heavily on individual transactions and services rather than the wider picture

Take a user-centred approach to everything for users

When I worked in DfE, we wanted to apply user-centred ways of working to a wider range of things the department did. We also wanted to join up policy intent and user needs to achieve specific real-world outcomes, like recruiting more teachers or attracting more people into apprenticeships.

We wanted to bring together policy and delivery and make the whole department fit for the digital age , not just have a digital team as a siloed set of specialists working in very different ways to everyone else.

We wanted to do things differently, not just make things more efficient.

This required us to think about how to apply the principles of user-centred design and delivery to a broader set of things, and how to help everyone in the department develop the skills, mindset and understanding they would need to do that. That means digital skills, tools, mindset and ways of working for all public servants, not just digital teams.

We did this because we reasoned that if we applied user-centred practices more widely in the organisation we'd have:

fewer bad ideas making it into delivery

more things working better for people

better policy outcomes

less failure and waste

We developed 6 user-centred practices for all civil servants, regardless of their particular profession or area of work:

This approach helped us link together policy outcomes with user needs and apply user-centred practices to a wider range of things ‒ 2 of the limitations of only using these approaches within digital teams. But it still didn't give us ways to understand and respond to complexity.

Co-design things with users

In the Future Farming and Countryside Programme, we've made a commitment to co-design our policies and services with users, not just design things for them.

That means that we work actively with farmers, land managers and other users, all the time. We identify and work through problems together through meaningful, ongoing discussions, rather than solely relying on formal consultations about policies and observational user research relating to specific transactions.

We respect, value and understand our users' lived experiences and insights, rather than focusing only on their specific needs relating to an individual transaction or service.

For example, we've been working with farmers, inspectors, land managers, environmental groups to understand what's wrong with the way environmental and other farming rules are set and enforced, and how we might improve things. We're now going to run some experiments to try out different ways of operating to achieve better outcomes in ways that work better for our users.

We're also running 70 tests and trials , where farmers and land managers are leading projects to try out ideas and ways of operating to help inform the design of our new policies and services. There are about 3,000 farmers involved in these tests and trials.

This co-design approach is helping us to try out a range of different ideas, in partnership with our users, all at the same time and at a small scale so that we can learn relatively quickly and efficiently and apply that learning to the design of our services as we roll them out more widely.

A few lessons

There has been plenty written about how to do effective co-design - I won't repeat that all here. Some of the main lessons for us so far have included:

be honest and direct: set clear constraints, make it clear what's up for discussion and what's already decided, don't waste people's time with things that look like co-design if you aren't really going to act on what you learn

build trust: people aren't used to government officials working in a genuine co-design way, you will need to build trust over time by demonstrating that the co-design work is for real

this stuff is hard because it's hard, and you won't always get it right - learn and adapt your approach as you go

co-design is not about pleasing everyone all the time, it's about respecting and taking everyone's views and experiences into proper consideration - you will still have to make some hard decisions that not everyone will like or agree with

Right approach, right context

I'm not advocating a single approach for all circumstances here or saying that co-design is always the best way. The approach you take will depend on the particular context you're working in and what you're trying to achieve. But I do want to suggest that we could probably take a co-design approach with more things, more of the time, and that if we did we might get much better results.

If you'd like to follow our work, to our programme blog, "Future Farming" .

# data[].attributes.attachment_link.authors[].name
Janet Hughes - Programme Director for the Future Farming and Countryside Programme, DEFRA

# data[].attributes.attachment_link.description
Hi, I'm Janet. I used to work at the Government Digital Service (GDS) and have also worked on digital and other types of delivery in various other bits of government, including NHS Digital and the Department for Education (DfE).

# data[].attributes.headline
The incompatibility of Nudge and Co-Design as tools for policymaking

# data[].attributes.subject
The incompatibility of Nudge and Co-Design as tools for policymaking

# data[].attributes.attachment_link.readable_text
The use of nudge theory to inform policy interventions in response to COVID-19 has re-opened debates over the politically paternalistic nature of governing by 'nudges' and has given momentum to calls to include the more participatory elements of co-design into policymaking. Emma Blomkamp and Colette Einfeld suggest in seeking to combine mutually exclusive elements of each practice, academics and policymakers risk blurring lines of authority and public trust in policymaking.

The COVID-19 pandemic has by necessity renewed interest in innovative approaches to policymaking. Notably in two areas, nudge and co-design. Nudges were used to encourage social distancing in India and elsewhere. Co-design has been promoted as a way to improve public trust in policy responses to the pandemic in the UK, Canada and Australia. Elements of each approach were also brought together to promote handwashing in Bangladesh. Increasingly, academics and practitioners are calling for nudge and co-design to be used together.

Both nudge and co-design promise more effective outcomes for governments by taking into account the public's 'real' behaviour, to overcome biases in traditional top-down approaches to policymaking. However, as we explore in our recent paper, both approaches stem from different understandings of society and politics. This raises questions over whether in practice these approaches are compatible and complementary in policy design, or contradictory to the extent they could undermine trust in government.

Both nudge and co-design promise more effective outcomes for governments by taking into account the public's 'real' behaviour, to overcome biases in traditional top-down approaches to policymaking.

Nudge is an approach to public policy that changes the context in which decisions are presented to citizens in order to encourage a particular choice. To illustrate, a nudge might change the presentation of information in letters sent to dangerous drivers, aiming to increase payment of the fine and reduce reoffending. This nudge would likely then be tested using a Randomised Control Trial.

Co-design uses creative and participatory methods to engage citizens, stakeholders and officials in an iterative process to respond to shared problems. If we again use the example of road safety, a co-design approach might start by reviewing literature and data, then use this knowledge to inform a range of design, research and decision-making activities with a co-design group. The co-design group could bring together community representatives, drivers targeted by the policy, government stakeholders and experts. Options for policy implementation would be generated, tested and reshaped in an iterative process with the co-design group.

Nudge and co-design have been used in the areas of obesity, tax compliance and road safety. Our examples above are based on instances of co-design in New Zealand and nudge in the UK. Both approaches use experimentation, are seen to reflect real world behaviour, and address wicked problems.

A recent academic article claims it is possible and desirable to "co-design behavioural public policy". The Nudge Unit at an American University and a New Zeal#and health authority claim to have co-designed nudges in healthcare. The Behavioural Insights Team in the UK are also co-designing nudges, as part of their recent work encouraging STEM careers for women.

Whether these examples strictly adhere to nudge and co-design principles is debatable. Despite calling its approach a 'design lab', there was no evidence of designerly practice in the recently published case study of 'co-designing nudges', which was led by an academic research and policy team. An American Medical Association article on designing nudges refers to "stakeholder co-design", although the top-down communication process it describes is far from what is normally considered as co-design.

The appeal of co-designing nudges stems from criticisms of paternalism in nudge theory. Nudges are arguably inherently paternalistic, since experts are making decisions about which choice will make the individual better off. Governments then guide people to the decision without public consultation. For advocates, co-design offers the potential to reintroduce different perspectives and an element of shared power into the policymaking process. This enthusiasm for combining these approaches makes it a good time to encourage reflection on whether they can be used together.

We argue that the underlying logic and philosophy of co-design and nudge are contradictory. Informed by the radical roots of participatory design, co-design techniques aim to deliberately flatten hierarchies, include more diverse forms of knowledge and experience in the policymaking process, and increase the agency of the people most affected by a policy. Nudge, on the other hand, promotes a different model of state-citizen relations. Advocates of the approach argue that people do not always make good decisions, and so it is government's role to "steer" them towards the desired behaviour. Nudge approaches privilege quantitative and scientific evidence, whereas co-design also embraces lived experience and creative expression.

To effectively combine nudge and co-design, a policymaker needs to simultaneously see themselves as an expert who has the authority to guide how others should behave, as well as not impose their views on others, seeing people as the experts in their own lives. This muddling of authority and expertise could potentially add to confusion on the role of government and exacerbate declining trust in experts and institutions.

To effectively combine nudge and co-design, a policymaker needs to simultaneously see themselves as an expert who has the authority to guide how others should behave, as well as not impose their views on others, seeing people as the experts in their own lives.

Let us take the example of road safety again, and imagine trying to "co-design a nudge", illustrating some of the questions and contradictions to consider. In the process, a policymaker may draw on academic evidence on how to reduce speeding, and take this to a co-design group, presenting them with potential nudges to explore. The statistical, population-level evidence on the causes of road safety brought to the co-design group may conflict with the diverse lived experiences of group members. The policymaker then has to decide if they are the expert in selecting evidence, or if people are the experts in their experiences. They would also need to decide whether to test one selected intervention through an RCT or adapt numerous lower fidelity prototypes through an iterative user testing and development process.

Further, the co-design group may contest the problem framing, reframe it as a structural rather than behavioural issue, and/or suggest an alternative response, such as community-led initiatives, rather than a nudge aiming to steer people towards a particular behaviour. Does the co-design group have the status and space to challenge, not only the solution and whether nudges are the most appropriate intervention, but also the policymakers' understanding of the problem? If nudge is the predetermined policy tool, but co-designers expect to be able to shape the output, might this undermine the process and trust in the government?

This example demonstrates that, while nudge and co-design have similarities, when applied together, their underlying assumptions and philosophies can cause conflict and contradictions. We hope that, by critically reflecting on the differences and similarities in approaches such as co-design and nudge, researchers and practitioners can use and evaluate policymaking methods and tools with a greater awareness of their implications for policy effectiveness and political trust.

# data[].attributes.attachment_link.title
The incompatibility of Nudge and Co-Design as tools for policymaking

# data[].attributes.attachment_link.description
The use of nudge theory to inform policy interventions in response to COVID-19 has re-opened debates over the politically paternalistic nature of governing by 'nudges' and has given momentum to calls to include the more participatory elements of co-design into policymaking.

# data[].attributes.attachment_link.readable_text
Unfortunately with the increasing use and profile of the word "co-production", there is also increasing misuse. To understand what co-production is, it is important to understand what it is not.

Co-production builds upon a range of similar approaches such as consultation, engagement and co-design. Clearly showing the difference between co-production and these related, but different, approaches is important if real co-production is to be put into practice.

The ladder of co-production is a useful way of making these important distinctions.

*If you wish to re-produce or reference the ladder of co-production in publications, kindly notify us at info@tlap.org.uk We ask that you reference the tlap website as the source of the material with a hyperlink to this webpage.

Consultation, engagement and co-design encourage people to input by asking for their ideas, experience and opinions (See Developing a new day care service). Co-production is different because it also needs people's actions. This can happen through 1-2-1 relationships with professionals where people play an active role in shaping and implementing their own support, or in wider peer or community support between people and professionals. Co-production means that power is shared more equally between those who use services and those who provide them. Everyone's skills and personal resources are put to use.

Quite often words describing co-production are mixed up with words describing co-design. Co-design and other ways of involving people using services in the design of services mean that the planning of services is done jointly, but this doesn't always lead to involving people using services, families and communities in the delivery of the service - i.e. actually making it happen. The following table by Tony Bovaird which is cited in Boyle and Harris helps to explain this (opens new window) :

# data[].attributes.attachment_link.description
Unfortunately with the increasing use and profile of the word "co-production", there is also increasing misuse. To understand what co-production is, it is important to understand what it is not. Co-production builds upon a range of similar approaches such as consultation, engagement and co-design.

# data[].attributes.body
Our-Place-guide-to-Co-design-FINAL.pdf (traverse.ltd)

# data[].attributes.headline
REBLE - Research Enabled by Lived Experience

# data[].attributes.subject
REBLE - Research Enabled by Lived Experience

# data[].attributes.attachment_link.readable_text
We wanted to embed inclusivity, equity, justice, diversity, and co-production meaningfully and authentically in our work.

In February 2021, we funded a pilot lived experience advisory panel to inform and shape our inclusive research and engagement practice. Once established, the panel chose to change its name to REBLE.

The REBLE researchers are ten people who bring different life experiences, expertise, and perspectives on public services.

They work with us to make sure our assumptions and interpretations of research and engagement are grounded in the experiences of those accessing services. They sense check our ideas and advise us on design.
About the REBLE researchers
The REBLE researchers are ten people who are passionate about sharing their lived experience, expertise, and perspectives to improve inclusive practice and co-production in research and engagement.

Coming from a range of different ethnic backgrounds and ages, some members have physical or learning disabilities, or long-term health conditions which affect their everyday lives.
Their expertise
Each member brings unique expertise on a range of topics, from mental health, disability and being a carer to anti-racism, sexuality, and discrimination.

Their expertise also covers experience in campaigning and research, as well as being part of lived experience groups.

"Members of the Traverse team joined us during one of our REBLE meetings. They too shared their life experiences with us. I remember typing in the chat that this was a day I would remember for years to come. It was extraordinary, I felt a wave of acceptance and belonging. I felt safe with these people, just as I had felt in previous months in our own REBLE meetings.

"It can feel confusing to me at times to think and talk about being disempowered. It is experienced as a feeling, a feeling of injustice that hurts my chest and makes my cheeks flame. Becoming a REBLE researcher with its focus on spending valuable time at a patient pace with each other is the answer to a promise that I silently made to myself - perhaps others did too, in terrifying, lonely dark times. The promise is that there will be a time in the future where you will have a voice, that the things you write and speak about will be listened to on a deeper level and that there is a better way." Steph - REBLE researcher

REBLE with a cause

On 25 November 2021, we hosted an event called REBLE with a cause to share our learning to date about setting up a lived experience in research panel.

Over 80 people joined us at the event, which was co-run by the REBLE researchers, who shared their own reflections and learnings alongside colleagues at Traverse.

Through a powerful interactive forum theatre approach, we explored inclusive practice - and how research can benefit from lived experience, co-production and power sharing. The forum theatre performance drew directly on the lived experience of REBLE researchers and their participation in research.

In this report, we share graphic recordings from the event which tell the story of the forum theatre performance and interactions. We also share reflections on the event from a range of perspectives and hopes for the future.

Listen to the event below.

# data[].attributes.attachment_link.description
We wanted to embed inclusivity, equity, justice, diversity, and co-production meaningfully and authentically in our work. In February 2021, we funded a pilot lived experience advisory panel to inform and shape our inclusive research and engagement practice. Once established, the panel chose to change its name to REBLE.

# data[].attributes.attachment_link.provider_name
The Australian Centre for Social Innovation

# data[].attributes.attachment_link.title
Unpacking co-design - The Australian Centre for Social Innovation

# data[].attributes.attachment_link.description
In Australia today 'co-design' has become almost interchangeable with 'consultation'. However ,the concept of co-design and the range of design based traditions that sit under the co-design banner have something very important to bring to place-based social innovation. Unfortunately, very few people have got to experience the true benefit of design-based approaches because co-design is rarely practiced authentically.

# data[].attributes.headline
Co-design with Citizens and Stakeholders

# data[].attributes.subject
Co-design with Citizens and Stakeholders

# data[].attributes.body
Co-design with Citizens and Stakeholders.pdf (governanceinstitute.edu.au)

# data[].attributes.body
Unpacking co-design - The Australian Centre for Social Innovation (tacsi.org.au)

# data[].attributes.headline
Systemic design practice for participatory policymaking

# data[].attributes.subject
Systemic design practice for participatory policymaking

# data[].attributes.body
Full article: Systemic design practice for participatory policymaking (tandfonline.com)

# data[].attributes.attachment_link.readable_text
Abstract

Abstract
As the complexity of policy problems is increasingly recognized, and participatory approaches gain popularity, policy workers are applying different methods to engage a wide range of stakeholders and citizens in policy development and implementation. Alongside burgeoning interest in various forms of design and systems thinking, systemic design has emerged as a descriptor for a practice that integrates dialogue, design and co-creation for sensemaking and decision-making. As an approach to participatory policymaking, systemic design involves creating the conditions for stakeholders to more meaningfully participate in building shared knowledge and taking collective action. This article puts forth a new practice framework for systemic design in public policy and social innovation. It distills insights from the author's experience and knowledge as a researcher, evaluator, practitioner and educator in the design and delivery of public policy and human services. The five core domains of the practice framework-principles, place, people, process and practice-are based on established understandings of design-led, systems-informed and participatory approaches to policymaking, as well as knowledge from critical practice reflections, recent research and evaluation reports. The relevance of the practice framework is illustrated through a case study of a design-led approach to a community services policy in New Zealand. Examples from the case study demonstrate some of the benefits and challenges of systemic innovation and participatory policy design.

Introduction

Government departments and policy workers around the world are adopting the processes, principles and tools of systems thinking, human-centred design and co-design. These approaches reflect the participatory turn in public policy and human services, recently described as a "new paradigm" of public administration (Torfing, Sørensen, and Røiseland 2019) and a "zeitgeist" in the context of health systems (Palmer et al. 2019). The spread of public sector innovation labs both reflects and contributes to the popularity of "designerly" methods, which emphasize empathy and creativity alongside abductive forms of reasoning (McGann, Wells, and Blomkamp 2021, 2). A recognition of contemporary planetary crises and complexity, notably the interconnected nature of "wicked problems" (Buchanan 1992), also sees systems thinking brought to bear on policy issues. This is exemplified in transition design, which extends the skillsets and methods of service design and social innovation to a longer-term horizon and transdisciplinary practice that advocates societal transition toward more sustainable futures (Irwin 2015).

There is great variance in the terms and techniques being used, causing confusion among scholars and practitioners alike (Blomkamp 2018; Jones 2014; Haynes et al. 2020). Over the past decade, systemic design has emerged as a useful descriptor for a practice that integrates design thinking and systems practice. While the term is not widely used or understood, systemic design can be defined as "a design-led practice that integrates dialogue in co-creation for sensemaking and decision making" (Jones 2018, 45). It may also be called systems (oriented) design, systems-led design, systems-aware co-design, design for complexity or system(s/ic) innovation.

The participatory approach of systems and design practices holds much promise for policymaking, yet there are significant challenges to embedding these methods in public sector organizations. The logic and application of collaborative and design-led modes of policymaking challenge traditional governmental relationships based on "power, control and expertise" (Durose and Richardson 2016, 35). They represent a departure from the classical view of the public sector as "the sole provider of public goods" (Torfing, Sørensen, and Røiseland 2019) as well as from siloed performance-based systems with narrow targets based on New Public Management (Tonurist et al. 2020). Research and evaluation specifically on (human-centred) design for policymaking (Bason 2014, 2017; Kimbell and Bailey 2017; Lewis, McGann, and Blomkamp 2020) and systems thinking for policy (Ryan and Leung 2014; Bijl-Brouwer and Malcolm 2016; Haynes et al. 2020) is beginning to emerge. In-depth case studies of design-led policy practice are rare but much in demand.

This article offers both a case study and practice framework to bridge the gaps in the theory and practice of systemic design and contemporary participatory policymaking. It aims to provide useful knowledge, especially to support practitioners interested in the application, limitations and potential of creative and participatory methods to improve public policy. Jones's ( 2018) framework for co-creation practice in systemic design is informed by practical case studies and contains relevant considerations for, and observations of, problems in systemic design practice. Yet it remains scientific in orientation and practitioners may find it difficult to understand and apply. By practitioners, I mean people responsible for developing public policy, including public servants, elected decision makers, and contractors or consultants. This article will help bridge the significant "gap between the theoretical knowledge embodied in systems thinking, the historical knowledge of how actual [policy] systems have changed in the past and the practical knowledge needed to make systems change happen in the real world" (Leadbeater and Winhall 2020, 47).

The "5 Ps" practice framework for systemic design in public administration and social innovation offers an accessible, flexible, comprehensive and contemporary approach. Recognizing that systemic design is an emerging, pluralistic practice, it would be inappropriate to propose a fixed methodology (Sevaldson and Jones 2019; Bijl-Brouwer and Malcolm 2020). Instead, the framework offers broad guidelines to accommodate transdisciplinary practice in diverse contexts. Inspired by practice frameworks in social work and human services, it "integrates empirical research, practice theories, ethical principles and experiential knowledge in a compact and convenient format that helps practitioners to use the knowledge and principles to inform their everyday work" (Connolly and Healy 2009). The aim is to build common understanding and improve the capability of professionals and organizations to implement systemic design practice and effectively cooperate and collaborate in complex engagement situations (Jones 2018). The five core domains of this practice framework-principles, place, process, people and practice-are based on established understandings of design-led, systems-informed and participatory approaches to policymaking, as well as knowledge from critical practice reflections and previously unpublished research and evaluation reports.

Methodology: a practice-based approach to participatory policy design

This research approach and outputs are similar to those of Bijl-Brouwer and Malcolm ( 2020) who present principles derived from case studies of systemic design in public and social innovation. As well as initially focusing on design-led practice, our principles are similar, except Bijl-Brouwer and Malcolm ( 2020) draw more extensively on living systems literature, while power is explicitly considered in the "5 Ps" framework. This article presents a more comprehensive framework to understand systemic design in practice in the public sector, emphasizing context and the roles and capabilities of key individuals.

The relevance of the practice framework is illustrated through a case study of community services policymaking in New Zealand, which demonstrates some of the benefits and challenges of participatory policy design. Examples from the case study suggest a growing appetite for the principles and practice of systemic design within this organization. It offers a clear and comprehensive example of a design-led policy process, with the outputs generated since adopted as official public policy, as well as an increasing appreciation of "designerly" ways of working in government. It is a unique example of design-led practice within the social policy branch of a large government organization, rather than within a public innovation lab, which has been the focus of much recent literature on design-for-policy (Lewis, McGann, and Blomkamp 2020).

The community services policy project was led by a government organization in New Zealand from 2016 to 2018 to address its prior lack of consistency in providing and supporting community facilities. The project resulted in a policy framework designed to guide strategic and operational decision-making and improve organizational practice, notably in terms of community relationships and facility management. Although the policy project was not explicitly described as "systemic design," like the case studies of Bijl-Brouwer and Malcolm ( 2020, 403), a key feature was its experimental design-led methodology. The project team trialed methods not previously used in their organization for designing policy in collaboration with internal and external stakeholders. Examples from the case study effectively represent systemic design as a practice that integrates dialogue, design and co-creation for sensemaking and decision-making (Jones 2018, 45).

The case study formed part of academic research on efforts to build and spread human-centred design capability among public sector workers in Australia and New Zealand. Between September 2017 and June 2018, the author collected documents and conducted interviews (in person and by phone), and spent ten days on site at the government organization. I conducted a total of 17 semi-structured interviews with all team members described in the "People" section below. Each interview took between 20 and 90 minutes, was digitally recorded, then transcribed in full. The 13 individuals interviewed included the Principal Policy Analyst (lead policy designer-several interviews), two senior managers (project sponsor and authoriser) project team members from different departments and external consultants (design coach and communication designer). I also observed two meetings involving the Principal Policy Analyst and external service providers. Participants gave consent on the condition that the individuals and organizations involved would be deidentified.

The documents, interview transcripts and observation notes were analyzed thematically to capture detailed information about policy-related practice (Yanow 1996). This abductive approach recognizes the political and institutional contexts in which policy is made and the multiple forms of knowledge and practice needed to work effectively in this field. The analysis of the case study and creation of the practice framework were informed by my experience and knowledge as a researcher, evaluator, practitioner and educator in the design and delivery of public policy and human services. This includes having led projects for public purpose organizations as a social innovation and strategic design consultant in New Zealand and Australia. Alongside this practical experience, as a research fellow at The University of Melbourne, I have conducted academic research on design-for-policy and public sector innovation, building on earlier research in cultural policy, local governance, community wellbeing and outcome evaluation. This experience of connecting different forms of policy-relevant knowledge (Tenbensel 2006) has enabled me to develop a practice framework for systemic design that bridges theoretical knowledge, practice-based evidence and ethical principles.

The "5 Ps" framework incorporates systems thinking, human-centred design and participatory design theories and practices to articulate what needs to be taken into account when tackling complex problems and designing for social change. This practice framework explicitly brings a systems lens to build on previous work, notably recognizing the importance of people and place in addition to the key features of co-design as principles, process and practical tools (Blomkamp 2018). Early sharing and testing of the "5 Ps" framework suggests it could be a useful point of reference for planning and commissioning systemic design work, teaching and practice reflection, as well as evaluating process and impact. The framework has been iterated and represented as the Systemic Design Practice Wheel (see Figure 1) following application and feedback by workshop participants with varying experience in systemic design (Blomkamp 2020a). RSD9 workshop participants considered it most useful as a structured planning or reflective tool for intermediate and experienced practitioners, rather than as a teaching framework for introducing people to systemic design.

A systemic design practice framework for policymaking

The Systemic Design Practice Wheel has five core domains: principles, place, process, people and practice (see Figure 1). As a framework for reflection or planning, key questions for practitioners to consider in relation to each domain are:

Principles: Why and how does this work need to happen? What matters most?

Place: Where does this work sit and fit? Which level are we working at?

People: Who needs to be involved? What resources and support do they need?

Process: How will we structure and organize our approach?

Practice: What specific methods, techniques and tools will enable us to reach our objectives and follow our principles?

Each of these domains is introduced below, before further details are provided from the case study. The above questions and related guidance can be tailored for different stages of the work-invitation, preparation, navigation and completion-as well as the maturity of the practitioner.

Principles

The principles are the values and core guidelines that underpin and characterize a systemic design approach. A synthesis of learnings from practice, they can be applied in similar contexts to improve the likelihood of success in future design practice (Bijl-Brouwer and Malcolm 2020). The principles presented below are based on a number of sources, in particular practice-informed principles of co-design (Blomkamp 2018) and systems change (Lankelly Chase 2020; Abercrombie, Harries, and Wharton 2015), analysis of systemic policy design case studies (e.g. Ryan and Leung 2014) and Jones's ( 2014) systemic design principles, which "were drawn from the generalization of systems principles applicable to design, and design principles developed as guidelines from systems theory." As Jones ( 2014, 2018) explains, the principles of systemic design "provide theoretically-sound guidance ... for practitioners to enhance engagement and evolve better practices," notably in the context of facilitation at "higher levels of complexity". While my abductive approach to identifying specific principles makes it likely they will apply in various contexts of public policymaking and social innovation, they could also be exchanged or adapted for other sets of principles to suit the place and process of a particular project.

The six core principles of the 5 Ps framework are each presented below, then later described in the context of the community services policy project.

1. Purpose-driven. Like other forms of design and innovation, systemic design aims to produce a particular result or change the future. This is akin to the idealization and purpose finding principles (Jones 2014) and the outcomes focus in co-design (Blomkamp 2018). By identifying an ideal state or future vision, the participants in systemic design articulate a common goal and are motivated to act together to achieve a desirable outcome. As Leadbeater and Winhall ( 2020, 32) put it, "The purpose should provide the point around which people, activities and resources are organized."

2. Recognizing complexity. Systemic design is founded on the recognition of complex adaptive systems, which have features such as self-organization, feedback loops and emergence. The mindset and behavior informed by this principle is: "People view themselves as part of an interconnected whole... Everyone wants the system as a whole to work, and knows they cannot control it." (Lankelly Chase 2020). This means acknowledging the dynamism and interconnectedness of activities, relationships and problems with multiple definitions and causes (Jones 2014). Complexity requires us to work together to build understanding of the system(s) we are seeking to influence and to remain open to reframing problems and boundaries as new understandings emerge.

3. Self-determination. Taking a rights-based or strengths-based approach is a value of ethical systemic design practitioners. Enshrined in the International Covenant on Civil and Political Rights, and particularly relevant for First Nations, the right to self-determination is essentially the right of people to determine their own destiny. Applied to individuals in co-design, this is often expressed as, "people are the experts in their own lives" or "experts of their experience" (Sanders and Stappers 2014, 30). In systems thinking, self-determination is connected to the self-organizing principle and social systems theories that see all stakeholders as designers of the system (Jones 2014). This aligns with the demands of dialogic design, that we protect the autonomy and authenticity of each participant (Jones 2018).

4. Equalizing power. Systemic design aims to reconcile ever-present power relations between individuals and organizations by eliciting and incorporating a variety of perspectives (Jones 2018). As Banathy ( 1996, 2) puts it, systemic design can help to achieve "truly participative democracy" by enabling people "to take part in decisions affecting our lives and guide the activities that can enrich the quality of our lives and add value to the systems in which we live." There are two main dimensions to this principle. Firstly, following the democratic tradition of participatory design, equality of voice is actively promoted through creating the conditions for genuine participation. This entails a shift in the role of the lead designer or policy practitioner from "expert" to "enabler" (Ryan and Leung 2014), and "finding ways to give voice to those who may be invisible or weaker in organizational or community power structures" (Luck 2018). Secondly, in recognition of systems behaviors and self-determination, leadership is distributed and decision-making devolved as much as possible (Lankelly Chase 2020; Leadbeater and Winhall 2020; Yunkaporta 2019; Abercrombie, Harries, and Wharton 2015).

5. Inclusive collaboration. Systemic design engages diverse worldviews, leverages the expertise of each participant, and enables stakeholder collaboration to create implementable plans and ethical action (Ryan and Leung 2014; Jones 2018). This reflects the foundational cybernetic principle of "requisite variety," which can be applied in systemic design to achieve the optimal selection of diverse stakeholders to participate in dialogue (Jones 2014). In plain language this can be expressed as "getting the whole system in the room" (Jones 2014) or "engage multiple actors" and "do it together" (Abercrombie, Harries, and Wharton 2015). The selection of a variety of participants and strategies for their involvement should take into account the different kinds of knowledge they hold, including lived, professional and specialist expertise (Blomkamp 2018).

6. Adaptive learning. Systemic design requires the creation of space, mindsets and mechanisms for collaborative problem-solving and mutual learning. Continuous feedback and a flexible accountability structure are needed to enable ongoing adaptation throughout the design process, from the early phases of problem framing through development to implementation (Jones 2014; van Buuren et al. 2020). Feedback loops are a natural part of systems, and iteration is integral to design practice, but feedback processes may need to be carefully designed and coordinated in order to effectively learn from and adapt to actions and responses in the system.

Place

Acknowledging the context, constraints and connections of a project or issue helps practitioners and participants to contribute meaningfully. This domain can be understood as "time and place," which "are usually the same word in Aboriginal languages" (Yunkaporta 2019, 66). A systems lens encourages us to consider when and where the work fits in relation to other activities, initiatives, organizations and networks, as well as within broader, interrelated human and ecological systems.

Articulating the scope clearly helps all involved to understand the constraints and boundaries of the work. The pitfalls of drawing a system boundary either too narrowly or too widely are noted by Leadbeater and Winhall ( 2020, 18), who explain that "a first step is to draw the boundaries of the system in a way which makes system change a viable activity." In a public sector context, particular attention must be paid to the authorizing environment and enabling conditions. As Buuren and coauthors (2020, 15) articulate, with regard to design-for-policy, "Politics, but also legal principles, the need for accountability and public bureaucracies, not only set specific conditions on processes of co-creation and co-design but also limit the room for creativity."

This dimension of the practice framework could be called "placement" (Jones 2014), in the sense used by Buchanan ( 1992) in his seminal depiction of four "orders" of design as:

1. Symbolic and visual communications

2. Artifacts and material objects

3. Activities and organized services

4. Complex systems and environments

Whereas conventional design practice focuses on communication (first order), products (second order) or services (third order) in a market context, the increasing demand for systems-informed design occurs within the contexts of large organizations, industry consortia and healthcare systems (Jones 2018). Another way to consider context and scale is with regard to working across the three levels of system change-micro (niche), meso (regime) and macro (landscape)-as articulated by Frank Geels ( 2005). Each placement, level or order of systemic design requires a different strategy along with "skill and coordination of distinct methods, design practices, collaboration skills, and stakeholder participation" (Jones 2014). These related aspects are explored within the domains of people and practice.

People

The quality of systemic design depends on various factors related to the people involved. The individuals and roles involved may vary throughout any systemic design approach, depending on the place and process phase (Jones 2018). There are nonetheless some key roles needed for systemic design to be impactful: "entrepreneurs" (and creative designers); "insider-outsiders" (sometimes called social intrapreneurs); "convenors" (leaders and facilitators); and "commissioners (and enablers)"; not to mention the "supporting cast" of participants, key stakeholders and predecessors (Leadbeater and Winhall 2020, 40-41). The people domain draws attention to the personal commitment and wellbeing of systemic design practitioners; their roles, strengths, mental models, biases and positionality; and the involvement of and connections between other participants. This domain is all about power, care (for self, team, organization and community) and relationships. The principles of inclusive collaboration, self-determination and equalizing power are particularly important here-as illustrated in the case study analysis below.

Process

Systemic design practice typically follows an innovation process. This could be based on a design thinking model, such as the "double diamond" (Design Council 2007), systems change framework, branded methods such as MG Taylor, another innovation process framework and/or models of project management such as Lean or Agile. The concern for context, distributed leadership and adaptive learning in systemic design means there is no universal process that can be applied to every situation. Rather than a stage-based linear process, it can be more helpful to think of overlapping "spaces" of design, which Bason ( 2017, 80-83) names, "exploring the problem space," "generating alternative scenarios" and "enacting new practices". We adapted this model in a recent policy design project, as shown in Figure 2, with simpler names for the key spaces, summarized as: "understanding," "imagining" and "making" (Blomkamp 2020b). Whichever process model or schema is used, it will offer an overarching organizing strategy with different spaces, phases or stages to navigate, which will also connect with methods for coordinating, documenting and communicating action.

A design-led approach differs from the dominant model of "the policy cycle," also known as the rational-comprehensive model (Bridgman and Davis 1998). In a conventional approach to policymaking, problem identification is conducted through desk research and data analysis. A design-led process focuses on the lived experience of the people involved or affected by an issue, to understand the problem and its context. Taking a systemic approach often means going beyond the initial brief to explore the root causes of an issue, as well as consider different perspectives and the interrelatedness of problems (Bijl-Brouwer and Malcolm 2020, 394), so that the policy developed can effectively intervene in the system. Both design- and systems-led approaches often result in a reframing of the problem before potential options or solutions are explored (Bijl-Brouwer and Malcolm 2016; Ryan and Leung 2014; Jones 2014). In addition to the described differences in models of policymaking, the design process cannot be entirely delineated or predicted in advance, which challenges dominant approaches to project planning. All of these challenges are illustrated in the case study analysis below.

Practice

As a design-based practice, rather than primarily an analytic tool or theory, systemic design is a craft with practical tools and techniques to enable participation, reflection and action. Like other forms of design, systemic design "is not a single method that can be applied like a boilerplate from one situation to the next" (Luck 2018). The practice of systemic design is shaped by the context in which it operates, as well as the resources and norms within those systems and the skills and limitations of individual practitioners (Jones 2014, 2018). It includes relational practices of helping, building, challenging, sharing, creating and innovating (Hancock 2019, 31).

Adopted and adapted from relevant disciplines (such as systems thinking, participatory design, community engagement, human-centred design, action research, appreciative inquiry, organizational design, dialogic design and principles-based evaluation), systemic design capabilities are developed through learning, action and reflection. Jones ( 2014) has identified five key methods of systemic design practice, which enable interventions within the third and fourth order of design. These are all illustrated in the case study analysis: human-centeredness, convening stakeholders, dialogic process, iterative inquiry and multiple design actions over time.

Policy design case study

As well as providing concrete examples for each domain of the practice framework, the community services policy case study illustrates several challenges for adopting and applying participatory, systems-aware and design-led approaches to policymaking.

Principles

The primary purpose of the community services policy project was to improve how the government organization provides and supports community facilities. A secondary aim was to build capability in and test a design-led approach to policy development and stakeholder engagement. Visual depictions of the policy issue helped project participants and stakeholders develop a holistic understanding of its complexity. One diagram (with concentric rings showing all the teams involved and data points indicating the volumes of work) in particular helped participants to understand the issue, identify their role in the system, and recognize the complicated organizational relationships surrounding community facilities. A team member described how participating in the project shifted their perspective to see relationships, rather than a document, as the most important output of a policymaking process. This aligns with Bijl-Brouwer and Malcolm's ( 2020, 396-367) observation that a distinguishing feature of systemic design, especially when compared with human-centred design, is its concern for human relationships.

The inclusive methods of the design-led approach, discussed further below, allowed the policy to develop in a real-life context, with input from people who would be responsible for implementing it. The Practice section illustrates how various methods and tools were used to "help different participants to express their needs and visions" (Luck 2018), which had the effect of equalizing power relations. While a wide range of stakeholders, including representatives of community organizations, were involved in the policy development process, this did not include citizens until the consultation phase. This, and the challenges of indigenous engagement, described in the next section, illustrate some of the limitations of participation and self-determination commonly faced in a government context (see, e.g. Blomkamp 2018).

The design-led process involved multiple feedback loops between project team members and stakeholders, which supported practice improvements and generated strategies for policy implementation. The mutual learning through the project helped participants to assess and question whether they were working in a meaningful way. The engagement process and materials fostered knowledge transfer between community members, policymakers and other stakeholders. The walkthroughs in particular (see Practice below) further encouraged a collaborative approach to embedding learning in the organization.

Place

The community services policy project was particularly complex in that it involved all orders of design. Policy design fits within the fourth order, which directly corresponds with systemic design. The focus of this policy was on the third order activities of the organization, which required consideration of "governance, operations, product line and service strategies, human resources, and all internal systems" (Jones 2014). Key outputs of the project were documents and posters (second order), which followed the visual style guidelines of the organization and represented ideas in new, visual ways (first order).

Systemic design practice (fourth order) may only be appropriate and effective in certain circumstances and environments. The case study suggests that conditions for a systemic design approach to policy include:

An organization that allows for experimentation;

A project team open to new ways of working, including democratic approaches to decision-making and power-sharing;

A project/team leader with experience of design and skills in the use of participatory and creative methods, as well as having the authority, autonomy or permission to apply these methods; and

A policy requiring significant consultation or engagement with stakeholders and/or community members.

This is not a definitive list of prerequisites for systemic design for policy; rather it identifies key elements from the case study as enabling conditions or key factors for success. Particular constraints in this organization were:

Ways of working that did not accommodate design-led practice;

Limited capacity and capability in systemic design practice;

Inconsistent understandings of methods (language, techniques and tools); and

Lack of foresight to engage Māori in the policy design process.

The lack of support and ownership from senior management is often identified as a key challenge with design-led approaches in government (Bason 2014; McGann, Wells, and Blomkamp 2021), as the process and outputs are unlikely to gain sufficient influence and impact if not supported by the authorizing environment. Some participants considered that the lack of understanding of the value and principles of human-centred design from some managers and politicians, or the view of it as "a fad," was a limiting factor in this project. An interesting outcome from the project, however, was that its perceived success encouraged wider adoption of design-based methods throughout the organization. This reflects how a systemic design project is not only constrained by its environment, but its self-organizing and emergent properties can shape that context too (van der Bijl-Brouwer and Malcolm 2020, 397).

The constitutional conventions in a state also set some boundaries and principles for policy development. Under the Treaty of Waitangi, government organizations in New Zealand have obligations to work in partnership with Māori ( mana whenua). The policy design team were surprised to discover early in the project that the government organization had few, if any, partnerships with marae (traditional gathering places for Māori that continue to be important community facilities and sites of cultural knowledge). The project scope had not initially included any focused engagement with Māori, so an additional workstream, dedicated project team and Māori design contractors were added. The additional work involved in rebuilding relationships with Māori put additional pressure on the project team and timelines. Despite the challenges and inability to resolve historical injustices, the Māori engagement workstream effectively informed the policy development. A team member explained, "some of the things we heard about what good relationships look like has totally made its way into the mainstream policy. And that wouldn't have happened had we not spent the time that we have with Māori over the last year." These examples of indigenous engagement illustrate an important element of how systemic design needs to respond to place, or context, in colonized countries.

People

The community services policy project brought together staff from different parts of the organization to work across silos; as one team member explained, "it breaks down those artificial divisions". The systemic design approach enabled the team to build and strengthen internal relationships to access information and understand different aspects of the policy. This was achieved by having people involved at the operational and policy levels of the organization, as well as developing relationships with community stakeholders.

Key human resources and capabilities involved in the project included, with reference to the roles identified by Leadbeater and Winhall ( 2020) in brackets:

Lead policy designer: the Principal Policy Analyst instigated and managed the project throughout all phases. As lead policy designer (inside-outsider), they used sophisticated project management, institutional knowledge and policy analysis along with facilitation, strategy, design thinking and related creative skills. This person had a unique mix of creative capabilities and analytical skills that may not always be found in one individual.

Design coach/co-facilitator: a consultant was hired to support the policy design and facilitation activities. An expert facilitator (convenor), they used various methods and facilitation skills throughout the process to guide the project team to think visually and to forge positive group dynamics, as well as providing moral support and strategic advice to the lead policy designer.

Visual designer: as the visualization of concepts and production of materials is an essential part of design-led policymaking, a communication designer (entrepreneur) was contracted to communicate the ideas in simple yet nuanced and effective ways.

A multi-disciplinary, cross-organizational team of "the right people": the project team included staff from different departments at the operational and policy levels who were committed to building internal and external engagement and had the ability to "work differently".

Other important roles included Māori design consultants and two senior managers (project commissioners). Apart from the Principal Policy Analyst, participating staff remained in their usual role and were not relieved of any responsibilities, which made it difficult to fully participate in a systemic design process. One staff member commented, "A real challenge for bureaucratic organizations that are politically driven is being able to fund that amount of flexibility in frontline staff to contribute." External consultants or contractors, like the design coach and communication designer, are often used in these contexts to increase capacity and provide expertise, especially when organizations are fairly inexperienced in systems or design-led methods.

Apart from representatives of community organizations in the discovery phase (see Figure 3), it is notable that citizens were not included in the policy development process. The lead policy designer consistently acknowledged that the project took a "design-led" approach but was not an example of "co-design," since they did not actively engage different users and stakeholders throughout the process: "I don't think it's co-design if you haven't got all the people in the room, who are basically representing all the different users in the system." In this way the project did not overcome typical power imbalances and information asymmetries between governments and policy beneficiaries. Through greater involvement of affected citizens, the project could have gained more diverse insights and reframed problems "in more acute and nuanced ways 'than professionals acting alone'," (Fung 2015, 5; McGann, Wells, and Blomkamp 2021, 5). The designerly methods nevertheless allowed stronger participation of citizens at the consultation stage and of internal stakeholders throughout the policymaking process.

Process

The community services policy project involved two key phases, modeled on the "double diamond" (Design Council 2007): discovery (understanding) and design (imagining and making). They were accompanied by the additional indigenous workstream, and followed by public consultation and decision-making by elected politicians (see Figure 3). The designerly practice of exploring problems and causes, before fully defining the issue to be addressed, goes against conventional approaches to policy, and was not fully supported by staff accustomed to traditional modes of policy analysis. While the double diamond is commonly used and understood as a heuristic to represent the design process, in reality a design-led process is iterative and messy. Following a systemic design process can result in the scope and focus of the project changing regularly. As one internal stakeholder stated, "You know the process loosely that you're going to use-that mentality is really hard for some people [in this organization] to take and possibly also support their staff to go on that journey because they want to know what's going to happen at each touchpoint." A team member similarly reflected on the challenge of an iterative approach, "Your project manager has to be open to the fact that this means that this either will take longer or that some of your stuff is more flexible at the end, or there might be parts that compress and expand; even if you need to end up here by this date, it's more movable." The flexibility needed made it difficult to fully implement a design-led process in this bureaucratic context.

Several participants commented that the design-led approach had not proven to be quicker or cheaper than regular practices of policy development. One manager suggested that elected decision-makers desire an efficient, not a creative, process. They argued that, unless politicians were calling for design-led methods, there was no mandate for using them. Another participant suggested that a design-led discovery phase may be useful at the start of a policy project, but then traditional policy analysis should take precedence. A culture of risk aversion and overly bureaucratic processes are commonly identified as challenges for change and innovation in government, especially for systemic design practice (Bason 2014; McGann, Blomkamp, and Lewis 2018).

Practice

The community services policy project demonstrates the importance of selecting, adapting and applying appropriate techniques and tools to engage a variety of stakeholders, build shared knowledge and maintain momentum over an extended period of time. Practical tools and designerly methods (in italics below) were used during the project by the lead policy designer and co-facilitator to enable numerous stakeholders to meaningfully participate in dialogue and co-creation.

A variety of facilitation tools and techniques enabled participation and fostered group dynamics. Relational practices and resources such as flipcharts, sticky notes, Lego, picture cards, sketching, games and role play were used by the experienced co-facilitators to keep meetings and workshops lively and to facilitate discussion and learning. At the same time, they helped to build trust within the team and ensure that all participants felt safe and comfortable interacting and contributing.

Empathy interviews were a data collection technique applied to gain an in-depth understanding of the needs, experiences and aspirations of community members, staff and other stakeholders. Conducted by staff from different departments, the empathy interviews served to explore the policy problem from different perspectives and build early engagement with users and stakeholders in defining the issue together as a "shared problem". They also provided a rich source of stories and natural language quotes. With the emphasis on hearing the voice of "users," staff from operational and policy levels put themselves into a deep listening mode through the interviews. By analyzing the interview findings as a project team, one member explained, "We really understood what different people's motivations and influences and things that were obstructing them were." Another participant said the interviews enabled them to develop a "depth of knowledge really quickly" and "having that story context around people's issues and how the policy can help address a lot of those things was really helpful."

Personas, scenarios and vignettes were generated to represent typical policy users and situations, showing the personal, operational and systemic impacts of experiences in a human-centred way. These fictional representations were easy to recognize and relate to, helping stakeholders to understand the impacts of the current approach and work through possible implications of a new policy framework. At the same time, they made it safe for participants to share their experience without identifying particular individuals or organizations.

Graphic visualization and communication tools, such as infographic diagrams and sketches, were valuable for getting complex information and key messages across in a simple and engaging way. The visual representation of different stakeholders helped people to see both their role and relationships as well as the complexity of the system surrounding the issue. By disseminating information in images that told stories, rather than only through words, participants were able to engage with a lot of content quickly.

Walkthroughs were interactive sessions where stakeholders were invited to view the work in progress and to participate in providing feedback and validation on findings. Considered a particularly effective method for engaging stakeholders throughout the project, the walkthroughs shared the voices and views from the community and demonstrated how insights, data and feedback gathered along the way were taken into account. They allowed stakeholders, including elected members, to see work in progress and provide feedback in early phases of the project, well before the usual consultation period. This offered a less formal and more convenient validation process, as people could review and comment on components of the policy in their own time and in a range of formats.

Several participants commented on the subsequent adoption of walkthroughs and other design methods throughout the organization as a result of exposure to this project. While some expressed concern about people without the skills, time or experience to effectively execute walkthroughs adopting this technique; on the whole, the interest and take-up of this practice was seen as evidence of a design-led approach spreading through the organization. The community services policy walkthroughs were especially successful because they pictorially presented complex problems and systems, curating experiences for the people involved and showcasing the flow of information in carefully designed visual materials. The development of the final written policy document extended and complemented the visual and narrative elements developed in earlier phases of the project to continue communicating complex and abstract ideas in engaging ways. The case study thus demonstrates the ongoing importance of esthetics within first and second order "design as making," even in the complex landscape of the fourth order of "design as social transformation" (Aguirre Ulloa 2020, 39-43).

The openness of design practice encouraged experimentation and risk-taking. For some team members, design was code for "different ways of doing things". Participants described design as a methodology that took its own shape to fit the project purpose or as a creative multidisciplinary practice, but not necessarily involving only design-based methods. One team member praised the flexibility of the approach for encouraging self-determination and different ways of thinking: "Because [in government] you get very locked into processes and systems and ways of working, and what I like about this is [it's] different and we can create it."

Conclusion

Overall, the design-led approach of the case study project had benefits for policymaking, particularly by building a human-centred understanding of issues and systems, and by involving a range of people in innovative and engaging ways. The experience of taking part in the project encouraged staff to be in a continual state of readiness for change, since they saw the value in not starting with predetermined problems and were eager to experiment with different methods. There was not, however, consistently strong understanding or support for this approach throughout the organization, especially by senior managers. It was at times challenging for the organization to provide the requisite skills and expertise for systemic design, or to adapt to different ways of working. None of this is surprising given that this was the first time the organization had experimented with a design-led policy development process. As Evans and Terrey ( 2016, 257) have written on co-design for policy, this approach "requires particular skills of observation, negotiation and empathy that are often in short supply in many public sectors."

Participants also noted that the collaborative approach to building relationships and understanding demanded more time than usually required by staff in the early phases of policy development. As has been argued elsewhere, concerns "about the requisite time and cost of co-productive policy approaches should be taken into account when managers decide which policy issues warrant a more participative approach" (McGann, Wells, and Blomkamp 2021, 16). Other research on co-design, co-production and public sector innovation has similarly highlighted the challenge of introducing designerly methods and participatory practice into the hierarchical and bureaucratic culture and structures of government (Bason 2014; McGann, Wells, and Blomkamp 2021; Lewis, McGann, and Blomkamp 2020; Blomkamp 2018; Durose and Richardson 2016; Aguirre Ulloa 2020). Despite the challenge and cost of participatory approaches, systemic design may nevertheless "pay off in the longer run in terms of establishing solutions that enjoy a higher degree of endorsement and robustness" (Dieckmann et al. 2020, 158).

The case study illustrated the core domains of the "5 Ps" framework for systemic design: principles, place, people, process and practice. It is expected that by reflecting on their work in relation to each domain of the framework, practitioners could identify constraints, guiding principles and practical options to pursue. Using the narrative form of news journalism as an analogy, the practice framework could be used to tell a story of why (principles), where and when (place), who (people), how (process) and what (practice) is, was, or will be involved in participatory policymaking. In particular, this can highlight missing elements and likely challenges to overcome in practice. Further research could explore the value and limitations of applying the practice framework in specific circumstances, not only in participatory policymaking but in other collaborative approaches to systems change. It would be particularly relevant to consider its application to cases that more fully involve the public in systemic design.

Acknowledgments

Some case study analysis was conducted by research assistant Primatia Wulandari. The paper was strengthened following helpful comments by the journal's peer reviewers and discussion at workshops at the National University of Singapore (March 2020) and at RSD9 (October 2020). The author sincerely thanks the participants in the case study research, especially the key informant (Principal Policy Analyst), for their openness and time.

Disclosure statement

In accordance with Taylor & Francis policy and my ethical obligation as a researcher, I-Emma Blomkamp-am reporting that I am a consultant whose business may be affected by the research reported in the enclosed paper. The research was developed prior to my current role, when I was fully employed at The University of Melbourne, and where I continue to be an Honorary Fellow.

Additional information

Funding

The case study research presented in this article was supported by The Policy Lab at The University of Melbourne and some analysis conducted by research assistant Primatia Wulandari.

References

Abercrombie, Rob , Ellen Harries, and Rachel Wharton. 2015. "Systems Change: A Guide to What It Is and How to Do It." New Philanthropy Capital (NPC). https://www.thinknpc.org/resource-hub/systems-change-a-guide-to-what-it-is-and-how-to-do-it/.

Aguirre Ulloa, Manuela. 2020. Transforming Public Organizations into Co-Designing Cultures: A Study of Capacity-Building Programs as Learning Ecosystems. Oslo School of Architecture and Design. https://aho.brage.unit.no/aho-xmlui/handle/11250/2654789.

Banathy, Bela H. 1996. Designing Social Systems in a Changing World. New York: Springer.

Bason, Christian. 2014. "Introduction: The Design for Policy Nexus." In Design for Policy, edited by Christian Bason, 1-8. Surrey, UK: Routledge.

Bason, Christian. 2017. Leading Public Design: Discovering Human-Centered Governance. Bristol: Policy Press.

Bijl-Brouwer, Mieke van der , and Bridget Malcolm. 2016. "Developing a Systemic Design Practice to Support a Regulatory Agency in Addressing Complex Problems." In Proceedings of RSD5 Symposium. Toronto.

Blomkamp, Emma. 2018. "The Promise of Co-Design for Public Policy."In Routledge Handbook of Policy Design, edited by Michael Howlett and Ishani Mukherjee, 59-73. New York: Routledge.

Blomkamp, Emma. 2020a. https://rsdsymposium.org/.Introducing and Testing a Systemic Design Practice Framework. India: Systemic Design Association.

Bridgman, Peter , and Davis Glyn. 1998. Australian Policy Handbook. 2nd ed. Sydney: Allen & Unwin.

Design Council 2007. "Eleven Lessons: Managing Design in Eleven Global Brands." http://www.designcouncil.org.uk/sites/default/files/asset/document/ElevenLessons_Design_Council%20%282%29.pdf.

Durose, Catherine , and Liz Richardson. 2016. Designing Public Policy for Co-Production: Theory, Practice and Change. Bristol: Policy Press.

Geels, Frank W. 2005. Technological Transitions and System Innovations: A Co-Evolutionary and Socio-Technical Analysis. Cheltenham: Edward Elgar Publishing.

Jones, Peter. 2014. "Systemic Design Principles for Complex Social Systems." In Social Systems and Design, edited by Gary Metcalf, 1:91-128. Translational Systems Science Series. Verlag: Springer. https://www.academia.edu/5063638/Systemic_Design_Principles_for_Complex_Social_Systems.

Jones, Peter. 2018. "Contexts of Co-Creation: Designing with System Stakeholders." In Systemic Design, edited by Peter Jones and Kyoichi Kijima , 8: 3-52. Translational Systems Sciences. Tokyo: Springer Japan. doi:10.1007/978-4-431-55639-8_1.

Lankelly Chase. 2020. "System Behaviours." Lankelly Chase. 2020. https://lankellychase.org.uk/our-approach/system-behaviours/.

Leadbeater, Charles , and Jennie Winhall. 2020. Building Better Systems: A Green Paper on System Innovation. Denmark: ROCKWOOL Foundation. systeminnovation.org.

Palmer, Victoria Jane , Wayne Weavell, Rosemary Callander, Donella Piper, Lauralie Richard, Lynne Maher, Hilary Boyd, et al. 2019. "The Participatory Zeitgeist: An Explanatory Theoretical Model of Change in an Era of Coproduction and Codesign in Healthcare Improvement." Medical Humanities 45 (3): 247-257. doi:10.1136/medhum-2017-011398.

Tenbensel, Tim. 2006. "Policy Knowledge for Policy Work." In The Work of Policy: An International Survey, edited by Hal K. Colebatch, 199-215. Lanham: Rowman & Littlefield.

# data[].attributes.attachment_link.title
Systemic design practice for participatory policymaking

# data[].attributes.attachment_link.description
Abstract Abstract As the complexity of policy problems is increasingly recognized, and participatory approaches gain popularity, policy workers are applying different methods to engage a wide range of stakeholders and citizens in policy development and implementation.

# data[].attributes.headline
The limits of agile - can we apply it to policy making?

# data[].attributes.subject
The limits of agile - can we apply it to policy making?

# data[].attributes.attachment_link.title
The limits of agile - can we apply it to policy making?

# data[].attributes.attachment_link.description
On Saturday, I went to UK Govcamp 2015, an all-day event where 200 people interested in innovation, data and digital in the public sector (among other things) get together to talk about - well, whatever they want to talk about …

# data[].attributes.headline
An introduction to Open Policy Making and design

# data[].attributes.subject
An introduction to Open Policy Making and design

# data[].attributes.body
An introduction to Open Policy Making and design - Open Policy Making toolkit - Guidance - GOV.UK (www.gov.uk)

# data[].attributes.attachment_link.readable_text
Open policy making helps civil servants create and deliver policy that meets the demands of a fast-paced and increasingly digital world. It means that policy is more informed and better designed for both the government and users by:

Open policy making is about developing and delivering policy in a fast-paced and increasingly networked and digital world through:

using collaborative approaches in the policy making process, so that policy is informed by a broad range of input and expertise and meets user needs

applying new analytical techniques, insights and digital tools so that policy is data driven and evidence based

testing and iteratively improving policy to meet complex, changing user needs and making sure it can be successfully implemented

Read more about Open Policy Making on our blog and twitter

Be open to new ideas and new ways of working

Listening to new ideas and engaging people will give you new insights and help you see problems from a different point of view. Design thinking and agile working are examples of new approaches that can help seeing a problem from a user perspective.

Read Policy Lab's introduction to design in policy making for more information on design thinking and policy making.

Be humble

Gathering evidence, information and a broad range of views is vital. It is important to be humble about what you know, and what you don't know. Open policy is about recognising that you are not the only expert and that you do not have monopoly on good ideas and policy development. Where good ideas exist, it is your job to find them.

Be prepared to be told something is not working and open to new suggestions, data and evidence. This will help you to understand different points of view and make iterative improvements.

Understand the real needs of users

Understanding user needs will help you develop policy that works for the people it impacts. Begin by understanding how users currently experience a policy and go from there.

Questions like:

How do users want to experience a policy once implemented?

What are their needs from the policy?

What is going to make a user experience positive or negative?

help you to understand who your audience are and how they work.

Involve the public

Engaging with the public will help you understand and gauge the public mood. The people with the most knowledge of how services and policies are working are those that it experience it first hand. Their experiences can help you to understand the needs of citizens and create more informed ideas.

Work with experts and engage with new knowledge

Experts can help you to better understand a policy problem, review possible solutions and challenge assumptions. Experts include:

User researchers and ethnographers who work with users to understand their experiences and problems.

Designers can help to create innovative policy ideas throughout the policy cycle and facilitators can help you talk to the public and engage in an open and safe environment.

Academic experts and subject matter experts can bring new knowledge to a policy area.

Digital experts open up the possibilities of engaging at scale and analysing large volumes of evidence at pace.

Test and use evidence to improve your policy as you go along

Test and and iterate policy solutions to make sure they will work for users in the real world. This will help to take a policy from the idea stage to a deliverable product that works.

Use data to learn, prove and succeed

Use data to generate new insights and learn what is working and what is not. It can tell you about who your audience and users are and help you understand what the problem is from their perspective.

Contents

Policy Lab: an introduction

Policy Lab is bringing new approaches to policy-making. From data science to user-centred design, it provides fresh thinking and practical support, working as a research and design testing ground for policy innovation across government.

The Lab is funded by departments and works on projects coming from all parts of government.

Open policy making is important to the Lab's approach. Policy Lab creates a neutral space for policy-makers to collaborate across departments and engage with the public and external experts in key policy areas. This involves a rigorous, collaborative process, working in close partnership with policy teams and using a range of innovative tools and techniques.

The practices of Policy Lab are described throughout this toolkit and in the Policy Lab Methodbank.

Policy Lab support is best suited to addressing intractable, complex, systemic policy problems that require fresh thinking that can lead to potentially transformative solutions.

So far, Policy Lab practice has involved 3 main areas of focus:

providing new policy solutions through inspiring practical projects

building the skills and knowledge of the policy profession and wider civil service

inspiring new thinking and innovation in policy through writing and experimenting

The Lab's approach is agile, flexible and iterative and can help departments in many ways.

The Lab:

supports policy teams to identify new insights into the needs of service users

generates ideas that can stimulate innovation and transformational change

acquires knowledge and expert opinion to inform policy development

creates opportunities to make policies more deliverable through testing and prototyping

produces efficiencies and cost savings

Policy Lab has so far worked on 15 practical projects ranging from policing in a digital age to family mediation and the future of ageing. As of October 2016, 4,000 civil servants have been involved in lab lights, sprints, policy schools and learning and development training courses. The Lab has also engaged with over 5,000 people inside and outside government through talks, workshops and its online presence.

Policy Lab offers lab taster sessions, which are an opportunity to work up a project idea in an introductory workshop for policy teams who may be interested in running a lab project. These are usually followed by policy sprints, which are more intensive, collaborative workshops over 1 to 3 days designed to help teams accelerate a project.

Lab full projects can run from 3 months to a year and involve working intensively with service designers, ethnographers, data scientists and subject specialists on complex challenges. Policy Lab also runs experiments designed to develop a number of policy 'firsts' for government. The Lab invites departments to set a challenge or get involved to try new ways of working through one-off trials of new and emerging techniques.

If you are interested in working with Policy Lab email policylab@cabinetoffice.gov.uk

Themes of Open Policy Making

Contents

Agile policy making: an introduction

Agile is more of a mindset than a tool or technique. It was initially conceived by software developers in the 1990s, and has been adopted in digital service development in government, but it can be applied to policymaking too.

It works well where things are unclear and non-linear, when you don't know with certainty what might happen. It asks you to put the user first, thinking about what they need, rather than what might be easiest for our organisations. You can then co-design and develop ideas iteratively, with feedback from users improving them.

It encourages multi-disciplinary teams, recognising that fresh perspectives create better ideas. You must be flexible and adaptive, open to new ideas, opportunities and feedback.

Agile in digital services

GOV.UK and all digital services on it are built using agile methodology. The Government Digital Service has an extensive introduction to Agile, which can be very helpful for policy makers too.

Examples of agile policymaking

Policy Lab worked with the Department of Health and the Department for Work and Pensions to support people to manage their health conditions and stay in work.

They organised this work in a series of short policy sprints (short project bursts). The first was a 3-day session with a multi-disciplinary team including policymakers, data scientists, ethnographers, designers, GPs, employers, Jobcentre staff and community organisations. They spent time understanding the problem and developing practical research and co-design solutions with users.

The plan was flexible with weekly 'stand up' meetings to adapt the plan according to the insight the research was generating and from user feedback from the prototype policies. This helped to create better ideas through recognising different perspectives and listening to user responses.

Contents

Designing with users: co-design

Open Policy Making is about working with users to understand their problems and create policy that works for them.

The process of creating products, services or policy with users is called co-design.

Co-design is a way of working that creates policy directly with the people that policy impacts. Co-design and Open Policy Making believe that are experts in their own experiences, and can bring different points of view that inform and innovate a policy's direction. Co-design puts users at the core of policy creation so that policy is more informed and responsive to their needs.

This often takes the form of workshops, hack days and idea jams. These events let users explain their current problems and then work with policy makers and designers to come up with solutions for the future.

Co-design can work at any point in policy design, however it is most commonly used at the beginning of a policy to help understand where a policy needs to focus.

Co-design and policy making

Co-design can be done in a number of ways with policy making. Policy Lab and the UKTI Ideas Lab use co-design techniques to make sure that all the policy they create is designed around the experience of the user.

Hack days and ideas jams are events that can run for hours or days and are a popular tool for co-design working. They bring in experts, users and outside knowledge to approach problems from new perspectives and develop solutions quickly.

Workshops are another way of co-designing policy. Tools like challenge setting, sketching and change cards can quickly improve understanding of a problem and the direction any solution needs to take. Instructions on how to use the tools are found in the Low Cost Tools section of this toolkit.

Open data is also an important part of co-design. Open data used at hackathons can bring in outside expertise and help you to design innovative solutions, quickly and easily.

Reading and information

There is a large amount of literature on co-design. Steve Hilton's book More Human looks at how co-design and open policy can impact policy and government.

UX magazine has an in depth look at co-design and how to do it.

Policy Lab has blogged about what design in policy making means and Dr. Lucy Kimbell has also published a free downloadable book that examines what design means to policy making.

Contents

Open data: an introduction

More data is being generated than ever before, from apps and devices people use everyday, to the administrative data that government departments keep on their services and customers. Open data can benefit policy in numerous ways. It:

brings in external thinking through hack days to open eyes of ministers and officials to design better solutions. This has been used at places like MoveMaker and SkillsRoute

can help policy makers work with voices of citizens and create choice for users. Ofsted's school dashboard has done this with schools to help parents choose

creates a different relationship between citizen and state, increasing awareness and sometimes encourages engagement

Not all data is fully open. Like Open Policy Making there are scales of openness. The Open Data Institute (ODI) spectrum helps you to see how open you can be. Read more information about Open Data from Open Knowledge Foundation's School of Data and Open Data Institute.

Examples of open data

Citymapper is one of the best known examples of open data. It takes data from transit authorities and the public sector around the globe to make cities easier to use and 'reinvent the transport app for the world's most complicated cities'.

There are also prototypes on GOV.UK that highlight how open data can be used to report on the success of a policy. These prototypes are not working yet but they do show how policy can work with digital and open data to improve understanding of the policy. View the prototype on housing and free schools

Why you should use data to understand

Data science techniques, such as clustering and predictive analytics, can help to identify an issue quickly, and allow you to make interventions earlier to avoid a major problem occurring.

For instance, by analysing comments and feedback data from GOV.UK, it's possible to see where most complaints are occurring, who is involved and how to mitigate against the issues reported in them.

One example of this is passports. If complaints are increasing in frequency, and often contain words relating to the online service failing, it is possible to undertake maintenance at an early stage to avoid complete service failure.

How to use open data

1. Get the data

Open data is often useful to further augment existing data to improve your knowledge and cross reference information. Open data is free and available from UK Data Archive, Data.gov.uk and the Office for National Statistics.

Often the best approach is just to explore. There are many places that hold openly available data that can be used for analytical purposes. Policymakers should seek data and not be overly prescriptive about traditional methods of understanding policy issues.

2. Use data to identify policy issues

Data visualisation techniques can make your data more compelling and show where policies are failing to reach their intended targets.

An example would be the visualisation of the LSYPE (Longitudinal Study of Young People in England) survey data to show transitions between different aged cohorts. By viewing data in this way it is possible to find trends, such as the large number of people moving from family care into unemployment between the ages of 20 and 21. This highlights an area to look at further and and begin to create policy interventions around.

Data visualisation, analysis and open data are often specialist tools and working with experts is often necessary.

Ethical implications and privacy concerns in using data

There are many ethical and privacy considerations when using data. Data that is heavily aggregated will be safe to use, but will not always make for compelling or useful analysis. Personal-level data is great for bespoke tailoring of services and interventions, but creates potential issues around privacy and protecting the anonymity of citizens.

Policymakers should think about what they are trying to achieve and how it could impact on the privacy of citizens. They should ensure that they are working within the knowledge and information management guidelines set out by their departments, and the terms of the Data Protection Act.

This guidance provides a broad overview of the methods and techniques available to conduct user research. Research can incorporate both qualitative and quantitative techniques.

Qualitative techniques are intensive and often small scale. These include focus groups and one-to-one interviews, and are typically used to explore and analyse unstructured data.

Quantitative techniques involve higher-volume research, and include online surveys, face-to-face interviews, and involve a structured approach to data collection and analysis.

Read more about user research in the Digital Service Manual.

Ethnographic research usually involves observing target users in their natural, real-world setting, rather than in the artificial environment of a lab or focus group. It aims to gather insight into how people live; what they do; how they use things; or what they need in their everyday or professional lives.

Read more about ethnography and how to do it in the government service design manual.

Contents

Measuring success in open policy: an introduction

There is no single way to measure the impact or success of a policy; each policy should be evaluated individually and according to its nature. It is important that the measurement of success is built into the design of policy from the outset.

Many policy makers talk about measuring success, but it's vital that you're clear on what constitutes success. For example, getting 20,000 young people onto a youth programme is an achievement but it is not an impact. Have they gained anything from it? Has it had any negative impacts? Is the programme being run as intended?

Some initial practical questions to consider

what evidence do I need?

are there certain questions which will be asked of me after implementation in order to meet objectives?

is there an opportunity to add to an existing evidence base?

who is this programme likely to impact and in what ways?

what am I trying to achieve? Consider creating a theory of change

how can I work with my main stakeholders and analysts?

is there already evidence I can use to inform the development of my policy/programme eg held by What Works centres?

who is doing work in this area, eg academics, think tanks, that I can link up?

Examples from around government

In 2014 to 2015 the Behavioural Insights Team worked with HM Revenue and Customs (HMRC) to understand how they could encourage small businesses use government programmes that they could benefit from. They investigated how HMRC could use existing communications channels differently to prompt and inform.

One option considered was whether they could contact businesses at the same time as they would for other matters (such as VAT returns). As there was some concern that providing extra information might detract from the core HMRC message, they ran a trial.

The trial used almost 400,000 HMRC emails to communicate to small- and medium-sized enterprises (SMEs) about the programmes they might benefit from. Businesses were randomly allocated to 2 groups; those that received the old type of email message and those that that received the new email message, which contained information on government programmes. A variety of ways of conveying information within the email messages were also used.

The trial showed that simplifying the messages worked very effectively. But the team also found that some messages were more effective than others. Telling firms that their type of organisation had been chosen to receive information on the programmes was the most effective of all. The sheer number of emails also significantly increased sign-ups to the programmes, demonstrated by the peaks in demand in the period following the release of emails.

Types of success and impact measurement

Measuring and evaluating impact must be considered upfront at the policy design and scoping phase, and then continuously throughout policy delivery. To evaluate successfully, certain measures which need to be put in place, and questions which need to be asked, before the policy or intervention begins. Examples of these are:

establishing a baseline: what is the state of a policy before any changes to it are made, or before any new policy is designed and introduced?

establishing a control group: do I need to measure what would have happened anyway if certain changes hadn't been made to demonstrate it is my design/policy which has been effective and not something else?

How to measure success and impact

When considering an evaluation or impact assessment it's always worth involving analytical colleagues for their advice. Whether you want to conduct a small scale, in house evaluation with minimal budget, or commission something larger out such as some randomised control trials, analytical advice and guidance is likely to be useful.

It's also important to involve those delivering and designing the policy in the process from the beginning. This will help you to understand what is feasible and will hopefully secure buy-in to the evaluation process.

Government guidance

# data[].attributes.attachment_link.title
An introduction to Open Policy Making and design

# data[].attributes.attachment_link.description
Open policy making helps civil servants create and deliver policy that meets the demands of a fast-paced and increasingly digital world.

# data[].attributes.headline
These cheap tools can help make policy more open. 

# data[].attributes.subject
These cheap tools can help make policy more open. 

# data[].attributes.attachment_link.readable_text
Contents

Acting: an introduction

Acting is a way of testing new policies and ideas for quick feedback and help. This can be done by organising workshops where policy ideas can be tested by people acting out the proposed idea, and then a discussion taking place around the scenes being acted.

Examples from around government

During April 2015, Policy Lab and Policy Profession organised and held three prototyping workshops where people acted out early ideas for a policy. The format allowed people to express concerns about the early stage concepts.

Because much of the detail of the concepts was undefined yet, people filled in the gaps or responded by making interpretations and assumptions. It was this ambiguity that created a deep level of engagement and response.

When to use acting as a tool

Acting works best when you have an early idea or concept that you want to test with experts or users. It can be a great way of quickly updating your knowledge and ideas to make sure they answer user needs.

If you haven't created some policy ideas yet then acting will not be of much use unless you wish to use it as a way of showing an idea to interested parties.

How to use acting to test policy ideas

1. Before a workshop you should meet with the actors to develop a rough script that explores the policy idea. It can be useful to make simple props for the mock scenes and do a run through. You should be clear about which parts of the scenarios the actors must stick to and where they can improvise.

2. Writing a scripted scene for each scenario that your actors are supposed to highlight will help them to keep to the point and show the exact policy idea.

3. At the workshop, you should introduce the scenarios and ask the invited audience to think about a particular question for each scenario. The actors can improvise with guidance prompts and stop at important points so that users and experts who are watching can bring up points of concern or topics of debate.

4. You should then engage the audience in discussion and be responsive to the audience's interests and guide the actors to change what they are doing. After seeking permission, it is a good idea to document the workshop with video and photographs to use as evidence later.

Contents

Challenge setting: an introduction

Challenge setting lets you work with users and groups of people to define the challenges that are important to your policy.

Challenge setting frames a person's fears and needs as questions that begins with 'How can we'. The challenges that are created can then become collective issues to debate and investigate further.

When to use challenge setting

Doing a hope and fear card exercise helps frame the problem for users. You should usually do this before challenge setting.

Use challenge setting at the beginning of a project to help you understand what people need and want from a policy area. The tool works well alongside other tools like journey mapping and idea days or policy jams.

Using challenge setting at an early stage of a project helps you look at your problem from a users' perspective and understand their needs and desires. It also helps you to find people to work with to design policy.

Challenge setting works with users and experts as well as any other interested party.

How to do challenge setting

To start using challenge setting you will need thick marker pens and challenge setting cards.

Challenge setting cards are A5 sized with 'How can we...' written on them. There should be space for people to write a challenge. Challenges should be as short as possible and no longer than 4 lines long.

To do challenge setting, give members of the discussion the cards and ask them to write down their aims or needs from a policy, event or experience. If you have done a hopes and fears exercise you can ask them to base their 'how can' we on some of the hopes and fears you have previously discussed.

You should then discuss everyone's challenge setting as a group. Placing everyone's challenge on a scale often helps to show the variety of challenges you will discover. The scale could be small to grand challenges or cheap to expensive depending on the policy area.

Contents

Challenge panels: an introduction

Challenge panels use experts to look at weaknesses and ideas to improve them current or future policy. Policy experts can test and provide constructive challenge to the policies from a real world perspective.

A challenge panel is one of the simplest ways to organise feedback and testing of a policy before it is launched as it only requires inviting experts in the field.

How to run a challenge panel

Challenge sessions work best when a policy team is prepared to explain their position, is open to challenge, and willing to listen.

Challengers must be focused, able to raise issues from non-government perspectives, and willing to provide tough constructive criticism.

If the event is being facilitated by someone who is not involved in the policy and willing to direct the discussion, drawing up key themes where policy is underperforming is made easier.

There should be a small number of independent, diverse challengers; for example people who used to work in the team, other government departments, think tank members, journalists, business specialists, parliamentarians, non government organisations, economists, scientists or community groups.

You shouldn't be defensive or respond to every point - challenge panels are about listening and being open to new ideas

If your policy area is sensitive, consider setting up an agreed confidentiality document that states your expectations are for non-disclosure of information.

Contents

Change cards: an introduction

Change cards are questions that help people think outside of the box and discuss possible directions of a policy. They are cards with questions on them like 'what if we had no budget?' and 'what would a start-up do?' that help people think outside of the box with policy ideas.

When to use change cards

Change cards are useful in the early stages of designing a policy when you are trying to understand what users need and want from a policy area. They are also useful when you have gone through all the usual ideas and need to think differently.

You can combine change cards with other tools to quickly understand and generate ideas with users and experts. Role play, challenge panels and hack days all work with change cards.

How to use change cards in a 1-hour workshop

Before the workshop create some change cards on A5 pieces of card or use the Policy Lab Change Cards. You will need to decide whether the cards pose a question with words, or stimulate debate with images. Find a space to work where you can stick things on the wall to enable discussion and free flowing ideas as you present cards to the group. Ideally find a space with several tables and chairs so that people can work in groups of about 4 to 6 people. Here are some example questions you could use to get you started:

what if we had no money? / what if we had an unlimited budget?

what if people were our only resource?

what if we did the opposite?

what if we exaggerated the idea?

what would we do in 2040? / what would we have done in 1920?

what would we do if there were no computers? / what would we do it we had to provide it all online?

how would a child design it?

what if we merged 2 ideas?

what would an entrepreneur do?

what would they do in the USA?

what would the public want us to do?

what would we do if we couldn't legislate? / what would we do if we could only recommend best practice?

At the workshop

You should give people permission to be creative and work differently. An icebreaker exercise or facilitators can help you make the session flow more freely.

During the exercise people should define their own challenge and organise themselves into teams with people from different backgrounds to stimulate new ideas. Doing a hope and fear card or knowledge safari before you start is often a good idea. You should then discuss the policy area as a group before individually working with change cards.

People should look at the change cards they are given and then present their ideas one by one, explaining the criteria they are using. After the exercise you can invite people to vote about which ideas work best and discuss together which ideas to take forward.

Contents

Data tool cards: an introduction

Data tool cards are questions that help policy makers work with data scientists to think and discuss possible data that they might need to include in their policy project.

Data tool cards have questions on them like 'what would be proxy data for your project?' and 'What would someone's phone tell you about your policy?'. The cards and questions are presented to policy makers to help them understand how data can impact their policy area and discuss what data they may need.

Policy Lab has created two types of cards that can all be used together.

When to use data tool cards

Data tool cards are useful in the early stages of designing a policy, when you're trying to understand what data you will need to understand users, the policy problem and also measure success and impact.

Data cards focus on facts and inspiration through words rather than images. Interesting facts are used to try and help policy makers see the importance of data in their policy development.

How to use data tool cards

You should use data tool cards during or after you have generated some ideas. The cards work best when they 'disrupt' the thought process.

1. Once everyone have spent a bit of time developing an idea, ask them to pick up a change card to help them push their thinking in a different direction - they can pick as many or as few as they like.

2. Get teams to present their ideas one by one, with other people giving constructive feedback and explaining the criteria they are using.

3. Invite participants to vote about which ideas work best.

4. Discuss together what ideas seem to work and what you want to take forward.

Guerrilla user testing is a low cost method of user testing. The term 'guerrilla' refers to its 'out in the wild' style, in the fact that it can be conducted anywhere eg cafe, library, train station etc, essentially anywhere where there is significant footfall.

Read the full advice and information for guerilla testing from the Government Digital Service

Contents

Hope and fear cards: an introduction

Hope and fear cards use images to inspire people to express their objectives and concerns for a policy or a problem. The images act as metaphors for people's hopes and fears - helping them articulate what they think already, and potentially encouraging them to think differently. They can be used with anyone involved in a policy project and are a good way of quickly understanding a group's way of thinking about a particular problem.

When to use hope and fear cards

You should use them at the earliest stage of policy development so that you understand the aims and needs of your team members, stakeholders and users.

How to use hope and fear cards

You can create your own cards or use some standard Hopes and Fears Cards made by Policy Lab.

If you want to design you own, start by picking as many pictures as possible. Inserting a small text box into the picture to encourage people to use a word or phrase to describe their hope and fear.

Leaving too much space risks people writing long sentences when the image should to do the talking.

During the workshop:

1. Lay all the cards out on the table - try and spread them out so that each picture is visible.

2. Provide large coloured pens for writing: these work better than biros as they stop people writing too much and are much easier for everyone to read.

3. Decide whether you want to address hopes or fears first.

4. Give participants a set period of time (say 5 minutes) to pick the card that speaks to a hope/fear they have for the project/issue/challenge.

5. Ask them to write 1 or 2 words in the space on the card that summarises their hope or fear.

6. Depending on the size of the group, bring everyone together to share their hopes/fears, or ask people to get into groups and share among themselves.

7. Display the cards on a wall or table where participants can return during the day.

8. Repeat the process for fears/hopes.

9. Reflect on the 2 sets of cards: is there anything interesting or surprising, any particular clusters, or vastly differing views (for example one person's hope being another's fear).

10. Encourage participants to revisit the hopes and fears throughout and at the end of the workshop to add to, remove or change what is there.

Contents

Journey mapping: an introduction

Journey mapping helps you to understand a user's experience of a service or policy over a period of time. By plotting the experience of a user you can understand the interactions and touch points that people have regardless of department or policy boundaries.

Journey mapping puts you in the shoes of users. This can help clarify the exact and various components of that journey and help you to join up different experiences and policies that people use.

An example of journey mapping could be the experience of someone falling ill and leaving work. Journey mapping will visualise their experiences across multiple government departments, charities and private companies. The government will see this as many experiences, but to the user it is one experience that may need improving.

Examples from around government

Policy Lab has worked with civil servants on journey mapping to examine how to better support people who have a health condition and are at risk of leaving work because of it.

It also used journey maps to describe the experiences of people who have been a victim of a crime. It used the technique to understand how, and why they do or do not report crime to the police.

Policy lab has also used journey mapping to help scope out a project to support people going through divorce or separation to use mediation services rather than going to court.

When to use journey mapping

Use this tool when you are at the early stage of understanding the policy problem and user needs. You can also adapt the tool and use it when interviewing people by asking them to map out their journey to help explain their experiences.

Journey mapping works well when you have little time or money, and you want to understand an issue from the perspective of citizens, users or frontline staff. It can also be useful when you want to involve a range of people in exploring an issue, some of whom have deep knowledge of the lives of people and some of whom may be operations or delivery-focused, or when your understanding of a policy problem is still unclear.

How to do journey mapping at a workshop or hack day

Work with groups of people who interact with your service or policy area and ask them to map their journey for you. User journeys can be fictional or real life - this will depend on the people you are working with.

What you'll need:

long rolls of paper

sticky notes

bluetack

good quality fine point marker pens - they make everyone's drawings legible and better looking

What you need to do:

1. Put several long pieces of paper on the wall before you start - 1 for each team.

2. You could also start with a persona creating exercise to help participants think about the issue as a whole.

3. Ask teams to pick 1 person whose journey they want to map - they should map out on the long piece of paper the main phases of this person's journey as they experience the issue.

4. Encourage participants to maintain a strong focus on the person's activities and their interactions with service touch points during the journey.

5. Ask participants to describe things in the user's terms and language, rather than that of government. If data is available from research, share this and invite people to use it. If not, use team members' knowledge to create a rich, complete picture of a specific user interacting with public or other organisation or service over time.

6. Prompt participants to provide lots of detail, however apparently mundane or unimportant. What is obvious to one person may provoke valuable insights in another.

7. Ask them to identify emotional highs and lows.

8. Get people to identify opportunities for improvements to service touchpoints or other kinds of interaction, or where no touchpoint exists but it would improve the experience.

9. Invite participants to share their journey maps with one another.

10. Discuss themes emerging across all the ideas people have generated.

Journey mapping with interviews and shadowing

You can also map people's journey by shadowing, interviewing and researching the experiences of people and mapping their journeys afterwards. This can help you to map exact experiences of your policy or service.

Other resources

An introduction to evidence safaris

An evidence safari lets everyone involved in a project look at all the data, evidence and knowledge surrounding a policy issue. An evidence safari sets a level playing field and can help a team to see the gaps in their knowledge that may need to be filled with further research, or look at all the evidence available and form a direction for the project or generate ideas in response to challenges. It is also helpful in clearly grounding the project, and other exercises like using personas, in evidence.

This tool works best alongside other tools like challenge cards, hope and fear exercises and journey mapping.

When to use an evidence safari

You could use an evidence safari at the beginning of a policy problem or project or after you have collected together fresh evidence. At the beginning, the 'diagnose' phase, this will help make sure that everyone involved has a similar understanding of a project, and will help bring in new evidence and data. At the develop phase, this will help people discover any fresh insight (e.g. data science, stakeholder interviews or ethnography) gathered and help inspire ideas.

In most cases, you will need to spend some time making sure that there is a large amount of data available to everyone involved in the safari, and that the data is presented in an easy to understand way. This does not mean giving everyone a report for them to read. You should break down the data into quick facts or graphs, photos or quotes. This can take some time.

If you have limited time and data, you could do a low-fi version where you ask everyone in the workshop to write down three to five bits of evidence or tacit knowledge. Then, ask them to group them into different clusters in different areas of the room. And hey presto! A pop-up safari. This is not as robust, but would give you indications of where to go and find the evidence.

How to do an evidence safari

1. Get the evidence

You should begin by collecting all the evidence available. You should break down your evidence into different aspects that you may wish to explore in your evidence safari. These could be:

Official Government statistics and reports

Stakeholder reports

Academic research

Images from a service safari (where you go and experience the service or policy yourself, a bit like mystery shopping)

Quotations from qualitative research

Innovation and ideas (including from other countries)

Media stories and social media views on the policy area

How people are talking about it on Twitter

2. Make the evidence easy and simple to view

Once you have collected all of the evidence you should break it down and easy to read.

When you run your evidence safari it will be placed on walls for people to walk around and explore. You will therefore need to make sure that the evidence is easy to understand, read and view. You should make sure that the font is big enough and each sheet of A4 paper you stick to the wall has one piece of evidence on it. Using different colours to highlight the different evidence themes can help.

Breaking down large amounts of data and turning it into simple and easy to use evidence can take some time, but is essential to a safari working well.

3. Display the evidence

Find a space with enough room for everyone involved in the project and evidence safari to walk around relatively freely. Then stick each sheet of evidence to the wall.

If you have grouped your evidence into themes you will want to use a different wall or part of a wall for each evidence theme. This is so different groups have space to discuss the evidence they are looking at without being crowded out by the next group. You will need to make sure that the themes are easy to see and understand.

4. Organise your workshop

Before you begin make sure you have A3 paper, post-it notes and sharpies.

5. Give people tasks

When people arrive for the evidence safari break them down into groups. These groups will be given an aspect of the policy problem to focus on. You could ask people to look for opportunities and barriers or to examine the experience of a policy. Some example questions are:

What is the current experience of the policy?

What are the drivers of this policy problem?

Who are the people that experience this policy?

Who will be the future users of this policy?

What future demographic changes could impact this policy?

What are other countries / organisations doing?

What are people's views on this policy?

You could also give groups different personas, and ask them to go round and pick out the relevant information that would help to create a user journey for them.

People should then move around the evidence in groups, spending about 3-5 minutes with each theme.

They should collect their findings, views or thoughts on post-it notes or paper as they move.

6. Feedback to everyone

After everyone has looked at the evidence each group should feedback what they found to the everyone else so that knowledge is shared from multiple perspectives. You could use a challenge setting exercise to help people think outside of the box. It might also be useful to focus on what evidence was missing from the evidence safari, as this could be an area that will need more research

Journey mapping and challenge setting exercises can work well after a evidence safari.

Contents

Prototyping: an introduction

Prototyping is trying an idea out to see how it might work, before the pilot stage. Prototypes can take many forms, from simple physical models to role play and more elaborate digital or physical mock-ups. Deciding on the right approach depends on the questions you want to answer, the stage you are at in a project and the resources you have available.

Why you should prototype policy

1. It can save money. Prototyping can spot and fix design flaws that can be costly, even in small scale pilots.

2. It makes abstract concepts visible and tangible in the context of the lives of users. This helps policy advisors to understand what a proposed solution might be like, raising assumptions and issues early on.

3. It gives confidence about the likely benefits and implications of a proposed direction or solution, so that it can be trialled on a larger scale in a more rigorous way.

Examples from around government

In a Home Office workshop involving senior police, academics, civil servants and victims' representatives, groups used simple craft materials, Lego and video to quickly build and share ideas on new ways to report crime. They then combined these ideas into prototypes of solutions, and discussed them to clarify which ones they valued most.

This worked well because the home office was able to try an idea on a small scale.

Building on early rapid prototyping of the first example, the Home Office is now working with Sussex and Surrey Police to try out a new online crime reporting tool and use feedback to improve it.

1. Prototyping national insurance letters HM Revenue & Customs wanted to come with up with new ways of communicating with young people about National Insurance. They ran a workshop where young people re-designed the letters and worked with a group of policymakers, youth engagement specialists and frontline staff to improve these prototypes. The resulting versions of the letter were then presented to and discussed with another group of young people to find out which would be most effective.

2. An example of what not to do. A local authority which spent £200,000 on a recycling pilot that was a failure because people did not understand the uses for the different coloured bags that were provided. If they had tested the bags on a single street first, they could have avoided a costly mistake and redesign the experience to work with the users and avoid confusion and wasted spending.

When to prototype

Prototyping is especially important at the early stage of developing policy, when an issue is not defined, there are many possible solutions, and the cost of changing your mind is low. Prototyping is typically used iteratively, improving continuously, to build up confidence in a proposed solution so that an idea is ready for a larger scale trial.

Prototyping can accelerate a project by putting ideas through their paces by making them tangible and exploring their implications early on and make policy more deliverable by finding out very early on which ideas work or don't.

You should use prototyping to:

get feedback from people who have knowledge of the policy area, especially the lives of people connected with it, including frontline staff

understand what a policy change might be like for the people who will be directly affected by it

build momentum and interest in a policy area and in possible solutions

Issues to be aware of

Prototyping requires some clear thinking about what you are trying to learn. You might start with an idea about how people will respond to the concept (deductive reasoning) and use prototyping to test it. Or you might also have a more general proposal as part of a prototype - "what if the solution was like this?" and be very open to people's different responses.

Prototyping for policy making is different to trialling or piloting. Prototyping is often 'quick and dirty', involves lots of variables and does not produce statistically valid results. Trialling or piloting usually has fewer variables, clear outcome measures, and an experimental protocol to produce reliable evidence, with a sufficiently large sample size. Prototyping builds evidence, momentum and a public interest in the issue. It often generates new insights, whereas trials produce evidence in response to a clearly defined question.

Types of prototyping

There are many types of prototyping that can be used at various stages of policy creation to test and express ideas and solutions. Check them out in the sections below.

Additional resources

Contents

Introduction

A touchpoint is an interaction between a service and a user which could be digital, material or person to person. For example a touchpoint could be a letter, a website or app, a form someone fills in to claim or apply for something, or a session with a care assistant or other service professional. Prototyping takes very little time, uses cheap materials and helps people develop and assess ideas when they are still at an early stage.

Examples from around government

HM Revenue & Customs (HMRC) wanted to improve the way they communicated with young people about National Insurance (currently a letter sent out a few months before someone reaches the age of 16).

A process of rapid touchpoint prototyping took place over a month. Working with a specialist service design agency and Policy Lab, HMRC ran a workshop where young people re-designed the letters. The agency combined these ideas and produced 5 letter prototypes. Another workshop with a group of policymakers, youth engagement specialists and frontline staff reviewed and improved these prototypes.

The resulting versions of the letter were then presented to and discussed with another group of young people to find out which one would be most effective.

When should you use touchpoint prototyping?

Touchpoint prototyping usually happens after you have done some research into a policy area and understood problems from the perspective of people directly affected - service users, frontline staff or volunteers. You might create a user journey map to understand how people currently experience the service and identify high and low points. Through this, you will identify touchpoints that you want to improve and explore ideas about how to do this.

Once you have identified potential ideas for newly designed touchpoints, new touchpoints or new services that are worth exploring in more depth, you can mock them up and test them on service users or others who have first hand knowledge.

This could be a prototype letter, printed screenshots of websites, or a set of role play cards for a new type of conversation between a service user and provider. These tests will provide immediate feedback to use to improve your idea.

How to do touchpoint prototyping in a 1-hour workshop

You will need materials to help people express their ideas. Some examples are:

Lego

small cardboard boxes

sticky notes

marker pens

bluetack

scissors

coloured card

paper

sweets (not to eat, but used as small, colourful items)

You'll also need a space to work where you can stick things on the wall, ideally with several tables and chairs that people can work at in groups of 4 to 6

At the workshop:

give people permission to be creative and work differently with an icebreaker

set a challenge or ask people to define their own challenge

get people to self-organise into teams, ideally with people from different backgrounds

give people a clear timeframe to work within and set expectations about what kinds of things they can make

invite them to respond to the challenge by exploring it from the perspective of people involved; come up with ideas using the materials provided and be ready to present their idea for feedback, for example using simple role play to show how someone would interact with or walk through their model or mock-up, take photos or use smartphones to video each presentation.

get teams to present their ideas one by one, with others giving constructive feedback

you might invite participants to vote about which ideas work best

discuss what ideas seem to work and what you want to take forward

After the workshop:

summarise the ideas generated

share the photos and videos

Contents

Introduction

Experience prototyping is similar to touchpoint prototyping but rather than looking at one touchpoint it involves more than one touchpoint in the user's experience of a policy or service.

Experience prototypes help quickly communicate and explore what an intended future experience could be like, so that it can be continuously improved. They help explore the possibilities and implications of future services which might involve personal interactions or situations in which someone is interacting with many organisational touchpoints.

Prototyping experiences allow a project team to explore, share and refine solutions involving key people such as service users or frontline staff, who have first hand knowledge. They also help build support from other stakeholders, explore assumptions and reveal unintended consequences.

The learning from prototyping helps build confidence for choosing a policy direction which might then later be followed by a larger scale trial, or help avoid costly mistakes by discovering a suggested proposal won't work at later stage.

Examples from around government

A project for Cornwall Council aimed to create new ways to give unemployed people over 50 access to information about local work opportunities. Following interviews with older people who were out of work, but who did not visit job centres, several ideas were generated.

One of these proposed building on the connections that people have with shop staff they see regularly on the high street, adapting the traditional Cornish idea of the 'huer' who announces from the cliff top the arrival of fish in the sea. The service concept was then explored in more depth through experience prototyping.

Designers from agency ThinkPublic created an experience prototype with local pharmacists, bookstore owners, news agents and retailers whose day-to-day work involves significant contact with the public. They were invited to be a new kind of huer. Over a few days, the 'High Street Huers' were asked to identify people looking for work or in need of health care assistance. They were invited to use their interactions with customers or visitors to their shops to talk and to establish the most relevant and useful services for them. Then, if people required more information, the huer jotted down their contact details, and passed them on to the relevant Cornwall Council service provider,to follow up. Through prototyping this experience on a high street for a few days, the project team identified which of the high street huers were trusted as intermediaries for these kinds of conversations about public services.

Additional resources

Contents

Sketching: an introduction

Asking policy makers to sketch their ideas disrupts conventional ways of working where people rely on words.

Asking people to sketch things instead of writing brings out new ideas that are more easy to understand and improve. Research into how designers go about their work shows the act of sketching or making is where ideas emerge.

Examples from around government

Policy Lab has helped civil servants and users to use sketching to come up with new solutions to the lack of new homes being built.

HMRC has organised collaborative with involve young people that used sketching to redesign the letters that give them their National Insurance number. They also used sketching to generate ideas for other parts of their lives (like school) where they could learn about the importance of National Insurance.

When to use sketching

The main reasons to do sketching are:

to generate lots of ideas fast

to create a broader range of ideas, including some unlikely ones that may help you reframe current understandings of the issue

to create a shared understanding of the opportunities quickly

to get and keep people engaged in a project

to broaden the range of participants able to contribute to solution finding

How to use sketching

1. Ask people to sketch touchpoints - points at which users, frontline staff, or other people involved such as family members or volunteers, interact in relation to the issue you are considering. They don't have to just be from government or public services.

2. These touchpoints could be:

web pages

leaflets

smartphone apps

emails and text messages,

signage

products, packaging

media, eg websites, TV programmes or adverts

environments and places such as homes, offices, clinics, cafes, schools or shops public spaces such as parks, sports or community centres, bus stops or trains

mundane things in the built environment such as posters, signs or bins

journeys or cartoon strips

Probably many or most of the participants in a workshop will not be that good at drawing, which helps to break down hierarchies. Keep the focus more on what people draw, rather than the quality of the drawing and there's more room for surprise.

Contents

Social media can help you to understand people's views and needs, and work with them to design policy.

Working with people through social media can help you to connect with their opinions and existing experiences. You can use it to crowdsource ideas and quickly test how ideas will be received and used in the real world.

Social media can be very useful for a quick link to the people your policy will impact. Running ideas, questionnaires or consultations through social media can significantly increase the number of people you reach.

Remember that by choosing to work with people on social media you are choosing to work with a small selection of the audience.

Social media analysis is helpful for analysing your engagement or finding existing data from social media platform.

the NHS Constitution consultation used social media to spread the word about changes to the NHS constitution

the Public Sector Efficiency Challenge used social media and crowdsourcing to share ideas about where money could be saved across the civil service

UKTI's Export Jam used social media to ask people what their experience of exporting was and for their ideas to improve it

the Government Digital Services and HM Revenue & Customs uses Twitter to help people when they have a problem with a service or policy

The Government Digital Service Social Media Playbook is the government guide for using social media. It outlines the available platforms and the ways that government should speak and use them.

Choose the most appropriate channel

Combine a range of different channels depending on who you need to reach.

You may wish to use a traditional open social media site like Facebook or Twitter, or something more closed like Hackpad or LinkedIn groups. You should find one that works for you and your users.

Remember this is a dialogue not a monologue

You should not simply broadcast what government wants people to hear. Any social media campaign should create a dialogue where government responds to individual people.

Social media is a democratic forum and people may want to express their displeasure with the government or a particular policy.

Influencers

Find the influential people who are already talking about your area on Twitter. Follow them and talk to them to see if they will help you increase debate around your crowdsourcing project.

Provide relevant information

Tweet useful information to get the conversation flowing. Infographics, pictures and links to relevant articles will make people more likely to read your tweets and more likely to engage with your question because they have some background data to work with. Ask them to share their information as well.

Closed/open

Most social media is open about the ideas and conversations taking place.

For some specialised or sensitive discussions you might want to use a more closed environment. You can use a members-only groups on LinkedIn, Slack and Hackpad to let people to discuss things which might be commercially sensitive or that they might not want to share openly.

On very sensitive topics such as sexual assault, you may need to let people to comment anonymously, ask them if you can use their contribution anonymously and remove any identifying features from any information they allow you to publish.

# data[].attributes.attachment_link.description
Acting is a way of testing new policies and ideas for quick feedback and help. This can be done by organising workshops where policy ideas can be tested by people acting out the proposed idea, and then a discussion taking place around the scenes being acted.

# data[].attributes.attachment_link.readable_text
So you're organising a hack day or hackathon? Here are some basic requirements to make your event a success, and avoid the common pitfalls that could otherwise ruin it. If you cannot provide any of the following, make it clear to guests before registration. Attendees are generally forgiving when clear communication is given. These are only guidelines, but exist to help you run a hack day in the easiest possible way.

Once you know when and how your event will take place, you'll want to tell the world about it. At a bare minimum, you should decide on a canonical place where all public information about the event lives - this might be a dedicated web site, an event on an existing event online service or some other place which is publicly accessible.

Once you've decided where that location is, use tools like Twitter and Facebook to make people aware of the event, and also consider which Google Groups and mailing lists developers relevant to your event may be hanging out. Don't spam them, though - nobody enjoys that.

Registration Optional

If you require your attendees to register, make it clear what information is required for signing up and what the deadline for the registration is. If there's a maximum number of attendees the venue can handle, communicate how the selection process works and when/how people will be notified if they've been granted attendance or not.

If there's a waiting list for the event, make sure to explain how it works.

Most registration forms lets applicants enter name, contact details and food preferences/allergies (if applicable). Depending on your event and your venue, you might want to ask for additional information - this manifesto lists a few other topics which might make sense to add to the registration form.

There are also a number of things you probably should not ask for, like gender, religious preferences etc.

Location

The venue should be relatively easy for people from outside of town to locate, with good public transport links. If it's difficult to reach, try to provide alternative means of transportation, such as coaches to/from local transport hubs throughout the event. Provide a full address, and if necessary, additional instructions to all attendees well in advance of the event.

Include instructions/contacts/getting in arrangements, too (i.e., what to do at reception/security desks).

Print big signs that will guide your attendees to the venue (and in some case inside the venue).

Accessibility

Hack day venues should be accessible to everyone: at a bare minimum, there should be step-free access to all common parts of the building. If an attendee is blind or deaf, they should be able to participate too. Make sure your event signup forms mention accessibility and ask what accommodations might be needed.

Insurance

Does your venue have the relevant insurance? Are guests' belongings protected in case of fire or theft? If not, you should make them aware of the risks so they can take extra precautions.

Do you, as an organizer have adequate insurance?

Security

Attendees will often be bringing a lot of expensive kit: laptops, tablets, etc. Make sure they have some security. If the venue does not have some form of lockers, can you run a cloakroom system?

Secure sites and photo ID

Are the rooms going to be available to non-attendees? Work out whether you are running a secure site and whether people need to bring photo ID; if so, tell them when they sign up and remind them close to the event date. For some people, showing photo ID may be a deal breaker. Not everyone will have 'government issued ID', either. Providing proof of address may be hard for some.

Weekend or Not?

Most hack days are held at the weekend as this allows people to attend without taking time off work. If you are planning a hack day that begins on a Friday, remember that this may limit the number of people willing to travel after work.

Checking for clashes

One of the most frustrating things for attendees to see is two similar events on the same day in the same area. To avoid this, check places like Lanyrd, Eventbrite, Meetup, and ask on Twitter "is anything going on in X on X?". Remember that people may be travelling long distances for hack days, so even if an event is a few hundred miles away, you are still diluting your potential audience.

Sleeping provision

If attendees are staying overnight, then a separate (dark, quiet) area should be available away from the hacking should people decide to sleep. If possible, this should be several areas potentially including dedicated areas, for example male/female/mixed, minors (+chaperones?), snorer/non-snorer, night-owls/early birds.

Showers Optional

Despite the stereotypes, most geeks like to keep clean. Only a few venues are able to provide showers - but if you can, you should. Make sure you tell attendees well in advance so they can bring towels and toiletries.

Your 4MB DSL isn't enough

Hack days have special requirements: don't just trust anyone who tells you that "it'll be fine". Think about the networking issues, and verify that they work for the kind of capacity you are going to have. People from the venue or their commercial partner will tell you all sorts of things you want to hear but keep in the back of your mind that they may not have any clue what they are talking about. Given the importance of network access, if you are operating a commercial event consider requiring network performance as part of your contract with venues and suppliers.

Rock solid WiFi

Many commercial WiFi providers plan for much lower use than actually occurs at hack days. The network should be capable of handling at least 4 devices per attendee.

Minimal firewalling

As a minimum: ports used for SSH (22), DNS (53 - TCP & UDP), VPN (47, 500, 1701, 1723, 5500), HTTP (80), NTP (123 - UDP), HTTPS (443), Submissions (587), IMAPS (993), MQTT (1883), CVS (2401), Subversion (3690), XMPP/Jabber (5222), IRC (6665-9), and Git (9418), must be open and accessible, without the need of proxy servers, and preferably without perimeter-level malware checks.

Subnet / DHCP capacity

Subnet and DHCP server must have capacity for allocating 6 IPv4 (and optionally, IPv6) addresses (preferably a /29) per hacker.

If NAT is not used, attendees should be informed in advance.

WiFi security

Use WPA2 security over an open WiFi network, to offer some basic protection against the monitoring of other users' network traffic.

No device isolation

Attendees should be reminded that when connected to the network, their devices may be accessible to all other devices on the network so they would be wise to secure them.

Organisers should make it clear that they absolve themselves of liability for any intrusion or damage caused over the network. Attendees will want to share resources with other attendees (version control, for example) but while it would be wise for them to do so in a controlled manner it is ultimately up to each individual to be responsible for the security of their own devices, connected or otherwise.

Attendees should be encouraged/advised to run their own firewalls, too, remembering that IPv6 exists.

IP Whitelisting

For events with 150 attendees or more, you will need to contact organisations such as Twitter, Google, and Freenode in advance to inform them of the increased number of connections from your IP range; they may have an existing process that you should use.

In many cases, rate-limiting is handled by authentication, not IP address (ranges).

For Freenode, email iline@freenode.net with dates, expected number of attendees and IP addresses if possible.

QoS and Monitoring

Traffic shaping and monitoring should be employed to help prevent a few users abusing the connection to the detriment of all others.

Deep packet inspection should not be used.

Ethernet to the Chair Optional

In case of WiFi collapse (or if you can't provide adequate WiFi at all) you should have ethernet available at the chairs. Have at least 100 megabit ethernet feeding into a gigabit backbone arranged in an (extended) star topology. If you go ethernet-only, announce this up-front. Regardless if you provide cables or attendees bring their own, you should have a big pile of spare cables at the venue.

Single subnet Optional

Both WiFi and ethernet should share a single subnet, allowing devices to communicate regardless of how they are connected to the network.

Power sockets

You will need a minimum of 1.5 power sockets available for every seat. That's a minimum; many attendees will bring two, three, or more devices that require separate power. Have spare cables available. Make sure you have access to the fuses of all used power circuits, and review power requirements with the venue.

Portable Appliance Testing

As stated above, your attendees will be bringing two, three, or more devices making full PATs for every device an impossible endeavour. But if you are providing a lot of power strips it can be worth getting them tested.

Isolated power for hardware hacks Optional

Isolating the power for those doing hardware hacks is generally a good idea, don't let one soldering iron knock out the power for the entire venue.

Dietary requirements

Not everyone in the technical community is hyper-carnivorous. Be sure to check with your attendees for dietary requirements: food allergies, vegetarians, vegans and people with dietary restrictions. Make provisions to ensure they are provided for equally. If you're on a budget, prioritise allergies and vegan alternatives; the vegan alternative will satisfy most non-allergy based requirements. Common food allergies include milk, eggs, nuts, fish, shellfish, soya, and wheat (gluten).

It's important to note that that vegan meals exclude all animal products, not just meat, and therefore it should exclude eggs, cheese, milk, yoghurt, honey and much more.

Breakfast

If your event starts before 11am you should provide breakfast to attendees, either continental or cooked.

Lunch

You should provide a lunch for each day of the event, this should consist at least of sandwiches (or suitable alternatives for gluten intolerant folk).

Dinner

If your event runs later than 6pm or overnight you should provide attendees with a proper evening meal, not sandwiches or snacks (and not everybody likes/eats pizza).

Overnight

If your event runs through the night, you should arrange for food to be available throughout the night, either by periodically refreshing it or supplying something that can still be eaten after it has gone cold.

Drinks

A selection of coffee, green and black tea (plus milks, sugars), energy drinks, caffeinated and non-caffeinated soft drinks, juices and water should be freely available to attendees throughout the event.

Snacks

Chocolate, sweets, biscuits, fruit, crisps, etc. should be freely available to attendees throughout the event. Try to ensure there are healthy options too. Fresh fruit, nuts and vegetable platters can provide sustainable high energy levels to developers through out an event.

Alcohol

Should be served as if you are the owner of your local bar because, in fact, you are. Check licensing rules with the venue, including where and when alcohol can be served and consumed and what time you need to stop serving. If you have underage attendees, make sure they're not served alcohol. Penalties for failing to do this could range from a fine to imprisonment for the person providing alcohol and loss of licence for the venue.

Presentations

Presentations by API and dataset providers should be kept to a minimum, try to make it a quick introduction involving the company name, the type of data / platform they have, and who to speak to for more information. Have technical - not marketing - people give the presentation as they will be asked technical questions, both on stage and during the rest of the event.

Working APIs

The APIs pitched to attendees should be checked in advance by someone technical to confirm that they are suitable, work, and are of a high enough quality.

Multiple clients

All APIs should provide client libraries in at least three major languages unless there is good reason for them not to (such as the target platform only supporting a single language).

(And if you haven't got the client libraries written, maybe you need an internal hack day to write them!)

Clean datasets

Datasets provided for use in the hack day should, where possible, be checked in advance by someone technical to ensure they are well structured and in a sensible format (CSV, XML, JSON, RDF etc., rather than Excel spreadsheets and PDFs).

Thorough documentation

All APIs and datasets should have thorough documentation, detailing format, any codes or abbreviations, as well as the underlying assumptions and business logic of what it represents. The documentation should be checked by someone technical before announcing the API.

Local downloads

Datasets -- especially large datasets -- should be available over the network from local servers without the need for registration, and where possible available on USB keys for even faster transfer.

Make data explorable

Try and ensure that large datasets are accompanied by smaller sample files so attendees can check them for suitability and write parsers using a small number of records before downloading and importing multi-gigabyte files.

API Keys

If an API requires a key to use it, then make sure that someone is on site and able to authorize the requests quickly, and with as little fuss as possible.

Licensing

Get clear licensing information from API and dataset providers ahead of time, and communicate those licences clearly to the attendees, remembering that those in the technical community often have strong opinions about open licences for data and code.

Longevity

Attendees are often averse to building on top of platforms and datasets that could disappear or will incur usage fees after an event, try to avoid companies making resources available only for the duration of the event and communicate clearly the risk of building on top of any alpha or beta APIs.

Hack submissions

If possible, have a shared, public listing of the hacks that have been submitted. This helps the judges, the attendees and bloggers/journalists who are covering the event. Hack submissions should be as straight forward as possible. You can use Google Docs, Etherpad or a wiki. Make sure that all attendees have write access and can concurrently edit. Plan for the hack listing to stay around as a historical record. Alternatively, there are hack submission services that will handle the logistics for you like Hackathon.io, WeHack.it, or Devpost.

Anyone who hacks should be a allowed to demo at the end of the event, regardless of the quality or completeness.

Time limit

Each demo should be given a fixed time limit, standard times are 60, 90, 180, and 300 seconds. Tell presenters ahead of time, let them know how much time remains (either half time cards or an on-screen count down), and don't let them run over.

Clear expectations

Try and communicate clear expectations for the demos to all attendees from the beginning of the event. Some attendees will become frustrated when they see others demo-ing paper prototypes or Photoshop mockups when they believed a working implementation was required. If hacks do not meet these base requirements, they should not be able to win a prize.

Audience vote

If you have an "audience choice" prize, someone will attempt to game the system, either technically or by running an obnoxious "vote for me" campaign. Keep the prize for it relatively small, and give attendees a one-time token (a code on the reverse of their pass for large events, or a physical item for smaller ones) in order to vote.

Judge selection

The judges for your hack day should reflect the nature of your event. For standard hack days, judges should be primarily technical or knowledgable in the relevant thematic area (eg. musicians judging Music Hack Day). Try to avoid having businesspeople (VCs, non-technical founders, journalists, etc.) judge technical events, as it encourages hackers to build prototype businesses, not interesting hacks. Your judging panel should be chosen to allow participants to win a hack day based purely on technical merit, even if the hack is completely impractical as a business.

Judge previews Optional

If possible, let the judges meet all the teams for a few minutes, a couple of hours before the presentations. This means people don't need to cram their whole idea in 60 seconds, but instead can demo the product to their peers knowing the judges understand how technically awesome their product is. This also allows for judges to fully inspect if a product was actually implemented, or is just a bunch of screenshots - and how technically advanced their project is.

Support

There should be staff on hand to help users set-up quickly for their demos, such as connecting to the projector. If the equipment at the venue allows it, have one demo presenting while the next is setting up.

Video connectors

For the demos, either have a set of adapters handy for (Apple Mac) Thunderbolt, Mini DisplayPort, VGA, DVI, and HDMI video outputs, or make sure attendees know the projector's inputs ahead of time so they can make sure they have the relevant adapters.

Projector info

Information on projector resolution, contrast, etc should be provided to attendees and speakers ahead of the event.

Hardware and mobile hacks

If mobile or hardware hacks are expected, equipment to demonstrate them on the device should be present. Have a camera feed or visualiser available on the big screen. State ahead of time if that'll be provided, to allow these to be demoed adequately.

Not every hack day needs prizes

Some people go to hack days to learn, or to play. Having one team go home with enormous prizes and many other teams go home with nothing may sour the end of the event. Make sure it's not all about the money/toys.

Prizes per team member

Prizes must be divisible to encourage teamwork: it is very hard to slice an iPod into three bits. If you wish to give away physical goods there should be enough that they may be given to each member of a reasonably sized team.

Available on the day

Prizes should be on site on the day of judging and ready to be handed to attendees as the winners of the various prize categories are announced. Attendees should not need to claim a physical prize after the event unless said prize is dependent on winner. If the prize is not available on the day, clear printed instructions on how to claim must be provided instead.

No cost to claim

Attendees should not be required to pay a subscription or sign a contract in order to claim or make use of their prize. Requiring prizewinners to do so is likely to induce negative feelings in the winner towards the company providing the prize and the hack day organisers.

Avoid service prizes

Discounts and credit for an API, face time with a company's CEO, or development resources to continue a hack make for poor prizes (with obvious exceptions such as of companies offering music/video streaming services or online stores) and are likely to just disappoint the winning team. When organizations/individuals wish to pursue further development of a hack they should speak to the winning team separately.

Don't make people feel unwelcome

Avoid sexism and other discriminatory language or attitudes. Don't make any assumptions about your attendees. Get someone who is demographically very different from you to check your marketing material through to see if it makes sense and isn't offensive to someone who doesn't share your background.

Many event attendees expect events to have and enforce a code of conduct. Having such a code encourages diversity of participation and can help to create and maintain a welcoming environment. It is highly recommended to adopt a code of conduct for your event. One such example is the Hack Code of Conduct, and there are several other appropriate examples. You can either reference your chosen code of conduct directly or create your own customized version. Make sure to communicate clearly to attendees ahead of time the existence of this code and the enforcement policy behind it. In addition, the code should be linked prominently from the event web site and promoted by the event organizers and by members of the event's community. The code should be known to and practiced by event staff and volunteers and enforcement procedures should be clear.

Make assumptions around privacy clear

Tell people up front what level of privacy they should expect during the event.

If you allow people to take photographs, tell photographers to respect people's privacy if they do not wish to be photographed: if someone objects to being photographed, they have a right to have that objection taken seriously.

Sleeping areas should be explicitly marked as no photography as sleeping people are unable to implicitly consent to being photographed by not objecting.

Do not charge attendees

If developers are willing to donate their time -- which could otherwise be used for paid work or hanging out with family and friends -- they should not be charged for tickets. It shows a complete disrespect for the value of their time, and is likely to leave a bad feeling towards the organisers and sponsors. The developers are providing the sponsors a service in testing and showing the possibility of their APIs, data, products and services.

Intellectual property

The intellectual property rights of hacks produced at the event should belong to the attendees, and not to the organisers, sponsors, or partners, with the obvious exception of (a) hack days based around the further development of an open source project, or (b) commercially run hacks.

Intellectual property can be a big issue; it is best to make expectations clear in advance, both to developers and to sponsors or data providers.

What are you after?

Have some idea what counts as success, but don't be too tight: people will surprise you! Some hack days are specifically focused on one language or tool, others on a theme, while others are meant for a particular open source project or problem that needs solving. The point of the event will inform how you do marketing, judging and many other aspects of the event.

This is not a rock concert!

Some hack days have hired bands or other entertainment. This is usually a bad idea. People are at the event to hack, and hacking requires extended periods of concentration. You don't need silence, but if you are thinking about putting on entertainment you may not understand exactly what the point of a hack day is. If you do insist on putting noisy entertainment on, don't be surprised if the people actually trying to build things go and hide in the lobby to carry on with what they are doing.

Emergency plans

Make sure you have a plan for attendees who are injured, fall ill or suffer any other emergencies. Have at least one designated first aider on call. Make sure you know where the first aid box is, and check it's appropriately stocked. Know the procedures for calling an ambulance, sounding the fire alarm, evacuation point, and logging incidents in the venue. Make sure attendees know who to turn to for help; a phone number, a few bright crew shirts and a central crew desk or wardrobe go a long way. A central point of contact will also help you deal with inevitable electric, network, venue- or food-related questions.

Handling problematic people

It's unlikely that you'll have to throw someone out, but just in case know what the procedures are, what you're legally able to do and what kind of misbehaviour would actually trigger ejection. Some events have instituted safe space and anti-harassment policies due to widely reported incidents of sexual harassment.

Keep your crew safe too

Working at a hack day, as a volunteer or otherwise, can be a stressful and demanding job. Make sure your crew are safe, happy and well-fed throughout, and that you've got enough staff to cope with emergencies or unexpected dropouts!

Finally, remember to have fun!

If you follow all these guidelines, prepare well in advance, and get plenty of rest before the big day, you'll maximize your odds of having an event that's not just successful, but enjoyable as well. Good luck!

# data[].attributes.attachment_link.description
So you're organising a hack day or hackathon? Here are some basic requirements to make your event a success, and avoid the common pitfalls that could otherwise ruin it. If you cannot provide any of the following, make it clear to guests before registration. Attendees are generally forgiving when clear communication is given.

# data[].attributes.attachment_link.readable_text
We've written about delivery-driven government as a large-scale vision of how all governments, from city hall to the nation's capital, could not just modernize their technology but build policy to meet user needs. Delivery-driven government in all levels of government is what we're striving for, but we're already seeing amazing examples of it working on the ground.

At Code for America Summit, Ashley Meyers and Maria Benjamin took the stage to tell us their story of delivery-driven government in action: how a new digital service changed a government team's approach to policy itself. Ashley is a product manager with San Francisco Digital Services, and Maria is the Director of Homeownership in the Mayor's office of Housing and Community Development.

"We built a new digital service - but more importantly, we started changing policy to serve user experience."

It's no secret that there is a shortage of housing in San Francisco. The city has built policies that give preference in affordable housing lotteries to vulnerable populations, and wanted to build a digital service to implement changes to get people into those units. They also took that opportunity to redesign the entire experience of applying for affordable housing, away from loads of paperwork (and paper raffle tickets!) to a user-friendly, streamlined digital experience - one that saw 86% digital take-up in its very first pilot. In doing so, they were able to examine what was working in delivery and what wasn't, make changes to the policies themselves to address those issues.

"What we're able to do is take agile development and not just use it for web-based products but use it for policy and procedure to see if it's working for the people."

This adaptable, flexible approach only became possible through culture change - a shift to the idea that they needed to not just make things easier for government staff but for the people they serve. They took the new system and used it as a catalyst to change policies and procedures that had been in place for an incredibly long time. And now that the culture has shifted, their sights are set even higher: on using the delivery data to inform the actual housing they build.

To hear more about the project, watch the video below.

# data[].attributes.attachment_link.description
We've written about delivery-driven government as a large-scale vision of how all governments, from city hall to the nation's capital, could not just modernize their technology but build policy to meet user needs. Delivery-driven government in all levels of government is what we're striving for, but we're already seeing amazing examples of it working on the ground.

# data[].attributes.headline
Towards ethical principles for participatory design practice

# data[].attributes.subject
Towards ethical principles for participatory design practice

# data[].attributes.attachment_link.readable_text
ABSTRACT
The aim of this paper is to identify the kinds of ethical issues practitioners have to consider in participatory design (PD) practice and use these to inform a set of ethical principles, which can be used to guide practice. It begins by discussing the ways in which PD, as a discipline, has engaged with ethical issues in the past and looks at how other disciplines have identified ethical principles to support ethical practice. It, then, draws upon interviews with researchers who have the experience of collaborative and PD projects to identify ethical considerations that they have encountered in their work and uses these to propose a set of ethical principles. Finally, it ends with a discussion on how a principles-based approach can be combined with other approaches, such as the cultivation of virtues, to support ethical practice in PD.

Acknowledgements

I would like to acknowledge Dr Alison Hay and Angela Tulloch for their input and support during the writing of this paper; also, in particular, I would like to acknowledge the contribution of the design researchers who shared their experiences with me for this research.

Disclosure statement

No potential conflict of interest was reported by the author.

# data[].attributes.attachment_link.title
Towards ethical principles for participatory design practice

# data[].attributes.attachment_link.description
ABSTRACT The aim of this paper is to identify the kinds of ethical issues practitioners have to consider in participatory design (PD) practice and use these to inform a set of ethical principles, which can be used to guide practice.

# data[].attributes.headline
Creativity-based Research: The Process of Co-Designing with Users

# data[].attributes.subject
Creativity-based Research: The Process of Co-Designing with Users

# data[].attributes.attachment_link.title
Creativity-based Research: The Process of Co-Designing with Users

# data[].attributes.attachment_link.description
Co-design workshops help designers partner with users to include their perspective in knowledge development, idea generation, and product development.

# data[].attributes.attachment_link.description
Edinburgh Living Lab is a data and co-design research lab. We bring together researchers and citizens to use data-driven design thinking on city challenges.

# data[].attributes.headline
Co-creation vs. crowdsourcing: Which drives greater business value?

# data[].attributes.subject
Co-creation vs. crowdsourcing: Which drives greater business value?

# data[].attributes.attachment_link.readable_text
The time has come for the world's innovative brands to reach beyond crowdsourcing and fully embrace co-creativity.

A new era of customer engagement and participation has dawned and co-creativity is at the heart of this highly-collaborative business model. While crowdsourcing may have decentralized some aspects of marketing and innovation through open calls to the public, co-creativity has actually democratized the relationship between brands, employees and customers by building a trusted community of collaborative peers.

Co-creation is an ongoing, collaborative process between a company's employees and customers. It is the key component of participation marketing, and involves a social collaboration between a group of people.

But to the uninitiated, it can sound a lot like crowdsourcing.

It's not. With crowdsourcing, a company poses a problem to the public hoping it sends back a viable solution. And while 'crowdsourcing' might still garner some buzz, especially with its recent foray into TV via CBS' Wisdom of the Crowd, its long-term value and benefit to brands and consumers alike is fleeting.

On the flip side, co-creation requires building a community of a brand's most enthusiastic customers and employees, and regularly interacting with this community of peers to innovate and solve problems together. By collaborating on the regular, both brand loyalty and advocacy are ignited and maintained for a much longer time.

We could sum things up by saying crowdsourcing is an "us and them" project, while co-creation is a "we" process; however, we thought a more detailed, visual analysis was in order.

Check out our infographic below and discover more of the key difference between co-creativity and crowdsourcing. After all, it's the difference between transforming a 'crowd' into a devoted community of passionate, creative consumers vs merely reaching out to an already very crowded and noisy online space.

# data[].attributes.attachment_link.title
Co-creation vs. crowdsourcing: Which drives greater business value?

# data[].attributes.attachment_link.description
The time has come for the world's innovative brands to reach beyond crowdsourcing and fully embrace co-creativity. A new era of customer engagement and participation has dawned and co-creativity is at the heart of this highly-collaborative business model.

# data[].attributes.attachment_link.readable_text
Card sorting is a method used to help design or evaluate the information architecture of a site. In a card sorting session, participants organize topics into categories that make sense to them and they may also help you label these groups. To conduct a card sort, you can use actual cards, pieces of paper, or one of several online card-sorting software tools.

Benefits of Card Sorting

Card sorting will help you understand your users' expectations and understanding of your topics. It is often most useful once you have done some homework to find out about your users and understand your content. Knowing how your users group information can help you:

Build the structure for your website

Decide what to put on the homepage

Label categories and navigation

Open and Closed Card Sorting

Depending on your needs, you may choose to do an open or closed card sort. They differ as follows:

Open Card Sort: Participants are asked to organize topics from content within your website into groups that make sense to them and then name each group they created in a way that they feel accurately describes the content. Use an open card sort to learn how users group content and the terms or labels they give each category.

Closed Card Sort: Participants are asked to sort topics from content within your website into pre-defined categories. A closed card sort works best when you are working with a pre-defined set of categories, and you want to learn how users sort content items into each category.

You may also choose to try a combination of the two. You could conduct an open card sort first to identify content categories and then use a closed card sort to see how well the category labels work.

Choosing a Technique

One on Ones are in-person sessions with an observer. Participants think aloud while sorting, giving a clearer picture of their reactions and thought processes.

This type of sort may be completed with physical cards or with online card-sorting software and the facilitator looking on and asking questions as needed.

n/a

Group / Independent - Concurrent In-person Sessions have participants sort a set of cards independently. The facilitator may brief the participants at the beginning and debrief the participants at the end, but the participant works alone for most of the session.

Because of the limited interaction, you can have many sessions at the same time with one facilitator.

You must have as many sets of cards as concurrent sessions or have each participant at a separate computer if using online card-sorting software.

Group - Concurrent In-person Sessions have participants sort a set of cards as a group. The facilitator may brief the participants at the beginning and debrief the participants at the end, but the participant together for most of the session.

Working collaboratively, or as a team, may quickly bring about grouping and labeling of main content areas.

Group dynamics might come into play and should be monitored.

Remote, Computer-based Sessions require participants to work independently. Participants sort the cards independently on their own computers. You can do open or closed card sorts remotely. Several software programs exist to help you with large-scale remote card-sorting studies. Using the software is an advantage because it analyzes the data for you.

Allows you to have many participants in many locations.

You do not get information on why participants sort the cards the way they do, because you cannot see the participants or hear them thinking out loud.

Best Practices for Card Sorts

Limit the number of cards. It is tempting to want the participant to sort "ALL" of your content, but be mindful of participant fatigue. We would recommend 30 to 40 at the absolute outside, especially for an open sort.

If possible, randomize the order of presentation so that each piece of content has a chance to be sorted earlier in the session.

Provide the participants with an estimate of how long the card sort will take before beginning the session to help them better gauge the required time and effort.

Consider the benefits of requiring participants to complete your sort. For an open sort, if possible consider requiring them to sort the cards, but perhaps not to label them, since that might be the more challenging part of the task, providing you have limited your items as suggested in point 1)

Consider an open sort as part 1 and a closed sort as part 2 of your process. One allows you to learn what goes together, while 2 allows you to really test out your labels to see if they are intuitive to your participants.

How to Conduct a Card Sort

Prepare the cards

1. Create your list of content topics. Topics can be phrases or words, very specific or more general. As a suggestion, limit yourself to 50-60 topics or less. This means there might not be a card to sort for every page on the site.

For a new site, list the content topics of types of information that you are likely to have on the site

For an existing site, list the most important / popular types of content

To create this list:

Review the content listed in your content inventory.

Identify the most important or most frequently used content

2. Decide whether you will be doing a physical card sort or using online card-sorting software.

If you are using online card-sorting software, consult the software instructions.

If you will be conducting a card sort using physical cards, write each topic on a separate index card.

Use self-adhesive labels and a word processor. The cards will be neat, legible, and consistent. You'll have the list of topics in the computer for later analysis.

Number the cards in the bottom corner or on the back. This helps you when you begin to analyze the cards.

Have blank cards available for participants to add topics and to name the groups they make when they sort the cards.

Consider using a different colored card for having participants name the groups.

Set-up the session

1. Plan about one hour for each session, longer if you have many cards.

2. Arrange the space.

For paper card sorts, ensure the participant has enough room to spread the cards out on a table or tack/tape them up on a wall. A conference room works well.

For online card-sorts, ensure there is a computer with an internet connection available as well as room for both the participant(s) and facilitator to sit comfortably.

3. Plan to have the facilitator or another usability team member take notes as the participant works and thinks aloud.

4. As with other techniques, arrange for payment or other incentives to thank the participant for spending the time and effort helping you.

Lead the session

1. Show the participant the set of cards. Explain that you are asking for help to find what categories of information should be on the site's homepage and what those categories should be called.
In an open card sort, explain that you want to see what groupings of cards make sense to the participant, and that you will ask for a name for each group of cards once the participant has grouped them.
If you are conducting a closed card sort, explain that you want to see how the participant thinks the cards fit within the defined groups.

2. Ask the participant to talk out loud while working. You want to understand the participant's thoughts, rationale, and frustrations.

3. Let the participant work. Minimize interruptions but encourage the participant to think aloud. Allow the participant to:

Add cards - for example, to indicate lateral hyperlinks or additional topics.

Put cards aside to indicate topics the participant would not want on the site.

4. If, at the end, the participant has too many groups for the homepage, ask if some of the groups could be combined.

5. Ask the participant to name each category.

In an open card sort, give the participant a stack of different colored cards. Ask the participant to use the colored card to name each group. Ask what words the participant would expect to see on the homepage or second-level page that would lead the participant to that particular group of content items.

In a closed card sort, asking about word expectations, their final card organization, and other follow up questions can provide valuable insight and observations for your research.

6. At the end, thank the participant and give the payment or other gift if promised.

Remote sorting sessions

1. Create your list of content topics. Topics can be phrases, words, etc., and can be very specific or more general. It might be tempting to have a card for every topic on your site, but in this case, more might not be better. Consider the cognitive load on the participant. You want them to be as on task for your first card as your last. As a suggestion, limit yourself to 50-60 topics or less.

2. Prepare the cards according to the software instructions.

3. Email your participants a link to the study. Provide instructions for the sort (whether open or closed) and let them know approximately how long the session should take to complete.

4. If a comment box is available, urge participants to use the field to record any observations or questions. While you will not be able to answer them in real time for the participant, these comments can be useful for your analysis.

5. Thank the participant for his or her time and provide instructions for receiving payment or other gift (if promised).

Analyze Your Data

1. Prepare your data for analysis.

If you used online card-sorting software, consult the software instructions. The software will analyze participant data in a variety of ways.

If you used physical cards for the test, either photograph the sort or use the numbers on the cards to quickly record what the participant did. Photograph or write down the names the participant gave to each grouping and the numbers of the cards the participant included under that name. Then you can reshuffle the cards for the next session.

Create a computer file for each session to gather a complete picture of the detailed site map each user creates.

Work from your original list of topics and move topics around to recreate each participant's groupings and enter that participant's name for the groupings.

If you used a physical card sort, you can also take a photograph of the finished card sort for reference later.

2. Analyze qualitative information based on user comments.

3. Analyze quantitative information based on:

Which cards appeared together most often

How often cards appeared in specific categories

4. For a less detailed analysis of the results, use your notes and recordings of the participants' names and card numbers under each person's name to find commonalities from different sessions.

5. For a more detailed analysis, consider using an Excel spreadsheet to show the relationship between the cards or use one of the available software programs to analyze your data.

6. Pull together your findings in a report to share with your team and stakeholders.

After you analyze the data from card sorting, you should have useful information for structuring the information architecture of the site. You should use the results of your card sort to help you define the navigation of your site.

Best Practices for Card Sorts

Limit the number of cards. It is tempting to want the participant to sort "ALL" of your content, but be mindful of participant fatigue. We would recommend 30 to 40 at the absolute outside, especially for an open sort.

If possible, randomize the order of presentation so that each piece of content has a chance to be sorted earlier in the session.

Provide the participants with an estimate of how long the card sort will take before beginning the session to help them better gauge the required time and effort.

Consider the benefits of requiring participants to complete your sort. For an open sort, if possible consider requiring them to sort the cards, but perhaps not to label them, since that might be the more challenging part of the task, providing you have limited your items as suggested.

Consider an open sort as part 1 and a closed sort as part 2 of your process. One allows you to learn what goes together, while 2 allows you to really test out your labels to see if they are intuitive to your participants.

References

# data[].attributes.attachment_link.description
Card sorting is a method used to help design or evaluate the information architecture of a site. In a card sorting session, participants organize topics into categories that make sense to them and they may also help you label these groups.

# data[].attributes.headline
Joint decision making, co-production and democracy

# data[].attributes.subject
Joint decision making, co-production and democracy

# data[].attributes.attachment_link.readable_text
About Joint decision making, co-production and democracy

Wellbeing in practice can be:

1. A goal for society or progress like the HMT Green Book describes wellbeing as 'social welfare'.

2. A framework, often based ongoing conversation about what matters, using outcomes and indicators.

Personal agency, power, responsibility and control are a part of our wellbeing: freedom to choose what we do in our life is second biggest driver of wellbeing. How organisations work with people to feel control and agency over their lives and communities can make a difference.

Joint decision-making encompasses or overlaps with concepts like democracy, devolution, open policy making, community empowerment, co-creation, co-production, inclusion and diversity, voice and accountability, rights, power and responsibilities, agency and control.

Mar 17, 2022 | By Nancy Hey

Mar 10, 2022 | By Laura Kudrna, Oyinlola Oyebode, Sarah Atkinson & Sarah Stewart-Brown

Feb 17, 2022 | By Nicola Robbins

Feb 3, 2022 | By Nancy Hey

In partnership with

# data[].attributes.attachment_link.title
Joint decision making, co-production and democracy Archives - What Works Wellbeing

# data[].attributes.attachment_link.description
Wellbeing in practice can be: A goal for society or progress like the HMT Green Book describes wellbeing as 'social welfare'. A framework, often based ongoing conversation about what matters, using outcomes and indicators. Personal agency, power, responsibility and control are a part of our wellbeing: freedom to choose what we do in our life is second biggest driver of wellbeing.

# data[].attributes.headline
Planning co-design sessions was a nightmare… until we built this template

# data[].attributes.subject
Planning co-design sessions was a nightmare… until we built this template

# data[].attributes.attachment_link.readable_text
Protecting and progressing projects during COVID-19

Robyn Johnston, perfectly demonstrating our typical feelings after one of our old-style planning sessions. This new toolkit is a vast improvement. 📸 Photo: Jonty Fairless

This is a follow on article from my SDN Academy course.
W e've created, tested, and refined a new co-design planning toolkit in Nile. And now we're sharing it with the community.

It's simple, visual, and the best one we've ever used. It lets us quickly collaborate on workshop plans, effectively coordinate objectives and collaboratively build activities and materials.
So here are some of the things we learnt while building it. Read on to learn how it and can improve outcomes in your workshops, whether in person or remote (the link to the toolkit is at the bottom of this article.)
Not only that, it's colourful, easy to digest for facilitators and stakeholders, and robust enough to use to drive the session itself.

What are co-design workshops?

They're a coming together of users, actors and stakeholders, making all of them part of the design process. It brings together the right knowledge, perspectives and (design) skillset in the same room for an intensive design session.

Why do we do them?

Co-design sessions often allow for rapid creation, validation, and development of ideas and concepts, while ensuring synchronous buy-in from different actors and stakeholders.

The success of a co-design workshop isn't (just) down to charismatic and inclusive facilitation...it's (at least) as important to design, plan & coordinate the workshop beforehand.

Our old co-design planning toolkit. Cumbersome and document-heavy

Here's what we used to do:

Yeah. We spent a lot of time on documentation.

The output was comprehensive, but inscrutable to project stakeholders. The workshop plan and the associated activities and templates felt disconnected. And in the end, it was all so cumbersome that it ended up being a barrier to re-use: it was less effort to build from scratch than comb through hundreds of files and templates to find what you need.

Like so many things, this worked OK-ish for a long time. It was a pain, but who's got the time to rebuild something like this?

And then I decided to go on holiday.

It was a good holiday you guys

During the busiest co-design period on my project. It was an accident. Honest.

When I was planning to cycle the Majorcan Colls, my project team needed to run a series of customer co-design sessions in different bank branches, in different cities, with the public, after hours.

It looked like it was going to be complicated. They're an incredibly experienced team, but I wanted them to feel fully prepared - like they could pick up the workshop plan and know exactly what they needed to do.

The old set of templates and documents wasn't going to cut it. Finally - this was the motivation I needed to redesign this stalwart of our practice.

A co-design workshop is a service where you - as the workshop designer and facilitator - have a central role in delivering the service. The Co-design Workshop Planning Toolkit is a key touchpoint.

1. Save time and effort by creating efficient collaborative planning and enabling delegation

2. Act as a single view of everything, with the workshop plan and templates in one place

3. Avoid information overload by giving project stakeholders the right amount of details to feel confident

4. Deliver consistent outcomes by aligning workshop facilitators and achieving consistency across groups

5. Enabling re-use of activities by letting us build a library of tried and tested workshop activities and templates across our team

Inspiration for my co-design toolkit: a recipe card from Hello Fresh. Everything you need, clear instructions, and easy to digest at a glance.

This is the toolkit. It's this simple. Link at the foot of the article

I started by making a wishlist for what we wanted this specific touchpoint to do for different actors involved in the workshop. I wanted it to:

So that's what I built:

It provides a single view. Of everything.

First, I've built the whole toolkit in Miro.

Miro is nearly perfect for this, with nigh-on infinite real estate and scalability to lay out your plan, alongside unrestricted file storage. You can drag and drop and organise everything in one place, accessible and downloadable by anyone you choose.

The toolkit I built and refined gives you a birds-eye view of a workshop, without compromising on the detail. We gather timings, objectives, activities - even facilitator scripts and wall layouts if required -together in one place. We can build packing lists, to-do tracking, logistics information and action assignments all on the same board.

Glance at the section progression, or get deep into the activity breakdowns. Nobody gets lost anymore.

This is perhaps the most valuable outcome from our new toolkit. You never lose anything.

We're not getting paid by Miro to advertise this... but damn it's a good product. Well done.

Gives project stakeholders as much as they need

Look at all this reusable goodness

This isn't just a tool to enable facilitators to run the workshop. We use it to get alignment and buy-in from internal and external stakeholders.

Because it's visual, colourful and linear, stakeholders can quickly take in the shape of the session, and then focus in to the details as required. It's supremely accessible, no matter what level of detail stakeholders are looking to get into.

Makes reuse and recycling easy

Think beyond a single workshop. You can now collect and collate all your tried and tested workshop toolkits together into a single, massive resource board, ready for anyone in your team to reuse.

It'll change your life.

# data[].attributes.attachment_link.title
Planning co-design sessions was a nightmare... until we built this template

# data[].attributes.attachment_link.description
This is a follow on article from my SDN Academy course. W e've created, tested, and refined a new co-design planning toolkit in Nile. And now we're sharing it with the community. It's simple, visual, and the best one we've ever used.

# data[].attributes.body
https://cardsforhumanity.idean.com/

# data[].attributes.attachment_link.description
Cards for Humanity is a practical tool for inclusive design. Deal the cards to get a random person and their needs. Your challenge is the work out how to design for this person.

# data[].attributes.attachment_link.readable_text
The Verify team has been supporting local councils with end-to-end transformation of a service. It's the first time GDS has had the opportunity to co-design with local government.

We wanted to see how councils could use GOV.UK Verify to let users prove their identity when accessing council services. Here's how we've been developing local service patterns to do just that.

What are local government service patterns?

GDS has been working on design patterns for central government services for a while. These patterns recommend best practice for designing parts of a service, like naming a service or how the start page should look.

With local government there are hundreds of authorities delivering the same kinds of services. So, there's an incredible opportunity to produce better services everywhere by co-designing best practice and guidance for all councils to use.

GDS created local service patterns by working with a small group of pilot local authorities to discover best practice. They're an iterative resource that future local authorities can use and add to. Co-designed and focusing on user needs, they've been reviewed by GDS privacy, fraud and accessibility experts. This will make it easier for councils to deliver excellent services that meet the government standard.

The patterns can be understood as iterative policy - documenting the evolution in best practice for tackling both common and diverse local issues.

How we created the patterns

We're currently running 2 pilots to transform 2 local services using GOV.UK Verify. We're developing end-to-end patterns for each of these services:

First we ran a collaborative discovery with more than 15 councils to find out what the current services looked like and how well they worked for users. This involved training our local authority colleagues in user research techniques, collectively interviewing more than 150 users, and mapping out the different user journeys to find pain points.

In alpha, we used that understanding to create 'example services' for the fictional council of Argleton, made quickly using the GOV.UK Prototype Kit.

Councils then fed back on whether these prototype example services could be implemented locally, based on their policy and technical set-up. We then iterated the prototypes.

Next, we tested them with real users to see how we could improve usability. We used the Argleton prototype and localised variations, branded to look like pilot council sites - enabling councils to research local policy with residents. We did 6 rounds of usability testing centrally. Each of the councils also conducted their own testing. As a result, we did more research than most central government service teams do in alpha.

Throughout the process, we documented design decisions, creating a new format specifically for local services. We also discussed these user research-based changes with council service teams using GitHub.

How we've improved the end-to-end journeys

While no two councils are the same, we've found their services have more similarities than they do differences. By collating user research, co-designing the same service and building it at the same time, we've achieved far more together than we could have individually.

This collaboration enabled us to identify essential steps of a service and highlight policy changes to make the service better. We couldn't have done this without a dialogue between councils.

There's a lot we're proud of. Here are some examples of best practice we've introduced:

1. We're using the Start page, Confirmation page and Check before you start patterns from the GOV.UK Service Manual.

2. The older people's bus pass service includes embedded webcam functionality, which users found easier than photo upload.

3. Parking permits service users can print a temporary parking permit and use it straightaway.

4. Based on user research we removed journey steps, by helping some councils change their policy around vehicle details and documentation.

5. The patterns use GOV.UK Notify to send confirmation messages and reminders, plus of course GOV.UK Verify to prove identity and their eligibility.

Moving forward to live

We've iterated the pattern and prototype to the stage where the majority of user needs found in research have been met. Councils that were able to continue into beta are now following this guidance to produce best practice services.

The next step will be for participant councils to go live with their redesigned services. We'll continue to iterate the pattern together from findings identified in live environments.

We'd love to hear your feedback on the patterns and the prototypes so that this work stays as living best practice. Get in touch with us or, if you're on Github, you can open issues here.

# data[].attributes.attachment_link.authors[].name
Sūryanāga Poyzer - former senior service designer, GDS & Homes England

# data[].attributes.attachment_link.title
Designing the first service patterns for local government

# data[].attributes.attachment_link.description
The Verify team has been supporting local councils with end-to-end transformation of a service. It's the first time GDS has had the opportunity to co-design with local government. We wanted to see how councils could use GOV.UK Verify to let users prove their identity when accessing council services.

# data[].attributes.attachment_link.readable_text
New in 2019 - Project Springboard builds on the the work of the Co-production Project Planner.

Co-production is a process that creates change. It is a way of working with, rather than doing to, people and communities to achieve better outcomes.

A co-production project sees people who access support as assets and builds on people's existing capabilities. In addition, it breaks down the barriers between people who use services and professionals. Co-production projects are planned, developed and delivered by a group of people who have different backgrounds and interests.

The outputs of a co-production process can be huge, such as services, processes and pathways, or small, such as a community hall's furniture, a poster, a blog page. Done well, the co-production process can support organisations and individuals become agents for change.

This free resource focuses on putting ideas into action. You don't need to be an expert in co-production prior to planning a project. Work through the guide and see how you get on.

Co-production project planner

The resource comprises:

1. project planning workbook to take you from initial ideas to running co-production workshops

2. guide to planning co-production projects

3. suite of Iriss facilitation tools

We hope you enjoy using the resource. Download the tool in three parts from this page.

Project Springboard

Project Springboard builds on the work of the Co-production Project Planner which we released early 2018. It gives you bite-sized chunks of information on what to think about when in the initial stages of a project.

You don't need to be an expert in co-production prior to planning a project. Work through the tool and see how you get on.

The tool covers the following:

Project springboard- clarifying your idea, exploring power dynamics and thinking about timescales

Asset bank- what resources do you already have and what else do you need to find to complete your project?

Team builder- who do you need in your co-production team and how will you get them together?

Workshops that work- thinking through your co-production sessions to make sure you get the most out of them

You may also like to follow the Making Co-production Happen blog by Josie Vallely.

We're keen to know how it made a difference, so please provide us with your feedback.

# data[].attributes.attachment_link.description
New in 2019 - Project Springboard builds on the the work of the Co-production Project Planner. Co-production is a process that creates change. It is a way of working with, rather than doing to, people and communities to achieve better outcomes. A co-production project sees people who access support as assets and builds on people's existing capabilities.