# data[].attributes.headline
HRA guidance on sharing evidence with the public to improve transparency 

# data[].attributes.subject
HRA guidance on sharing evidence with the public to improve transparency 

# data[].attributes.attachment_link.title
Make it Public: transparency and openness in health and social care research

# data[].attributes.headline
Creative techniques to communicate reviews

# data[].attributes.subject
Creative techniques to communicate reviews

# data[].attributes.body
https://systematicreviewsjournal.biomedcentral.com/articles/10.1186/s13643-022-01935-2

# data[].attributes.attachment_link.readable_text
Over the last 5 years, we have explored a number of creative communication methods, to extend dissemination beyond traditional academic mechanisms and share our research with the diverse range of people who could benefit from the findings. Our general approach is based on evidence based guidance [ 4, 5, 6, 7], principles of good dissemination [ 1, 8, 9] and examples of best practice relevant to the communication of systematic review findings [ 10, 11, 12].

Our aim in this paper is to (i) share and describe our experience of using a variety of creative communication tools (illustration, podcasts, infographics, blogs, briefing papers and board games and social media shareable content) and (ii) reflect on their use in different situations within the review process and thus to enhance learning and discussion amongst the systematic review community.

Underpinning our approach are three key questions:

Consideration of these questions will help guide decisions in involving evidence users in developing a dissemination plan that outlines key messages, identifies potential audiences and explores suitable format(s) and outlets for sharing messages which make the most of existing networks and opportunities.

At the heart of our work in this area is (i) an awareness of the need for producing different formats for different people, (ii) the importance of images to attract attention, and to aid learning and recall and (iii) a desire to communicate complex messages in easily accessible formats to foster enthusiasm and ultimately action. We are mindful of the need for evaluation to identify issues, learn from them and improve. To date, we have focussed primarily on informal formative and process evaluation as an ongoing process through consultation and feedback with representatives of those whom we hope will engage with our messages.

In the remainder of this paper, we provide examples of our experience in developing and using six communication tools for systematic review findings, illustrations, podcasts, blog posts, briefing documents, boards games and social media shareable content, before reflecting on our learning about creative communications.

Illustration

Illustrations are drawings or images that give a visual explanation or interpretation of a topic or context. They have been used as a means of communication for centuries, adapted for different specialties and using a range of techniques. Today, illustration often sits under a broader concept of graphic communication whereby numerous design techniques can be brought together to communicate a topic in a variety of formats for different audiences.

'...today the discipline of illustration is often lauded by the way it enriches and expands the lives, imaginations and sensibilities of individuals. In fact, it's known to create and interpret cultural capital in all of its forms, most notably by its contribution to education and learning, research and new knowledge.' Professor Alan Male (Emeritus Professor of Illustration, Falmouth University), Keynote address, International Conference on Illustration and Animation, 2015 [ 13].

We have used a range of illustrative techniques at several stages of the review process.

"Graphic Medicine" is a relatively new terminology coined to encapsulate the use of comics within healthcare for the purposes of education and patient care [ 14]. Comics can provide an opportunity for patients to share their experiences of medical care, treatment or living with an illness, which provides them with the opportunity to learn more about their illness and access support from peers living with a similar condition [ 14]. On the other side of the practitioner-patient relationship, comics also support practitioners to gain additional insight into the patient experience [ 15]. Comics, as a form of illustration, have the potential to create a shared narrative across different audiences and convey complex and/or emotion information in an engaging and accessible format [ 16]. We wanted to adopt the principles of this approach within the academic field to engage evidence users with the work that we do.

Scribing, sometimes known as Graphic Facilitation [ 17, 18], is a technique that uses a scribe to bring together key messages, ideas or insights from an event or meeting. Scribing usually involves the team working collaboratively with the scribe giving them as much information as possible beforehand for the context of the event, the intended aims, and what is expected of the output created. Scribing can take place in person or online, and with the scribe working on the illustration as the event is happening, it is possible to see progress immediately, dynamically reflect live events, actively engage attendees and make changes on the go.

Infographics aim to provide information and/or data in a form that is quick and easy for the audience to engage with and as such can use a variety of techniques to share information within the same output/product [ 19, 20, 21, 22, 23, 24]. Infographics are often highly technical pieces of communication produced by trained professionals to simplify and ease communication of complex knowledge [ 24].

Example 1: parent-to-parent support interventions for parents of babies cared for in a neonatal unit (PaReNt)

Stage of review: protocol development

This example is based upon a review evaluating evidence regarding the effectiveness and experiences of support groups for parents of infants admitted to neonatal wards [ 25]. We developed a plain language protocol summary (PLPS) which was intended to be a more accessible version of the scientific protocol [ 26]. Scientific protocols are usually written using technical language for the people conducting research. We wished to develop a more accessible version to support the parent advisory group to more fully engage with the review process by enhancing their understanding of the review rationale and processes, thus empowering them to share their knowledge and experiences.

Whilst the process of developing the PLPS helped us to identify the most important messages in a format which was definitely more concise than our full protocol, the final output was still well within our academic comfort zone. It was a three-page written document, and although it felt like an important step in improving communication from our perspective, our parent advisory group were nonplussed. On reflection, we felt we could do more to make the information accessible and useful. We decided to explore the potential for using illustrative techniques to convey the rationale, aims and methods of our project to parents and members of the public. Drawing upon the information provided within the PLPS, a researcher and artist from outside the project team (LS) created a series of illustrations to emphasise key points. These illustrations were then arranged in a series within a one page document through consultation with the Parent Advisory Group and project team.

Stage of review: dissemination

We asked artist Rae Goddard from Scriberia ( Scriberia.com) to attend and live draw two meetings where the results of the project were discussed, the final project meeting with the research team and the final meeting with the parent advisory group. Rae created a summary of key findings that evolved and developed during the meetings. The images captured a record of what was said including key quotes. We used the images to reflect on the parents' experiences but also as mementoes to thank and acknowledge the parents for their contribution to the project. The final output (Fig. 1) involved interpreting both researcher and parent narratives to create a one-page visual summary of the project process, findings and implications for further clinical practice and research. This illustration was included within a newsletter for the BLISS charity, and postcards of the image were sent to neonatal units across the UK, along with an electronic image file with permission to be printed as posters for display on neonatal wards. Parts of the final image were also incorporated within a Briefing Paper summarising the review findings.

Example 2: multicomponent hospital-led interventions to reduce hospital stay for older adults following elective surgery

Stage of review: dissemination

We used Graphic Medicine to support the dissemination of findings from a systematic review [ 27] aiming to evaluate the effectiveness and cost-effectiveness of interventions to reduce the length of stay for older adults admitted to hospital for planned procedures. A researcher within the project team (LS) created a series of illustrations to accompany brief text excerpts which summarised the rationale, methods, findings and implications of the review. We circulated the final two-page document to local patient groups and clinical contacts. The black and white format was intended to be suitable for photocopying and sharing within hospital waiting rooms in a way which minimised use of hospital resources. We also incorporated individual images into conference posters and the project Briefing Paper for clinicians and service commissioners. Due to time-limitations, it was necessary to focus on producing one comic which could be shared with a variety of stakeholder groups. We would like to use the images to create materials which identify and communicate messages tailored to the specific needs of different groups to enhance the project's accessibility and relevance to different stakeholders.

Example 3: improving continence in children and young people with neurodisability

Stage of review: interpretation of findings; dissemination

Example 4: using creative techniques to communicate systematic reviews

Stage of review: dissemination

In this final example, we worked with a sketch note artist, Grace Elizabeth, to create an infographic (Fig. 3) that describes the various methods of creative communication discussed within this paper and the advantages and disadvantages of using them. Development of the infographic involved an iterative process of discussing and sharing key points with the artist and editing and amending draft illustrations. We also shared draft versions of the infographic with individuals unconnected with the paper to check external understanding. The infographic was used to present our work at a national conference. Discrete sections of the infographic have also been used on social media.

Reflections on using illustration

Overall, illustration can be an interesting and engaging method to involve key stakeholders with both the process of conducting systematic reviews and sharing the findings. Whilst producing illustrations can be labour intensive and require additional time (if utilising in-house expertise), or financial resources (if hiring a professional artist), we have found that the outputs can be used flexibly across a range of academic resources and can be an easy and cost-effective method to share project information and findings with evidence users. To maximise the usefulness of this type of output, we recommend the final illustration represents collaboration and a shared understanding between researchers, evidence users and artist.

Podcasts

Podcasts offer the opportunity to condense the information presented in a fairly lengthy document into a few minutes of audio, whilst adding a conversational tone and connecting with evidence users through a non-visual medium [ 29]. Podcasts also broaden accessibility and allow the consumption of information on the move and without looking at a computer screen [ 30].

Example 1: interventions to improve the mental health of young people with long-term conditions

Stage of review: dissemination

We produced several podcasts as part of the dissemination of the findings from a complex systematic review of interventions to improve the mental health of young people with long-term conditions [ 31]. We initially recorded two podcasts: one to allow members of the research team (researchers, clinicians, young people and their parents) to describe the findings of the systematic review and one for researchers, young people and their parents to reflect on the process of involvement in the project.

These podcasts were resourced by the dissemination budget allocated to the project. We were able to travel to London to record conversations with the young people involved in the project and their parents. We also had the support of a science communications specialist with professional recording equipment who guided us through the process, from developing the script to producing good-quality audio and editing the audio clips. This support was invaluable in producing a seamless and professional product from multiple recording sessions.

The finished products are hosted on our project page, are available on SoundCloud and were shared via Twitter alongside the release of the final report. Our discussions with the young people and their parents indicate that producing the podcasts was a rewarding experience not least because it provided an opportunity for their voices to be heard on a subject that they felt strongly about. They also enjoyed being able to make a practical contribution to the dissemination of the research.

A further podcast about the project was recorded and featured alongside a blog by the 'Mental Elf' (@MentalElf) following the 2018 Cochrane Colloquium in Edinburgh, UK. This involved Erin Walker, Patient and Public Involvement Lead, and Katrina Brooks, a young person from our advisory group, discussing their experiences of being involved in the project.

Example 2: Exeter HS&DR Evidence Synthesis Centre

Stage of review: dissemination

Having previously produced a successful set of podcasts, we were aware of the value of such an approach. However, we found it to be a fairly time-consuming and resource-heavy endeavour, requiring recording equipment, multiple meetings and technical expertise, which we considered prohibitive. That was until we were inspired to pilot a regular research podcast by a twitter discussion, when a member of the public professed that it would be nice to hear academics talk about their work. We wanted to know whether a podcast could effectively confer the findings of a research study and whether it would be achievable with a low-tech, low-resource solution. We found that this was achievable and have now produced two podcast episodes to accompany outputs produced by the Exeter HS&DR Evidence Synthesis Centre.

The first described the findings of a rapid review of evidence to inform the Independent Review of the Mental Health Act (1983) in late 2018, with a single speaker describing the work. This was quick and easy to do, using a podcast producing app called Anchor on a smartphone. Anchor allows audio to be recorded and re-recorded with the touch of a button, in the comfort of the office or home, without the requirement of additional equipment (although this can be added to enhance quality).

For the second episode, we described the findings of a large review of interventions aiming to reduce length of stay in hospital for older adults undergoing planned surgery. Rather than a single speaker, we wanted to include a range of voices including clinical stakeholders and members of the review team.

In order for this to run smoothly, we needed to ensure that all speakers had clear instructions about their contribution to the overall episode, had access to the required technology (smartphone) and could contribute with minimal time commitment. By combining separate recordings we were able to have one researcher 'hosting' the podcast, asking questions of the other contributors and varying the delivery of the episode. We promoted links to the podcast on social media alongside the final report and links are available on our project pages.

Reflections on using podcasts

If done well, podcasts can engage the audience, allowing easy access to key messages and a connection with members of the research team [ 32]. Access to research findings is enhanced by providing an alternative to visual dissemination materials, and the conversational tone of podcasts may allow the audience new insight into the research [ 29]. We found it was possible to produce adequate quality podcast episodes with ease and were able to combine the perspectives of various contributors remotely. In our first example, it was particularly rewarding for young people and their parents to be actively involved in producing dissemination materials about research in which they have been involved and were passionate about. In our experience, podcasts can be recorded in a variety of settings, without access to a computer or professional recording equipment. We have found that without regular content to promote the podcast, numbers of listeners can be relatively low. For example the podcast hosted on the established 'Mental Elf' blog has been listened to over 400 times, whilst those hosted on our relatively unfrequented research pages have had 40-50 listens each. If choosing to capture audio from a group discussion, or when looking to produce higher quality audio, additional recording equipment may be required. It may also be time-consuming to record if several re-takes are needed or interruptions take place. Time to become familiar with the software and some degree of training may be required to allow contributors to record clips remotely, although this can be done quite quickly. It can also be helpful to have a 'non-academic' listen to the final podcast prior to publication, to ensure its tone and language is interesting and engaging. Finally, a podcast series is likely to be more effective and to build up a listener base with regular episodes, but it can be difficult to keep the momentum going-particularly if waiting for the end of a project to produce the next episode.

Blog posts

Example 1: blog series on the perspectives of patient-initiated appointment systems for people with long-term or chronic conditions

Stage of review: dissemination

In 2020, we published a Cochrane systematic review [ 36] that looked at the use of patient-initiated appointment systems in place of consultant-led appointment systems for people with chronic and recurrent conditions being managed in secondary care (hospital outpatients department). The review itself had taken some time to finally bring together and publish and during that time the idea of patient-initiated clinics was becoming increasingly talked about. We knew that the review findings would be well publicised in various forms with the help of the Cochrane team, and we had also published a briefing paper on the review to increase accessibility for a wider audience. So, for the blog series we decided to focus on the intervention itself and reflect on how it was received by a range of stakeholders: a service-user, a clinical nurse specialist, a consultant, a service manager and a health service commissioner). Whilst this could have been incorporated into one post, it would have been very long so we decided to make a blog series-one post per perspective-that could be published over a number of days or weeks. An additional advantage of this approach was that each new post not only directed readers to the review (and associated dissemination outputs) but also to the other blog posts in the series, those that had already been published or those that were forthcoming. We created tweets to share the blog posts to a wider audience and also asked our stakeholders to share the links. The series was well received in terms of comments and re-tweets.

Example 2: a single blog post on social work practice in the UK

Stage of review: scoping

In 2017, our Exeter Health Services and Delivery Evidence Synthesis Centre was commissioned by the HS&DR programme to undertake a systematic review of the effectiveness of different models of social work practice in the UK. Our first task was to become familiar with the area and to explore the available evidence. In doing this, we realised that there was very little evidence available that could be synthesised and the project did not progress. We felt, however, that a summary of our scoping of the topic might be a useful resource for others and could also provide some insight into the processes that we use to become familiar with a topic. We posted the blog post on our team blog-Sifting and Sensemaking-in July 2017 and shared the link via both our Twitter feed and via email to those who had been involved in the scoping process. The blog post was well received in terms of comments and re-tweets and allowed the review team to feel that although the project was paused due to lack of available evidence, their work in reaching that decision could still be useful.

Reflections on using blogs

Blogs can be used in lots of different ways and at different stages of the review process to engage with the audience. We have found that preparing blog posts, especially with other stakeholders, can help to clarify key messages, provide an outlet for discussion and increase the diversity of people who engage with review findings. However, it can be time consuming in terms of identifying and liaising with the contributors, preparing the posts on the blog site, finding the appropriate images and devising tweets and emails to promote them. Publishing the methods and outcomes during the review process can help to improve the transparency of the work undertaken by the team, something which is important to establish trust between scientists and evidence users.

Briefing papers

Short, plain language summaries of systematic reviews are an increasingly common dissemination output [ 9]. In keeping with the underlying purpose of the approaches discussed in this paper, we define briefing papers as short, accessible, engaging summaries of systematic review findings designed to go beyond an academic audience. Guidance on preparing briefing papers for policy makers suggests a 1:3:25 page format with one page of take home messages, a further 3 pages that describe the methods, findings and implications in more detail and a full (25 page) report [ 37].

Example 1: Exeter HS&DR Evidence Synthesis Centre: the Briefing

Stage of review: dissemination

For this programme of work, we have developed a template that combines the 1- and 3-page summary to create a 4-page document, tailored to the anticipated audience and focused on the findings and implications with little methodological detail. The front page presents a complete summary of the review with key take home messages and the following pages provide explanation of context, brief methods, findings and implications for practice, policy and research. We have also used this template in a number of other projects and found it to be easily adaptable to different review topics and audiences.

Example 2: how do "robopets" impact the health and well-being of residents in care homes?

Stage of review: dissemination

For this review, we created a newspaper style briefing paper, The Woofington Post, to share our findings with care home staff. Methodological detail was minimised to dedicate space to communicating key messages arising from the research. Based on our experience of working with care home staff, our intention was to produce a short, easy to read and visually appealing briefing paper that would be suitable for staff to read on breaks. We shared this briefing paper predominantly in hard copy format in response to feedback that many care home staff do not spend their breaks in front of computers or devices. The Woofington Post is also available electronically and the visually appealing format has proved popular on Twitter.

In both examples, we provide easily accessible links to signpost the reader to the full report, other related resources and the contact details for the research team and thereby provide further information for those who are keen to have a more in depth understanding of the research.

Reflections on using briefing papers

Briefing papers often form the mainstay of our communication activities as they provide a quick summary that is easy to share both electronically and in hard copy format. We have found them to be a useful device in communicating with, for example, creative colleagues to inform discussions about producing other outputs and, with support groups and third sector organisations to facilitate communication and enable effective decision making about the suitability of the research for newsletter or website pieces. The process of creating briefing papers itself can also be beneficial in creating clarity within the team about the nature of the important messages and their implications. The written form may not be accessible to all and this type of output can be easily overlooked amongst the volume of written/electronic information we are exposed to on a daily basis.

Board games

The well-known concept of a board game is a fun way of engaging people with many subjects. For example, educational board games help learners increase knowledge and stimulate discussion [ 38]. As an effective way of breaking down complex information into an easily accessible and enjoyable format, there is potential for board games to be used to explain research processes and research findings [ 39]. This works on two different levels: as a visual metaphor or at the level of game mechanics. The former uses visual clues to evoke the concept of the board game, for example the idea of a game path can dress up a clinical looking flow-chart and immediately communicate the notion of progress, as well as detours and short-cuts. This is a variation of an illustration but uses familiar board game aesthetics to evoke a playful genre (and possibly mood for the audience). It allows the most important pieces of information to be embedded within the design of the 'board'. The second level is not just about the visuals, i.e. it is possible to make some project outcomes look like a board game at the same time as building in game mechanics to make them playable. This raises the engagement factor of the communication-as members of the audience become players they might experience the frustration of encountering a chance card triggering an unfortunate event, for example. It is also possible to add information to and through gameplay and rules-for example have different rules for different demographics to highlight special risk factors of those demographics.

Example 1: Pet Purrsuit-a board game to share evidence on the impact of animals in care homes

Stage of review: dissemination

We then worked with a local artist and designer to design and create a visually appealing and professional looking prototype board game, Pet Purrsuit which incorporates images of care home staff, residents and animals.

Initial observations from playing the game with researchers and care home staff suggest that the board game is an engaging way of learning which encouraged the players to think and engage and apply the research findings to their own context. Conversations occurred during the gameplay, as the players read the content of the 'treats' and 'paws' cards aloud. Several aspects of our approach to designing the game helped make it easy for people to engage including basing the visual appearance and the rules on familiar games and permitting play to stop and resume later. We believe that playing the game may also have the potential to flatten the power hierarchy between researchers and research users.

Reflections on using board games

In this example, the board game was used for dissemination but this method of communication could provide an opportunity to engage with stakeholders in other stages of the review process, e.g. in explaining the scope and remit of a review albeit this would require a less polished product. Researchers should note that designing and developing a board game is resource and time intensive, in this case, requiring a professional artist and consultation with stakeholders. There is also the possibility that the concept of a board game as a way of engaging with research may be viewed by some as trivialising the research and engagement may be limited if playing the game requires facilitation. People are motivated to play games for different reasons [ 42], and it may be a challenge to develop a game which appeals to all audiences. As a result of the global coronavirus pandemic, we have had limited opportunities to explore these issues in depth with care home staff.

Social media shareable content

Social media shareable content is content that can easily be shared on social media as a means of signposting the audience to more substantial outputs. Using visual images has been shown to be effective in increasing engagement with social media [ 43, 44, 45] which is perhaps not surprising given that the use of images to improve attention to and recall of information is a well-known approach in health education [ 46]. The use of visual snapshots of the findings of systematic reviews is a technique frequently used to share Cochrane evidence [ 47] in a dissemination product they refer to as blogshots.

Example 1: World Alzheimer's Day

Stage of review: dissemination

We recently created and shared a series of Twitter cards in order to share our relevant research in celebration of World Alzheimer's Day (Fig. 5). Each card has a similar design and contains a short statement of the main findings from the review, a visually appealing image and a link to the published paper. We shared the cards on Twitter over the course of the day.

Reflections on using social media shareable content

Social media shareable content is relatively quick to create using existing content. In this example, we were able to highlight a collection of previously published research relevant to World Alzheimer's Day in a cohesive style. This form of dissemination enables the sharing of key messages from a published paper in an eye-catching format. As this example demonstrates, it can also be used to promote previously published research (rather than new outputs) as part of a themed collection. It can, however, be challenging to summarise the key points with the necessary degree of nuance in the space available.

# data[].attributes.attachment_link.title
Bursting out of our bubble: using creative techniques to communicate within the systematic review process and beyond - Systematic Reviews

# data[].attributes.attachment_link.description
Background Increasing pressure to publicise research findings and generate impact, alongside an expectation from funding bodies to go beyond publication within academic journals, has generated interest in alternative methods of science communication. Our aim is to describe our experience of using a variety of creative communication tools, reflect on their use in different situations, enhance learning and generate discussion within the systematic review community.

# data[].attributes.headline
A review of patient involvement in publications/research summaries

# data[].attributes.subject
A review of patient involvement in publications/research summaries

# data[].attributes.attachment_link.readable_text
Research article

Open Access

Published:

Lauri Arnstein 1,

Anne Clare Wadsworth 1,2,

Beverley Anne Yamamoto 3,4,5,

Richard Stephens 6,7,

Kawaldip Sehmi 8,

Rachel Jones 9,

Arabella Sargent 10,

Thomas Gegeny 11 &

Karen L. Woolley orcid.org/0000-0003-4626-7723 1,12,13,14

Research Involvement and Engagement 6, Article number: 34 (2020) Cite this article

4059 Accesses

8 Citations

48 Altmetric

Metrics

Abstract

Background

There are increasing calls for patient involvement in sharing health research results, but no evidence-based recommendations to guide such involvement. Our objectives were to: (1) conduct a systematic review of the evidence on patient involvement in results sharing, (2) propose evidence-based recommendations to help maximize benefits and minimize risks of such involvement and (3) conduct this project with patient authors.

Methods

To avoid research waste, we verified that no systematic reviews were registered or published on this topic. We co-created, with patients, a PRISMA-P-compliant protocol. We included peer-reviewed publications reporting the effects of patient involvement in preparing peer-reviewed publications or results summaries from health research studies. We searched (9/10/2017) MEDLINE, EMBASE and the Cochrane Database of Systematic Reviews, and secondary information sources (until 11/06/2018). We assessed the risk of bias in eligible publications and extracted data using standardized processes. To evaluate patient involvement in this project, we co-created a Patient Authorship Experience Tool.

Results

All nine eligible publications reported on patient involvement in preparing publications; none on preparing results summaries. Evidence quality was moderate. A qualitative synthesis of evidence indicated the benefits of patient involvement may outweigh the risks. We have proposed 21 evidence-based recommendations to help maximize the benefits and minimize the risks when involving patients as authors of peer-reviewed publications. The recommendations focus on practical actions patient and non-patient authors can take before (10 recommendations), during (7 recommendations) and after (4 recommendations) manuscript development. Using the Patient Authorship Experience Tool, both patient and non-patient authors rated their experience highly.

Conclusions

Based on a systematic review, we have proposed 21 evidence-based recommendations to help maximize the benefits and minimize the risks of involving patients as authors of peer-reviewed publications.

Peer Review reports

Plain language summary

This research project is a systematic literature review. This is a structured way to look at all the published research on a topic.

This review looked at patient involvement in sharing results from health research studies, including clinical trials. Results from health research studies are shared in different ways, such as:

The researchers wanted to know:

What are the benefits and risks of involving patients in sharing results?

What recommendations could help to increase the possible benefits and reduce the possible risks?

What was the experience of patient authors taking part in this review project?

The researchers searched for published research from different sources. They found 167 articles, of which nine were relevant.

The researchers found that:

Based on the evidence they found, the researchers made 21 recommendations.

Overall, the patient and non-patient authors involved in this review project had a good experience.

These results suggest that patients should be involved in sharing research results. There are now recommendations on how to involve patients as authors on publications.

See Additional file 1 for an infographic plain language summary.

Background

The importance of ethical and substantial patient involvement in medicines development is being acknowledged by stakeholders [ 1, 2]. Patients are becoming informed, empowered and active partners in their healthcare, and are calling for increased involvement across the entire medicines development life cycle [ 3, 4]. In response, stakeholders are recognizing the need for internal process and cultural change. Regulators, including the European Medicines Agency and the US Food and Drug Administration, have declared patient involvement to be a priority, and payers are increasingly taking the patient perspective into account in reimbursement decisions [ 1, 5]. The pharmaceutical industry is beginning to empower the patient voice in medicines development, partnering with international patient advocacy organizations and healthcare professionals to establish ethical frameworks and principles to facilitate patient involvement [ 6, 7, 8].

In conjunction with this patient-empowered evolution, patients are increasingly seeking health information from peer-reviewed publications, and this demand is likely to increase [ 9, 10]. Consistent with this interest from patients and their advocates, some medical journals are facilitating greater involvement of patients as authors, editors, peer reviewers and readers [ 11, 12]. In addition, increasing attention is being focused on involving patients in plain language summaries of publications of clinical trials and other types of research publications, building upon the European Medicine Agency's regulatory requirement for lay summaries of clinical trial results [ 13].

However, in comparison to patient involvement in other phases of the health research life cycle, such as study design and recruitment, patient involvement in publications has lagged behind. Not only does the level of patient involvement in publications seem low, but the potential benefits and risks of patient involvement are not well understood. Such insights are needed to define evidence-based best practices, thereby enhancing the potential benefits and minimizing the potential risks of involving patients as publication partners (e.g. as authors or contributors).

Our objectives were to: (1) conduct a systematic review of the evidence on patient involvement in results sharing, (2) propose evidence-based recommendations to help maximize benefits and minimize risks of such involvement and (3) conduct this project with patient authors.

Methods

Protocol, registration and definition

This systematic review was registered in the PROSPERO database (PROSPERO 2018 CRD42018084452), conducted according to a prespecified protocol and reported in compliance with best-practice reporting guidelines for systematic reviews [ 14] and research involving patients [ 15]. For this study, we employed the PICO model to define the key elements in our research, which included peer-reviewed publications that have investigated the effects of having patients (P - Participants) engaged as authors of or contributors to (I - Intervention) peer-reviewed publications. We quantified the number of these publications (Outcomes) and compared this with the number of publications on patient involvement in the preparation of lay summaries of clinical trial results (Comparator).

To minimize the risk of research waste, we searched (5 June 2017) the PROSPERO database to ensure we were not duplicating a planned or ongoing systematic review. We also registered our review on SYNAPSE, the Patient Focused Medicines Development (PFMD) repository for patient engagement initiatives ( https://synapse.pfmd.org/initiatives/patient-involvement-in-preparing-clinical-research-peer-reviewed-publications-or-results-summaries-a-systematic-review).

Within this study, 'patient' was defined in broad terms, based on an existing definition [ 16, 17] and input from our patient partners. For this research, 'patient' refers to "people having or at risk of having medical condition(s), whether or not they currently receive medicines or vaccines to prevent or treat a disease" as well as "the family and those voluntarily caring for those with the medical condition(s), patient advocates and patient groups."

Eligibility criteria

We included peer-reviewed publications that studied the effects of patient involvement for sharing results of health research studies (all phases and all designs) either through a peer-reviewed publication or through a lay summary of clinical trial results (i.e. consistent with the regulatory requirements described in the EU Clinical Trials Regulation 536/2014 [Article 37] [ 13]). We considered publications, in any language, that were detected in searches of electronic bibliographic databases or secondary sources (see Information sources and search strategies section). We did not include unpublished data or abstract-only articles.

Information sources and search strategies

We used primary and secondary information sources. For our primary source, we searched three electronic bibliographic databases (MEDLINE, EMBASE and Cochrane Database of Systematic Reviews) using the OVID database search interface. We used Medical Subject Headings (MeSH) and words/phrases related to patient involvement in health research peer-reviewed publications and lay summaries of clinical trial results. An experienced search strategist conducted the search of the three databases on 9 October 2017 (limits: 1 January 2015-9 October 2017; see Additional File 2 for full search strategies). References were exported to a reference management software program (EndNote) and saved into a project-specific library within EndNote. Duplicate publications were removed.

We supplemented our primary information source with secondary information sources. These sources comprised the repository of patient involvement in research literature curated by the Patient-Centered Outcomes Research Institute (PCORI), PubMed's 'similar articles' listings and the authors' personal reference files. A search of the PCORI repository was conducted on 10 June 2018 via PCORI's Engagement in Health Research Literature Explorer (limits: dissemination, 1 January 2017-10 June 2018); a search of the top 10 PubMed articles most related to eligible articles retrieved from our primary information sources was conducted on 11 June 2018; and authors' personal references were scanned up until 30 May 2018.

Study selection

We assessed publications for eligibility using a standardized process and a dedicated assessment team (three authors with predefined roles). One author (KW) selected potentially eligible articles based on a title and abstract screen, then two authors (LA, TG) independently selected eligible articles based on a review of the full text of each potential article. Differences in assessment were resolved by consensus among the assessment team.

Data collection process and data items

Based on the protocol, we prepared a spreadsheet template to collect data from all eligible publications. The template was pilot tested and refined before use. Data were collected using a standardized process and a dedicated data collection team (three authors with predefined roles). Two authors (RJ, AS) independently extracted data from each publication and a third author (ACW) verified 100% of the data extracted for the primary outcome measure from every eligible publication, and verified 100% of the data extracted for the secondary outcome measures from a random sample of 25% of the eligible publications. Data (Additional File 3) were extracted for publication title, journal, year, authors, first author's country, research sponsor, number of patient authors, patient author background, patient author recruitment process, author communication process, extent of patient involvement in preparing the publication or lay summary of clinical trial results, how patient involvement outcomes were assessed, benefits (to patient authors, other authors, the project), risks (to patient authors, other authors, the project) and best practice recommendations.

Risk of bias in individual studies

We anticipated that there would be few randomized controlled trials that directly addressed our research question. As per the protocol, the Cochrane Collaboration tool for assessing bias risk ( https://handbook-5-1.cochrane.org/chapter_8/table_8_5_a_the_cochrane_collaborations_tool_for_assessing.htm) was to be used to assess randomized controlled trials. The Newcastle-Ottawa Scale was used to evaluate the risk of bias for other study designs, although we recognize this instrument does have limitations ( http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp).

Summary measures

Our primary outcome measure was the number of peer-reviewed publications that investigated the effect of patient involvement on preparing peer-reviewed publications.

Our secondary outcome measures were:

1. a.
The number of peer-reviewed publications that investigated the effect of patient involvement on preparing regulatory-standard lay summaries of clinical trial results;

2. b.
The quality of the evidence reported in the eligible publications;

3. c.
The number and the background (e.g. patient experts, clinical trial participants, patient advocacy group members) of patients contributing to the preparation of the publications or lay summaries of clinical trial results;

4. d.
The type of patient involvement (e.g. as authors, as non-author contributors);

5. e.
The number and type of patient involvement outcomes assessed (e.g. benefits, risks, best practice recommendations, other).

Synthesis of results

As anticipated in the protocol, we did not synthesize data quantitatively due to heterogeneity in the eligible publications. We conducted a qualitative narrative synthesis of the data to describe patient involvement outcomes.

Additional analyses

To assess the quality of the patient authorship experience in the present study, one author (KW) customized the original PFMD Patient Engagement Quality Guidance [ 16], which PFMD co-created with over 100 people from 51 organizations, representing patient associations, industry, academics, researchers, and external experts. The resulting draft Patient Authorship Experience (PAE) self-assessment tool was then updated with co-author feedback. The final PAE tool evaluates eight publication-relevant domains using a bipolar, five-point, psychometric, Likert scale (strongly disagree to strongly agree) [ 18]. The total score is determined by adding up the individual scores for each question and calculating a percentage out of a maximum score of 80. Two versions were created: one for patient authors and another for non-patient authors. All authors in this study completed the PAE tool.

To document the involvement of patients in this project, the GRIPP2 Short Form was completed [ 15].

Results

Quantity of evidence on patient involvement

The number of peer-reviewed publications that reported the effects of patient involvement on preparing peer-reviewed publications was low. An extensive search of primary and secondary information sources yielded only nine publications (Fig. 1). Of these nine publications, nine reported on the effects of patient involvement in preparing peer-reviewed publications; none reported on the effects of patient involvement in preparing regulatory-standard lay summaries of clinical trial results.

Flow of screened and eligible publications (PRISMA Flow Diagram)

Quality of evidence on patient involvement

The quality of evidence on patient involvement in preparing peer-reviewed publications was moderate. There were no published systematic reviews or randomized controlled trials focused on this topic. Assessments based on the Newcastle-Ottawa scale, indicated six of the nine studies were good quality and three were fair/poor quality (Table 1).

Type of patients involved and type of involvement

Patients were involved as authors in six of nine publications, and as contributors in eight of nine publications; the majority of them were members of patient and public involvement (PPI) groups (Table 1).

Evidence on benefits and risks of patient involvement

From the evidence reported, patient involvement was associated with benefits and risks, which we categorized into those that affected: (1) patients, (2) others (primarily researchers and clinicians) and (3) the actual research projects (Table 2). Based on the number of effects and the overall impact, the potential benefits of patient involvement appear to outweigh the potential risks. Key benefits include validation of research need, relevance and value; improved materials and processes; improved reporting and dissemination; enhanced credibility and trust; and development of new knowledge and skills. Risks focused on increased resource needs and opportunity costs; navigation of power dynamics; and risk of nonrepresentative insights from less diverse PPI groups.

Evidence on best practices recommendations for patient involvement

Based on the evidence extracted from the eligible publications (Additional file 3), we developed a list of 21 recommendations that could help maximize the benefits and minimize the risks of involving patients in preparing peer-reviewed publications (Table 3). Each recommendation is linked to the eligible publications that provided the evidence to support that particular recommendation. To enhance practical use, the recommendations are categorized into actions that research teams could take before, during and after manuscript preparation.

PAE self-assessment tool

Overall, authors in this study rated their co-authorship experience highly. The average PAE tool score was 86% for patient authors (69 of 80; N = 3) and 90% for non-patient authors (72 of 80; N = 6). The authors identified areas to enhance future collaboration; for example, fewer assumptions on the part of the research team about patient authors' prior experience and knowledge of publications and greater awareness of the extent of patients' emotional investment in research involvement.

Reporting of patient involvement in this project

Consistent with best-practice reporting for patient involvement in research [ 15], the involvement of patient co-authors in this project has been documented using the GRIPP2 Short Form (Table 4).

Discussion

To our knowledge, this is the first systematic review of evidence on the effects of patient involvement in sharing results from health research studies. We identified nine publications that reported benefits and risks when patients were involved in sharing results through peer-reviewed publications. From this systematic review, we have proposed 21 evidence-based recommendations to help maximize the benefits and minimize the risks of involving patients as authors of peer-reviewed publications.

Consistent with the drive for greater involvement of patients in the development, regulation and reimbursement of medicines [ 1, 2, 3, 4], there is growing recognition that patients should be more involved in sharing research results. However, in contrast to the proliferation of general guidelines on how to involve patients in the medicines life cycle [ 2, 3, 6, 7], there is limited guidance on how to involve patients in sharing results. Traditionally, results from all types of health research have been shared through peer-reviewed publications in medical journals. These publications have usually been authored by medical research scientists and clinicians. More recently, European regulatory requirements have prompted the development of lay summaries of results from certain types of clinical trial studies [ 13]. These regulatory-standard lay summaries are usually authored by research sponsors (e.g. pharmaceutical company staff or service providers). We reasoned that if patients are going to be more involved in sharing results through peer-reviewed publications or regulatory-standard lay summaries, then evidence-based guidance was needed to optimize patient involvement. This was the rationale for our systematic review.

From our systematic review, we found that evidence to guide patient involvement does exist, but it is limited in both quantity and quality. We identified publications that reported the benefits and risks of involving patients in preparing peer-reviewed publications, but we did not identify any publications that reported the effects of involving patients in preparing regulatory-standard lay summaries. Consequently, we have limited our evidence-based recommendations to how to involve patients as authors of peer-reviewed publications. As these appear to be the first set of evidence-based recommendations on this topic, we hope that developers of new or updated guidelines on how to prepare peer-reviewed publications will take our recommendations into account. For example, the Good Publication Practice guidelines (published in 2003 [ 28], 2009 [ 29] and 2015 [ 30]) are one of the most widely used guidelines for publishing industry-sponsored research [ 31], and are due to be updated. As many users of these guidelines will have limited or no experience with involving patients in publications, our evidence-based recommendations may help them maximize the benefits and minimize the risks of embracing this 'innovation' in publications. From our real-world experience of working with patient authors on this systematic review, it would have been helpful for our team (patient and non-patient authors) and our project if we had been able to follow these recommendations, before, during and after preparing this manuscript. We would also encourage guideline developers to address other areas of patient involvement in the publication ecosystem (e.g. inviting patients to serve on publication steering committees, co-creating plain language summaries of publications with patients, thanking patients for their participation in research studies, partnering with patients to raise awareness of research presentations and publications that are most relevant to patients).

From our systematic review, we also found that it was challenging to search the literature and to identify which authors may have been patient authors. Until these two issues can be addressed, research on this topic will not be as efficient or effective as it could be. For example, without the use of standardized keywords to interrogate databases (e.g. MEDLINE), we may have missed eligible publications. The lack of standardized keywords in an emerging area of research is not surprising, and we encourage stakeholders to propose and reach consensus on suitable keywords to enhance future literature searches. We suggest that even the adoption of 'patient author' would help research in this area. We did find that the yield of potentially eligible publications was higher from searching the PCORI's Health Literature Explorer database, as some level of curation had already taken place. We encourage researchers with an interest in this topic to use the PCORI database ( https://www.pcori.org/literature/engagement-literature). In terms of identifying which publications had patient authors, we found that there was no consistent or clear way that patient authors were described. We typically had to review the author affiliation, methods and acknowledgements sections of publications to detect which, if any, authors represented the patient perspective. Even if patient authors were identified, sometimes their listed affiliation was a university or hospital. A subsequent Internet search helped to clarify and confirm their role as a patient author (e.g. leadership role in a patient advocacy group, community advisory board). Understandably, these groups might be affiliated with a university or hospital and, quite rightly, this affiliation is given. However, it would help research in this area if such affiliations were complemented by a description of patient representative roles.

In addition to conducting this systematic review and proposing evidence-based recommendations, an important goal of this project was to partner with patients, as authors. We developed a number of tools, with patients, to help facilitate the involvement and evaluation of patient authors in publications and to help share our research findings with the patient community and other interested stakeholders. The PAE tool, with versions for patient authors and for non-patient authors, was a publication-specific adaption of the Patient Engagement Quality Guidance [ 7]. This guidance was developed by the not-for-profit organization, PFMD, with input from a large and diverse group of stakeholders [ 7]. We have made the PAE tool available for others to use for free ([ 18]; Additional Files 4 and 5). We welcome feedback on this tool, and hope that it helps patient and non-patient authors evaluate how well they worked together and where they might enhance their partnership. We have also developed, with patients and other stakeholders, a Plain Language Summaries of Publications Toolkit ( https://www.envisionthepatient.com/plstoolkit/) [ 32]. We have also made this Toolkit available for others to use for free and welcome feedback on its use. The patient authors of this systematic review played a key role in using this Toolkit to co-create the plain language summary for this systematic review.

Although we have based our recommendations on a systematic review of the literature, we recognize that our research does have limitations and that additions to the literature over time will necessitate updates and expanded perspectives relevant to the expanding evidence. We found nine eligible publications and these all focused on the effects of patient involvement in preparing peer-reviewed publications, not regulatory-standard lay summaries. As more patients become involved in sharing results through publications or lay summaries, we encourage them and their non-patient partners to evaluate their experiences and publish their findings. We need to keep building the evidence base to guide best practice. We also encourage other researchers to expand and update the evidence base (e.g. by examining the grey literature; repeating our search as more papers on this topic are published).

We also acknowledge that while our recommendations are based on evidence, they need to be tested in 'real-world' situations. We have reflected on the recommendations and have recognized how applicable and valuable they would have been to this project, had they been available at the start. We have also started to apply these recommendations in new projects involving patient and non-patient authors, and the initial feedback from patient and non-patient authors has been positive. In terms of global use of these recommendations and the tools we developed, we welcome collaboration with individuals around the world who may be able to assist with translation and cultural adaptations.

Conclusions

Published evidence on the effects of patient involvement in sharing results from health research studies is limited. From a systematic review of the literature, we have summarized the reported benefits and risks of involving patients in preparing peer-reviewed publications of health research studies. Based on this evidence, we have proposed 21 recommendations for involving patients before, during and after preparing a manuscript for publication. Understanding and learning from the experience of both patient and non-patient authors is equally important, and we piloted the PAE self-assessment tool as an initial step towards unlocking these insights. Researchers, patient partners and publication professionals may draw on these resources as they start or continue to collaborate on ethical, substantial patient involvement in publications.

Availability of data and materials

Original articles are available through their respective publishers, some as open access.

Abbreviations

PAE:

Patient Authorship Experience

PCORI:

the Patient-Centered Outcomes Research Institute

PFMD:

Patient Focused Medicines Development

PPI:

patient and public involvement

References

Download references

Acknowledgements

The authors acknowledge the insightful patient representative contributions from Antonio Ciaglia (employed at the International Alliance of Patients' Organizations during study concept development), as well as the professional support from Envision Pharma Group colleagues (literature searching and editorial assistance). The authors also thank the pioneers (patients and their research partners) who have and continue to advocate for ethical and substantial involvement of patients in medicines development-they have inspired our research.

Funding

No external grants were used to fund this review. All work was completed on a voluntary basis.

All authors acted in a voluntary capacity. Envision Pharma Group provided limited financial support for administrative expenses.

Author information

Authors and Affiliations

1. Envision the Patient - Envision Pharma Group, Suite 5.11, 5th Floor, 1 Lyric Square, London, W6 0NB, UK

Lauri Arnstein, Anne Clare Wadsworth & Karen L. Woolley

2. Alligent EU - Envision Pharma Group, Wilmslow, UK

Anne Clare Wadsworth

3. Osaka University, Osaka, Japan

Beverley Anne Yamamoto

4. Hereditary Angioedema Japan (Registered NPO), Hyogo, Japan

Beverley Anne Yamamoto

5. Hereditary Angioedema International (Registered NPO/Charity), Fairfax City, VA, USA

Beverley Anne Yamamoto

6. Consumer Forum, National Cancer Research Institute, London, UK

Richard Stephens

7. Research Involvement and Engagement, London, UK

Richard Stephens

8. International Alliance of Patients' Organizations, London, UK

Kawaldip Sehmi

9. Swii.ch Health, Manchester, UK

Rachel Jones

10. Curo - Envision Pharma Group, London, UK

Arabella Sargent

11. Engage - Envision Pharma Group, Southport, CT, USA

Thomas Gegeny

12. ProScribe KK - Envision Pharma Group, Tokyo, Japan

Karen L. Woolley

13. University of Queensland, Brisbane, Queensland, Australia

Karen L. Woolley

14. University of the Sunshine Coast, Maroochydore DC, Queensland, Australia

Karen L. Woolley

Contributions

All authors participated in the research, were actively involved in preparing the manuscript and provided approval for submission.

Authors' information

LA is Patient Partnership Liaison at Envision Pharma Group.

ACW is a Global Business Unit Head at Envision Pharma Group.

BY is a patient author and Professor in the Graduate School of Human Sciences, Osaka University.

RS is a patient author and member of the NCRI Consumer Forum and several other UK and European patient groups. He is also Co-Editor-in-Chief of Research Involvement and Engagement.

KS is a patient author and CEO at International Alliance of Patients' Organizations.

AS is Value Strategy Lead at Envision Pharma Group.

RJ is the Medical Director and Founding Partner at Swii.ch Health Ltd.

TG is a Senior Scientific Director at Envision Pharma Group.

KW is the Global Lead for Patient Partnerships at Envision Pharma Group; Adjunct Professor at the University of Queensland and the University of the Sunshine Coast; and Editorial Board Member for the journal Research Integrity and Peer Review. Author ORCiD iD: 0000-0003-4626-7723.

Corresponding author

Correspondence to Lauri Arnstein.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

RS is Co-Editor-in-Chief of Research Involvement and Engagement. KW is an Editorial Board Member for Research Integrity and Peer Review. BY has received consultancy payments from CSL Behring, Shire and Biocryst, and expenses for presentations from CSL Behring, Shire, Biocryst and Takeda. All authors are advocates for greater involvement of patients as partners in medical research, including legitimate authorship of medical research publications.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.
Reprints and Permissions

About this article

Cite this article

Arnstein, L., Wadsworth, A.C., Yamamoto, B.A. et al. Patient involvement in preparing health research peer-reviewed publications or results summaries: a systematic review and evidence-based recommendations. Res Involv Engagem 6, 34 (2020). https://doi.org/10.1186/s40900-020-00190-w
Download citation
Keywords

Patient and public involvement

Patient participation

PPI

Research reporting

Systematic review

Authorship

Medical writing

Clinical trials

Health research

Patient author

# data[].attributes.attachment_link.title
Patient involvement in preparing health research peer-reviewed publications or results summaries: a systematic review and evidence-based recommendations - Research Involvement and Engagement

# data[].attributes.attachment_link.description
Background There are increasing calls for patient involvement in sharing health research results, but no evidence-based recommendations to guide such involvement. Our objectives were to: (1) conduct a systematic review of the evidence on patient involvement in results sharing, (2) propose evidence-based recommendations to help maximize benefits and minimize risks of such involvement and (3) conduct this project with patient authors.

# data[].attributes.body
Read the blog https://kmalliance.co.uk/drawing-in-the-experts/

Tweet thread with individual images: https://twitter.com/dr_know/status/1548986384727277568?s=20&t=qik6XEvmK0oqCxxGm7yosA

# data[].attributes.headline
Making research documents clear through working with patients

# data[].attributes.subject
Making research documents clear through working with patients

# data[].attributes.attachment_link.title
Making it clear and relevant: patients and carers add value to studies through research document reviews

# data[].attributes.attachment_link.description
Making it clear and relevant: patients and carers add value to studies through research document reviews - Author: Kristina Staley, Joanne Ashcroft, Lisa Doughty, George Szmukler

# data[].attributes.headline
INVOLVE guidance on writing a plain language summary

# data[].attributes.subject
INVOLVE guidance on writing a plain language summary

# data[].attributes.body
On inaccessible academic writing!:

# data[].attributes.attachment_link.readable_text
If Eve L. Ewing were a superhero, her special power would have to do with words. She is that rare academic able to combine scholarship, poetry, and writing for Marvel Comics.

An assistant professor of the sociology of education at the University of Chicago, published a poetry collection called Electric Arches in 2017; a nonfiction book, Ghosts in the Schoolyard, in 2018 about racism and school closings on Chicago's South Side; and a year later, a second poetry collection, 1919. She also writes the series for Marvel, tweets prolifically under the nom de plume of Wikipedia Brown, and has a forthcoming book for elementary readers, Maya and the Robot.

A proud product of the Chicago public schools, she and Nate Marshall co-direct Crescendo Literary, a partnership that develops arts events and educational resources in Chicago. And in the wake of the Black Lives Matter protests, she has spoken about race and social injustice both on Twitter and in interviews like this one with NPR.

In short, she was a perfect candidate for the Scholars Talk Writing series to discuss her work and her approach to academic and nonacademic writing.
Ghosts in the Schoolyard is political, passionate, and personal. For whom did you write this book?
Ewing: Audience is always the first, and most important, question for me. Having an audience in mind helps you stay accountable to what you're trying to do. It helps keep you honest, and it helps you decide the terms of your success or failure in advance, before the thing is out in the world. I think it's important to every aspect of the writing process, from conceiving an idea to revision.

One challenging aspect of Ghosts was that I knew from the beginning that I was trying to write for multiple audiences whose needs might be at odds with one another:

I wanted my academic colleagues in sociology and education to read the book and to respond favorably, and I wanted to do all the things you have to do as a junior professor in a first book - flex my muscles a little bit and show that I had some command of the conversation intellectually.

I also knew that I wanted the book to be useful as a teaching text for graduate students and undergraduates, and to be a textbook for people who are trying to teach about racism or urban-education policy. So there were a few concepts I had to make plain and transparent for them.

And I wanted the book to be useful to people who care about education and don't know where to start, folks who don't necessarily see themselves as experts but who want to be informed citizens.

The final and most important audience was school folks themselves - organizers, teachers, community members. I wanted this book to be useful to them, to articulate things that they already know in a convenient and accessible way. I wanted to offer a tool to save them from having to marshal the same arguments over and over, and allow them to just skip a step and get everyone on the same page.

In writing the book, I made a conscious decision: If I ever found myself at a fork in the road regarding those audiences, it would be that final group that would win out. I really needed the book to work for them. And that was scary because I truly didn't know if academic colleagues would regard this book with disdain or disregard, or think I just wasn't very smart because I didn't write a much more opaque book.

For some academics, inaccessibility is the coin of the realm. For some you prove your expertise by restricting your own legibility to as few people as possible. I just took a deep breath and accepted that this was the choice I was making and if those people didn't like the book, c'est la vie.
What was your process for revision?
Ewing: I always try to think of specific people. For this book, I thought about my mom. She read the first chapter, and then I asked her to basically recount it back to me. When she was able to do that, I felt like I had achieved something. And I thought about the principal at the school where I had been a teacher. When I revised it, I would stop and ask myself whether those two people would understand what I meant or whether I had to clarify something.

The first people to read a full manuscript were my students at Stateville Correctional Center, many of whom grew up in the places I write about. They're not shy about sharing their opinions, so I showed up in class ready for them to tell me all the things I needed to fix, and they were so laudatory. One of them said, "Thank you for writing this book about my life, about my history." So then I was able to tell myself that - whatever happens - if no one else likes this book, at least the people whose lives have been most impacted by these injustices liked it.
You fully inhabit the first-person in the book. Can you talk about refusing claims of "objectivity"?
Ewing: Well, the obvious point that nevertheless bears repeating is that nothing anyone writes is objective - it's just that some people have the privilege of maintaining the facade of objectivity because their bodies and their perspectives and their beliefs and their epistemologies are seen as neutral and normal, not as deviant, or even really as noteworthy.

Not only do they seem hell-bent on convincing other people of the charade of their own objectivity, but some of them actually seem to really believe it about themselves. I find that amazing. I think it's a funny thing to still be litigating in 2020.

Of course my work is subjective. I'm drawing on the legacies of generations who did this work before me, drawing on critical race theory, drawing on Black feminism, drawing on lots of phenomenological thinkers. The fact that this is still a point of dispute - or that it's even worthy of comment - is its own fascinating bit of sociological data.
One of your major influences as a writer was Gwendolyn Brooks. Which academic authors have been models for you?
Ewing: So many. One thing I don't like is when people refer to "academic" versus "creative" writing. Academic writing is a creative act! I've learned so much about writing from so many types of scholars. Of course, the most obvious is my graduate adviser at Harvard, Sara Lawrence-Lightfoot, who is renowned as a brilliant writer and who deeply values beauty in writing and always encouraged me to do the same, both in word and by example.

Also, when I was in graduate school, I took advanced statistical analysis with a professor named John B. Willett. He thought very intentionally about his pedagogy, and one thing he always drove home was the importance of clear writing - even in a statistics class. Especially in a statistics class, where your imagined audience is less likely to have the context or expertise to evaluate your work, so you need to be really honest with them. He always told us to imagine "a naïve but intelligent audience" - people who maybe don't have the technical background, but they care and they're sharp, so if you explain things they're going to follow you. I think about that idea a lot.

Mary Pattillo was the person whose work I read and thought, "This is the book I want to write." Not just in terms of content, because she writes about the South Side, but in terms of style, her flexibility in moving across discipline and drawing on whatever tools she needs to make a point.

Then the ethnographers - Rubén A. Gaztambide-Fernández's Educating Elites is an ethnography that I love. Laurence Ralph is a good friend of mine, and when it was time to write the index for his first book, Renegade Dreams, I convinced him to let me do it. I had never written an index before, but I just thought I could figure it out, and he let me take a shot at it. So I got to know that book intimately, and it taught me so much.

This feels lofty or self-important to say, but The New Jim Crow: Mass Incarceration in the Age of Colorblindness, by Michelle Alexander, has been a big influence on me because of the way Alexander is able to draw a compelling narrative in that book. She took ideas that were fringe and radical to many people, and lots of scholarship both from her own work and that of her predecessors, and she wove it all together in a way that has been so convincing to so many people and, I believe, truly changed society.

And then of course, the OGs: Zora Neale Hurston, W.E.B. Du Bois, Ida B. Wells. They were "interdisciplinary" before anybody, and they wrote for so many audiences because they understood that that's a necessary practice for our liberation.
So ... Marvel comics?
Ewing: Writing for Marvel has been a lot of fun, and it's just a totally different sense of scale. The number of people who bought Ghosts in the first year was considered a really big success for an academic book. If I sold the same number of comics, it would be considered a failure. It's just a good reminder to not put too much stock into things like that because it's all so arbitrary and relative.

It's also been fun because there are quite a few sociological topics and ideas that I sneak in there, especially in my Outlawed, which is about Congress passing a law that makes it illegal for young people under 21 to be superheroes. Lots to unpack there!
Any advice for grad students about writing?
Ewing: You know, one thing I find odd is that - even though the majority of academics have to make their way in the world by writing - a lot of scholars don't think of themselves as writers or think much about the craft of writing.

Even if you don't think of yourself as a writer, just attending to it a little bit is something all scholars can benefit from. Just that mental shift is a useful intervention.

# data[].attributes.attachment_link.description
If Eve L. Ewing were a superhero, her special power would have to do with words. She is that rare academic able to combine scholarship, poetry, and writing for Marvel Comics.

# data[].attributes.headline
NPC guide for charities on using data viz/ infographics 

# data[].attributes.subject
NPC guide for charities on using data viz/ infographics 

# data[].attributes.attachment_link.readable_text
Charities are getting better at planning their evaluation, collecting and analysing data, both qualitative and quantitative.

But finding the best way to present this data can be tricky.

Choosing the right charts, graphs or infographics to communicate the story your data is telling can influence how well your audiences engages with your findings, and how much can be learnt from them.

Here we draw on advice and recommendations from NPC's measurement seminar held on 14 June 2017 in London.

This paper will:

introduce you to different data visualisation options

explore the benefits

Look at how organisations have approached this.

# data[].attributes.attachment_link.title
Data visualisation: What's it all about?

# data[].attributes.attachment_link.description
Charities are getting better at planning their evaluation, collecting and analysing data, both qualitative and quantitative. But finding the best way to present this data can be tricky. Choosing the right charts, graphs or infographics to communicate the story your data is telling can influence how well your audiences engages with your findings, and how much can be learnt from them.

# data[].attributes.headline
The easy read version of Moving Forward, from Keele PPI

# data[].attributes.subject
The easy read version of Moving Forward, from Keele PPI

# data[].attributes.attachment_link.readable_text
Download PDF
Download the PDF
A guide for the public on the latest physiotherapy research for the health and wellbeing of people with muscle, bone and joint pain
This was published in association with the Chartered Society of Physiotherapy, Keele University and Versus Arthritis. All information in these materials are based on latest evidence and trusted clinical guidance. However, they were produced before the current global COVID-19 pandemic. We would therefore like to remind you to continue to adhere to guidance from the government and healthcare authorities when using or recommending these materials. All up to date guidance in the UK can be found here: https://www.gov.uk/coronavirus

Introduction

Physiotherapy is important to help people manage pain in their muscles, bones and joints and to be as physically active as possible. There has been a large amount of research carried out into physiotherapy treatments over the past 10 years.

This is an easy-to-read guide which will tell you about some of the latest physiotherapy research relating to your problems with muscles, bones and joints (sometimes called musculoskeletal problems). This guide will give you information about treatment or exercise plans developed through research. You can use this information to discuss the treatment options available in your local area with a health care professional, such as your physiotherapist, doctor, nurse or pharmacist.

The guide has been developed for the public and patients, by patients and physiotherapists. It is intended for use by the public and can also be a quick reference guide for health care professionals. The work has been carried out by the National Institute for Health Research (NIHR), Keele University and the Chartered Society of Physiotherapy. It focuses on adults and covers seeking help from a physiotherapist for musculoskeletal problems through to treatment you may have at the hospital or community clinic. It also covers advice which will help you to manage your condition yourself at home. It does not cover broken bones and falls.

In my experience, research evidence is rarely or never shared with us patients. I think if it were then that might encourage us to take the exercises more seriously, if we could see the evidence of them being more effective.

Jennifer Bostock, Patient

How do I use this guide?

This guide gives you more information to help you make informed choices about getting help, understanding your treatment options and learning how to stay well in the longer term.

This has been created from a larger guide called Moving Forward - Physiotherapy for Musculoskeletal Health and Wellbeing. The larger detailed guide is aimed at physiotherapists but contains more information on the studies mentioned in this version. If you are interested in finding out more, you can read the full guide here.

What do I do if these treatments aren't available in my area?

Physiotherapy services can vary from area to area but you do have the right to access treatment which is based on best available evidence. You can ask your GP, physiotherapist, nurse or pharmacist for more local information.

I'm thinking about how I can get treatment for my muscle, bone, or joint problem. What do I need to think about?

It may help you to think about the following questions:

"Do I know how and where I can access physiotherapy - from hospitals, community services, local GP surgeries, workplaces, community centres and leisure centres?"

"Can a physiotherapist explain how the treatment will help me to manage my pain and everyday activities, including work?"

"Can GPs and physiotherapists advise me on exercise videos and websites and refer me for exercise at leisure centres and community physical activity groups?"

"Do I know what to do at home and after physiotherapy, to help me to continue to improve and keep me as healthy as possible?"

"What do I need to do to motivate myself to seek help, seek treatment or advice, and stay well?"

Why is research important?

We need research to provide evidence that treatments work. It is equally important to show if some treatments are not effective in the way we thought they could be.

Research involves looking into a specific problem using a scientific method. This is done by turning the problem into a question and then doing the research to answer the question.
Research can help address common problems. Research can help people back to work.
One in four people are affected by musculoskeletal conditions, which are the leading cause of pain and disability in the UK.
Research can save the NHS money.
30.8 million working days were lost in 2016 due to musculoskeletal problems - the second leading cause of sickness absence at work.

Caring for patients with musculoskeletal conditions is the third largest part of the NHS spend.

Seeking Help

Routes to access physiotherapy:

GP - physiotherapy

First Contact Physio

Self referral

Telephone appointment

Research has shown us...

...that patients can access physiotherapy themselves directly

Some areas of the UK allow patients to refer themselves directly to physiotherapy without having to go through their GP

This is beneficial because it usually allows quick access to physiotherapy care

Getting initial advice from a physiotherapist over the telephone (sometimes known as PhysioDirect) is safe and just as good as it is seeing them in person

Face to face appointments can be made if necessary

...that physiotherapists can prescribe medication

...how to categorise people so that they can have the right care

What can I do when seeking help?

Contact your GP surgery for a consultation with a First Contact Physiotherapist if your surgery has one

Ask your GP surgery how you can directly access physiotherapy over the phone. This is sometimes known as PhysioDirect

You can ask your pharmacist, physiotherapist, GP or nurse for more information about medications

You can talk to your health care professional about the treatment that you both think is right for you

First Contact Physiotherapists (FCP) have advanced skills and are able to care for people with a wide range of bone, joint and muscle conditions

In areas where First Contact Physiotherapy is available, you can go directly to see a FCP at your GP surgery without needing to see a doctor

The FCP can assess you and diagnose what's happening, talk to you about medications and treatments, and give expert advice on how best to manage your condition

They can refer you to specialist services if necessary

Jane's story

Jane has suffered with back pain for the last six weeks, which has become worse. This has stopped her from doing things she enjoys, such as her ballroom dance classes and has resulted in her being off work for the past week. Jane went to her GP practice to make an appointment with her doctor, and the receptionist advised that she could have an appointment directly with a First Contact Physiotherapist based at the practice. During her appointment, the physiotherapist discussed Jane's symptoms and how her low back pain affected her, and they completed a short questionnaire together. The questionnaire helped the physiotherapist to give Jane the most appropriate treatment for her.

The physiotherapist gave her some initial advice about managing her pain, before referring her for group based talking therapies and exercise. After several sessions, Jane started to learn ways of coping with her back pain, pacing daily activities and continuing with her exercises, which helped to keep it at a manageable level. The physiotherapist also helped her to talk to her employer and get back to work. She now goes ballroom dancing once a week, which she loves.

For further information about the research studies which supported Jane to seek help, get treatment and stay well, go to the full review.

Getting Treatment and Staying Well

Treatment given by physiotherapists

Exercise and activity

Advice about your condition

Medication advice

Work advice

Support for self management

Research has shown us...

... physiotherapists can help people at work

Physiotherapists can spot issues that make it hard for people to return to work or keep fit for work

They provide work related advice and support for people who find it hard to work with a muscle, bone and joint problem or return to work after having time off

In some areas of the UK, physiotherapists provide advice over the phone and follow up with a face to face appointment if needed

They can help people to have fewer days off work, perform better at work and give people improved confidence when returning to work

This saves money for society by keeping people in work

... physiotherapists can help with exercise

... physiotherapists can give you useful written advice

"Having being diagnosed with osteoarthritis in both knees I was given a booklet titled 'A guide for people who have osteoarthritis.' This booklet has been well researched by people from various organisations and sufferers of this painful and distressing condition. I found it to be very informative as it covered all aspects of osteoarthritis, offering not only sound advice on everyday living, but ways and means of coping with associated problems. Its use of simple language made it easy to read and the diagrams were a useful visual aid. All in all a helpful, practical and informative guide that all adults over the age of forty-five with osteoarthritis should have access to."

... group based exercise can work for knee pain

Education and exercise groups can help people with long term knee pain, for example, the ESCAPE-Pain programme. Participants receive:

Exercise plans

Strategies for self-managing knee pain

Myth busting

Encouragement to keep doing regular activities after the programme

Although this is group based, advice is tailored to each participant

Classes can take place in leisure centres and physiotherapy departments and are value for money

After completing the programme participants are able to move better and do more which improves their quality of life

Participants take less pain relief and don't need to see their GP as much

... classes plus exercising at home are important

People who exercise at home as well as going to physiotherapist classes have greater improvement in walking, balance and lower limb strength compared to just completing home based exercise

... combined physical and emotional approaches are helpful for some people with low back pain

... physiotherapy can help with shoulder pain

Most people receive treatment for shoulder pain from GPs or physiotherapists

People who experience common causes of shoulder pain see greater improvements in pain and function with physiotherapist-led exercise compared with advice and exercise leaflets

... physiotherapy can support recovery from knee surgery

... physiotherapy can also help people online

There are resources online which support recovery after knee surgery. When a patient uses this tool and goes to face-to-face physiotherapy, their confidence and motivation increase

... acupuncture is not advised in low back and knee pain due to osteoarthritis

Although acupuncture is not currently recommended in the UK for people with low back pain or in osteoarthritis, continued research will help inform future knowledge

... physiotherapists can promote good health

Physical activity can help people to avoid developing other conditions

A key role for a physiotherapist is to provide motivational support to help people to continue with exercises and activity outside of the clinic or hospital

Work

"When I started the programme I couldn't see how the pain in my limbs was going to go. My first goal was just to stand straight. Now I'm not hunched over and I'm beginning to walk properly. I'm still stiff, I've still got arthritis but what I don't have is the pain. I'm not afraid to go out, I'm not afraid to cross the road....I feel better mentally as well as physically."

What can I do when accessing and getting treatment?
Surgery

If work is a concern, talk to your physiotherapist or health care professional
Written Information
Exercise

If you have to have surgery, talk to your physiotherapist about exercise and access to physiotherapy to help you to recover. The availability of physiotherapy after discharge from hospital varies between local areas

Ask for written health information to support you to get better, for example https://startback.hfac.keele.ac.uk/patients/ or https://jigsaw-e.com/patient-focus/guidebook/

Be cautious when using internet search engines about the quality of the information. Some online resources may not be accurate, if in doubt, seek advice from your health care professional

See 'further information' at the end of this document for places you can find out more about research and physiotherapy

Abdulla's story

Abdulla's knee has been troubling him for a few years. His knee pain is worse in the morning but by the time he's up and showered it feels a bit better. Abdulla is an IT specialist and sits all day. He doesn't feel confident exercising and also worries that moving may make his pain worse. His wife suggested he visits his medical practice and from there he booked an appointment with a First Contact Physiotherapist, who asked him some questions and looked at his knee, so that she could fully understand his problem.

Abdulla was told he had osteoarthritis. The First Contact Practitioner had completed specialist training on osteoarthritis and talked Abdulla through his condition, providing him with an information guidebook and told him about a local group based exercise class called ESCAPE-Pain. After this conversation, Abdulla realised that something had to change if he was going to feel better, so he decided to give it a try. Abdulla went along to the local community centre twice a week for six weeks and was given exercises that were tailored to him by a physiotherapist who works in the community. He learned that losing weight and moving more would help his pain, rather than making it worse, which is not what he had thought. He was also able to talk to other people in his situation who gave him some advice on how to fit exercise into his daily life. He has lost a bit of weight and is feeling more positive.

For further information about the research studies which supported Abdulla to seek help, get treatment and stay well, go to the full review.

Staying Well

Treatment given by physiotherapists

Exercise and activity advice

Advice about your condition

Medication advice

Work advice

Support for self management

What can I do when trying to stay well?

You may have ups and downs in the long term, but physiotherapists can help you over time - speak to them if you have any ongoing concerns

Remember to stay as active as possible

Further Information

This guide has been created from a more detailed and comprehensive guide called Moving Forward - Physiotherapy for Musculoskeletal Health and Wellbeing. This larger guide is aimed at patients, local NHS organisations, physiotherapists and NHS commissioners, but contains more information on the studies mentioned in this Easy Read version. If you are interested in finding out more, go to https://evidence.nihr.ac.uk/themedreview/moving-forward-physiotherapy-for-musculoskeletal-health-and-wellbeing/

Are you interested in being involved in research? If you would like to be involved in (but not participate in) research, go to: https://www.peopleinresearch.org/ . Being involved in Patient and Public Involvement and Engagement gives you the opportunity to use your experience of living with a musculoskeletal condition and advise researchers on how to keep their work patient focused and relevant. If you want to be a participant in research (i.e., receive the treatment that is being trialled in a research study) then you can visit: https://bepartofresearch.nihr.ac.uk

For accounts of real life patient and public involvement, visit Keele University School of Medicine's Patient and Public Involvement and Engagement pages: https://www.keele.ac.uk/ppie/.

To learn more about how evidence from research is implemented into real world clinical healthcare, go to: https://www.keele.ac.uk/pcsc/research/impactacceleratorunit/

To find out more about physiotherapy and physical activity, visit the Chartered Society of Physiotherapist's Patient and Public pages: https://www.csp.org.uk/public-patient

# data[].attributes.attachment_link.description
Download PDF Download the PDF This was published in association with the Chartered Society of Physiotherapy, Keele University and Versus Arthritis. All information in these materials are based on latest evidence and trusted clinical guidance. However, they were produced before the current global COVID-19 pandemic.

# data[].attributes.attachment_link.readable_text
Today Cochrane launches new guidance to help people select images to share alongside Cochrane evidence. You can read more in 'Choosing images for sharing evidence: new guidance from Cochrane'

Knowledge Translation (KT) in Cochrane is pleased to announce the launch of ' Choosing images for sharing evidence: a guide' - a new resource developed by Cochrane UK for people choosing images when sharing Cochrane evidence or news.

Images really matter. They are a vital part of telling the story we want to tell when we share evidence. Images can help our target audience to see that content is relevant to them and to make an immediate connection with the topic. Thoughtful image choice can inform or misinform, attract or repel, and they can help challenge assumptions and change the conversation about health conditions. The images we choose also have the potential to reinforce the message that Cochrane is a trustworthy source of reliable information to support health decisions and to encourage people to engage with the evidence.

An early lesson for Sarah Chapman, who has co-authored the guide along with Selena Ryan-Vig, was her choice of image to accompany evidence on weight loss: a warm, positive image of parents with an overweight teenager, the three sitting together and smiling. A reader told her it was such a contrast to negative images which often accompany stories about obesity and was key to them feeling able to share the article with their son.

So while choosing images is a daily task for many working within Cochrane, particularly those involved with dissemination, we should never underestimate their impact, or the challenges!

To support those selecting images, Cochrane UK has developed 'Choosing images for sharing evidence: a guide'.

The guidance is based on, and complements, Cochrane's Checklist and Guidance for disseminating findings from Cochrane intervention reviews. It has been tested with a diverse group of people and is applicable to materials in different languages and ¬¬cultural settings. It features a 15-item checklist and accompanying guidance with many examples of images that may work well, as well as those to avoid, with detailed explanations.

The guide also includes:

a list of sources of images, many of which are healthcare specific and free to use

technical advice to ensure that appropriate permission is sought, credit is given, and to make sure that images are high quality rather than fuzzy

advice on making images accessible to people with impairments, through the addition of alt-text

worked examples of choosing images which illustrate a range of challenges and considerations where the choice is not straightforward.

Read the full guide

# data[].attributes.attachment_link.title
Cochrane launches a new guide for choosing images for sharing evidence

# data[].attributes.attachment_link.description
Today Cochrane launches new guidance to help people select images to share alongside Cochrane evidence. You can read more in 'Choosing images for sharing evidence: new guidance from Cochrane' Knowledge Translation (KT) in Cochrane is pleased to announce the launch of ' Choosing images for sharing evidence: a guide' - a new resource developed by Cochrane UK for people choosing images when sharing Cochrane evidence or news.

# data[].attributes.attachment_link.title
Comic launch and art exhibition challenge obesity stigma - ARC West

# data[].attributes.attachment_link.description
A new comic exploring how our culture stigmatises larger body sizes is launching in Bristol on 19 July, as part of an exhibition running from 17 to 21 July. The Weight of Expectation, or WoE, tells the story of how stigma associated with bodyweight and size gets under the skin and is felt in the [...]

# data[].attributes.attachment_link.description
An accessible summary of the international evidence on teaching 5-16 year-olds.

# data[].attributes.headline
Academic paper - measuring reach, relevance, relationships, refinement in PPI

# data[].attributes.subject
Academic paper - measuring reach, relevance, relationships, refinement in PPI

# data[].attributes.attachment_link.readable_text
Abstract

Objectives

To review the progress of public involvement (PPI) in NIHR (National Institute for Health Research) research, identify barriers and enablers, reflect on the influence of PPI on the wider health research system in the UK and internationally and develop a vision for public involvement in research for 2025. The developing evidence base, growing institutional commitment and public involvement activity highlight its growth as a significant international social movement.

Design

The 'Breaking Boundaries Review' was commissioned by the Department of Health. An expert advisory panel was convened. Data sources included: an online survey, international evidence sessions, workshop events, open submission of documents and supporting materials and existing systematic reviews. Thematic analysis identified key themes. NVivo was used for data management. The themes informed the report's vision, mission and recommendations, published as 'Going the Extra Mile-Improving the health and the wealth of the nation through public involvement in research'. The Review is now being implemented across the NIHR.

Results

This paper reports the Review findings, the first of its type internationally. A range of barriers and enablers to progress were identified, including attitudes, resources, infrastructure, training and support and leadership. The importance of evidence to underpin practice and continuous improvement emerged. Co-production was identified as a concept central to strengthening public involvement in the future. The Vision and Mission are supported by four suggested measures of success, reach, refinement, relevance and relationships.

Conclusions

The NIHR is the first funder of its size and importance globally to review its approach to public involvement. While significant progress has been made, there is a need to consolidate progress and accelerate the spread of effective practice, drawing on evidence. The outcomes of the Review are being implemented across the NIHR. The findings and recommendations have transferability for other organisations, countries and individuals.

Keywords: patient and public involvement, public engagement, health policy

The National Institute for Health Research (NIHR) is the first funder of its size and importance globally to review its approach to public involvement.

The breadth of the evidence collected including from patients, carers and the public, NIHR facilities and institutions, other funders and research organisations and international initiatives.

Evidence-based policy development that is now being implemented.

Review primarily focused on research activities of the NIHR.

Further exploration required to assess equivalence of themes in international contexts.

Background

Introduction

Public involvement is becoming an increasingly important feature of health research, nationally and internationally. Public involvement-as defined by INVOLVE and adopted for the National Institute for Health Research (NIHR) Review in England is undertaken 'with' or 'by' patients or members of the public, rather than 'to', 'about' or 'for' them. It can mean people becoming members of the research team, or part of reference groups, involved in key discussions and decisions, sharing their unique knowledge, expertise and perspective. For example, they may be involved in identifying key research questions, planning study designs, selecting appropriate outcome measures, collecting data, analysing and interpreting data, disseminating and implementing results. This active involvement is different from people participating as passive subjects in clinical trials with little contribution to identifying its need, designing, conducting or interpreting the trial. It also differs from public engagement which creates a dialogue between researchers and the public to improve public awareness and understanding about research. The intention of public involvement is to prioritise and create research that is relevant, acceptable and appropriate from the patient or public perspective. It may be more likely to be implemented, creating greater impact on health and well-being, particularly if patients also have an active role in implementation. It can also help avoid waste in research by ensuring it focuses on issues of importance and benefit for patients, so maximising the potential for democratic accountability to the wider public, who fund a significant proportion of UK research.

Public involvement is growing as a movement in the UK, Canada, Australia, Europe and the USA. For instance, in the USA, the Patient Centred Outcomes Research Institute (PCORI) encourages patients to submit research questions, provide input on funding applications, participate in events and become an ambassador, reflecting many aspects of NIHR activity. The developing evidence base, and growing institutional commitment to public involvement, highlights its growth as an international social movement, gathering strength and creating significant changes in how research is conducted. Public involvement is focusing on how individuals, communities and patient groups can co-produce with researchers and health professionals, knowledge that will underpin their care and treatment. The potential benefits of public involvement in research and on researchers, patients and the wider community have been identified. The beneficial impacts of public involvement on research, researchers, patient and communities include the: identification of patient-relevant topics; grounding of studies in the day-to-day reality of patient experience, enhancing the relevance and appropriateness of studies; identification of patient important outcome measures and solving challenges in securing informed consent. For patients and the public benefits include: feeling listened to and empowered; increased confidence and self-worth and enhanced skills for self-management. Patients involved in research can also benefit in a number of ways which can improve their experience of care. In summary, public involvement has been found to have a significant role to play in improving the effectiveness and efficiency of research and community and patient empowerment are seen as critical elements in helping the NHS meet future challenges.

Nonetheless, in spite of the emerging evidence base for public involvement over the last 20 years, and a noticeable increase in the number of papers published more recently, challenges remain. These include the quality and utility of the evidence base for practice, including poor conceptualisation, varied definitions, limited capture or measurement of progress of public involvement (PPI) impact and relatively few studies looking at later outcomes of PPI in research. A significant difficulty is inconsistent reporting of PPI, with studies often providing partial reporting of their aims, methods and results of PPI in their studies, limiting our understanding of them.

In addition, the practice of PPI is not unproblematic and there is still a significant need to attend to the cultural barriers that inhibit PPI from being completely embedded in research. A recently launched International PPI Network is attempting to create significant culture change in the world of research. In addition, we need to acknowledge that PPI is not always a positive experience with negative impacts reported, particularly on the people involved, if carried out poorly. In addition, the tokenism that can exist has been highlighted and the narrowness of current PPI models, with few organisations mentioning empowerment or addressing equality and diversity in their involvement strategies. The potential for poor practice and negative impact made it even more important we undertook the Review to find out how far we have progressed and to understand the current barriers as well as the enablers.

The UK context

In the UK, the NIHR pioneered a strong policy approach to public involvement including high level support from the Chief Medical Officer. It also established an organisational infrastructure and system for its advancement, delivery and support and INVOLVE, the NIHR funded national advisory group for the promotion and advancement of public involvement. The resulting environment has enabled public involvement to flourish and become a strategic priority for NIHR. Professor Dame Sally Davies (Chief Medical Officer) said,

'No matter how complicated the research, or how brilliant the researcher, patients and the public always offer unique, invaluable insights. Their advice when designing, implementing and evaluating research invariably makes studies more effective, more credible and often more cost effective.'

The need for the Review

After 10 years of the NIHR promoting and advancing public involvement across its growing infrastructure and associated activities, there was a need to review progress within a UK and international context and to develop a vision for the future and to identify cultural and organisational development required to fulfil the vision of public involvement.

This was particularly important because the extent to which policy support for PPI in health research results in any actual influence on health research agendas also remains unclear. In addition, progress has been relatively slow in funders recognising the importance of funding the substantive development of the PPI evidence base, as opposed to funding the practice of PPI as a stream of activity within a study.

As a result, the 'Breaking Boundaries Review' was announced by the Department of Health on 31 March 2014 and reported as 'Going the Extra Mile' a year later. It was the first such Review by the NIHR of its public involvement work and the first of its type internationally. It was designed as an open and collaborative exercise involving patients, the public, other funders and partners nationally and internationally. The Review Group also felt the need for the policy review to be evidence-informed in examining progress made and in developing a vision for the future.

Aims of the Review

1. To review progress made in public involvement in research the UK.

2. To develop a vision for public involvement in research for 2025 and set objectives for the NIHR's leadership in public involvement.

3. To identify cultural and organisational development required to fulfil the vision of public involvement as an embedded component of health research in NIHR.

Methods

Review Panel

A Review Panel was established to shape the Review. Members' expertise included research, policy, research management and patient and public involvement. All members of the panel, including the service users were involved in the planning of the Review, design of the survey, analysis and interpretation and in planning the evidence sessions. Three members were service users. A full list of members is provided in (online supplementary appendix 1).

Ethical aspects

While formal ethical approval was not sought through an NHS ethics committee for this policy review, it was conducted according to Health Research Authority (HRA) principles of good ethical conduct in research which were applied to relevant stages of the Review. Respondents were invited to read an information sheet about the Review before participating. All respondents were assured of anonymity and confidentiality, unless they gave explicit permission to be quoted. Any identifying information was removed from quotes used within the main report and publications. All submissions were stored on the NIHR CLAHRC Northwest London Imperial College computer system in password protected files.

Collection of evidence, experiences and perspectives

The Panel carefully considered the type of evidence and information required to address the aims of the Review. The intention of this policy review was not to undertake a review of literature but to be informed by key studies and systematic reviews. There were no formal criteria for inclusion. All members of the Review Group were asked to identify key papers they thought were relevant to the Review. Moreover, the expert Panel also recognised the importance of developing a rigorous process of data collection and analysis, to contribute to high quality evidence-informed policy recommendations. However, it was also felt that a wider collection of evidence, experience and perspectives was necessary, in order to adequately address the Review questions and to meet the NIHR's public involvement values and principles. Five key approaches were selected to facilitate the breadth of evidence collection, nationally and internationally, summarised in box 1.

Box 1

1. Online questionnaire.

2. Audio and video evidence.

3. Document review.

4. International, third sector and industry representatives evidence panel sessions.

5. Workshops, meetings, social media.

Online questionnaire

A survey monkey online questionnaire was developed in collaboration with the Panel to minimise respondent burden and maximise response. Five key questions were posed to respondents. These were felt by the Panel to align with the aims of the Review and its key themes. The survey questions were also made available as a downloadable word document which could be completed electronically or by hand and posted. A purposive sampling strategy was used to identify a wide range of potential respondents, including individuals and organisations, with the intention of maximising variation in response. Individuals and organisations targeted included patients and members of the public, researchers, clinicians, researchers, user-groups, patient organisations, charities and policy makers nationally and internationally. The initial email with the link to the on-line survey was sent to a range of individuals and organisations, who were asked to cascade it to others nationally and internationally. It was also available on the NIHR INVOLVE website. It was not possible to identify a final sample size because the email was cascaded through the public involvement community and within the NIHR.

Audio and video evidence

Potential respondents to the call for evidence were offered the opportunity to submit evidence in other formats including audio and video, although no respondents opted for this.

Document review

Key documents including papers from the NIHR INVOLVE bibliography such as key systematic reviews and grey literature were used to underpin the Review. No systematic review was undertaken due to limited resources. Instead Review Group members and respondents provided key papers, reviews and reports to provide appropriate background and ensure the underpinning evidence base was considered.

International, third sector and industry evidence

In addition to written submissions, the Review Panel requested input from international colleagues, the third sector and pharmaceutical industry. In total, three panels convened, one panel focusing on international perspectives, one focusing on industry views and one focusing on third sector opinion. Participants were selected based on the knowledge of Review Panel members. The role of the panels was to provide perspectives, insights and any relevant information rather than to have an active involvement role. A set of questions were developed by the Review Panel to support discussion with invited experts which focused on the broader impact of NIHR's public involvement strategy, progress in different sectors, perspectives on how successful the NIHR had been, gaps in provision and areas where it had been less successful.

Workshops, meetings, social media

Members of the Review Panel joined four workshops hosted by the research team undertaking a key NIHR PPI study called RAPPORT in order to gather evidence. Meetings were held in London, Cambridge, Bristol and Newcastle. Social media was used to publicise the Review, generate debate and encourage submission. An additional workshop was conducted with representatives from medical charities hosted by Parkinson's UK in London. The discussions from workshops, meetings and social media provided a wider context for the Review and its final recommendations but they were not included as part of the NVivo analysis.

Patient and public involvement

Box 2 reports PPI using the BMJ Open criteria and (online supplementary appendix 2) reports PPI using GRIPP2 Short Form.

Box 2

* How was the development of the research question and outcome measures informed by patients' priorities, experience and preferences?

The question was identified by the Review Panel who included patients. Patients had a key role in shaping the review questions, the methods, the interpretation of the data and the formation of key recommendations.

* How did you involve patients in the design of this study?

Patients shaped the design of the review, contributing to the design of the methods for data collection. Patients particularly emphasised the importance of qualitative data collection to capture experiences and perspectives.

* Were patients involved in the recruitment to and conduct of the study?

Patients were involved with other panel members to identify and recruit participants. The survey link was cascaded through snowball sampling by patients and PPI leads to key contacts and organisations nationally and internationally.

* How will the results be disseminated to study participants?

The study findings will be disseminated through multiple channels including publication, meetings, conferences, social media and through the dissemination plan for NIHR to actively implement the recommendations.

* For randomised controlled trials, was the burden of the intervention assessed by patients themselves?

Not applicable.

* Patient advisers should also be thanked in the contributor ship statement/acknowledgements.

Patient contributors are thanked in the acknowledgment statement.

Analysis

Data submitted to the Review via the online survey, by email and post was managed using NVivo software for analysis. Thematic analysis was used to identify key themes emerging from the data. Information provided through other methods was not included in this analysis, but rather provided wider context. A particular focus was on identifying common issues and whether narrative patterns emerged across themes and whether any patterns related to the source of the evidence. Once a submission was received, it was logged and given a unique number and saved to the electronic password protected folder on the Imperial system. Initial thematic analysis was conducted by RM to identify recurrent or common themes. This included responses to the Review questions and the submission of any 'open' evidence. A formative summary was developed by RM of emerging themes, which included a high level summary in the context of the volume and sources of evidence. SS, VM and SD checked meaning and interpretation. The emerging themes were discussed with the Panel to check the interpretation of categories and themes. In order to further structure the analysis RM, SS and VM developed the emerging themes into a coding framework. The data were then analysed according to this framework. Development of themes continued until data saturation, the point at which no new major themes are evolving. As the key themes were identified, SD, with RM, SS and VM identified broader conceptual themes which captured core components of the evidence submitted and provided the conceptual underpinning of the future vision and mission. Panel members also drew on the wider evidence which was documented from the discussion with the international, industry and third sector panels, the regional RAPPORT workshops and workshop with medical charities. Two meetings were held with the Review Panel to scrutinise all available evidence, review interpretations of data and prioritise the report themes.

Results

Eighty-two responses were received from an individual, institutional, organisational or collective perspective with some submissions representing the combined views, with reporting respondent characteristics. These included submissions from different parts of the NIHR, medical research charities, universities, industry and third sector bodies. A total of 538 people responded to the online survey. Oral evidence sessions were held with colleagues from USA, Denmark, Germany, Canada and Australia.

Key aspects of Review results are reported in this paper, focusing on positive impacts of PPI, barriers to PPI and then explore how PPI can be undertaken differently. Future delivery is considered and the resulting vision and mission are presented.

NIHR and INVOLVE as positive influences

The evidence indicated that the NIHR's commitment to include the public in research activity has strengthened over the last 10 years and that the presence and activities of INVOLVE has been important in achieving this. In addition, patients and carers reported a range of positive impacts including gaining insight into the research process and learning more about conditions and treatments. They also reported positive relationships with researchers and welcomed the opportunity to gain new experiences, knowledge, skills and contacts. For example:

'It has given me a platform to represent the views of carers and service users in the design and implementation of research. It has given me a role in life as a lifelong carer I have often felt apart from the world of work and have before my PPI work floated without a purpose.' ID 156 Public

Researchers identified a range of positive impacts including changing their research focus to make it more relevant to patients, altering study designs to take account of experience and improved recruitment. Researchers reported feeling more purposeful and connected to the potential beneficiaries of research.

'It has helped to keep my research close to the concerns of service users. Working with service user researchers in designing studies has been important in keeping the research questions and methodology focused on the concerns of those who will ultimately benefit.' ID 332 Researcher/Academic

Barriers to public involvement in research

Respondents identified a range of ongoing barriers to public involvement including public awareness, attitudes, resources, infrastructure, recognition, reward and payment and resources and training.

Public awareness

Although there was greater awareness of public involvement in research, it was felt that opportunities were not accessible to the wider population. Evidence submitted by those working in public health particularly emphasised the risk of reinforcing inequalities and missing opportunities to improve health in communities with the most to gain.

'I think the whole "public involvement" side of things is very good at the moment. However, the information (online) about it, such as the opportunities available and how to apply, could be simplified'. ID 32 Public

Many commented on the need for a high profile and well-crafted communication campaign to raise awareness of health research and demystify the activity in a way that the general population could engage with;

'People need to know what is out there, how they can get involved and why it's happening. The acronyms, that then need to be spelt out and explained along with the many avenues an opportunity comes from, suddenly gets too difficult to decipher unless you're an academic or a clinician... ID 227 Other

Resources

Variability in the availability and allocation of resources to support involvement was a common theme. There was frustration that funding to support relationship building and partnership work ahead of preparing funding applications could be difficult to obtain, but was vital in providing an acceptable standard of good involvement practice in the early stages of research design.

Training and support

Many respondents commented on the need for training and support for public involvement. There was broad agreement that a basic level of support should be available to anybody who becomes involved and a minimum skill level and knowledge about public involvement should be incorporated into researcher training. It was acknowledged there is still significant development needed to embed PPI into the research culture in terms of training.

'Currently the training provided is basic, to explain what PPI is and help researchers plan how to proceed (I have taught on such workshops). ID 74 Researcher/Academic

'Training early career researchers in good involvement practice would help increase confidence and understanding of public involvement and reduce the likelihood of bad involvement experiences... ID 19e Charity

Inconsistency in approach

Some respondents identified difficulties of translating evidence of effective PPI into practice and noted the evolution of ad hoc practice. Many individuals and teams work independently of each other even within the same organisation, institution or region although there are areas where a more collaborative approach is emerging. For some there is a desire to introduce standards while for others a systematic but flexible approach which addresses key elements such as 'why', 'how' and 'who' would be more helpful.

Making all involvement opportunities task specific, time-limited, with clear expectations and guidance on what people should expect from being involved and how their input will be qualified (eg, two-feedback/appraisal process on how people are performing). Providing information on outcomes of previous, relevant research and examples of how PPI was crucial to the effectiveness of the research trial. 'ID 91 Public Involvement Lead/Specialist

While frameworks for planning evaluations exist, the approach to evaluating PPI was varied and inconsistent.

One would be at the start of a study, to plan ahead how to evaluate the impact of PPI on the research, and on the contributors (cf. the PiiAF - Public Involvement Impact Assessment Framework document). The second would be, with other researchers and PPI representatives acting as 'critical friends', to reflect on a study at the end and thus to work out what to do better next time. ID 74 Researcher/Academic

Some respondents highlighted increasing pressure to demonstrate the impact of PPI and ensure it forms part of a University submission to the Research Excellence Framework, the evaluation of research quality in England and Wales.

Leadership

A supportive, competent and influential leadership was perceived as critical to the successful delivery of involvement. Respondents commented on the value of experiential knowledge of public involvement in leaders. Conversely, perceived lack of first-hand experience of PPI and limited or absent empathy with patients were thought to diminish the status of some research leaders. It was suggested that champions of involvement are required from outside established involvement teams to promote changes in organisational and institutional culture.

Challenges

A number of respondents reported poor experiences with PPI including a general sense of frustration from engaging with research, understanding the NIHR and how it links to services. There was also confusion around how to access information and opportunities about becoming involved, suggesting a varied picture of personal practice, organisational commitment and institutional culture.

'I wholeheartedly agree with the intentions and principles of PPI... Unfortunately, I think that lip service is given to PPI by some academics. There is a lack of transparency about how service users who are involved in research studies are selected, approached, recruited and what biases might be operating.' ID 15 Researcher/Academic

...Some organisations are in a frenzy of PPI because they know they have to do it not because they want to. ID 260 Public

Scepticism, professionalisation and confusion

Respondents reported a range of challenges when they undertook public involvement, including scepticism about its value, uncertainty about its underpinning theoretical concepts and unclear practice standards. Challenges also included individuals feeling confusion, apprehension and anxiety about how to conduct involvement in a way that demonstrated a positive impact. Researchers were sometimes wary of using experienced advisers because they perceived that the very experience those individuals started from may evolve and be diluted over time. Others felt the development of such specialist expertise was important and had a beneficial impact.

'PPI architecture tends to call for a small number of individuals to make a massive commitment. This means it is hard to find people who can do it and those who do come forward are probably not representative of the wider population. We should try to design more distributed systems which are less clunky and more dynamic (more 'Web 2.0'). Instead of periodic half-day meetings, break things up into smaller modules/components that can be distributed among more people so it is less of a burden for each person. This could allow more people to get involved and it would democratize PPI.' ID 216 Public Involvement Lead/Specialist

While a range of barriers were identified and challenges were identified, respondents recognised that progress in developing and embedding PPI across the NIHR had been made. This had raised the profile of public involvement, established aspects of good practice and made a difference for patients and their families by ensuring research was more meaningful and focused on improved outcomes.

Doing public involvement differently

Respondents identified new ways of approaching involvement, reflecting a broad range of experience now emerging across the NIHR. A number of key areas for future development emerged.

Practice standards

There was a perceived need to consolidate and use the available evidence to identify gaps in knowledge. The use of continuous improvement was suggested as way to improve practice standards alongside peer review, performance management, self-regulation and independent regulation. The practice standards have now been launched by INVOLVE and were developed through a consultation process. They provide important guidance and form the basis for continuous improvement.

Diversity and inclusion

Current involvement practice was perceived by some as being exclusive and not always fully meeting the requirements and goals of equality legislation. Respondents suggested a range of improvements:

'Shorter interactive and more accessible involvement so that everyone can join'. ID 525 Young People Advisory Group Researcher Adviser

'This is difficult for many organisations. Seeing role models like themselves-old/young, non-white, not wearing grey suits-all these would help. People from unrepresented areas may believe that it's not for the likes of them to get involved so showing people who are like them, getting on and making a difference, is likely to be helpful.' ID 29 Public

Respondents also commented on the need for involvement to more closely reflect diversity in the population. It was felt that if leaders and role models were promoted and recruited from varied backgrounds, this would encourage more people to become involved.

'Be more aware of community centres, faith centres as sources of research participants. Acknowledge public health expertise in their local communities; community support officers etc. Get Healthwatch involved. Local radio stations (eg, we have had health/health research message put over local Punjabi radio) Research in the evenings? Weekends? Think differently about when research is done and where it is done. Think who are we going to get participating at that time? The times are usually convenient for the researchers rather than the participants. Make it clear that research studies welcome those with access and mobility difficulties.' ID 240 Other

The future design and delivery of public involvement in NIHR

Flexible evidence-based methods

Some respondents suggested that the methods of involvement should be evaluated for their effectiveness. For example, the common practice of inviting one or two patients to join committees was perceived by some to be of limited value and likely to become less attractive as an approach. Many respondents felt that knowledge of the 'ingredients' of effective involvement needed to be developed.

'Involve in the design and delivery as wide a constituency as possible-those with 'knowledge', 'experience' and 'expertise', but also those who may be able to assist by asking questions, because they have different backgrounds.' ID 23e Researcher/Academic

Better identification of the key points where involvement makes an impact was also regarded as important, particularly in relation to deciding research priorities, funding decisions, and translating findings into real benefits for patients. The need for greater openness and transparency in facilitating conversations with the public was also considered important. This would enable patients or members of the public to identify more collaborative or user-led approaches.

'One of the most widely mentioned 'metrics' of improved Public Involvement (PI) would be a growth in collaborative or user-led research. Suggestions for other specific indicators included: routine PI sections in annual reports and evaluation of PI in NIHR funded research project reports; increased representation of people from minority groups; and better recruitment to trials (the latter two suggestions being offered by public contributors).' ID 15e RDS collective

Third sector representatives and community voluntary organisations were identified as potential partners who could more effectively engage with people locally and nationally;

'The voluntary sector could play a key role in both the design and delivery of NIHR funded research. NIHR could establish much stronger links between research charities (such as the Wellcome Trust, Cancer Research UK, the McPin Foundation) and NIHR funded bodies in order to jointly commission and fund research.' ID 35e Voluntary Sector.

Developing a future vision

Many respondents, while recognising progress made so far, expressed the desire to be ambitious for the future. For some, this meant refining practice. For others, it was much more about reframing the purpose of involvement entirely, working differently and recognising the positive connections between engagement, involvement and participation.

Valued practice

Respondents felt that the debates about the need for public involvement should mature into discussions about what forms of involvement work in particular contexts. Individuals wanted to place their focus on improving the quality of their PPI in creating relevant research.

'PPI should be routine-how things are done, not an optional extra. This should be embedded throughout the NHS so that all users of NHS services can expect that research evidence (is) supported by robust PPI. PPI isn't simply an issue for research but for patient care, too.' ID 15e RDS collective

'By ten years, public involvement should have a much greater profile than what it has now. Members of the public and patients should know that we actively do research in an array of disease areas or conditions and that there are many opportunities for them to take part in this.' ID 20e Public Involvement Lead/Specialist

Key concepts

Analysis of the themes emerging from the evidence submissions and synthesis of the data and discussion with the Review led to the development of a mission and vision as presented in (online supplementary appendix 3). Three concepts for measuring success were suggested:

Reach: the extent to which people and communities are engaged, participating and involved in NIHR research including the diversity of this population.

Refinement and improvement: how public involvement is adding value to research excellence as funded by the NIHR.

Relevance: the extent to which public priorities for research are reflected in NIHR funding and activities.

In addition to these three concepts, as the implementation of the recommendations has progressed, a fourth theme has emerged, relationships. This has been recognised as a significant determinant of success in strengthening public involvement.

Underpinning these concepts is support for the principles of co-production as the basis of the NIHR's approach in the future. These draw on the Boyle definition which emphasises the importance of developing close collaborations based on valuing people as assets with knowledge; recognising the expertise and perspective people bring to involvement; promoting good relationships and networks; a perception that all people involved can benefit from public involvement; recognising that involvement often involves an exchange of some type; the process of involvement is important and requires facilitation; that there is a need to change some of the professional boundaries that may inhibit more collaborative forms of work.

Implementing the Review

In addition to the vision and mission, the Review led to a range of recommendations presented in (online supplementary appendix 4), designed to strengthen co-production and collaboration at the heart of research.

These recommendations are now being actively taken forward across the NIHR. 'Going the Extra Mile' was signed off by the Chief Medical Officer, Professor Dame Sally Davies, in September 2015 with an instruction to NIHR leaders, organisations and staff to support its implementation. This position has been supported with the decision by the Department of Health to regularly audit the NIHR's progress in public involvement using the report recommendations as its starting point. Lines of responsibility and accountability for public involvement have been strengthened accordingly.

The INVOLVE Co-ordinating Centre's future work programme reflects the priorities highlighted in the report and is the NIHR's national lead of diversity and inclusion, learning and development and community (incorporating co-production). A national champion for diversity and inclusion has been appointed. The UK continues to be the only country where national government funds and supports an organisation focused on public involvement in research.

INVOLVE, in partnership with the NIHR's Research Design Services (RDS) organisation, is in the process of supporting and developing regional networks to facilitate collaborative working at local and regional level. These will connect with existing fora and partnerships and will reach across traditional research, services boundaries. Work is ongoing to refresh the way in which the NIHR presents its public involvement work beginning with the new corporate website.

A set of national standards designed to improve the quality and consistency of public involvement in research have been launched. These are based on a set of values and principles for public involvement published by INVOLVE in 2015 and a series of workshops to discuss how best to evolve standards that organisations can operationalise and against which progress can be assessed. A number of NIHR organisations will pilot these standards. This work will feed into emerging thinking about current reporting requirements on initiatives and how these can be improved in ways which will best promote continuous improvement. The Review Panel considered and rejected the notion of a formal regulatory regime for public involvement in favour of an approach which supported and encouraged organisations and their staff.

The programme of reform that is now underway is on top of ongoing innovation in public involvement activity generally. The expansion of the Patient Research Ambassadors Initiative (PRAI) across the NHS, the involvement of young people in research, promotion of public contribution to research through its 'OK to Ask' campaign, and growth of the James Lind Alliance Priority Setting Partnerships programme are all flagship initiatives which continue to receive support from the NIHR within the new strategic framework and approach.

Discussion and conclusion

Arguably the NIHR is the leading public research funder globally when it comes to the steps it has taken to make public involvement a core principle for how it funds and supports research excellence. It is perhaps inevitable that it should therefore be the first to attempt a review of the size and scale of 'Going the Extra Mile'.

While the main focus was England, its messages have potential relevance for other countries developing their public involvement, reflecting wider societal changes towards a democratisation of research that enhances the quality of research. Public involvement in the NIHR has made significant progress in the last decade, enabled by a strong policy and infrastructure and implemented by a community of practitioners who recognise the value of actively involving patients and the public in research. We acknowledge the Review was limited to some extent by the lack of a formally conducted review of the literature and would recommend this for future policy reviews.

The Review identified a range of barriers including limited awareness of opportunities, lack of diversity, resistant attitudes to involvement, inconsistent levels of resources, systems that work in different ways, patchy training and support and variable organisational implementation. A key finding from the Review is the need for a step change, increasing the rate of change and with a greater focus on embedding public involvement in research culture, so that it becomes 'business as usual'. The NIHR implementation plan is now starting to address this need but its ambition needs to be recognised as the rule and norms of research need to change for involvement to properly flourish. The Review was strengthened by the involvement of the Review Group, including the service users who ensured there was strong PPI in this example of evidence-informed policy development.

While the focus was national, there are important international implications from the Review. Co-ordination and collaboration across organisations, funders and systems nationally and internationally to deliver high quality public involvement is vital. Public involvement needs to be underpinned by a strong evidence base which enables the development of effective practice that is continuously improved and creates a positive impact. The promotion of opportunities alongside the creation of greater diversity of individuals involved will help ensure a wide range of voices are heard.

In analysing the evidence gathered for the Review, four new key concepts emerged of importance to the field of PPI; reach, refinement and improvement, relevance and relationships. Relationships was added, in the implementation phase, as an additional key concept, vital to the delivery of the future vision. Reach refers to the extent of involvement, engagement and participation, ensuring diversity among members of the public who become involved. To achieve 'reach' researchers and research may need to work closely with the public to develop new ways of working to ensure diversity and inclusion are embedded within involvement. Relevance is focused on the extent to which public priorities for research are reflected in funding and activities. In an era of limited public funding, there is an ethical imperative to ensure public monies are spent on research that patients feel has most relevance to their lives and the beneficial impact it may create. Relevance also refers to ensuring the research questions in a study are focused on what is acceptable and appropriate from the patient perspective. Drawing on evidence to refine practice through continuous improvement underpins attempts to develop relevance.

The thematic analysis underpinned the development of the vision of ' Going the Extra Mile', of a population actively involved in research to improve health and well-being for themselves, their family and their communities. The mission of ' Going the Extra Mile ' is of the public as partners in everything we do to deliver high quality research that improves the health, well-being and wealth of the nation. Underpinning this future mission is the principles of co-production, which emerged as a new and important way of understanding the step change required in public involvement. At its heart is the co-production of knowledge and evidence through the creation of ways of working, cultures and systems that support this. From a research perspective, co-production offers a way of constructing 'complete' knowledge that includes all relevant aspects of a concept.

'In recent years an approach to research that embeds active participation by those with experience of the focus of that research has been championed both from the human rights perspective, that people should not be excluded from research that describes and affects their lives, and from a methodological perspective in terms of rigorous research: '... knowledge constructed without the active participation of practitioners can only be partial knowledge.'

We emphasise that co-production emerged from the Review and during the process we were not able to explore the concept fully. This in-depth exploration has been conducted by INVOLVE, drawing on a review of literature to inform the development of guidance on co-production and is now published.

In conclusion, 'Going the Extra Mile' challenges researchers, research and the organisations and institutions that fund and promote it to go further in working alongside citizens. The Rome Declaration on Responsible Research and Innovation in Europe in November 2014 emphasises the need to evolve a more inclusive approach to research.

'Hence, excellence today is about more than ground-breaking discoveries-it includes openness, responsibility and the co-production of knowledge.' p. 1

Our vision for the future is ambitious and may take many years to achieve. At its heart, there is a fundamental reorientation of research, its focus, how it is undertaken and how knowledge is created. As others have said, 'if PPI were a drug, it would be malpractice not to prescribe it'. The benefits of co-production could lead us to a new era in research, one that is focused on the co-production of knowledge that benefits humanity in a new and fundamental way. Our ambition is that the 'Going the Extra Mile' Review escalates such paradigm shift and contributes to changing the nature and role of research, for the benefit of patients, public and wider society.

Acknowledgments

We would like to thank the Going the Extra Mile Review Panel (online supplementary appendix 1) who worked very hard to shape the Review and participated in all aspects of it. We would like to thank the patients who formed part of the Panel and other patient advisors. Finally, we would like to thank everyone who participated in the generation of evidence for the Review and shaping its recommendations and conclusions.

Footnotes

Contributors: SD was chair of the Review Group, led on the development of the strategic vision and mission and recommendations in collaboration with the Review Group. SS was vice-chair and supported SD. RM led on the operational aspects of the Review, including the analysis. VM, SS and SD were involved in analysis. SS led the writing of the analysis paper, drawing on the Going the Extra Mile policy report written by SD and RM in collaboration with the Expert Advisory Group.

Funding: This work was supported by National Institute for Health Research. SS is part-funded by CLAHRC WM. RM is fully funded by NIHR CLAHRC NWL. The 'Going the Extra Mile' policy review was funded by the National Institute for Health Research. This article presents independent research funded by the National Institute for Health Research (NIHR) under the Collaborations for Leadership in Applied Health Research and Care (CLAHRC) programme for North West London and West Midlands.

Disclaimer: The views expressed in this publication are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care.

Competing interests: None declared.

Patient consent: Not required.

Provenance and peer review: Not commissioned; externally peer reviewed.

Data sharing statement: No additional data available.

Author note: Dr. Virginia Minogue was at NHS England until October 2017.

References

2. National Institute for Health Research. Going the Extra Mile: Improving the nation's health and wellbeing through public involvement in research. The final report and recommendations to the Director General Research and Development/Chief Medical Officer Department of Health of the 'Breaking Boundaries' strategic review of public involvement in the National Institute for Health Research. 2015.

6. Mockford C, Staniszewska S, et al. . A systematic review of the impact of patient and public involvement on health and social services. International Journal of Quality in Health Care 2012. [ Google Scholar]

11. Coulter A, Collins A. Making Shared Decision-Making a reality. No decision about me, without me. The Kings Fund 2011. [ Google Scholar]

12. Robert G. et al. . Chapter 14 in [ Understanding and Using Health Experiences : Ziebland S, Coulter A, Calabrese J, Locock L, Participatory action research: using experience-based co-design to improve the quality of healthcare services. Oxford: Oxford University Press, 2013. Google Scholar]

16. Staniszewska S, Brett J, Simera I, et al. . GRIPP2 reporting checklists: tools to improve reporting of patient and public involvement in research. BMJ 2017;3:j3453. [[ PubMed] PMC free article] [ Google Scholar]

# data[].attributes.attachment_link.title
Reviewing progress in public involvement in NIHR research: developing and implementing a new vision for the future

# data[].attributes.attachment_link.description
To review the progress of public involvement (PPI) in NIHR (National Institute for Health Research) research, identify barriers and enablers, reflect on the influence of PPI on the wider health research system in the UK and internationally and develop ...

# data[].attributes.headline
NICE video on how they are changing their guidance to be more usable 

# data[].attributes.subject
NICE video on how they are changing their guidance to be more usable 

# data[].attributes.attachment_link.authors[].name
National Institute for Health and Care Excellence (NICE)

# data[].attributes.attachment_link.title
NICE Connect: Delivering our vision for the future

# data[].attributes.headline
UX guide on testing the Usability of documents 

# data[].attributes.subject
UX guide on testing the Usability of documents 

# data[].attributes.attachment_link.readable_text
1. Tell me in your own words.

Paraphrase testing is a great way to find out whether people understand the messages in a document....

In a paraphrase test, each participant goes through the document a bit at a time. After reading each bit, the participant tells you, in his or her own words, what that bit said.

Why

Paraphrase testing is a great way to find out whether people understand the messages in a document such as an email message, letter, notice, jury instruction, lease, warranty, or a Web page whose aim is to convey information.

This method also works for longer documents that people must read from beginning to end-such as a contract or legal brief.

You'll learn whether

the document's organization makes sense to your participants

your paragraphs and sentences are short enough for people to get all the parts

the words you use are ones your participants understand

You'll also hear whether participants use different words from those in the document.

How

Before you start the test, decide where the breaks between bits will be. A bit might be a sentence, a short paragraph, a list, or one provision of a contract. You want each bit to be both meaningful and small enough for your participants to grasp.

In a one-on-one interview-whether in person or remotely-ask a participant to open or go to the document.

Start by asking a few general questions such as:

What's your first impression of this document?

Who is it from?

What do you think it's about?

Then, ask the participant to read whatever you've decided is the first bit. For example: "Please read just the first paragraph, then stop."

We often ask participants to read out loud. That lets us hear what words the participant reads easily and words the participant stumbles over.

You might tell the participant that it's okay to be unsure how to say some of the words out loud. If your participants are not comfortable reading out loud, you can have them read to themselves. The important part is having them tell you, in their own words, what they just read.

At each stopping point, ask the participant, "Please tell me, in your own words, what that bit you just read means to you." Take notes on these points:

what they understood correctly

what they misunderstood

what they left out

words they used that are different from the words in the document

You and the participant continue in this way through the document-or the part of the document you're interested in-with the participant reading and then paraphrasing.

At the end of this test session, you might ask a few more questions, such as the following:

You might probe for the participant's understanding of specific words. For example, "The letter uses the words gainfully employed 'What does gainfully mean to you here?' or 'What would be an example of being gainfully employed?'"

"What would you do now that you've read this letter?"

"How do you feel now about the organization that sent this letter?"

2. Mark positives and negatives.

Plus-minus testing is a good way to get people's reactions to a document.

Dutch researchers Menno de Jong and Peter Jan Schellens developed the plus-minus testing technique, and they describe it this way:

"Participants are asked to read a document and put pluses and minuses in the margin for positive and negative reading experiences. After that, the reasons for the pluses and minuses are explored in an individual interview."

Why

Plus-minus testing is a good way to get people's reactions to a document. You can choose what plus and minus mean for your document, depending on its purpose and what you want to learn.

You can use plus-minus to

probe for your participants' opinions about what is clear and what is not clear to them

get people's emotional reactions to a document

investigate some other attribute such as confidence in the organization

We find that, when we ask for positive and negative reactions in general, we often get comments about wider topics such as the tone of a document alongside comments about whether the document makes sense.

You can also probe more deeply for specific emotional reactions. For example, you might want to find out how well the document transmits a brand's values. You could ask participants to write a plus sign where the document makes them feel positive about the brand and a minus sign for negative feelings.

Anything that has both positives and negatives can be the focus of a plus-minus usability test. Pete Gale describes using plus-minus to explore how a document did and did not make people feel confident about a government service, in his blog post " A Simple Technique for Evaluating Content."

How

Decide what you want to focus on for plus-minus testing. Write a short set of instructions so participants know what plus means for this usability test and what minus means. Give or send the instructions and the document to each participant.

If you are doing this testing in person or sending people the document on paper, you might ask participants to use pens with different colors-perhaps yellow to mark pluses and blue to mark minuses. (We avoid red and green to avoid issues for people who have color-deficient vision.) Ask participants to write comments about their pluses and minuses as they mark up the document.

For participants who are working online, we've found that it's easier for them to type + or - at the start of each comment. Most editing programs seem to make using highlighting a bit tricky.

If you send the document to the participants to mark up on their own time, we recommend arranging a brief interview to go over their general reactions and their specific markings.

3. Find an answer or do something.

Watch and listen as your participants try to use the document to find and understand the information.

You may be working on a document that people won't read from beginning to end. For example, people rarely read through annual reports, benefits handbooks, insurance policies, manuals, and many other types of documents. They refer to them only to get the answer to a question, check a specific fact, or follow some instructions.

Task-based testing of a document for which finding information is as important as understanding it is much the same whether the document is on paper or digital-on a Web site or in an app. Watch and listen as your participants try to use the document to find and understand the information.

Usability testing of Web sites typically uses techniques that were developed for usability testing of software and software documentation. Both of us were doing task-based usability testing for printed manuals and other paper documents before Web sites existed.

Why

The reader here is really a user who wants to read only what is necessary. Finding the right place is critical. The reader first finds where the information is; then perhaps skims and scans to read just enough to get the answer, check the fact, or follow the instructions.

How

Just as in typical usability testing for Web sites, think about what you want to learn. You might decide to write scenarios for specific tasks for your participants. Or you could ask each participant what task they might have that is relevant to this document. Give or send the document to each participant or have participants go to it online.

As with other techniques, you might start with the following general questions:

"What's your first impression of this document?"

"Who is it from?"

"What do you think it's about?"

Give the participants the scenarios, one at a time, or ask them to tackle each of the tasks they've identified themselves.

Watch and listen-whether remotely or in person-as they use the document. If the task is to get an answer or check a fact, the participants tell you what they found. If the task is to act-or follow some instructions-you'll watch and listen as they act-or explain how they would act if it's not something you can have them do then and there.

You might end with the same types of questions or surveys that you use when you're conducting usability testing for Web sites, products, or services.

How Do I Choose Which Technique to Use?

Let's return to the point about purpose. As Table 1 shows, The intended purpose of the document can help you decide how to test it.

Purpose of Document

Best Technique

To explain something in detail-whether to read now, refer to later, or act on after reading

Paraphrase testing

To give a general understanding of a topic

To create an emotion-for example: reassure, create confidence, or build trust

Plus-minus testing

To give answers or instructions

Task-based testing

Can I Use These Techniques Remotely?

Remote testing has the advantage that participants are in their own space. But that also means you're a virtual visitor in their space.

Yes, certainly. Both of us have tested the usability of documents in person and remotely.

Remote testing has the advantage that participants are in their own space. But that also means you're a virtual visitor in their space.

The UK Government Digital Service has thoughtful advice about conducting remote user research in their article, " Conducting User Research While People Must Stay at Home Because of Coronavirus." You'll find their good advice useful even when the pandemic is over.

Testing Remotely

You might want to look into tools that are specifically for remote usability testing-such as those in Justinmind's list of usability tools. However, you might not need a specialized tool. You can do live, remote testing using a free chat or video-conferencing tool. We've used Skype, Google Hangouts, and WhatsApp calls. They all offer enough video for a usability test.

Ideally, find out what tool each participant is most comfortable using and be prepared to learn that tool for that particular test.

You can share a document with your participants by screen-sharing or by creating a permanent or temporary location for a document that they can use online. If you need to keep a document confidential, secure it with a password or some other identification process.

One government organization refused to let Caroline use most of the popular tools on her government-issued notebook computer. But they let her use a separate, stripped-down computer that was not linked to their network. That way, she could do usability testing with the tools that the participants were most likely to want to use.

For more ideas, see the crowd-sourced tools and tips for remote research that user researchers at the UK Government Digital Service have put together.

Testing Remotely with Paper

Some participants might not use digital technology. You can still include them by using mail, paper, and your phone. Here's how:

1. Put the document into an envelope with the instruction: "Please do not open this envelope until our phone call."

2. Write a short letter with the arrangements for the phone call-including the time you'll call them, how long the interview will take, the purpose of the call, and how you will use the results.

3. Put the letter and the envelope containing the document into a larger, outer envelope.

4. Mail the whole package to the participant.

You won't be able to see these participants as they work through the document. During your interviews, ask each participant to have their document in front of them. Make sure that you have an exact copy in front of you, too. Explain to participants that they need to help you by telling you what they're looking at, so you can keep up with them.

If you opt for a plus-minus test, once you and a participant have finished your discussion about the document, ask the participant to mail the marked-up copy back to you. Make the mail-back part of this process easy for participants by including a stamped, self-addressed envelope in the package.

Bottom Line: Testing Documents Is Necessary and Powerful

Usability testing helps you find out how well your document works for the people who use it.

Documents often have very serious legal, economic, or medical consequences.

Usability testing helps you find out how well your document works for the people who use it.

Don't stop testing just because you have to test remotely.

Testing documents is fascinating. You'll learn what really works-and what doesn't-directly from people who need to use the document in real life.

Most of all, testing documents is powerful. When we create documents, we can become too familiar with them. The results from testing cut through that familiarity and help us-and our colleagues-make better decisions about what to change and what to keep.

Acknowledgments

We would like to thank Carol Barnum, Ella Botting, and Benjy Stanton for helping us by commenting on this column.

Reference

Menno de Jong and Peter Jan Schellens. " Toward a Document Evaluation Methodology: What Does Research Tell Us About the Validity and Reliability of Evaluation Methods?" IEEE Transactions on Professional Communication, October 2000. Retrieved May 3, 2020.

# data[].attributes.attachment_link.keywords[]
user experience ux strategy customer experience ui gui human-computer interaction ixd hci chi user-centered ucd user interface visual icon design guidelines patterns consulting

# data[].attributes.attachment_link.keywords[]
design usability evaluation review usability testing user requirements modeling research studies heuristic evaluation expert review competitive analysis consulting

# data[].attributes.attachment_link.keywords[]
information architecture ia navigation design consulting

# data[].attributes.attachment_link.keywords[]
user goals tasks task models modeling analysis personas scenarios

# data[].attributes.attachment_link.description
Paraphrase testing is a great way to find out whether people understand the messages in a document.... In a paraphrase test, each participant goes through the document a bit at a time. After reading each bit, the participant tells you, in his or her own words, what that bit said.

# data[].attributes.headline
Academic paper - disseminating recommendations to patients 

# data[].attributes.subject
Academic paper - disseminating recommendations to patients 

# data[].attributes.attachment_link.readable_text
8455 Accesses

14 Citations

41 Altmetric

Metrics

Abstract

Background

The aim of this systematic literature review was to assess what dissemination strategies are feasible to inform and educate patients about recommendations (also known as guidelines).

Methods

The search was performed in February 2016 in PubMed, Ebsco/PsycINFO, Ebsco/CINAHL and Embase. Studies evaluating dissemination strategies, involving patients and/or reaching patients, were included. A hand search and a search in the grey literature, also done in February 2016, were added. Searches were not restricted by language or publication type.

Publications that referred to (1) guideline(s) or recommendation(s), (2) dissemination, (3) dissemination with patients/patient organisations and (4) dissemination to patients/patient organisations were included in this article. Criteria 1 AND 2 were mandatory together with criteria 3 OR 4.

Results

The initial search revealed 3753 unique publications. Forty-seven articles met the inclusion criteria and were selected for detailed review. The hand search and grey literature resulted in four relevant articles. After reading the full text of the 47 articles, 21 were relevant for answering our research question. Most publications had low levels of evidence, 3 or 4 of the Oxford levels of evidence. One article had a level of evidence of 2(b).

This article gives an overview of tools and strategies to disseminate recommendations to patients. Key factors of success were a dissemination plan, written at the start of the recommendation development process, involvement of patients in this development process and the use of a combination of traditional and innovative dissemination tools. The lack of strong evidence calls for more research of the effectiveness of different dissemination strategies as well as the barriers for implementing a strategic approach of dissemination.

Conclusion

Our findings provide the first systematic overview of tools and strategies to disseminate recommendations to patients and patient organisations. Participation of patients in the whole process is one of the most important findings. These findings are relevant to develop, implement and evaluate more (effective) dissemination strategies which can improve health care.

Peer Review reports

Background

In health care, many guidelines or recommendations for the management of diseases are developed. These recommendations are primarily developed to inform health professionals to improve daily routines of medicine. Dissemination and implementation of these recommendations are often focussed on professionals [ 1], not on patients. As a result, in many countries, patients are not aware of the existence of recommendations, are not able to access the publications or do not fully understand the English language and the academic and medical terminology. Patients have therefore limited access to information to get an adequate understanding of their disease and treatment options. One way to empower patients to make more informed choices is the development and dissemination of patient or lay versions of the recommendations. Providing lay versions might be seen as a key component of good care [ 2], especially because patients increasingly want to be involved in decision-making processes [ 3]. Involving good-informed patients in their treatment decisions is assumed to lead to more personal comfort with the treatment decision [ 2], better treatment adherence and motivation, reduction of the number of interventions in some cases [ 4] and more control by patients [ 5].

Improvement of health care can be enhanced by the dissemination of recommendations that are easy to find and easy to understand by patients. Making these recommendations accessible for patients requires an extra effort from health professionals or patient organisations to translate the English version into another language and to adjust the content of the recommendations to the national context, and the specific information needs of patients without losing scientific rigor [ 6].

Studies on the dissemination of recommendations towards professionals are extensively described in the literature (e.g. [ 7- 12]). However, it is not systematically investigated which strategies are feasible for the dissemination of recommendations to patients. The aim of this systematic literature review is to assess the feasibility of dissemination strategies to inform and educate patients about recommendations or guidelines. This review is part of the European League Against Rheumatism (EULAR) project to develop a practical guide for patient organisations to improve the dissemination of EULAR recommendations to people with rheumatic and musculoskeletal diseases. This review will hopefully enable other national organisations of patients and health professionals to develop their own strategy to disseminate national or international recommendations to patients. In the context of this review, the word guidelines and recommendations are used as synonyms.

Research question

What dissemination strategies are feasible to inform and educate patients about recommendations or guidelines?

Methods

Searches

This systematic literature review (SLR) followed the process recommended by the Centre of Reviews and Dissemination [ 13]. The scope of the SLR was discussed by a EULAR Task Force representing eight countries, covering all regions of Europe. It comprised seven patient experts, six health professionals (three rheumatologists, three health professionals) and one dissemination expert. The group followed the EULAR Standardized Operational Procedures [ 14] and met twice.

The search terms and strategies were discussed in the research team (TA, MB, KS, MdW). A review protocol was developed by KS and JK, based on the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [ 15]. PubMed, Ebsco/PsycInfo, Embase.com and Ebsco/Cinahl were searched on 4 February 2016, all from inception, by KS and JK. The following terms were used (including synonyms and closely related words) as index terms or free-text words: 'guidelines' or 'recommendations' and 'dissemination' and 'patients' or 'consumers'. The full search strategies for all databases can be found in Additional file 1. Searches were not restricted by language, publication type or date.

Duplicate articles were excluded. All languages were accepted. A search in the grey literature was added, using the method of 'communication with experts' and 'snowballing' [ 16, 17]. The reference lists of articles from the search that fit the criteria were scanned for missing papers.

Study inclusion criteria

To be included in our final article, the article had to refer to (1) guideline(s) or recommendation(s), (2) dissemination, (3) dissemination with patients/patient organisations and (4) dissemination for patients/patient organisations. Criteria 1 AND 2 were mandatory together with criteria 3 OR 4.

Study selection and data extraction

After deleting duplicates, all articles (title and abstract) were screened for inclusion, independently by two reviewers (KS and MB). Discrepancies were resolved by discussion, supported by two of the authors (TA and MdW). Abstracts that met the inclusion criteria were selected for detailed, full text review. Reasons for exclusion were dissemination towards professionals instead of patients/patient organisations. The selected abstracts were complemented by articles and grey literature identified through a hand search.

Data synthesis and presentation

We found qualitative studies, surveys, descriptive studies, opinions, editorials and conference abstracts. Because of the nature of these studies, a statistical synthesis was not appropriate. Therefore, a content analysis [ 18] was done and themes were extracted, using codes. Both reviewers (KS and MB) conducted the analysis separately and then explored similarities and differences between the studies. The research team then synthesised and interpreted the evidence as it related to the purpose and aims of the review.

Quality assessment

Two reviewers independently assessed the methodological quality of included studies (KS, MB). The Quality Assessment Tool for Quantitative Studies was used for assessing the quality of the quantitative studies [ 17] (see Additional file 2). This tool leads to an overall methodological rating (strong, moderate or weak) taking important elements into account such as selection bias, study design, confounders, blinding, data collection methods, withdrawals/dropouts, intervention integrity and analysis [ 17].

We used the Quality Assessment Tool for Qualitative Studies to assess the methodological quality of the qualitative studies [ 19] (see Additional file 3). Five aspects were taken into account: the aims of the research; research methods and design; sampling; data collection and analysis and results, discussion and conclusions. This tool has similarities with the tool for assessing quantitative research and is recommended by Cochrane.

The level of evidence was categorised according to the design characteristics of available studies using an established hierarchy [ 20] (see Table 1).

Table 1 Categories of evidence [20]

Results

In total, we identified 47 articles that met the inclusion criteria, 43 through the SLR and 4 through the hand search. After reading the full text, 21 articles were included in this review (see Fig. 1). The articles are published between 2002 and 2014 but most of them are published between 2010 and 2014 (15). Most of the articles are published in 2013 (8 articles). The articles are mainly about the field of rheumatology, asthma/COPD and diabetes. The authors are from the Canada (7), Europe (7), USA (5), Russia (1) and Africa (1).

Most of the 21 papers can be described as opinion papers or papers that describe strategies that have been used or might be effective. Only a small amount of papers are based on a RCT or a survey or qualitative study. The 21 papers were assessed for the level of evidence and were scored level 3 (descriptive studies, such as comparative studies, correlation studies or case-control studies) or 4 (expert committee reports or opinions; Table 2). Only one study was assessed as having a level of evidence of 2B (controlled study). Although Cochrane suggests to exclude descriptive papers, editorials or opinion papers, we have included those papers because of the lack of articles with a level of evidence of 1 (meta-analysis of randomised controlled trials) or 2 (single randomised controlled trials). The analysis of the articles showed that, besides information about dissemination to patients or patient organisations, many articles paid attention to patient involvement in dissemination. This result section will, therefore, address both themes: information on dissemination to patients or patient organisations and patient involvement in dissemination.

Table 2 Characteristics of selected papers

Dissemination to patients or patient organisations

Dissemination plan

Lay version

Producing a lay version of the original recommendations is the second factor that may improve the dissemination of recommendations [ 6]. A lay version enables patients to better understand the goals of treatment, the different treatment options and the benefits and risks of each option. Patients who have access to lay versions are better equipped to prepare themselves for the consultation with their health care provider and are expected to become an active partner in their own treatment [ 6].

Boulet et al. recommend to take the following aspects into account when developing a lay version. First, the message should be customised to the target audience. The information should be made relevant for the target audience, patients in this case. Furthermore, the information in the recommendations should be consistent, unambiguous and credible [ 6]. A successful lay version provides clear, explicit and specific information [ 6, 23] and some key messages [ 2]. The information in the lay version has to be readable for patients. A well-known pitfall is that information is often too difficult for the general public [ 24, 25] and in particular for less literate persons [ 25]. Information can be simplified by using less medical and technical terms or by giving an explanation of the terms [ 24]. McGuire et al. stress the fact that patients do not speak the same language as health professionals [ 24]. Only after a while, they will become more familiar with the language spoken by professionals [ 24]. Based on a survey among professionals, McGuire et al. further recommend the use of familiar words of one or two syllables, the use of active voice in the present tense and the use of short sentences of 15 words or less, and short paragraphs of ten lines or less [ 24]. Finally, information about where more in-depth information can be found should be included in the lay version [ 6, 23].

For international guidelines, lay versions of guidelines should ideally be translated into different languages. Based on international recommendations and best practices, Azevedo and colleagues [ 26] suggest to follow three steps for the translation and cross-cultural adaptation of guidelines: forward translation, back translation and patient testing. In the forward translation step, two professionals/patients (no translators) independently translate the original version into the target language. The translations are then compared, and an agreed version is drawn up between the translators and those involved in the development of the original version. In the back translation step, the text is translated back into the original language with the support of the developers of the original version. It is then compared with the original and reviewed to ensure conceptual equivalence. The last step is the patient test phase. Ten adult patients are given the translated version and are interviewed about the interpretation and wording of each item. The results are reviewed, and any changes are integrated into the third and final version [ 26].

Combining strategies

The third factor that may lead to better dissemination is the simultaneous use of multiple tools and strategies. The use of different approaches can help to increase awareness and use among target populations [ 26].

Patient organisations can furthermore organise an annual national forum on a disease at which people share their experiences and take part in training and education programmes. Patient organisations can also provide telephone and online counselling and literature and other resources for patients and caregivers [ 6]. Allu et al. suggest to disseminate recommendations by providing automatic updates of new information and resources for patients who have signed in, by interactive internet-based lectures and by developing a variety of learning tools like posters, summaries, handouts, pocket cards and slide sets for patients [ 21]. Education to patients should furthermore be pro-active whereby face-to-face and contact by telephone can be used [ 28], just as education events for patients [ 29]. The use of internet and digital tools like websites and apps seems to be promising and necessary to reach patients [ 30, 31]. Guidelines should be easily searchable and accessible immediately [ 30, 31]. Another option is, according to Boulet et al., the development of community 'champions': through train-the-trainer sessions, community leaders are trained to become 'champions'. Those champions assist in the dissemination of information to patients [ 6]. Knowledge brokers can also be used. Knowledge brokers are persons who bridge the gap between researchers and the end users [ 32]. Allu et al. stress the importance of patient associations. Patients should be encouraged to become a member of the association and receive information and other support [ 21].

When different sources and tools are applied over a longer period, the dissemination can be described as most adequate because information will last longer when seen and heard more often [ 6]. There is less information about the impact of the use of different strategies. The reviewed publications provide clear descriptions of the used or recommended dissemination strategies, but it is mostly unknown to what extent these strategies were effective. We do, however, know that event-based knowledge mobilisation may be effective for the initial dissemination of guidelines. For sustainable adaptation, more efforts are needed [ 29].

The literature also describes a study in which patient outcomes were improved by the dissemination of guidelines through didactic educational meetings, a printed guideline and a patient brochure [ 33], but we do not know which of these elements has resulted in the improvements.

In conclusion, in order to be as successful as possible, dissemination strategies should be characterised by developing a dissemination plan at an early stage, developing a lay version and using multiple dissemination strategies to reach the patients. The overriding principle to make recommendations accessible for patients and to ensure that they are comprehensible and fit the context is that the dissemination strategy suits the target audience. It is stressed by several authors that this can best be achieved by direct involvement of the target audience: patients and their organisations [ 22- 25, 36]. How patients and their organisations can be involved is described below.

Patient involvement

In developing recommendations

An international working group of researchers, health professionals and patient representatives of the Guidelines International Network (GIN) has developed a toolkit about patient involvement in guidelines. The GIN supports patient involvement in guideline activities around the world. The toolkit is the result of a series of consultations, a literature review and the practice and experience of the GIN members [ 38].

In developing lay versions

There are different ways that patients or patient organisations can be involved in making a lay version [ 1]. EULAR has chosen to involve patients from different countries because of the international context of the recommendations. They have experienced that the involvement of patients with different native tongues enhances the likelihood that the lay version can be easily understood by many patients and that an English lay version can easily be translated into various languages because typical English phrasings are avoided. The GIN toolkit suggests that the translation of the English lay version in different languages should be done by patient organisations, using a heterogeneous group of patients with different disease status and educational levels [ 38]. Other authors [ 39] suggest to use a participatory (action) research (PAR) in order to involve patients in the development of guidelines/lay versions. Using such a PAR design may, according to the authors, result in culturally appropriate brochures for patients [ 39].

In disseminating recommendations

Van Eijk and colleagues state that a successful change of clinical practice in accordance with the recommendations requires an adequate or even better effective implementation strategy in which the key stakeholders delivering and receiving care (among others patients) are involved in the dissemination [ 40]. National patient organisations should, according to the GIN toolkit, disseminate the recommendations in their own countries. This can be done by using their own website, newsletters, brochures, other publications, phone calls, support groups, workshops, events, seminars, annual conferences, local or regional events, events for professionals and/or patients, press releases, print-ready ads, fillers or by including the recommendations in their information packages provided to their members [ 38]. Personal stories of patients in media can also help to raise awareness of new recommendations [ 38].

Who should be involved?

The GIN toolkit suggests that patients who have participated in the development of the recommendations can also actively contribute to the dissemination process [ 38]. Another suggestion is the establishment of permanent groups, networks or 'virtual panels' of patients [ 38]. The network members are alerted when new recommendations or patient versions are published. They can raise awareness by distributing lay versions to health professionals, patients, patient organisations and members of the public. The network should in this case include members with different backgrounds.

Conditions for involvement

Patients involved in the dissemination process have specific needs that should be taken into account. The GIN toolkit suggests (1) informing patients about their role before participating, (2) clarifying expectations about the specific role of the patients and the time commitment required, (3) giving a training in advance to prepare patients for their assigned role and (4) supporting patients during the process [ 38]. The training could be in technical areas such as how to understand the terminology or how to take part in the group effectively (e.g. assertiveness) [ 38]. Supporting patients can be done by providing networking opportunities for individuals or by providing a buddy [ 38].

Suggestions to make patient involvement more successful

Three suggestions aimed at creating patient involvement in the development and dissemination of recommendations are described in the literature. The first suggestion is the use of selection criteria in choosing patient representatives [ 22]. A criterion may be the ability to consider the evidence objectively and to make recommendations that do not depart from preconceived views or self-interests [ 22]. Second, involve patients in less traditional ways (e.g. as committee member) and choose for more innovative ways such as the use of new media that better fit the patients' role, expectations and capabilities. The development group may not include a consumer representative but may invite patients to review draft documents or attend a group meeting or internet forum to share their perspectives [ 22]. The third suggestion is training: provide patients with sufficient information and knowledge before and during the project. This empowers them to become effective partners in the dissemination and implementation process [ 38].

Discussion

This systematic review has identified 21 studies that describe how recommendations can be disseminated to patients and patient organisations. Such a review was missing and adds up to our knowledge on patient involvement in the development [ 2, 6] and dissemination of recommendations to patients and patient organisations and indirectly to professionals [ 7, 41].

At the start, we assumed a difference between dissemination towards professionals in comparison to dissemination to patients. In this discussion, we evaluate this assumption in the light of our results and existing literature.

This review shows that dissemination towards patients does not differ when it comes to the need to make a plan before or at the start of the project. It should take the needs and preferences of the target audience into account as well as contextual factors [ 7]. However, dissemination strategies to patients versus health professionals have to be different when it comes to the development process of recommendations, the language used and the methods and tools that are used for dissemination. First, recommendations should not be developed without the active involvement of patients.

Second, the original recommendations, often developed by and for professionals, need translation into a readable lay version for patients [ 6]. Not only are the scientific journals in which recommendations are published inaccessible for patients, they are also written in a language that cannot be understood by patients with low or moderate health literacy. The translation into a readable, comprehensible, relevant, consistent, unambiguous and credible version [ 6, 7, 16, 23- 25] for patients will increase the success of the dissemination.

In some cases, patient friendly decision aids may also be an adequate tool for dissemination of recommendations to patients although more research is needed to define condition where decision aids can replace or complement lay versions of recommendations.

This review suggests that patients can be involved in making this lay version [ 1]. The selection of patients may be challenging. Patients are expected to be able to review scientific evidence objectively and to approach the recommendations from a wider patient perspective rather than individual preconceived views or self-interests [ 22]. Ideally, a heterogeneous group of patients with different educational levels [ 38] should be involved in order to communicate to a diversity of patients [ 22, 23, 37]. In several rheumatology patient networks, we have recognised that white, female and higher educated patients are often overrepresented [ 42]. A concern is that such misrepresentation of patients might lead to lay versions that are not serving the wider groups of patients something that is confirmed by the study from South Africa [ 25]. It is therefore important to involve a diverse group of patients (age, gender, educational level, ethnicity), and for that aim, the adaptation of existing processes might be needed to make their participation possible. A final concern may be the question whether recommendations are still valid when being translated or adapted. A solution as used by EULAR is the check and formal approval of the adapted English version by the task force leader and the chair of the standing committee of patients.

Third, dissemination strategies need to be tailored to the needs of patients that use other channels for communication and education than professionals. Although the findings of our study regarding the value of a multifaceted and active strategies are much in line with studies on dissemination towards professionals [ 43, 44], the applied tools and methods are different. Similar to the condition of patient involvement in the development of recommendations, patients and patient organisations should be actively encouraged to support and contribute to the actual dissemination.

Not only for ethical reasons but also because direct involvement improves the dissemination [ 1, 22- 25, 36- 38]; patients should be involved in the whole process. In particular, patient organisations can play an important role in reaching out to the patient community and to solve difficulties to reach patients. They can provide information on their websites, can coordinate web-communities, can organise self-management trainings and other educational events and publish patient magazines, books and brochures. Larger organisations run national campaigns and are able to generate funds to promote education on the disease management of their members. Strategies where health professionals and patient organisations join forces will probably be even better. In these cases, information about the recommendations may reach the patient from different sources with the benefit of mutual enforcement.

Although patients are more and more involved in the development of guidelines [ 3, 45] (for professionals), their involvement in the dissemination process (towards patients and professionals) is still less common. This may be a result of the problems professionals expressed on patient involvement in the development of recommendations [ 2]. Professionals reported the discrepancy between the perspectives of themselves and patients as an important barrier in the development of recommendations. They find it difficult to reconcile the preferences of patients with their own views [ 2]. At the same time, patients find it difficult to affirm their views and experience in the presence of evidence-based information and existing power asymmetries wherein expert knowledge is appreciated more than experiential knowledge [ 46- 49]. Their lack of familiarity with the scientific and medical terminology made communication also more difficult [ 2]. By persisting the use of such language, professionals unintentionally exclude patients [ 50].

The involvement of patients is thus seen as important and as a key to success, but at the same time, it seems to be difficult to involve patients in an adequate way. Training and support for patients are described as helpful and are therefore needed to make the involvement of patients in the development and dissemination of recommendations successful [ 2, 22, 38] which in the end will lead to more successful dissemination of the guidelines to patients and patient organisations. Training for professionals should however also get attention since professionals also have to learn how to work effectively together with patients [ 51].

Strengths and limitations

The strength of this systematic review is the fact that it for the first time focuses solely on the target group of patients and patient organisations, a target group previously often ignored. A limitation might be the low level of evidence of most included articles. This has been the result of our inclusion criteria. Because we expected to find a small number of relevant articles, we chose to be inclusive regarding their methodological quality. For this reason, the results of this review should not only be handled with care, it should also encourage researchers to initiate evaluation studies that will provide knowledge about the effectiveness of dissemination strategies with higher levels of evidence.

Conclusions

Our study is the first systematic literature review that provides an overview of strategies to disseminate recommendations to patients or patient organisations and how to involve patients in this process. The findings were helpful to develop a practical guide for national patient organisations to promote the dissemination of recommendations among patients. The dissemination of recommendations towards patients should largely follow the same principles as dissemination towards professionals, although best practices are still scarce and procedures are not always followed. More attention should however be given to making a lay version which takes the diversity of the target group, patients, into account. Involvement of patients, not only in the dissemination of recommendation but also in the development of recommendations, is needed. The use of PAR and training for patients and professionals may be helpful to improve the quality of the participation of patients.

This review also identified a significant knowledge gap regarding effective dissemination strategies: More valid and credible research has to be conducted in order to obtain higher levels of evidence for the effects, efficiency and barriers of existing dissemination strategies and the role of patient organisations in that process.

References

1. 4.
Stacey D, Légaré F, Col NF, Bennett CL, Barry MJ, Eden KB, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2014;28(1):CD001431. doi: 10.1002/14651858.CD001431.pub4.

Google Scholar

2. 14.
Eular report. EULAR standardised operating procedures for the elaboration, evaluation, dissemination, and implementation of recommendations endorsed by the EULAR standing committees. Ann Rheum Dis. 2004;63:1172-6. doi: 10.1136/ard.2004.023697.

Article Google Scholar

3. 25.
Snyman M. Using the printed medium to disseminate information about psychiatric disorders. South African Psychiatry Review. 2004;7(4):15-20.

Google Scholar

4. 27.
Drouin D. Dissemination and implementation of recommendations on hypertension: the Canadian experience. Allergy, sthma Clin Immunol. 2010;6 Suppl 4:A10.

5. 33.
Maximov D, Lesnyak O. Implementation of the osteoarthritis clinical guideline: results of a cluster randomized trial in primary care. Ann Rheum Dis. 2013;71:307-8. doi: 10.1136/annrheumdis-2012-eular.2416.

Article Google Scholar

6. 38.
G-I-N Public Toolkit. Patient and public involvement in guidelines.

7. 40.
Van Eijk-Hustings Y, Buss B, Fayet F, Moretti A, Ndosi M, Ryan S, et al. Dissemination and evaluation of the EULAR recommendations for the role of the nurse in the management of chronic inflammatory arthritis. Results of a multinational survey among nurses rheumatologists and patients. Rheumatology. 2014;53(8):1491-6.

8. 41.
Gagliardi AR, Marshall C, Huckson S, James R, Moore V. Developing a checklist for guideline implementation planning: review and synthesis of guideline development and implementation advice. Implement Sci. 2015;10(1):205. doi: 10.1186/s13012-015-0205-5. Epub 2015 Feb 12.

Google Scholar

9. 42.
De Wit M. Patient participation in rheumatology research. A four level responsive evaluation. Thesis VU University Amsterdam. 2013

10. 43.
Prior M, Guerin M, Grimmer-Somers K. The effectiveness of clinical guideline implementation strategies-a synthesis of systematic review findings. J Eval Clin Pract. 2008;14:888-97. doi: 10.1111/j.1365-2753.2008.01014.x.a.

11. 46.
van de Bovenkamp HM. The limits of patient power. Thesis. 2010.

Google Scholar

12. 50.
Elberse J, Caron-Flinterman JF, Broerse JEW. Patient-expert partnerships in research: how to stimulate inclusion of patient perspectives. Health Expect. 2010. doi: 10.1111/j.1369-7625.2010.00647.x.

PubMed Google Scholar

Download references

Acknowledgements

We thank the members of the EULAR Task Force for their comments on the preliminary results of this systematic literature review and their suggestions for making the search better. We also thank Quince Blits, junior researcher, who was temporarily involved in the project. At last, we would like to thank EULAR for funding this project.

Authors' contributions

KS made the review protocol, conducted the literature search, formulated the inclusion criteria, selected the papers, assessed the level of evidence of the papers, conducted the content analysis/data synthesis and made the first draft of this paper. MB and MdW were involved in formulating the inclusion criteria, selecting the papers, assessing the levels of evidence of the papers, conducting the content analysis/data synthesis and writing this paper. JCFK performed the searches together with KS and wrote the method section of this paper. TA wrote the research proposal of which the systematic review is part of, was intellectually engaged in the whole process and supervised the project. She was actively involved in all steps of the review and writing of this paper. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Author information

Affiliations

1. Department of Medical Humanities, VU University Medical Center, Amsterdam, The Netherlands

K. Schipper, M. Bakker & T. A. Abma

2. EULAR Standing Committee of People with Arthritis/Rheumatism in Europe, Zurich, Switzerland

M. De Wit

3. Medical Library, VU University, Amsterdam, The Netherlands

J. C. F. Ket

4. Department of Medical Humanities/EMGO+, VU University Medical Center, Post box 7057, 1081 BT, Amsterdam, The Netherlands

K. Schipper

Corresponding author

Correspondence to K. Schipper.

Additional files

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Reprints and Permissions

About this article

Download citation

# data[].attributes.attachment_link.title
Strategies for disseminating recommendations or guidelines to patients: a systematic review

# data[].attributes.attachment_link.description
The aim of this systematic literature review was to assess what dissemination strategies are feasible to inform and educate patients about recommendations (also known as guidelines). The search was performed in February 2016 in PubMed, Ebsco/PsycINFO, Ebsco/CINAHL and Embase. Studies evaluating dissemination strategies, involving patients and/or reaching patients, were included.

# data[].attributes.headline
Academic paper - knowledge dissemination strategies to patients and professionals 

# data[].attributes.subject
Academic paper - knowledge dissemination strategies to patients and professionals 

# data[].attributes.attachment_link.readable_text
2497 Accesses

28 Altmetric

Metrics

Abstract

Background

While there is an ample literature on the evaluation of knowledge translation interventions aimed at healthcare providers, managers, and policy-makers, there has been less focus on patients and their informal caregivers. Further, no overview of the literature on dissemination strategies aimed at healthcare users and their caregivers has been conducted. The overview has two specific research questions: (1) to determine the most effective strategies that have been used to disseminate knowledge to healthcare recipients, and (2) to determine the barriers (and facilitators) to dissemination of knowledge to this group.

Methods

This overview used systematic review methods and was conducted according to a pre-defined protocol. A comprehensive search of ten databases and five websites was conducted. Both published and unpublished reviews in English, Spanish, or Portuguese were included. A methodological quality assessment was conducted; low-quality reviews were excluded. A narrative synthesis was undertaken, informed by a matrix of strategy by outcome measure. The Health System Evidence taxonomy for "consumer targeted strategies" was used to separate strategies into one of six categories.

Results

We identified 44 systematic reviews that describe the effective strategies to disseminate health knowledge to the public, patients, and caregivers. Some of these reviews also describe the most important barriers to the uptake of these effective strategies. When analyzing those strategies with the greatest potential to achieve behavioral changes, the majority of strategies with sufficient evidence of effectiveness were combined, frequent, and/or intense over time. Further, strategies focused on the patient, with tailored interventions, and those that seek to acquire skills and competencies were more effective in achieving these changes. In relation to barriers and facilitators, while the lack of health literacy or e-literacy could increase inequities, the benefits of social media were also emphasized, for example by widening access to health information for ethnic minorities and lower socioeconomic groups.

Conclusions

Those interventions that have been shown to be effective in improving knowledge uptake or health behaviors should be implemented in practice, programs, and policies-if not already implemented. When implementing strategies, decision-makers should consider the barriers and facilitators identified by this overview to ensure maximum effectiveness.

Protocol registration

PROSPERO: CRD42018093245.

Peer Review reports

Background

Knowledge translation (KT) is "the synthesis, exchange, and application of knowledge by relevant stakeholders to accelerate the benefits of global and local innovation in strengthening health systems and improving people's health" [ 1]. The process of KT ensures that evidence from research is used by relevant stakeholders, including healthcare providers, managers, policy-makers, informal caregivers, patients, and the public in the improvement of health [ 2]. While there is an ample literature on the evaluation of interventions aimed at healthcare providers, managers, and policy-makers, there has been less focus on patients and their informal caregivers.

"Patient-mediated" KT interventions are those strategies that involve patients in their own healthcare and have the aim to improve patient knowledge, relationship with the provider, the appropriateness of health service use, satisfaction with the provision of care experience, adherence to the recommended treatment, and other health behaviors and outcomes [ 3].

The Canadian Institutes for Health Research (CIHR), a leader in the science and practice of knowledge translation, have recognized four key elements in the process of KT: synthesis, dissemination, exchange, and ethically sound application of knowledge. For this overview, we will be focusing on dissemination as a core strategy in KT. Dissemination involves identifying the appropriate audience and tailoring the message and medium to the audience [ 4]. Dissemination of health-related information is the active, tailored, and targeted distribution of information or interventions via determined channels using planned strategies to a specific public health or clinical practice audience, and has been characterized as a necessary but not sufficient antecedent of knowledge adoption and implementation [ 5]. According to CIHR, dissemination can include elements such as summaries for/briefings to stakeholders, educational sessions with patients, practitioners and/or policy makers, engaging knowledge users in developing and executing dissemination/implementation plans, tools creation, and media engagement. Dissemination can be done through different information and communication technologies (ICT) based or not on the internet, i.e., videos, websites, brochures, decision aids, or art pieces.

There are many models and theories to explain what makes KT for healthcare recipients (and providers) effective [ 6, 7, 8, 9]. These theories have varying objectives, which range from information provision individually or to large audiences (e.g., mass media) to achieving behavior change through education or skills acquisition. When focusing on behavior change, the aim is to increase the capacity to use and apply evidence effectively, thus achieving better health outcomes including quality of life. Desired outcomes of these models include shared decision-making between patients, their families, and providers; patient-provider communication; self-efficacy; adherence; improved access; and cure or survival. Intermediate outcomes could include healthcare users' improved health knowledge, health behaviors, and physiologic measures; patient satisfaction; and reduced costs [ 10].

Further, in KT processes addressed to patients and informal caregivers, it is important to consider determinants or barriers at the level of healthcare recipients, i.e., knowledge, language, and cultural differences, skills deficits, attitudes, access to care and motivation to change, among others [ 7, 10, 11, 12]. Also, it is usual practice to combine multicomponent dissemination strategies such as a combination of reach, motivation, or ability goals.

For the purpose of this overview, we have focused on dissemination strategies aimed at healthcare users and their caregivers in order to improve health and wellbeing. We used the taxonomy developed by Lavis et al. to organize the results, which includes six groups of strategies that are explained later [ 13].

This overview addressed two specific research questions:

1. 1.
How effective are the strategies that have been used to disseminate knowledge to healthcare recipients (both for the general public and patients)?

2. 2.
What are the barriers (and facilitators) to disseminate knowledge to healthcare recipients (both for the general public and patients)?

Methods

This overview used systematic review methodology and adheres to the Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement [ 14]. A systematic review protocol was written and registered prior to undertaking the searches [ 15]. Deviations from the protocol are noted.

Inclusion criteria for studies

Studies were selected based on the following inclusion criteria.

Types of studies

Systematic reviews (SRs) that included quantitative studies of any design that provided information on the effectiveness of dissemination strategies. SRs of qualitative studies that describe barriers and facilitators to uptake of research evidence were also included.

Types of participants

We included studies that involved healthcare recipients as the main focus, such as the general public, patients, caregivers, or patient groups. We excluded studies where other users, such as practitioners, policy-makers, educators, decision-makers, health care administrators, and community leaders, were the main focus. We also excluded studies where the dissemination strategy was directed to participants with a single health issue, e.g., multimedia interventions to promote HIV testing. This was to ensure a more general approach to strategies for dissemination of knowledge to healthcare recipients.

Types of interventions

SRs that evaluated KT dissemination strategies aimed at healthcare recipients or caregivers were included. The dissemination strategies were defined based on the Health System Evidence taxonomy [ 13] for "consumer targeted strategies" as follow:

1. a)
Information or education provision: strategies to enable consumers to know about their treatment and their health.

2. b)
Behavior change support: interventions which focus on the adoption or promotion of health and treatment behaviors at an individual level, such as adherence to medicines.

3. c)
Skills and competencies development: strategies that focus on the acquisition of skills relevant to self-management.

4. d)
(Personal) Support: interventions which provide assistance and encouragement to help patients cope with and manage their health and ongoing medical issues, such as counseling and follow up on treatment efficacy.

5. e)
Communication and decision-making facilitation: strategies to involve consumers in decision-making about healthcare.

6. f)
System participation: interventions to involve patients and/or caregivers in decision-making processes at a system level.

The dissemination element could be written on paper (i.e., pamphlets, flyers, booklets), verbal (i.e., using telephone), or written or verbal using ICT (i.e., e-health, m-health, websites, multimedia, telemedicine, patient reminder, etc.). The dissemination could be done individually, in groups, or massively.

Types of comparisons

There were no restrictions on types of comparisons.

Types of outcome measures

We included outcomes related to the effectiveness of dissemination strategies addressed to health-care recipients, caregivers, or the general public, including change in knowledge, understanding, perception, attitudes, adherence to health recommendations, and behavior changes. Other proposed results were health status, access, use of services, social outcomes, user satisfaction, costs, and cost-effectiveness. Additionally, we considered barriers to uptake of research evidence through dissemination strategies at the level of knowledge, competency, health literacy, attitudes, access to care, and motivation to change.

Publications in English, Spanish, or Portuguese were included and there were no restrictions on the year of publication. Both published and gray literature were included.

Search strategy and sources of systematic reviews

A comprehensive search of ten databases and five websites was conducted. The databases searched for SRs were MEDLINE (Ovid); Embase (Ovid); ERIC (EBSCOHost); CINAHL (EBSCOHost); PsycINFO (Ovid); LILACS (BVSalud); and World Wide Science. The specialized sources of SRs were the Cochrane Library (including Cochrane Reviews, the Database of Abstracts of Reviews of Effects and Health Technology Assessment); Epistemonikos; and Health Systems Evidence.

Manual searches were conducted in Google and Google Scholar; EPPI-Center Systematic Reviews; Rx for Change ( https://www.cadth.ca/rx-change); and 3ie-International Initiative for Impact Evaluation. In addition to the above sources that included gray literature, we manually searched the System for Information on Grey Literature in Europe (Open Grey- http://www.opengrey.eu).

Electronic searches were conducted between 21 and 23 May 2018 and supplementary searches (reference lists, contact with authors, and gray literature) were conducted in January 2019. Databases were searched using keywords from keyword areas related to the participants, the intervention, outcomes, and study type-combined using "AND." Keywords were searched for in the title and abstract fields and using Medical Subject Headings (MeSH) terms where available (search terms and strategies for the electronic searches are in Additional file 1). Results were downloaded into the EndNote reference management program (version X8.2) and duplicates were removed.

Screening and selection of studies

Titles and abstracts were screened independently according to the selection criteria by pairs of review authors (EC, JB, and MO). The full text of any potentially relevant papers was retrieved for closer examination. The inclusion criteria were then applied against the full text version of the papers independently by two reviewers (EC and MO). Disagreements regarding eligibility of studies were resolved by discussion, and a third reviewer (JB) consulted when necessary. All studies which initially appeared to meet the inclusion criteria but on inspection of the full text paper did not meet the inclusion criteria are listed in a table "Characteristics of excluded studies" together with reasons for their exclusion.

Data extraction

Information extracted from included SRs were objectives, study designs and number of studies included, date of last search, intervention/strategy, participants, settings, country of studies, and financing source, as well as outcome measures, findings, barriers, research gaps, and theories or frameworks. Data extraction was shared between six reviewers (EC, MB, TT, EI, MH, and MO) and checked by a second reviewer (EC). Disagreements were resolved through discussion and consensus.

After extracting data from the included SRs, reviewers completed a matrix previously designed using the Health Systems Evidence taxonomy [ 13] for each of the six strategies down the left-hand side with the different outcome measures across the top. While we started with the list of outcome measures specified in the protocol we had to expand the matrix because we found more types of outcome measures than originally proposed. Classification was done by each reviewer (EC, MB, TT, EI, MH, and MO) and checked by a second reviewer (EC).

Assessment of methodological quality

The methodological quality of included SRs was assessed independently by pairs of reviewers using A MeaSurement Tool to Assess systematic Reviews (AMSTAR) [ 16]. Disagreements in scoring were resolved by discussion and consensus. For this overview, SRs that achieved AMSTAR scores of 8 to 11 were considered high quality, scores of 4 to 7 medium quality, and scores of 0 to 3 low quality. SRs of low quality were excluded. We did not find any SRs of exclusively qualitative studies to inform barriers and facilitators so could not use the Confidence in the Evidence from Reviews of Qualitative research approach to assess the confidence of findings from qualitative syntheses [ 17] as proposed in the protocol. Instead, we found SRs with a mix of study designs that included qualitative research so the overall quality of these studies was evaluated with AMSTAR tool.

Data analysis

Findings from the included publications were synthesized using tables and a narrative summary informed by the matrix of strategy by outcome measure. Meta-analysis was not possible because the included studies were heterogeneous in terms of the populations, strategies/interventions tested, and outcomes measured. Further, few studies informed effectiveness measures. Thus, to inform the main results, we developed effectiveness statements using four categories and standardized language as proposed by Ryan et al. [ 9]. The decision rules took into account the results, their statistical significance, and the quality and number of studies that support the result. The four categories are (1) sufficient evidence, (2) some evidence, (3) insufficient evidence, and (4) insufficient evidence to determine effectiveness (Additional file 2). Category 4 was used to inform research gaps.

Results

Search results

Study selection flow chart

Characteristics of included studies and quality assessment

The different strategies tested, and types of communication or dissemination tested in each of the SRs are shown in Additional file 5. When reviewing the included SRs, we found outcome measures that were not included in the protocol (or in the " Methods" section of this review). These included shared decision-making between patients, their families, and providers, patient-provider communication, self-efficacy and/or self-management, awareness, beliefs, clinical results, coverage, use of services, empowerment, less suffering or anxiety, persuasion, safety, social support and influence, quality of life, health status and wellbeing, hospitalizations, length of consultation, participation in health, sustainability, choice, addiction to media, and readability. More details are in Additional file 6.

Effectiveness statements

The effects of interventions are presented below by strategy according to the adopted taxonomy [ 13]. The SRs were divided into those testing the specific strategy alone (single) or in combination with other strategies (combined). Many reviews evaluated interventions involving multiple strategies and so contributed evidence to more than one category. More details are in Additional file 5, Table 1. The effectiveness statements are presented in Tables 1, 2, 3, 4, and 5 for those with "sufficient" or "some evidence" (categories 1 and 2 in Additional file 2). Those with "insufficient evidence" (category 3) are in Additional file 7 and those with "insufficient evidence to determine" (category 4) were used to inform the research gaps (Additional file 8).

Providing information or education

Communication and decision-making facilitation

Acquiring skills and competencies

Twenty-six reviews included this strategy but only five provided evidence that was useful for the development of the effectiveness statements [ 5, 9, 11, 22, 53]. One of these five reviews was of high quality [ 9]. See Table 3 and Additional file 7 for the effectiveness statements.

Behavior change support

Personal support

Thirty reviews included this strategy but only two provided evidence that was useful for the development of the effectiveness statements [ 9, 49]-both were of high quality. See Table 5 and Additional file 7 for the effectiveness statements.

Consumer system participation

Twenty-eight reviews included this strategy but only six provided evidence that was useful for the development of the effectiveness statements [ 10, 19, 23, 32, 42, 45]-two of these were of high quality [ 10, 45]. In relation to consumer system participation, no single strategies were identified. For the combined strategies, none had sufficient evidence and only one had some evidence of effectiveness, with the resulting effectiveness statement:

The combined strategies with insufficient evidence are listed in Additional file 7.

Barriers

We did not find any SRs of exclusively qualitative studies. Of the 44 included SRs, ten included qualitative research among other study designs. For the synthesis of barriers (and facilitators) to KT to healthcare participants, 31 SRs contributed information. The barriers identified were grouped following the type of communication used for the intervention or strategy (Additional file 5, Table 2). While this method of grouping barriers was not originally stated in the protocol, we found it to be the most logical way to group them due to the way in which barriers were reported in the included systematic reviews.

None of the SRs identified barriers to verbal communication specifically. However, in relation to patient advisory councils (which may use both verbal and electronic communication), the main barrier described was that the implementation takes a significant amount of time and resources for recruitment, holding meetings, and providing follow up [ 45].

For written information that does not require the internet, concerns were raised about motivation and awareness [ 29, 49], health literacy [ 40, 53], and comprehension and understanding [ 29, 40]. Other possible barriers that should be considered are the reliability and trustworthiness of the information [ 29], personal needs [ 49], and text complexity and design [ 40].

For information technology interventions in general, barriers raised include health literacy, privacy and information quality concerns, access to technology, and information design [ 42].

Computer-based strategies, whether internet-based or not, present as barriers difficulties in the management of technologies, mainly for the elderly [ 19], e-literacy [ 25, 31, 51], privacy concerns, consumer's personal feelings, socioeconomic factors [ 31, 41], and health literacy [ 31, 44]. Other barriers include reliability and trustworthiness in the information, lack of time and personal impairment [ 31], motivation and awareness, and information that does not meet personal needs [ 49], text complexity and lack of access to information [ 44].

Strategies that use multimedia not based on the internet bring difficulties like motivation, awareness, information that does not respond to personal need or without sufficient detail [ 49], problems of communication [ 35], and health literacy [ 18, 52] for their implementation.

Most of the studies did not discuss issues such as ethnicity, income level, or homelessness, which are important when considering the use of an internet-based technology to deliver an outpatient intervention. The long-term effects on individual persistence with chosen therapies and cost-effectiveness of the use of internet-based therapies and hardware and software development require continued evaluation [ 51]. Recent SRs have mentioned inequities such as lack of access to technology [ 42]. However, one review noted that a benefit of social media is that it can widen access to those who may not easily access health information via traditional methods, such as younger people, ethnic minorities, and lower socioeconomic groups [ 39].

For social media interventions, barriers on the individual level include health literacy [ 7, 42] and the risk of a deterioration in the relationship between health professionals and patients [ 7, 39], including the inability to meet the patients' emotional and information needs [ 46]. Other concerns include how the information is presented [ 7, 21], privacy, information quality, lack of internet access, trustworthiness in the information, information overload, and stigma about certain conditions [ 39]. Another highlighted barrier was the fact that the social content is used more than the educational content, i.e., participants use the social media to interact with other users more than as a means for self-education [ 38].

m-health (with mobile phone) strategies raise as the main barriers to its implementation issues such as e-literacy and lack of internet access [ 10, 27], health literacy [ 34, 44], socioeconomic factors [ 27, 50], privacy concerns, lack of personal skills [ 10], text complexity [ 44], and the time-consuming nature of the technology [ 54]. Another potential limitation of m-health could be that the delivery of interventions can be interrupted if the mobile phone is stolen or lost. However, the same limitations exist with many other forms of communication (e.g., postal mail may be delivered to the wrong address, email boxes may be too full to receive messages) [ 27].

Barriers to implementation of telemedicine are also related to e-literacy, privacy concerns, lack of internet access, and personal skills [ 10].

For the implementation of patient decision aids, which can include pamphlets, videos, or web-based tools, barriers detected include decision aids that do not meet the needs of the population, clinicians unwilling to use them, and clinicians and healthcare consumers without skills for shared decision-making [ 48].

Discussion

For this overview, we identified 44 SRs that describe the effective strategies to disseminate health knowledge to the public, patients, and caregivers. Some of these SRs also describe the most important barriers to the uptake of these effective strategies. The reviews that tested more general strategies were selected instead of those directed to a particular condition or setting. To our knowledge, this is the first overview of SRs addressing this objective.

While we reported the strategies and results according to the taxonomy adapted from the Health System Evidence database [ 13], we found that many strategies overlapped for both the type of intervention and the outcome measures. For example, interventions providing information or education could report outcomes related to behavior change or self-efficacy, and the primary intention could have been to increase knowledge. Situations like these were frequent and could be due to the use of combined strategies or to characteristics of the intervention itself, its intensity, frequency, or duration. The strategies reported in the included SRs could be directed to individuals or groups, in print or verbally, face to face, or remotely. In addition, interventions could range from single (e.g., a written information leaflet) to combined strategies. We considered a strategy to be combined when it used two or more verbal, print, or remote health information strategies (e.g., video, computer, and slide show presentations [ 11]), or different electronic communication types (based or not on the internet), such as telemedicine, ICT applications or ICT platforms [ 10, 42], or social networking like Facebook or Twitter [ 39, 46].

We found few SRs with a meta-analysis that could inform the magnitude of effects. Thus, an overall meta-analysis for each of the strategies could not be conducted, which is why we chose to adopt the approach proposed by Ryan et al. [ 9] and have presented the findings as evidence statements.

A key objective of the included interventions was to inform, improve knowledge, or to change health behaviors. To achieve behavioral changes, different strategies were used, such as training, coaching, or text messages. Factors that affected the effectiveness of the intervention included its frequency, intensity, and follow-up time. These factors are important to consider when implementing the chosen intervention strategies, including the applicability of the intervention in different modes of implementation and contexts.

Other examples of effective tailored interventions, such as those designed to improve communication or participation in decision-making between patients and healthcare providers, were the use of patient decision aids and patient information leaflets, provided electronically or not [ 10, 48, 49]. Interestingly, when coaching was added to patient decision aids, we found some evidence for improvements in knowledge and participation. Also, coaching, when compared to patient decision aids alone, increased values-choice agreement and improved satisfaction with the decision-making process [ 47]. In relation to satisfaction, we also found some evidence for improvement in patient satisfaction for interventions through multimedia before consultations designed to help patients with their information needs [ 35].

With regard to caregivers, in particular of patients with Alzheimer's disease, we found good evidence for the effect of a home safety toolkit for improvements in home safety, risky behavior, and caregiver self-efficacy [ 53]. For interventions that involved patients and/or caregivers in decision-making processes at a system level, we did not find sufficient evidence to make any statements. Further, few studies included a follow-up period longer than 1 year or reported retention rate, thus it is not known if behavior change results are sustained over time [ 32, 36, 42, 51].

Our second research question focused on barriers to the dissemination of knowledge to healthcare recipients, which are important to consider when implementing chosen intervention strategies. The barriers most frequently mentioned were related to ICT or to the information itself. For ICT, the main concerns were access to the technologies, including availability of the internet. On a personal level, the lack of skills for managing new technologies, privacy issues, lack of time, and deterioration of the doctor-patient relationship were also mentioned, especially when using social media or websites. As for the information itself, the lack of understanding or comprehension, the volume of information, text complexity and its design, information that did not meet the needs of the patients, and trustworthiness were the key barriers mentioned. While inequities were mentioned and were often related to the lack of health literacy or e-literacy, the benefits of social media were also emphasized, for example by widening access to health information, particularly for ethnic minorities and lower socioeconomic groups.

Strengths and limitations of the overview

Strengths of our overview were that only reviews of medium or high quality were included, as well as our focus on strategies that translated health information to patients and caregivers through different strategies and types of dissemination. Further, we focused on more general interventions rather than specific interventions, which are already abundant in the scientific literature and could be among the list of SRs that were excluded. While we did not find SRs of qualitative studies to analyze barriers to a better implementation of dissemination interventions, we did find considerable information and analysis of barriers in many of the included SRs. These included good quality studies on health literacy. Further, we were able to identify many research gaps that are detailed in Additional file 8.

Limitations of our overview include limitations in the included SRs, such as the lack of clear description of the interventions, setting or samples, and outcomes in some reviews. Further, not all of the included SRs used theories or frameworks to inform the strategies. Finally, due to the heterogeneity in the interventions and outcomes, a meta-analysis was not possible.

Achieving improvements in knowledge uptake or health behaviors is difficult and the literature of effectiveness for the different strategies in the clinical field has been presented using a range of frameworks, theories, or taxonomies. While work is underway to develop consistent taxonomies for the design and reporting of behavior change and dissemination and implementation interventions, such as the behavior change wheel [ 55], the theoretical domains framework [ 56, 57], and other taxonomies [ 58], these are not consistently applied in the existing literature. Further, few have been developed for patients or their caregivers, and there is more of a focus on implementation rather than dissemination. None of the developed frameworks were suitable for our context ( https://dissemination-implementation.org/viewAll_di.aspx). Thus, given that this overview was aimed at healthcare decision-makers, we chose to use the Health Systems Evidence taxonomy of Lavis and colleagues [ 13]. The advantage of using this taxonomy is that it makes it easier for healthcare decision-makers to find, understand, and use the evidence contained in the overview. Further, while there is debate about how best to measure the effectiveness of complex behavior change interventions [ 59, 60], these authors acknowledge that further work is needed. Until that work is conducted and consensus achieved, systematic reviews of randomized controlled trials (and other designs), as used in this overview, are the currently accepted best method.

Conclusions

This overview of systematic reviews has shown that a variety of dissemination strategies aimed at healthcare users and their caregivers can improve health and wellbeing in different ways. However, implementation of our findings will need to consider the particular context in which a strategy is to be implemented. This overview will help decision-makers choose the most effective dissemination strategies and will also inform them as to the factors that they should consider when implementing those strategies.

Implications for practice and policy

Those interventions that have been shown to be effective in improving knowledge uptake or health behaviors should be implemented in practice, programs, and policies-if not already implemented. The benefits of strategies such as e-health and m-health, including telemedicine, should be considered for knowledge dissemination and to improve health behaviors-especially in populations with lack of access to traditional sources of healthcare, including in remote or rural areas. The application of distance technology may strengthen the continuity of care between patient and clinician by improving access and supporting the coordination of healthcare activities from a single source. When designing KT strategies, not only the effectiveness of the strategy but also the characteristics of the interventions should be taken into account, such as the type of dissemination (electronic or not), frequency, intensity, and follow-up time. It is also important to ensure that the content of the messages is addressed to people with low literacy, low numeracy, and low e-literacy. The knowledge disseminated should be readable, comprehensible, relevant, consistent, unambiguous, and credible for patients. Moreover, patients should be invited to participate in its design. All of these strategies are likely to increase the success of the dissemination.

Implications for research

Future research should focus on the areas identified as research gaps in Additional file 8. In addition, researchers should ensure that the interventions tested are well described in their papers. Likewise, systematic reviewers should also ensure that they include a clear description of the interventions, settings, samples, and outcomes included in their reviews to facilitate their evaluation and implementation by decision-makers.

Availability of data and materials

All files supporting the conclusions of this article are included within the article or in Additional files 1, 2, 3, 4, 5, and 6.

Abbreviations

AMSTAR:

A MeaSurement Tool to Assess systematic Reviews

ARR:

Absolute risk reduction

CIHR:

Canadian Institutes for Health Research

ICT:

Information and Communication Technologies

KT:

Knowledge translation

MeSH:

Medical Subject Headings

NNT:

Number needed to treat

PRISMA:

Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RRR:

Relative risk reduction

SMS:

Short message service

SR:

Systematic review

References

1. 2.
WHO. Knowledge translation framework for Ageing and health. Geneva: Department of Ageing and Life-Course, World Health Organization; 2012.

Google Scholar

2. 9.
Ryan R, Santesso N, Lowe D, Hill S, Grimshaw J, Prictor M, et al. Interventions to improve safe and effective medicines use by consumers: an overview of systematic reviews. Cochrane Database Syst Rev. 2014;4:CD007768.

Google Scholar

3. 15.
Chapman E, Barreto JOM. Knowledge translation overview: strategies for dissemination with a focus on recipient health care. PROSPERO 2018 CRD42018093245. Available from: http://www.crd.york.ac.uk/PROSPERO/display_record.php?ID=CRD42018093245.

4. 21.
Akl EA, Oxman AD, Herrin J, Vist GE, Terrenato I, Sperati F, et al. Using alternative statistical formats for presenting risks and risk reductions. Cochrane Database Syst Rev. 2011;3:1-90.

Google Scholar

5. 32.
Health Quality Ontario. Electronic tools for health information exchange: an evidence-based analysis. Ont Health Technol Assess Ser. 2013;13(11):1-76. Available from: http://www.hqontario.ca/en/documents/eds/2013/full-report-OCDM-etools.pdf.

6. 41.
Revere D, Dunbar PJ. Review of computer-generated outpatient health behavior interventions: clinical encounters "in absentia". J Am Med Inform Assoc. 2001;8(1):62-79.

CAS PubMed PubMed Central Article Google Scholar

7. 44.
Schipper K, Bakker M, De Wit M, Ket JC, Abma TA. Strategies for disseminating recommendations or guidelines to patients: a systematic review. Implement Sci. 2016;11(1):82.

CAS PubMed PubMed Central Article Google Scholar

8. 59.
Campbell M, Fitzpatrick R, Haines A, Kinmonth AL, Sandercock P, Spiegelhalter D, et al. Framework for design and evaluation of complex interventions to improve health. BMJ. 2000;321(7262):694-6.

CAS PubMed PubMed Central Article Google Scholar

Download references

Funding

This study was supported by a grant from the Ministry of Health of Brazil (TED MS/SCTIE-Fiocruz #43/2016). The funder had no role in the design of the study, collection, analysis or interpretation of data or in the writing of the manuscript.

Author information

Affiliations

1. Fundação Oswaldo Cruz, Fiocruz, Brasília, Brazil

Evelina Chapman & Jorge O. Maia Barreto

2. Departamento de Ciencias Químico Biológicas, Universidad de Sonora, Hermosillo, Sonora, Mexico

Michelle M. Haby

3. Centre for Health Policy, Melbourne School of Population and Global Health, The University of Melbourne, Melbourne, VIC, Australia

Michelle M. Haby

4. Instituto de Saúde, Secretaria de Estado da Saúde de São Paulo, São Paulo, Brazil

Tereza Setsuko Toma & Maritsa Carla de Bortoli

5. School of Psychology Universidad Mayor, Santiago, Chile

Eduardo Illanes

6. Departamento de Medicina Interna, Facultad de Medicina, Universidad de La Frontera, Temuco, Chile

Maria Jose Oliveros

Contributions

EC and JB conceived the study and wrote the protocol. MH and EC conducted the searching. EC, JB and MO selected studies for inclusion. All authors contributed to data extraction and quality assessment. EC, JB, and MH contributed to the analysis and interpretation of results. EC drafted the first version of the manuscript, with input from JB and MH. All authors read and approved the final manuscript.

Authors' information

Not applicable.

Corresponding author

Correspondence to Michelle M. Haby.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Reprints and Permissions

About this article

Download citation

# data[].attributes.attachment_link.title
Knowledge translation strategies for dissemination with a focus on healthcare recipients: an overview of systematic reviews

# data[].attributes.attachment_link.description
While there is an ample literature on the evaluation of knowledge translation interventions aimed at healthcare providers, managers, and policy-makers, there has been less focus on patients and their informal caregivers. Further, no overview of the literature on dissemination strategies aimed at healthcare users and their caregivers has been conducted.

# data[].attributes.headline
Academic paper - focus groups with patients about plain language summaries 

# data[].attributes.subject
Academic paper - focus groups with patients about plain language summaries 

# data[].attributes.attachment_link.readable_text
We recruit and support patients, carers and members of the public to give feedback on health and social care research funding applications.

It's a nice feeling to know that you are part of something that will help change and improve things in the bigger picture.

-Rosie Forder - Public Reviewer

As well as members of the public, reviews are sought from health care professionals, researchers, health economists and other professionals. This means that you only need to give us your opinion based on your experience of being a patient, carer or user of health and social care services

What will happen if you are invited to do a review?

You can sign up to review potential research through our online Reviewer Match service.

Once you have registered as a public reviewer, you may be invited to review research proposals for one of two NIHR coordinating centres:

the NIHR Central Commissioning Facility (CCF)

the NIHR Evaluation, Trials and Studies Coordinating Centre (NETSCC).

The type of documents you may be asked to review include a research or commissioning brief and a research funding application.

What is a research or commissioning brief?

A research or commissioning brief is developed by the NIHR. It describes a potential area of research. A public reviewer will help decide whether the proposed research topic should be advertised. Researchers can then apply to do research in the advertised area.

What is a research funding application?

Researchers applying for NIHR funding complete a form describing what research they want to do and how they want to do it. A public reviewer will help to inform the funding committee as to whether the research should be funded.

You would read the documents at home and submit comments online.

The reviewing process

1. The proposal - We receive a document proposing a research study.

2. The right match - The research topic of the document is matched to a member of the public who has personal experience as a patient, carer or member of the public who is affected by the issues being researched.

3. The review - The patient, carer or member of the public reads and assesses the application's strengths and weaknesses.

4. The funding decision - Public and professional/scientific reviewers' comments are fed back to researchers applying for funding and to the research advisory committee which makes funding decisions.

Guidance and support

We've developed an interactive training course for the public on reviewing funding applications for research. The training includes a module on things to consider when reviewing a research commissioning brief or funding application.

You may wish to consider these ' top tips ' created with public reviewers and also hear from some of the people involved in reviewing.

Payment for reviewing work might affect those in receipt of state benefits or those working in the public sector. For free, confidential advice you can contact the Benefits Advice Service.

Guidance is available on what's involved in the review process for research proposals submitted to CCF or NETSCC.

# data[].attributes.attachment_link.keywords[]
there are many ways that you can help nihr by getting  involved in its research and other activities

# data[].attributes.attachment_link.description
We recruit and support patients, carers and members of the public to give feedback on health and social care research funding applications. It's a nice feeling to know that you are part of something that will help change and improve things in the bigger picture.

# data[].attributes.headline
A Plain Language Summary toolkit, cocreated with patients

# data[].attributes.subject
A Plain Language Summary toolkit, cocreated with patients

# data[].attributes.attachment_link.readable_text
We co-created our Toolkit at the PLS of Publications Workshop, which was held in London in November 2018 with support from Patient Focused Medicines
Development (PFMD).

The workshop brought patients, publishers, pharma, and publication professionals together to discuss the benefits, opportunities, and challenges of PLS.

# data[].attributes.attachment_link.description
We co-created our Toolkit at the PLS of Publications Workshop, which was held in London in November 2018 with support from Patient Focused Medicines Development (PFMD). The workshop brought patients, publishers, pharma, and publication professionals together to discuss the benefits, opportunities, and challenges of PLS.

# data[].attributes.attachment_link.description
People are living longer and many are enjoying healthy lives. However, a significant percentage of older people are particularly vulnerable to even relatively minor changes in their circumstances, and so need particular attention when admitted to a hospital. Comprehensive Care looks at the concept of 'frailty' in older people living in hospital.

# data[].attributes.attachment_link.readable_text
The NIHR has launched a new Centre for Engagement and Dissemination that brings together its activities in patient and public involvement, engagement and participation with its strengths in research dissemination.

Building on the work of the NIHR Dissemination Centre and NIHR INVOLVE, the new Centre will lead NIHR's work to make health and care research representative, relevant and ready for use.

The NIHR Centre for Engagement and Dissemination will further enhance the strong collaborative culture already established in engaging with patients, service users, carers and the public across the NIHR, providing coordination and thought leadership across the health and care system.

Dr Louise Wood CBE, Director of Science, Research and Evidence at the Department of Health and Social Care, said: "The NIHR's commitment to engaging patients, carers and the public is as strong as ever. I believe that bringing together NIHR's research dissemination activities with its leading role in patient and community engagement will help ensure that we strengthen the way research serves the needs of patients and those of the health and care system.

"By working together with individuals, groups and communities across the health and care system, we will maximise the relevance, reach and impact of our research."

The new Centre will focus on four key areas of activity:

Learning and capacity development for public and patient contributors, public contribution leads, researchers and evidence users

Building and stimulating an evidence user community

Producing accessible and audience-targeted evidence resources and sharing them across a broad stakeholder community

Running a series of innovation projects to harness the best engagement and dissemination ideas coming from the community.

These activities will be underpinned by engaging with patients, care users and the public throughout - as individuals, groups or communities.

The Centre will be led by the new NIHR Director for Public Voice, Jeremy Taylor OBE.

Jeremy Taylor is a former researcher, civil servant and charity leader. From 2009 to 2019, he was Chief Executive of the charity coalition National Voices, where he championed the vital contribution of patients and the public to decisions in health.

Jeremy said: "There has never been a more important time for health and care research to be shaped by what matters to people and to be translated into action that improves and sustains health and care. I am delighted to be joining the NIHR to lead the new Centre and this vital work".

To coincide with the launch of the new Centre, the NIHR has also established a new website - NIHR Evidence.

NIHR Evidence presents high quality summaries of findings from health and care research, making research informative, accessible, relevant and ready for use for all.

The new evidence resources that will be available on NIHR Evidence are:

Alerts: Short summaries of the latest health research presented in plain English, to promote the use of research to improve health and care

Collections: Overviews of recent research on topics or themes highlighted as important by members of the community

Themed reviews: Specially created reviews of recent research on topics of strategic importance for the UK health and social care system

Explore NIHR Evidence at www.evidence.nihr.ac.uk and on Twitter at @NIHRevidence.

You can get in touch with the new Centre on: ced@nihr.ac.uk.

# data[].attributes.attachment_link.title
NIHR launches new Centre for Engagement and Dissemination

# data[].attributes.attachment_link.description
The NIHR has launched a new Centre for Engagement and Dissemination that brings together its activities in patient and public involvement, engagement and participation with its strengths in research dissemination. Building on the work of the NIHR Dissemination Centre and NIHR INVOLVE, the new Centre will lead NIHR's work to make health and care research representative, relevant and ready for use.

# data[].attributes.body
We are using these as a framework to guide our PPI activity

# data[].attributes.attachment_link.readable_text
This project is addressing the second of eleven recommendations in the 'Going the Extra Mile' report ( http://tinyurl.com/llekart): "The NIHR should commission the development of a set of values, principles and standards for public involvement. These must be co-produced with the public and other partners. They should be framed in such a way, and with a clear set of self-assessment criteria, so that organisations across the NIHR see their adoption as integral to their continuous improvement in public involvement".

Over the last two years a UK-wide partnership, have been developing a set of standards and indicators for public involvement/engagement in research. The new, national standards for public involvement in research provide a framework for reflecting on and improving the purpose, quality and consistency of public involvement/engagement. They describe the building blocks for good public involvement/engagement and provide a baseline of expectations.

The partnership want to encourage people to start using the standards. During 2018/19, they will be working with 10 test bed projects ( 10 pilot sites) across the UK, to see how the standards work in practice in a variety of contexts.

You can download the Standards documents by clicking on the following links:
UK Standards for Public Involvement UK Standards for Public Involvement Summary UK Standards for Public Involvement - Easy Read Version
A Welsh version of the UK Standards for Public Involvement - Easy Read will be available shortly

You can find more information on the developing and piloting of the standards here.
Standards Consultation - Analysis

# data[].attributes.attachment_link.description
This project is addressing the second of eleven recommendations in the 'Going the Extra Mile' report ( http://tinyurl.com/llekart): "The NIHR should commission the development of a set of values, principles and standards for public involvement. These must be co-produced with the public and other partners.

# data[].attributes.headline
Academic paper - review Plain Language Summaries are inconsistent 

# data[].attributes.subject
Academic paper - review Plain Language Summaries are inconsistent 

# data[].attributes.attachment_link.readable_text
Research article

Open Access

Published:

BMC Medical Research Methodology 16, Article number: 61 (2016) Cite this article

2047 Accesses

8 Citations

24 Altmetric

Metrics

Abstract

Background

The aim of this study was to analyze whether Cochrane plain language summaries (PLSs) adhere to the Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS).

Methods

A systematic analysis of adherence to the measurable PLEACS items was performed for Cochrane PLSs published from March 2013 to the end of January 2015. Duplicate independent data extraction was performed. An adherence score was calculated for each PLS and for the Cochrane Review Groups (CRGs) that published them.

Results

Of the 1738 analyzed PLSs, not a single one adhered fully to the measured PLEACS items. The highest adherence was found for absence of details of the search strategy (99 % adherence), and the lowest adherence for an item mandating to address quality according to the GRADE system (0.7 % adherence). Overall adherence percentage of PLSs reporting reviews with included studies was 57 %. Different CRGs had a wide range of adherence scores.

Conclusions

Cochrane plain language summaries are highly heterogeneous with a low adherence to the PLEACS standards. Therefore, there is much room for improving the content and consistency of the PLS. A standardization of PLSs is necessary to ensure delivery of proper and consistent information for consumers.

Peer Review reports

Background

Systematic reviews are considered the highest level of evidence because they include systematic analysis of literature of a particular research question. As such, systematic reviews make evidence accessible and usable by a busy clinician, and particularly for experts engaged in making clinical guidelines in different fields of medicine. However, systematic reviews may be long and difficult to understand and therefore many readers will access only abstracts to get the key message [ 1]. In addition to conventional scientific abstracts, Cochrane systematic reviews have a plain language summary (PLS), which is aimed towards general public. The Cochrane PLSs are supposed to be clear, understandable and accessible, especially for the lay persons in particular field of medicine. It would be desirable to write PLSs in a standard format, so the Cochrane has published Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS) [ 2].

Since all Cochrane reviews are prepared and published in English, the Cochrane has recognized the need to promote evidence-informed health care by publishing its high-quality content in languages other than English. At the moment, there are 13 Cochrane translation teams around the world, managing translations into Chinese (simplified and traditional), Croatian, French, German, Japanese, Korean, Malay, Polish, Portuguese, Russian, Spanish and Tamil [ 3]. Most of them translate only PLSs. Therefore, a PLS is now an important knowledge translation tool, not only in English, but in numerous languages worldwide [ 4]. Therefore, it is now more important than ever to ensure high quality and homogeneity of the PLSs, and their adherence to standards.

These standards complement the Methodological Expectations of Cochrane Intervention Reviews (MECIR) project, which aims to standardize the production of Cochrane systematic reviews [ 5]. However, while reading a Cochrane PLS one often finds technical and scientific jargon that may be very difficult for consumers to understand. Additionally, it is unclear what proportion of the Cochrane PLSs is adhering to the PLEACS standards. Therefore, the aim of this study was to analyze heterogeneity of the PLSs and their adherence to the PLEACS guidance for writing PLSs. We hypothesized that the Cochrane PLSs are very heterogeneous considering their structure, length and information they contain, and that majority of them are not written entirely according to the PLEACS.

Methods

A systematic analysis of the PLSs' adherence to the measurable items of PLEACS was performed using a priori defined research protocol. Cochrane PLSs published from March 2013 to the end of January 2015 were searched. Withdrawn Cochrane reviews were excluded from the study. Six authors extracted data (MV, FS, AJK, MF, IP, IM). Duplicate independent data extraction was performed for each PLS. Independent data extractions were checked for consistency and discrepancies were resolved by another author (SD).

For analysis, we used the latest version of PLEACS, which is currently publicly available; the version 3.0 dated February 28, 2013. The PLEACS contains 14 items, of which 12 are marked as 'mandatory' and two as 'highly desirable' (PLS9 and PLS11). From those mandatory items, we identified measurable units eligible for independent assessment by reviewers and therefore the collected data do not necessarily correspond to the complete instructions provided in the PLEACS. Since the goal of the study was to assess the PLSs' heterogeneity, additionally we also analyzed word count of PLSs. Even though the PLEACS say: "It is highly desirable for the PLS to be 400 words and it should be no more than 700 words.", this sentence is located within the PLS1, which is marked as mandatory.

Certain elements that are indicated as 'mandatory' in the PLEACS were not analyzed, such as PLS4, related to 'Consistency', because the way it is described indicates that the key messages of the PLS need to be consistent with the text of review and the Summary of Findings table in the review, and therefore we deemed that this item is not measurable solely by analyzing the PLS.

Apart from the date of publication (month, year) and the Cochrane review group, the following 14 data items were collected from the PLS: 1) Is the title of the systematic review and the title of the PLS the same? ( the title should be restated in plain language), 2) Number of words ( recommended: 400 words; not more than 700 words), 3) Did they avoid technical terms and jargon - words that are specifically recommended to be avoided ((outcome, literature, case series, efficacy and effect size) as well as terms that may have slightly different meanings in medicine than in common usage (e.g. local, blinding, control, practice))?, 4) Is PLS narrative or structured ( headings are mandatory)?, 5) If structured, are the subtitles as recommended ( Review Question, Background, Study Characteristics, Key Results, Quality of Evidence)?, 6) If structured, how many recommended headings are missing?, 7) Is the search date indicated?, 8) Did they include details of the search strategy ( i.e. databases, search terms should not be included)?, 9) Are there population details?, 10) Is there information about the number of studies included in the systematic review?, 11) Is there information about the number of participants in included studies?, 12) Are there any complex statistical data used and not explained?, 13) Is quality of the included studies addressed?, and 14) Is quality addressed according to the GRADE system?

For empty systematic reviews data for items 9-14 were not collected. A summary adherence score was calculated for each PLS based on the 14-item scoring system indicated in the Additional file 1, where points were assigned for having mandated elements. Four items (1, 3, 8 and 12) were scored in reverse because for these, certain elements should not be present.

Based on this adherence scoring, each PLS with included studies could have a minimum of 0 points to a maximum of 19 points for adherence to the measured PLEACS items, while empty reviews could have 0-13 points. Overall adherence to the measured PLEACS items of PLSs and Cochrane Review Groups (CRGs) that published them was expressed as a percentage.

Since the latest version of PLEACS, which was used in this study, was published in February 28, 2013, we decided to analyze PLSs published from March 2013 up to the day of the start of this study so that we can analyze whether the consistency of PLSs improves over time.

Descriptive statistics (frequencies and percentages) for each domain were used to gain insight into the proportion of PLSs written according to the measured PLEACS items using Microsoft Excel (Microsoft Corp., Redmond, WA, USA). Correlation between time of publication and adherence score was calculated. Linear regression analysis was conducted to explore association between time of publication and adherence score using GraphPad Prism (GraphPad Software, La Jolla, CA, USA).

Results

We found 1799 PLSs published between March 2013 and January 2015, 61 were withdrawn, leaving 1738 PLS for analysis. Of those, there were 176 empty reviews with no studies included. None of the included PLSs was updated subsequently during the analyzed period.

Table 1 indicates how many of the PLS adhered to different PLEACS items that were scored in this study. The most successful adherence was to the item regarding the absence of details of the search strategy (99 % adherence) and absence of unexplained complex statistical data (98 % adherence), while the lowest adherence was observed related to addressing quality according to the GRADE system (0.7 % adherence).

Table 1 Number of Cochrane plain language summaries adhering to the PLEACS standards

Mean number of words in the PLS was 319 (median 304, range 46-1125). Few PLSs ( N = 8, 0.5 %) had a higher number of words than the recommended range between 400-700 words. However, a significant portion of the PLS had a lower number of words than the minimum mandated by the PLEACS ( N  = 1326, 76 %).

Quality scoring indicated that overall average quality score of 1562 reviews with included studies was 11 (range: 4-18 points) of maximum 19 points, indicating average adherence of 57 % with the analyzed recommendations. There were 10 (0.6 %) PLSs with adherence of 0-25 %, 307 (20 %) with adherence of 26-50 %, 1051 (67 %) with adherence of 51-75 % and 194 (12.4 %) with adherence of 75-100 %.

Among 176 empty reviews, average quality score for their PLSs was 7.6 (range 3-12) of maximum 13 points, indicating average adherence of empty review PLSs of 59 %. There were 2 (1.1 %) PLSs with adherence of 0-25 %, 32 (18 %) between 26-50 %, 122 (69 %) between 51-75 % and 20 (11 %) with adherence above 76 %.

Not a single PLS completely adhered to the standards.

Analysis of PLSs published by different CRGs showed that PLSs were published by 53 review groups in this period. Their output was very heterogeneous, ranging from publishing 3 to 159 Cochrane reviews during the analyzed 23 months. The range of total adherence percentage for different CRGs was from 43 % (Cochrane Incontinence Group, published 8 PLSs in the analyzed period) to 81 % (Cochrane Oral Health Group, 16 PLSs). A CRG devoted to consumers and communication (Cochrane Consumers and Communications group) had adherence to the PLEACS of 61 % (17 PLSs). We noticed that certain Cochrane review groups consistently use their own format of preparing the PLS, which is different from PLEACS items.

The value of Pearson correlation coefficient between time of publication and adherence score was 0.235, indicating a weak positive association. A linear regression analysis was used to test increase in adherence over time. The slope showed significant difference from zero (p < 0.001) suggesting increase in adherence over time.

Discussion

This study showed that Cochrane plain language summaries are highly heterogeneous, in terms of length and adherence to recommended standard for their writing. Cochrane is a global non-profit organization, dedicated to producing and promoting credible and accessible health information. The Cochrane thus represents an international gold standard for high quality, trusted information [ 6]. However, Cochrane systematic reviews are produced by numerous Cochrane review groups, which do not necessarily use the recommended editorial processes. It has already been demonstrated that methods of Cochrane reviews may vary, depending on the Cochrane review group which produced the review [ 7].

Cochrane plain language summaries were developed as a simple format of presenting key information from Cochrane systematic reviews to patients [ 8]. The PLS is considered a main building block for dissemination of the review to the end-users of health information. Therefore, it is commendable that there are efforts to develop a standardized language for describing statistical results, based on the effect size and quality of supporting evidence. Producing standardized high-quality PLS may enable more effective dissemination [ 9]. There are also different ideas on how to improve format of the PLS, based on the information that consumers might want to read [ 8, 10].

The latest version of the PLEACS standards for preparing the PLS was published on February 28, 2013. Even though there was another version of the PLEACS available before that, in this analysis we included only PLSs that were published after the latest version of the PLEACS. Our analysis indicates that the Cochrane PLSs are widely heterogeneous in terms of length and various items of the PLEACS standards. Standard scientific abstracts of Cochrane reviews all follow the same style. Having the same style and format for the PLSs may help consumers to find quickly information they are looking for in specific parts of a summary. Additionally, it is worth emphasizing that the majority of PLSs were shorter than recommended. Some of them had as little as 46 words - this PLS consists of three short sentences [ 11]. Such brevity may not necessarily allow proper explanation about what was done in a review and what the results mean to lay audience.

It is important to emphasize that in this manuscript we did not analyze the PLEACS themselves. It is possible that these standards for writing the plain language summaries are not ideal themselves and not evidence-based, but this is another topic. However, if standards exist in an organization, they should be followed uniformly.

Conclusion

Among 1738 analyzed PLSs, not a single one adhered fully to the measured PLEACS items. Therefore, there is much room for improving the content and consistency of the PLS. To improve consistency of information presented in the Cochrane PLSs and to enhance dissemination of evidence among consumers, Cochrane review groups should closely follow standards for writing simple summaries of Cochrane systematic reviews.

Abbreviations

CRG:

cochrane review group

PLEACS:

standards for the reporting of plain language summaries in new Cochrane intervention reviews

PLS:

plain language summary

References

1. 1.
Maguire LK, Clarke M. How much do you need: a randomised experiment of whether readers can understand the key messages from summaries of Cochrane Reviews without reading the full review. J R Soc Med. 2014;107(11):444-9.

Article PubMed PubMed Central Google Scholar

2. 2.
Cochrane. Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS). 2013. Available at: http://editorial-unit.cochrane.org/sites/editorial-unit.cochrane.org/files/uploads/PLEACS_0.pdf). Accessed 18 May 2016.

3. 3.
Cochrane. Cochrane Translations. 2016. Available at: https://community.cochrane.org/editorial-and-publishing-policy-resource/translation-projects. Accessed 18 May 2016.

Google Scholar

4. 4.
Puljak L. Using social media for knowledge translation, promotion of evidence-based medicine and high-quality information on health. J Evid Based Med. 2016;9(1):4-7.

Article Google Scholar

5. 5.
Cochrane. The Methodological Expectations of Cochrane Intervention Reviews (MECIR). 2015. Available at: http://editorial-unit.cochrane.org/mecir. Accessed 18 May 2016.

Google Scholar

6. 6.
Cochrane. About us. 2015. Available at: http://www.cochrane.org/about-us. Accessed 18 May 2016.

Google Scholar

7. 7.
Vucic K, Jelicic Kadic A, Puljak L. Survey of Cochrane protocols found methods for data extraction from figures not mentioned or unclear. J Clin Epidemiol. 2015;68(10):1161-4.

Article PubMed Google Scholar

8. 8.
Santesso N, Rader T, Nilsen ES, Glenton C, Rosenbaum S, Ciapponi A, Moja L, Pardo JP, Zhou Q, Schunemann HJ. A summary to communicate evidence from systematic reviews to the public improved understanding and accessibility of information: a randomized controlled trial. J Clin Epidemiol. 2015;68(2):182-90.

Article PubMed Google Scholar

9. 9.
Langendam MW, Akl EA, Dahm P, Glasziou P, Guyatt G, Schunemann HJ. Assessing and presenting summaries of evidence in Cochrane Reviews. Syst Rev. 2013;2:81.

Article PubMed PubMed Central Google Scholar

10. 10.
Glenton C, Santesso N, Rosenbaum S, Nilsen ES, Rader T, Ciapponi A, Dilkes H. Presenting the results of Cochrane Systematic Reviews to a consumer audience: a qualitative study. Med Decis Mak. 2010;30(5):566-77.

Article Google Scholar

11. 11.
Singh M, Jaiswal N. Dehumidifiers for chronic asthma. Cochrane Database Syst Rev. 2013;6:CD003563.

Google Scholar

Download references

Acknowledgement

The study was conducted within the project "Professionalism in health" awarded to Prof. Ana Marusic by Croatian Foundation for Science (HRZZ), No. IP-2014-09-7672. The HRZZ was not involved in any way in the design of the study, collection, analysis, interpretation of data and in writing the manuscript.

We are grateful to prof. Damir Sapunar for statistical advice.

Funding

None. Even though the study was conducted within a project, the study did not receive any funding from the project.

Availability of data and materials

All data used in this study are available from the corresponding author upon request.

Authors' contributions

LP conceived the study idea. LP wrote the first study protocol, which was revised by AJK and MF. The data extraction and comparison of independent extractions were performed by AJK, MF, MV, FS, IP, IM and SD. Data analysis were performed by AJK; SD and LP. LP wrote first draft of the manuscript. All authors critically reviewed the manuscript before submission. All authors read and approved the final manuscript.

Competing interests

The authors declare that they have no competing interests.

Consent to publish

Not applicable (the manuscript does not contain any individual person's data in any form).

Ethics approval and consent to participate

Not applicable (this study did not involve any humans or animals, only secondary data analysis of plain language abstracts that were already published).

Author information

Affiliations

1. Cochrane Croatia, University of Split School of Medicine, Soltanska 2, 21000, Split, Croatia

Antonia Jelicic Kadic, Milan Vujcic, Frano Saric, Ivana Propadalo, Ivana Marelja, Svjetlana Dosenovic & Livia Puljak

2. Faculty of Medicine, University of Tuzla, Univerzitetska 1, 75000, Tuzla, Bosnia and Herzegovina

Mahir Fidahic

Corresponding author

Correspondence to Livia Puljak.

Additional file

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License ( http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Reprints and Permissions

About this article

Cite this article

Jelicic Kadic, A., Fidahic, M., Vujcic, M. et al. Cochrane plain language summaries are highly heterogeneous with low adherence to the standards. BMC Med Res Methodol 16, 61 (2016). https://doi.org/10.1186/s12874-016-0162-y
Download citation
Keywords

Cochrane systematic reviews

Plain language summary

Dissemination

Simple language

Adherence

Quality standards

# data[].attributes.attachment_link.title
Cochrane plain language summaries are highly heterogeneous with low adherence to the standards

# data[].attributes.attachment_link.description
The aim of this study was to analyze whether Cochrane plain language summaries (PLSs) adhere to the Standards for the reporting of Plain Language Summaries in new Cochrane Intervention Reviews (PLEACS). A systematic analysis of adherence to the measurable PLEACS items was performed for Cochrane PLSs published from March 2013 to the end of January 2015.

# data[].attributes.headline
Blog by Karen Morley on trying to find health information online

# data[].attributes.subject
Blog by Karen Morley on trying to find health information online

# data[].attributes.attachment_link.readable_text
Karen Morley blogs about her experience of seeking help for her Obsessive-Compulsive Disorder (OCD) and how finding and using Cochrane evidence Cochrane Reviews are systematic reviews. In systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol effective and safe for treating back pain?). The studies are identified, assessed, and summarized by using a systematic and predefined approach. They inform recommendations for healthcare and research. was a turning point.
Without knowing what it was, I had experienced episodes of Obsessive-Compulsive Disorder (OCD) since I was an adolescent, usually when I was particularly stressed. But it was when I was caring full time for my mother, who had multiple conditions including dementia, that I had an unusually distressing episode of contamination-related OCD. When I took to the internet I was amazed to discover that the obsessive checking, washing and , abnormal anxiety were symptoms of an illness with a name - and to recognize other symptoms from my past that I now realised were related to it.

I read everything I could find...

I read everything I could find. I knew I should be careful about my sources, but I read fairly indiscriminately from forums, charity websites and a variety of health websites - some sensible, others less so - and that was a mistake.

I had a vague sense of stigma about mental illness and particularly about medication, so I was interested in talking therapies, especially Cognitive Behavioural Therapy (CBT) with Exposure and Response Prevention (ERP). What could I expect if I tried it? Unfortunately there were some out-of-date posts in which people described nightmare experiences: someone who had allegedly to sit for two hours with his hands in a toilet; a specialist who made his patients 'contaminate' everything in their house including their bedsheets; a popular self-help book with an example of an ERP goal of touching the toilet bowl without washing your hands, then all the 'clean' areas in your house, and then preparing a meal! The mere thought of this made my anxiety soar.

Seeking help for OCD

Nevertheless, since the OCD and anxiety were becoming increasingly hard to tolerate and making my carer's role difficult, with the ongoing and patient support of my brother I made a ten minute appointment with my GP. I told her I thought my OCD was situational and that I understood it was usual to try a talking therapy before medication. I don't recall that we had much discussion about this, though we did talk about respite care, something I knew my mother would hate. The GP gave me the contact details of the local Increasing Access to Psychological Therapies (IAPT) service and told me to refer myself.

It was a difficult telephone conversation that I had to arrange at a time my mother would not overhear. I described my situation, my symptoms, my feelings, my fears of CBT/ERP, I cried and I felt ashamed. The call handler talked to her supervisor and we agreed that I should try counseling first. The waiting list would be shorter at my GP's surgery, she said; she would write a letter and I should contact them. They told me that it would take at least six months to get an appointment.

Things didn't improve

Unsurprisingly, things didn't improve. My Carer Support Worker arranged for some funding for private counselling: I had to find someone from an online approved register. The support of this kind, sympathetic woman was only of limited relevance, largely because my OCD was quite severe. Unfortunately she didn't challenge my fear of psychological , saying she felt CBT/ERP seemed 'cruel' and when I talked about medication, told me an anecdote about her sister who hadn't tolerated it well. After ten sessions I called it off. I talked again with my brother and with a friend who had taken fluoxetine and was currently on citalopram. At the time there were confusing debates in the press and social media about the of these anti-depressants, some of which denied their and suggested tackling the perceived causes of depression and anxiety.

I wanted to see the evidence and know how good it was

And then I searched in the Cochrane Library. I knew what I found there would be evidence based, free from vested interest, recent and reliable.

That was what I needed. I wanted to see the evidence and know how good it was. I wanted to escape the confusion, the emotion (my own included), the forum postings and anecdotes.

I found two :
Selective serotonin re-uptake inhibitors (SSRIs) versus placebo for obsessive compulsive disorder (OCD) Psychological treatments versus treatment as usual for obsessive compulsive disorder (OCD)
They told me all the SSRIs in the were known to be more effective than in reducing symptoms in the short term. So they did work. The effect was described as modest but any improvement would be welcome. There might be unpleasant side effects, which was a worry, but the of common side effects for fluoxetine was close to .

Psychological treatments derived from Cognitive/Behavioural models were more effective than ' as usual'. So they worked, too, although a subgroup analysis suggested that people with more severe symptoms might not benefit as much. Antidepressants, not psychological therapies, were usually the first line of . That was a relief: I had somehow come to think I had a moral duty to slog through a psychological therapy and that taking antidepressants was a sign of weakness, but clearly that was nonsense. ERP was collaborative and the was negotiated with the patient - so no-one was going to make me prepare food after touching the toilet if I didn't want to - and really, who would?

The second review also referred to this one:
Behavioural and cognitive behavioural therapy for obsessive compulsive disorder in children and adolescents.
I knew I should be wary of this because my eighteenth birthday has long gone. But the said OCD was similar in children and adults. I was interested in the finding that neither medication nor BT/CBT was superior but there was evidence that medication and BT/CBT combined produced better than medication alone. It looked as if I was going to have to tackle the Exposure and Response Prevention.

It gave me a feeling of control

I made up my mind to be pro-active, grasp the nettle and discuss with my GP. It gave me a feeling of control I hadn't had before.

Unable to get a quick appointment with my GP, I asked for the duty doctor to phone me and advise me. I was lucky: psychiatry was his specialism and he was extremely sympathetic, bypassing reception and making me an appointment at his clinic. He asked me what I already knew about OCD and its . 'I've read a couple of ,' I said, casually. I was unprepared for the response. He perked up instantly and looked twice as energetic. It might have been a coincidence - he was an excellent doctor - but for the first time ever I experienced shared decision-making. It was tremendous. I felt like a partner in the consultation. He showed me where I could find information online. We discussed the potential benefits and risks of medication - it was customary to begin with fluoxetine - the dosage, the side effects, what we would do if I couldn't tolerate the . He turned his computer screen towards me so I could share the information and talked me through things as he looked them up. We discussed my circumstances, too, and what positive steps could be taken to help my mother and me.

For the first time ever I experienced shared decision-making. It was tremendous.

Cochrane was a turning point for me

Then he asked me, 'Do you think you're depressed?' I was surprised. I knew I was anxious. But as I went through a list of depressive symptoms, I realised I was. He already thought so, of course, but he didn't tell me: he asked me. He was listening to me. I felt valued; I could be active and exercise control, which I found particularly encouraging in my situation. This persisted through subsequent appointments when we discussed whether and when to increase the dosage. I made another call to IAPT to ask for CBT/ERP and (after several months) had some therapy via telephone: I felt much braver once the medication had begun to take effect. And yes, the whole process was negotiated and graded.

If I hadn't taken this route - if I had carried on with counseling - I am sure I would have had to give up my caring role, and I wouldn't be recovering now. Cochrane was a turning point for me, and I am utterly convinced of the value of evidence based medicine.

But you don't have to take my word for it. Try it for yourself.
Join in the conversation on Twitter with @CochraneUK or leave a comment for Karen on the blog.
References can be found here.

# data[].attributes.attachment_link.title
My OCD story: evidence-based medicine to the rescue! - Evidently Cochrane

# data[].attributes.attachment_link.description
Karen Morley blogs about her experience of seeking help for her Obsessive-Compulsive Disorder (OCD) and how finding and using Cochrane evidence Cochrane Reviews are systematic reviews. In systematic reviews we search for and summarize studies that answer a specific research question (e.g. is paracetamol effective and safe for treating back pain?).

# data[].attributes.headline
Blog by Karen Morley, Cochrane Consumer Review

# data[].attributes.subject
Blog by Karen Morley, Cochrane Consumer Review

# data[].attributes.attachment_link.readable_text
Cochrane Common Mental Disorders recently asked Karen Morley if she would give consumer peer review a go. In this blog, first published here, Karen provides an insightful and honest reflection of how she found the experience and notes important challenges that Cochrane and Cochrane groups can improve on.

Would I, as a new consumer volunteer for Cochrane Common Mental Disorders, have jumped in at the deep end and attempted a peer review if I hadn't been asked? I don't think so. But now I'm here, unsteadily dogpaddling, with two protocols and a systematic review under my belt, I thought I'd share some reflections and questions with you.

First came the Fear. I felt quite tiny in the huge, complex Cochrane organisation. And 'peer reviewing' sounded so imposing. I know there are consumers who are themselves researchers or health professionals, but how could I be considered a 'peer'?

Jess Hendon, the Managing Editor, was warm, kind and welcoming and assured me my lived experience was what counted. But did I have the knowledge, understanding and skills for the task? I had some experience of close reading and critical evaluation from my years as a secondary English teacher, and a basic understanding of systematic reviews from contributing to some Cochrane consumer learning resources that are currently in development, but would that be enough? I was afraid of being judged and afraid what I had to say might not be useful. Well, it was too late for dithering; a deadline was approaching: I took the plunge.

There was a checklist that gave some focus and structure. There was also some linked online support. At the moment this is quite brief and packed with detail and assumes some prior knowledge in the reader (for instance, 'interventions', 'outcomes', 'comparator' and 'settings' have specialist meanings that might be unfamiliar to a complete newcomer) and the language is rather formal. I was lucky to have already explored the subject a little and I wondered what it must feel like to be starting from scratch.

Despite the checklist I was still full of doubt. What exactly was expected of me? How should I express myself and in how much detail? A worked example would have been helpful. When I later saw the feedback from the authors on all the reviewers' comments I had a much clearer idea of the process and the kind of contributions that were acceptable. I felt strangely detached and lonely as I looked at my screen. It would have been nice to have the luxury of a mentor, or some buddies to bounce ideas off, or maybe to have had a group practice run. It would also be interesting to hear an author's view. (I've since had the pleasure of meeting Lindsay Robertson, which I found reassuring - thanks, Lindsay!) Last year I took part in a James Lind Alliance Priority Setting Partnership and I found it was illuminating to hear the comments of other patients, carers and clinicians: we have lots to learn from each other.

The protocol I looked at first would certainly be challenging for a consumer to read. I had to check my own understanding of 'monoamine neurotransmission', for instance. (That was quite a ride!)

I'd be interested to know whether, when consumer reviewers write about language and style, the same issues crop up frequently, and whether it might be desirable or feasible to collect them or to have some sort of glossary. Making suggestions for explanations or alternative expressions throws up a host of knotty problems. Who is the reader I'm writing for? What level am I pitching at? Which consumers read protocols - and systematic reviews - and why? How much can something be simplified without losing important nuances or even completely misrepresenting its meaning? Is there in fact a conflict between several different audiences and purposes? Which ones do the authors have at the forefront of their minds? Why doesn't a protocol have some kind of Plain Language Summary?

I suppose I felt the most confident about working on the language and less so about commenting on things like objectives and the criteria for considering studies. I knew there would be other contributors but I had a strong sense of responsibility: a real desire to write something that would actually be helpful to the authors. I have enormous respect for them and the huge, intricate, demanding task they have carried out. I don't want to waste their time.

But now to the heart of the matter: representation. 'Does the background address the hopes and concerns of people considering the treatment?' asks one of the prompts. My task is to put myself into the shoes of the most important end users of systematic reviews and protocols: the consumers. This is the biggest responsibility of all. I can do my best but I'm aware that, alone, I'm not up to the task. I am one old, white, middle class woman, with one experience of one manifestation of a wide-ranging mental health condition: one tiny part of a huge jigsaw. Cochrane needs many consumer peer reviewers to represent us in all our diversity. And not just reviewers. I appreciate my opportunity to make changes to the review before it's published, but this is the end of a very long journey. Have other consumers been involved before I see the review - in setting priorities or determining the research question or outcomes, say? How would I know? Shouldn't it be mentioned in the review somewhere?
So, consumers, you are needed! If you feel the stirrings of an inclination to get involved, please have a go at something - anything. You might feel reticent, but come and help lead the way until it becomes commonplace for us to be included and to shape research around all our hopes and concerns.
I am sometimes shocked when I think how comparatively young the modern evidence based medicine movement is. Similarly, it's easy to forget that the meaningful inclusion of consumers is really only beginning. I am delighted that Cochrane's revised 2018 Peer Review Policy means Cochrane Review Groups must now aim to have at least one consumer peer reviewer for each Cochrane Review. (If there were more of us, fellow consumers, it could be more than an aim!) I'm delighted and daunted in equal measure to be one of those peer reviewers, and enthusiastic about CCMD's plans to take this forward. It has been a tremendous pleasure to meet Rachel Churchill, Jess Hendon, Lindsay Robertson and Noortje Uphoff (well, I popped into her office!). The reception they have given me has been affirming and encouraging. I feel the burden, but I can see a great opportunity.

# data[].attributes.attachment_link.title
Confessions of a rookie consumer peer reviewer

# data[].attributes.attachment_link.description
Cochrane Common Mental Disorders recently asked Karen Morley if she would give consumer peer review a go. In this blog, first published here, Karen provides an insightful and honest reflection of how she found the experience and notes important challenges that Cochrane and Cochrane groups can improve on.