# data[].attributes.headline
What is Data Visualization in 3 minutes?

# data[].attributes.subject
What is Data Visualization in 3 minutes?

# data[].attributes.body
Limited and costly data, that was challenging to collect, predominated prior to the digitalization era; this short video helps explain why data visualization is so beneficial today with a surplus of inexpensive data that is difficult to process.

# data[].attributes.attachment_link.title
What is Data Visualization in 3 minutes?

# data[].attributes.body
This TEDTalk discusses how data storytelling can benefit your personal health, promote involvement in your own care, and enhance your own knowledge about management. Not only could this be beneficial for us as providers to explain trends to our patients, but educating them about how to take simple measurements, like daily temperature or blood pressure, can empower and motivate them to take control of their health. Storytelling with data, as opposed to just comparing today's A1C to the one 6 months ago, can really help paint a picture for our clients.

# data[].attributes.attachment_link.title
Show me the data -- becoming an expert in yourself: Talithia Williams at TEDxClaremontColleges

# data[].attributes.body
The author discusses the four main ethical principles, beneficence, nonmaleficence, autonomy, and justice, and some key principles of each. Integrating ethical aspects, the author focuses on caring as a central element (Verkey, 2021).  Varkey, B. (2021). Principles of clinical ethics and their application to practice. Medical Principles and Practice, 30(17), 17-28. https://doi.org/10.1159/000509119

# data[].attributes.attachment_caption
External factors influencing health informatics

# data[].attributes.body
As stated in the abstract by the authors, "This article provides an overview of the HL7 FHIR Architecture, including the workflow state, linkages, and various informatics approaches used in healthcare units. The article discusses future trends and directions in Health Informatics for successful application to provide public health safety. With the advancement of technology, healthcare units face new issues that must be addressed with appropriate adoption policies and standards" (Yogesh & Kerthikeyan, 2022).

Yogesh MJ, Karthikeyan J. Health Informatics: Engaging Modern Healthcare Units: A Brief Overview. Front Public Health. 2022 Apr 29;10:854688. doi: 10.3389/fpubh.2022.854688. PMID: 35570921; PMCID: PMC9099090.

# data[].attributes.attachment_link.readable_text
Abstract

In the current scenario, with a large amount of unstructured data, Health Informatics is gaining traction, allowing Healthcare Units to leverage and make meaningful insights for doctors and decision-makers with relevant information to scale operations and predict the future view of treatments via Information Systems Communication. Now, around the world, massive amounts of data are being collected and analyzed for better patient diagnosis and treatment, improving public health systems and assisting government agencies in designing and implementing public health policies, instilling confidence in future generations who want to use better public health systems. This article provides an overview of the HL7 FHIR Architecture, including the workflow state, linkages, and various informatics approaches used in healthcare units. The article discusses future trends and directions in Health Informatics for successful application to provide public health safety. With the advancement of technology, healthcare units face new issues that must be addressed with appropriate adoption policies and standards.

Keywords: health informatics, public health, information systems, health policy, public health systems

1. Introduction

Machine Learning is the fastest-growing topic in computer science today, and Health Informatics (HI) is the most difficult problem to solve ( 1, 2).

Emerging economies are increasing their investments in healthcare, which makes sense and encourages health professionals to adopt sound frameworks and regulatory standards, as well as health IT, to improve the quality and efficacy of care ( 3). In this expanding field, new age occupations can be established. This new field has the potential to be a lucrative career path in the future. With a clear flow of information across many medical subsystems, adoption of electronic health record systems (EHRs) will improve the health care system going forward ( 4).

Big Data is frequently employed in the field of health informatics, as new data is constantly pouring into the system, requiring analysis and interpretation in order to make rational decisions ( 5, 6).

In recent years, the healthcare industry has seen a rapid growth in medical and healthcare data, which can be used to improve facilities and public health care utilization and implementation by modern healthcare units using novel treatment and diagnosis methodologies, which gives citizens confidence in using the best public healthcare services available and aids governments in developing better healthcare policies ( 10).

Computerized systems for analysis and diagnosis were first adopted by health professionals. More recent technology are making it easier for people to make better decisions ( 11, 12).

Health Informatics' promise to improve public health activities would be fully realized if information science and technology were applied ( 13).

In a complex social and economic environment, the issue is thus how to increase the quality of offered healthcare services while lowering prices. Information and communication technologies (ICT) have been shown to help healthcare systems increase productivity, which has resulted in significant cost savings in operations and service delivery. For administrative and healthcare objectives, ICTs have already proven to be quite effective. New prospects for new medical equipment and systems are opening up as ICTs become smaller, quicker, wireless, and remotely controlled.

The Internet and the web have recently brought up new possibilities for increasing the response time of health-care services while also lowering costs. It is clear that we are in the early stages of a new era that will fundamentally alter the way healthcare services are provided. This will help us acquire the public's trust in using high-quality healthcare services.

New e-Health services and technology should be researched, developed, promoted, and disseminated with significant effort. With the present pandemic (COVID-19) sweeping the globe, increasing ICT use has demonstrated that healthcare will become more contactless in the future, with fresh means of treating patients and providing healthcare services emerging. This is a popular yet difficult research subject since it necessitates interdisciplinary competence ( 14).

The major purpose of Health Informatics is to increase our understanding of medicine and medical practice by using real-world medical data. In the subject of healthcare, health informatics is a blend of information science and computer science ( 15).

Big data in healthcare is intimidating not only because to its sheer magnitude, but also due to the variety of data types and the pace with which it must be managed. To gain people' trust and give quality healthcare services, all health service providers are now putting in extra effort to use the most up-to-date technologies to provide health services and advanced treatments.

Various requirements drive innovation in this industry, such as finding appropriate accommodation with standardization and coordinating the acquisition and implementation of newer healthcare systems and services on a national/international level.

With the present COVID-19 scenario, investments in this sector are gaining steam with new age healthcare units in many nations, and growing economies such as India and China will play a vital part in providing good and quality healthcare services in the future.

As a result, New Age Healthcare Units and Systems will play a critical role in dramatically lowering costs, making Public Healthcare Systems more dependable and instilling citizens' confidence in using inexpensive healthcare.

2. Related Work

Big data is a term used to describe a significant volume of data that is collected and stored today and has outgrown standard data management and analysis solutions. Solutions like Hadoop and Spark, according to Roger Fyre and Mark McKenney, have arisen to solve some of the big data concerns ( 16).

Machine Learning is crucial in the testing and development of various models that take into account clinical and other important medical characteristics for decision-making.Medical imaging, which incorporates capabilities such as image segmentation, image registration, annotation, and database retrieval, is one of the most famous examples of newer medical technologies that can be utilized for decision making in the future. For all of this, updated ML/DL models for speedier decision-making can be constructed ( 26).

Machine learning/data science researchers are in high demand for developing algorithms that adapt to changing data. Deep Learning (DL) is now being used to solve more difficult problems in the healthcare/informatics arena ( 26, 27).

Holzinger et al. ( 28) discussed many approaches to developing an explainable model for the medical domain were examined. Prediction explanations can be useful in a variety of situations, including teaching, learning, research, and even court. The demand for interpretable and explainable models is growing in the medical field. They need to be able to re-enact the decision-making and knowledge extraction processes. In their article, Ribeiro et al. ( 29) discussed how machine learning models are black boxes. Understanding the reasons for predictions can help to build trust. It can be used to assess model performance and construct better, more accurate, and correct models by providing insights into the model. Ribeiro et al. ( 29) propose the LIME algorithm for explaining predictions of any model. This article by Bahdanau et al. ( 30) deals with neural machine translation, but the model proposed can be used in a variety of other applications.

3. Introduction to HL7 FHIR Architecture

In the last two decades, electronic health records (EHR) have been widely implemented in the United States to improve health-care quality, increase patient happiness, and reduce health-care costs ( 31- 33). As growing countries such as India, China, and Bangladesh experiment with innovative ways to establish EHR systems, it will significantly aid in the development of effective public health systems.

FHIR's basic idea was to create a set of resources and then create HTTP-based REST application programming interfaces (APIs) to access and use these resources. FHIR uses components called resources to access and perform operations on patient's health data at the granular level. This feature distinguishes FHIR from all other standards because it was not present in any earlier version of HL7 (v2, v3) or the HL7 clinical document architecture (CDA).

The fundamental building blocks of FHIR are the so-called resources, which are generic definitions of common health care categories (for example, patient, observation, practitioner, device, condition). For data interchange and resource serialization, FHIR employs JavaScript object syntax and XML structures. FHIR not only supports RESTful resource exchange but also manages and documents an interoperability paradigm.

FHIR has grown in popularity and is being increasingly used by the health care industry since its inception. In 2018, six major technology companies, including Microsoft, IBM, Amazon, and Google, vowed to remove barriers to health care interoperability and signed a statement mentioning FHIR as an emerging standard for the interchange of health data. With incorporation of Substitutable Medical Applications Reusable Technologies (SMART), a platform for inter operable applications ( 34), FHIR can be expected to attract even more attraction in digital health in the future.

The use of FHIR for medical data transmission has the potential to deliver benefits in a wide range of disciplines, including mobile health apps, electronic health records (EHRs), precision medicine, wearable devices, big data analytics, and clinical decision support.

The primary goal of FHIR is to reduce implementation complexity while maintaining information integrity. Furthermore, this new standard integrates the benefits of existing HL7 standards (v2, v3, and CDA) and is projected to overcome their drawbacks. FHIR enables developers to create standardized browser applications that allow users to access clinical data from any health care system, regardless of the operating systems and devices used. Figure 1 represents the general architecture of FHIR ( 35).

3.1. FHIR for Patient Access to Medical Records

FHIR, or Fast Healthcare Interoperability Resources, is an HL7 standard for electronically transferring healthcare information. The CMS Interoperability and Patient Access final regulation, announced in 2020, mandates all CMS-regulated payers to use FHIR version 4. Unlike earlier releases, the fourth iteration is backward compatible, ensuring that software suppliers' solutions will not become obsolete when a new FHIR version is released.

The FHIR (pronounced "fire") standard defines a collection of HTTP-based RESTful APIs that allow healthcare platforms to exchange and share data in XML or JSON format. FHIR offers mobile apps, which users can obtain from the Apple App Store or Google Play in order to access their medical records and claims data.

FHIR's basic exchangeable data piece is known as a resource. Each resource is formatted similarly and contains roughly the same amount of data. It offers information about patient demographics, diagnosis, prescriptions, allergies, care plans, family history, claims, and so on, depending on the kind. They span the complete healthcare workflow and can be used independently or as part of a larger document.

Each resource is given a unique ID, and many parties-health systems, insurers, patients, or software developers-can access the underlying data element using an API. Figure 2 represents the data layers and resources of FHIR ( 35).

3.2. FHIR Resource

A resource is the smallest discrete concept that can be independently maintained and is the lowest feasible unit of a transaction ( 36). As a result, a resource is a known identity that provides useful data. Each resource has distinct bounds and differs from all others. A resource should be provided in sufficient depth to specify and enable the process's medical data interchange. The FHIR community has specified over 150 resources till date, according to the most recent FHIR version (R4) ( 37).

There are five key categories in which these resources can be found: (1) Administrative: location, organization, device, patient, and group; (2) Clinical: CarePan, diagnostics, medication, allergy, and family history; (3) Financial: billing, payment, and support; (4) Infrastructure: conformance, document, and message profile; and (5) Workflow: encounter, scheduling, and order.

FHIR is fast gaining popularity due to its dynamic properties. FHIR is projected to quickly become a symbol for clinical data interchange in the health-care industry.

3.3. Workflow Description

Workflow is a critical component of healthcare; orders, care regimens, and referrals drive the majority of activity in in-patient settings, as well as a significant amount of activity in community care. FHIR (Fast Health Interoperability Resources) is concerned with workflow when it is necessary to share information about workflow state or relationships, when it is necessary to coordinate or drive the execution of workflow across systems, and when it is necessary to specify permissible actions, dependencies, and behavior requirements.

3.4. Workflow State and Relationships

FHIR does not have to be used for workflow execution. Orders, care plans, test findings, hospital admissions, claim payments, and other documents can all be exchanged utilizing FHIR resources without the need for an FHIR transaction to solicit fulfillment of those orders or request payment of those claims. Because it necessitates a greater level of standardization, inter-operable support for workflow execution is a more advanced FHIR activity. Interoperable workflow execution necessitates the standardization of processes, roles, and activities across multiple systems, rather than just the data to be exchanged.

Even if FHIR is not used for workflow execution, there is still a requirement to standardize workflow data elements: how does an event or a result point to the order that allowed it? How are parent and child steps tied together? How does a care plan know which protocol it is following?

FHIR distinguishes three types of resources engaged in activities: requests, events, and definitions. Each of these categories is associated with a "pattern." Resources in that category are encouraged to follow their specific pattern. These patterns provide conventional elements that are common to the majority of resources in each category. Work groups are anticipated to align with common domain behavior and requirements as more authoritative than "desired" architectural patterns, therefore strict conformance is not necessary. When a pattern capability is assessed to be "not common, but nonetheless relevant" for a given resource, it may be supplied through extensions rather than core parts. Figure 3 represents the work flow relations of FHIR Standard ( 38).

4. Overview of Health Informatics

Health informatics involves more than merely automating routine tasks. With contemporary technology developments in machine learning and deep learning, it is possible to redesign systems using methodologies that were previously impossible or not even considered ( 26).

Such a study is computationally expensive and can now be handled by the latest IBM POWER9 processors with GPU capabilities, which was previously impossible because the data was not available in electronic form and the number of possible symptoms/incident patterns was too big to manage. Early detection of patterns that can assist anticipate what kind of treatment or diagnosis can be offered to such patients has improved dramatically ( 39).

In the future, modern healthcare units will make use of such a framework for successful treatment delivery for society as a whole, making effective use of data obtained from such systems by extracting insights that assist decision makers such as doctors, hospital owners, and health policymakers.

Some of the most pressing future difficulties in healthcare informatics will be addressed by developing systems based on the principles listed below, which are not commonly employed in this sector:

Traditional machine learning frameworks consider only one learner attempting to solve a single task. However, in many applications, there are multiple tasks that label the same data instances differently. When the tasks are related, the information learned from each task can be used to improve learning of other tasks. Learning relevant tasks concurrently, rather than learning each task independently, is thus advantageous. Multi-task learning makes use of the intrinsic relationships between multiple tasks to improve generalization performance. It benefits all tasks by leveraging task relatedness and shared information across relevant tasks ( 43).

Traditional machine learning technology has had a lot of success and has been used in a lot of practical applications, but it still has certain limits in some real-world settings. Machine learning works best when there are a lot of labeled training cases with the same distribution as the test data. In many cases, however, gathering sufficient training data is costly, time-consuming, or even impossible. Semi-supervised learning can help to alleviate this difficulty by removing the requirement for large amounts of labeled data. A semisupervised approach typically requires a small amount of labeled data and a large amount of unlabeled data to improve learning accuracy. However, in many cases, unlabeled instances are difficult to collect, making the resulting traditional models unsatisfactory ( 44).

- Multi-agent-Hybrid Systems Multi-agent systems are networks of interconnected autonomous agents in which the behavior of neighboring agents influences the dynamics of each agent.Because of the increasing importance of multi-agent systems, there is a growing interest in coordination control to ensure consensus, flocking, containment, formation, rendezvous, and so on. To better understand multi-agent coordination, a variety of dynamic models of agents have been developed over the last two decades. Furthermore, many mathematical methods are used in the analysis and control of multi-agent systems. For more information, see survey article ( 45) and the references therein ( 46).

- Representation Learning

Patient-specific data such as vital signs, medications, laboratory measures, observations, clinical notes, fluid balance, procedure codes, diagnostic codes, and so on are all included in modern EHR systems. Clinicians originally employed the codes and their hierarchies, as well as their associated ontologies, for internal administrative and invoicing functions. Recent deep learning algorithms, on the other hand, have attempted to project discrete codes into vector space, identify intrinsic commonalities between medical concepts, more accurately depict patients' health, and perform more precise predicting tasks. Word embedding and unsupervised learning have been used to examine medical concepts and patient representations in general ( 47).

Health informatics has a number of long-term benefits in terms of research and healthcare delivery that can be used to create a sustainable ecosystem. ICTs aid in the enrichment of relevant data for analysis and decision-making by health professionals. Following the pandemic (COVID-19), new age healthcare units will arise, with increased investment and research spending making public healthcare more accessible. As a result, solutions are required to manage the massive amounts of data created by medical equipment and healthcare systems, allowing for effective storage and retrieval in real-time data analysis and decision-making.

4.1. Informatics Approaches

To make electronic health data more easily usable for research, recent publications have identified the need for effective adoption and use of standards, essential data and research services, clear and consistent policies regarding data access and use, and transparent and effective governance structures ( 48, 49).

To achieve data quality criteria, electronic health data utilized in research frequently require standardized ontologies, additional contextual information, field transformations, and missing or contradictory data to be handled ( 50). For research-related data or functions, such as cohort identification and repeated extracts of source data over time, system development is frequently required ( 49). Organizations with expertise utilizing and enhancing their health IT infrastructure for research have shared their lessons learned in these areas, adding value to organizations with similar goals but less experience or resources ( 51).

For example, when preparing data for research use, organizations must understand the clinical context and structure of electronic health data, just as they do for other data uses such as decision support or population health. Individual researchers and data analysts can be relieved of their load by informatics support that spans research and operational usage of data. It is vital to evaluate and develop informatics tools and approaches by establishing processes that allow for coordinated governance and decisions informed by research users.

Investing in infrastructure to enable the use of electronic health data for research has also been shown to be beneficial to researchers by providing them with the necessary tools and expertise, to patients by providing clinical trial participation opportunities, to clinicians by enabling more rapid translation of research into practice, and to population health analysts by facilitating patient cohort views. In order to reduce project-specific IT costs, using health IT to assist research necessitates greater flexibility, increasing use of standards, and reusable ways for getting, preparing, and evaluating data ( 52).

Any use of operational data in research necessitates the establishment of a privacy and security framework, as well as data governance monitoring. Two initiatives, Informatics for Integrating Biology and the Bedside (i2b2) and Observational Health Data Sciences and Informatics (OHDSI), have developed informatics tools and approaches that allow researchers to query organizational participants and support transformation or analytics of relevant data to facilitate research ( 53, 54).

Specifically, i2b2 has standardized data models and distributed computational tools that enable for the anonymous identification of potential genomic study participants at the institution level. OHDSI also employs a single data model, which incorporates information such as health economics and health systems. The methodologies utilized in these programmes demonstrate the kind of functionality that may be required in health IT systems to better support research, as well as the types of concerns with the quality of electronic health data that regularly arise.

5. Status QUO in Health Informatics: An Indian Perspective

5.1. Health Care Delivery Systems

Despite having a solid telecommunication infrastructure, the existing systems are based on manual record keeping. The value of medical informatics in healthcare delivery has yet to be recognized by policymakers ( 55).

In countries such as India, health informatics is a new and emerging discipline. Its future prospects are very bright, thanks to the development of excellent infrastructure here. However, in order to implement a robust framework, this necessitates a multidisciplinary interaction with various stakeholders.

Information systems development could be another area where research is being conducted to improve the way data flows from various sources such as devices and medical equipment, allowing doctors and decision makers to make rational decisions on critical cases or equipment purchases in the future.

Digitizing all medical data also aids in the creation of a structure for patient-related data in a hospital that can be easily retrieved and searched. Finally, the development of some kind of electronic health record can be accomplished through the development of information systems ( 56).

5.2. Applications of Health Informatics

Health Informatics in India can become cost effective and ensure proper service delivery, which aids in beneficiary behavior change through the use of ICTs ( 57). We can create novel applications that can be used effectively by utilizing local talent and effective use of ICT in remote parts of India ( 58). Various governance issues can be addressed in the future with certain checks and balances in the data collection and analysis process ( 59). The following are some of the areas where health informatics can be used:

Epidemiological disease prediction

Awareness in Healthcare Processes

Healthcare in Remote areas

Electronic Health Records and its linkages with health systems

Development of Decision Support Systems (DSS)

Visualization tools for doctors

Recommendation Systems for Health Informatics

Precision Drug Prediction.

The potential of this emerging area has far-reaching benefits over a long period of time, and new and novel solutions can be built using various machine/deep learning models. There is a lot of work to be done in this area where we can use cutting-edge technology to aid/assist in the development of robust products and frameworks for public health policy ( 60- 63).

5.3. Future Trends and Directions in Health Informatics

These are a few of the current trends in the field of Health Informatics that can be used to develop sustainable products, services, and health-related policies for effective implementation across the country and internationally. A few of them are listed below, and many more trends may emerge in the near future as a result of discussions with multiple stakeholders.

Data standards and Interoperability

Processes to transform medical/clinical data

Toolkits and Pipelines: Data Management

Standardized Reporting Methodologies

Appropriate Use of Informatics Expertise.

More trends may emerge in the future, taking into account the most recent technological advancements. Because health informatics is a new and emerging field, more research challenges may emerge, bringing forth newer perspectives in the future. To solve more difficult problems in this area, future researchers will prefer machine/deep learning methods/models ( 26). There are numerous other research directions being pursued in relation to various aspects of health care data such as quality, veracity, privacy, and timeliness. The following are some of the most notable data characteristics of healthcare data ( 9, 64):

Real Time: The velocity of big data in health informatics reflects not only the rate of data collecting and processing, but also the timeliness of replies. There are various instances that call for a quick choice. For example, it would be immensely desirable to monitor and analyse a person's health condition in real time or near real time in order to predict potential disease. It would also be critical to raise the alert for a potential influenza outbreak by examining public health statistics. Although real-time analytic applications are still in their infancy in the big data era, they represent the strongest trend and most promising direction in health informatics' future ( 72).

Privacy: Data privacy is another major worry for future big data analytics in healthcare informatics ( 73). Although strong laws control more formalized EHR data, extra attention should be taken and standards should be enforced to regularize the use and dissemination of personal and sensitive information obtained from diverse sources. In addition to data privacy, there are a number of other challenges, like as data protection, data security, data safety, and the protection of doctors from liability resulting from manipulated data, that necessitate the use of specialized big data analytics to address these complicated constraints ( 73, 74).

6. Challenges in Health Informatics

There are numerous challenges in this area, particularly in a country like India, where a large population is denied affordable healthcare, which can serve as a starting point for developing and implementing robust health informatics applications, products, and Research and Development investments. The start-up/venture costs, people, and equipment are very high, and it is a niche sector in which many people are hesitant to venture into such a space/sector that can be leveraged making very good business sense ( 78).

To improve the way health services are delivered and implemented to the public, a multidisciplinary approach involving various sectors/stakeholders is required. These informatics systems can be used for a variety of purposes. Medical/healthcare data will be more structured and democratized by competent authorities.

The majority of stakeholders are unconvinced about the benefits of Internet technologies in health care and are unfamiliar with how to use such new technology ( 80). Concerns about security and privacy may arise as robust healthcare systems are developed in the future ( 11).

Proper techniques are essential to ensure patient data confidentiality, and system security may become a concern when data policy standardization increases. Many new challenges can be encountered while developing novel and innovative ways to promote public health through the use of information technology (IT) and other computing technological advances such as Cloud Computing, Data Visualization, and Medical Informatics. Future research is required because the healthcare/informatics domain is still in its early stages, where more difficult problems can be solved and newer products and services can be spawned as a business venture. Deep Learning in Health Informatics has its own set of challenges ( 26).

Despite significant investments in information technology, patient safety and productivity have not improved. Preventable medical errors are the third leading cause of death in the United States, after heart disease and cancer, killing over 400,000 people each year. These blunders cost the United States over one trillion dollars per year. To combat this disease, the federal government has mandated that the healthcare industry transition to electronic health records (EHRs) and use these records to improve patient processes and outcomes (i.e., meaningful use). Emerging economies have to allocate and give incentives to healthcare units/organizations to promote the use of EHRs.

The following are the main gaps and challenges to an effective pandemic response in health information management and health informatics:

A lack of standards for information exchange between providers and PHAs (Public Health Authorities)

Issues with data collection and data quality, particularly in terms of completeness and timeliness

Governance, Public Policies, and Regulations.

The latter included a lack of procedures to support efficient data sharing, contact tracking, and data governance, as well as providers' concerns about privacy regulations, which resulted in insufficient data sharing.

Governance and public policy hurdles stem from chronic underfunding of public health infrastructure, as well as a lack of adequate investments in resources (particularly qualified employees) and facilities. A key difficulty was also recognized as a lack of international coordination. Many overlapping and interrelated legal, ethical, scientific, technical, technological, health equality, and privacy elements influenced how health information was managed or mismanaged during the COVID-19 global pandemic. Other long-standing systemic difficulties in health information management will need to be addressed in order to operationalize many of the data and information system recommendations ( 81).

The financial investment required to design, execute, and sustain e-health programmes is a key difficulty in health informatics, and Anderson cited a lack of financial backing and high initial expenses as hurdles to implementing ICT in health care ( 82). While health informaticians and information professionals may see future benefits from investments in ICTs, health professionals and managers may be skeptical, especially if they are satisfied with current methods of working and wish to maintain the status quo, and may see such initiatives as diverting financial resources away from under-resourced clinical care ( 83).

Resistance to the establishment of ICT systems by health professionals and managers can lead to further issues once the systems are in place, and the restricted adoption of health informatics applications has meant that their potential is not always achieved. Decision support systems, for example, may be ignored or overridden, and evidence-based information may have limited applicability for an individual patient. Clinicians make life-changing judgments or act in life-threatening situations, and if they don't comprehend the reasons behind computer-based decision support systems, they won't trust them or use them effectively ( 84).

This highlights the importance of not only involving physicians and healthcare professionals in the construction of systems and the interpretation of outcomes, but also of providing adequate explanation and information at the point of care for healthcare practitioners to trust the systems ( 84).

As previously stated, identifying the types of information that clinicians require, as well as the methods by which they access and utilize information, is critical in ensuring that developments not only meet the needs of the users, but are also perceived to be valuable, so that health professionals and other users will want to maximize their potential. Addressing healthcare professionals', patients', and the public's concerns about data security, as well as threats to patient privacy and confidentiality, will be critical in developing online access to patient records ( 85). Greater security measures integrated into system design will help boost system confidence, however the chance of third parties getting access to sensitive patient-identifiable data remains a danger ( 82).

Another challenge that can stall the creation and execution of health informatics efforts is quality. The real and perceived quality of data entered into systems and then used for health care is vital not only for assuring system use, but also for the safety and well-being of patients. If data is not input, or is not entered correctly, the buildup of missing or low quality data discourages others from using the system and creates additional suspicion and skepticism about future advancements. The importance of accurate and correct data will grow as lifelong electronic records are established ( 84), both prospectively as individuals are born and retrospectively using data acquired over an existing person's lifetime to date.

As previously noted, the earlier development of small-scale information systems within individual departments or hospitals resulted in system incompatibility and difficulties communicating or transferring data when larger-scale systems were later built. One approach to addressing this issue is to increase interoperability and employ known electronic record architectures in the creation of new systems. In addition, the lack of data standards in health presents additional challenges for moving and sharing data between systems ( 82). Attempts to address these issues include the creation of information-management standards such as Digital Imaging and Communications in Medicine (DICOM), Health Level Seven (HL7), and terminologies and coding systems [e.g., the International Classification of Diseases (ICD), Read coding, and Snomed] to standardize the ways in which medical conditions and diseases are represented in computer-based systems and to attempt to codify the natural language used by medical staff.

The International Classification of Diseases (ICD) was created to provide a standard method of classifying medical diagnoses for epidemiology and health-care purposes; initially, it only included causes of death, but more recent versions have included causes of morbidity, and it is now in its eleventh version ( 86). There are numerous challenges, and many new problems can be solved using cutting-edge technologies such as ML/DL and cloud computing ( 26, 87- 92).

6.1. Health Data Standards Challenges and Possible Solutions to Them

Clearly, data standards are not lacking in the healthcare industry. SDOs (standard development organization) have created a plethora of them to address nearly every facet of communication between diverse health systems.

However, the simple fact that they exist and are available does not address all of the issues surrounding interoperability. We'll go over some of the more difficult standards issues, as well as potential solutions.

Medical coding speed and accuracy issues:

The manual work required to convert diagnoses, treatments, services, treatment plans, and other concepts into medical codes is undertaken by professionally qualified individuals. Computer-assisted coding systems are now used by coders. However, the translation process's speed and precision are far from flawless. To that aim, high hopes are placed on AI-powered tools capable of identifying proper codes and recommending them for expert evaluation. Currently, such intelligent systems speed up coding, but they cannot completely replace humans and automate the entire process.

Need for mapping between codes:

Each code in healthcare serves a specific purpose: SNOMED allows physicians to provide a thorough clinical picture of a patient being treated, whereas ICD-10 presents diagnoses quickly and CPT summarizes services. However, there are times when translation from one code system to another is required. As previously stated, SNOMED cannot be used for billing reasons and must be translated to ICD-10-CT. To overcome mapping issues, standard development groups experiment with various approaches.

Lack of compatibility between old and new standards:

To comply with existing interoperability regulations, hospitals must make content described by USCDI available via FHIR-based APIs. But, let's face it, the truth is: Most EHR systems were designed with previous standards in mind. Some of them are only capable of importing and exporting HL7 v2 messages. Others rely heavily on C-CDA materials. Neither v2 nor C-CDA are compatible with granular USCDI data elements or FHIR basic interchangeable data blocks-resources. As a result, hospitals will require additional digital technologies and human resources to extract data from legacy formats and convert it into FHIR and USCDI-compliant parts.

No two-way communication between patients and EHRs:

The FHIR standard enables patients to access health data through apps of their choice. However, because EHRs only allow read-only access to their systems, this is a one-way street. The software allows users to request information but does not allow them to modify or change it. Many industry experts believe that the next major difficulty for healthcare is a lack of two-way communication between medical apps and EHR systems. And, sooner or later, it will need the development of new data standards ( 93).

7. Conclusion

Research in this fresh domain of health informatics is vital because we need to be aware of the necessary discipline of health informatics where new discoveries can be realized effectively and via rigorous testing. There are no proven design blueprints for such a comprehensive infrastructure, and the goal is always shifting due to the nature of real-time data collecting from a variety of sources, such as patient/medical/equipment data. The future of health informatics is to create novel algorithms, models, and an ecosystem that is conducive to health professionals and decision makers.

Some of the advantages that will result from innovation in this new emerging area will be critical, such as:

- Case/Incident related Health Information Standards and Services

- Professionalism among Healthcare Units

- Various Innovative Industrial Processes

- Enhanced National and International Collaboration for Research

- Novel Prediction Models (Machine Learning/Deep Learning/Reinforcement Learning)

- New Products and Services related to Healthcare/Health Informatics Domain

- Storage and Retrieval Mechanisms with Healthcare Related Data

- Curated New/Novel Datasets for Research Community.

Big data is also playing an important role in health informatics, where a large amount of data related to healthcare is generated, assisting and assisting doctors and decision makers in making rational decisions regarding patient treatment and diagnosis. Newer computational technologies may emerge that will improve the way healthcare is delivered and implemented in the future, thereby vastly improving the public healthcare system. Policymakers can propose tried-and-true use cases that can be transformed into a meaningful framework in which all stakeholders can deliberate and decide on the best model for improving the public health care system.

Health informatics is a new field with many stakeholders involved in the design and implementation of sustainable public health systems and policies for the benefit of society as a whole.

Author Contributions

MY has written the manuscript with relevant information from the literature regarding the major challenges and gaps that the present Health Informatics system and public health system are undergoing. The manuscript was reviewed and inputs were given by JK to focus on Health Informatics and policy framework. All authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher's Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

References

3. Pete S. Congressional intent for the HITECH Act. Am J Managed Care. (2010) 16(12 Suppl HIT):SP24-8. [[ PubMed] Google Scholar]

19. Karun AK, Chitharanjan K. A review on Hadoop-HDFS infrastructure extensions. In: IEEE Conference on Information and Communication Technologies. Thuckalay: (2013). p. 132-7. [ Google Scholar]

54. Hripcsak G, Duke JD, Shah NH, et al.. Observational Health Data Sciences and Informatics (OHDSI): opportunities for observational researchers. Stud Health Technol Inform. (2015) 216:574-8. [[ PubMed] PMC free article] [ Google Scholar]

60. Spencer M, Eickholt J, Cheng J. A deep learning network approach to [[ ab initio protein secondary structure prediction. IEEE/ACM Trans Comput Biol Bioinform. (2014) 12:103-12. 10.1109/TCBB.2014.2343960 PubMed] [ PMC free article] [ CrossRef] Google Scholar]

Articles from Frontiers in Public Health are provided here courtesy of Frontiers Media SA

# data[].attributes.attachment_link.title
Health Informatics: Engaging Modern Healthcare Units: A Brief Overview

# data[].attributes.attachment_link.description
In the current scenario, with a large amount of unstructured data, Health Informatics is gaining traction, allowing Healthcare Units to leverage and make meaningful insights for doctors and decision-makers with relevant information to scale operations ...

# data[].attributes.headline
Infographics And Public Policy: Using Data Visualization To Convey Complex Information

# data[].attributes.subject
Infographics And Public Policy: Using Data Visualization To Convey Complex Information

# data[].attributes.body
Described as "storygraphics", the authors of the article present how infographics are a powerful tool for conveying scientific information and are effective when made using specific characteristics. There are a few examples shown to help illustrate these concepts. Even though this particular article is not directly related to healthcare, acknowledging the authors' conclusion is specifically applicable; presenting important research findings to decision makers may be achieved with infographic use, based on the various advantages discussed.

# data[].attributes.headline
Helpful Visuals for Creating an Infographic

# data[].attributes.subject
Helpful Visuals for Creating an Infographic

# data[].attributes.body
Firstly, 5 steps for creating an infographic are outlined. More impactful are the infographic comparisons presented using slight modifications from the original. There is a section on types of graphs and their use. As seen from the examples, it was very informative to see how a little white space, choice of color scheme, use of icons or symbols, and incorporation of neutrals can really change the impact of an infographic.

# data[].attributes.attachment_link.title
How to Make an Infographic in Under 1 Hour (2023 Guide) - Venngage

# data[].attributes.attachment_link.description
Only have one hour to spare? No worries. Learn how to make an infographic in just under an hour with 5 easy steps, using Venngage.

# data[].attributes.body
There are two tables with real life examples of PICO to help us understand how to build a research question. Then, the linked YouTube video discusses how to make your research more refined by using the PICO format. A key takeaway from this link is the inclusion or exclusion of terms to narrow or broaden your database search. Sometimes people, myself included, completely change their topic based on overwhelming or insufficient search results. Using PICO to clarify the question could help yield new results pertaining specifically to the desired research topic.

# data[].attributes.attachment_link.readable_text
Skip to Main Content

It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

As a result of Children's Wisconsin's new security protocol, all users on the CW network will need to register for an OpenAthens account to access library resources (including UpToDate, VisualDx, etc.) via the library webpage. Click here to register for an OpenAthens account or view more information.

PICO

Before you start your search, it is important to have a well-built question. One way to construct a well-built question is to use the PICO model. PICO stands for patient/population, intervention, comparison and outcomes.

Patient/Population

Intervention

Comparison

Outcomes

Who is your patient?

Age, sex, race or patient

Primary problem

Health status

What do you plan on doing for the patient?

Diagnostic test

Medication

Procedure

What alternative are you considering?

Another test, medication or procedure

Watchful waiting

What do wish to accomplish?

Accurate diagnosis

Relieve or improve symptoms

Maintain function

Example: Is adherence to the Mediterranean Diet associated with reduced risk of heart attack?

Patient/Population

Intervention

Comparison

Outcomes

Adult

History of heart disease

Typical diet

No comparison

Reduction in heart attacks

When you go to search the databases, you may not use all these terms in your search. For instance, you may start out using only the terms Mediterranean Diet and heart attacks.

If you retrieve too many results, add more terms to your search, or add more specific terms.

If you retrieve too few results, use fewer or more broad terms.

More information on formulating a PICO question

MCW Libraries
8701 Watertown Plank Road
Milwaukee, WI 53226
(414) 955-8300

# data[].attributes.attachment_link.title
LibGuides: Evidence Based Medicine: PICO

# data[].attributes.attachment_link.description
Skip to Main Content It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

# data[].attributes.body
This infographic helps to show the difference between exploratory and explanatory analysis. Even though exploratory analysis is done in the preliminary stage, earlier, it is more time consuming.

# data[].attributes.attachment_caption
Open notes infographic about provider and patient opinions

# data[].attributes.body
Interestingly, even back in 2015 when this infographic was made, 88% of providers and 99% of patients thought open notes would a good idea. As we have all learned and witnessed as nurses, patient involvement and interest in their care has a lot of benefits. Open notes could prove beneficial in increasing patient participation.

# data[].attributes.body
The authors researchers how much human behavior impacted health care security breaches and the number of records during an actual cybersecurity attack. They claim a data breach should be recognized as "when", not "if". Their analysis revealed loss and exposure to EHR, many times, resulted from unintentional insider threats. This is something we all must keep in mind as future FNPs.

# data[].attributes.attachment_link.readable_text
Abstract

The healthcare sector continues to be the industry suffering one of the highest costs of a data security breach. Healthcare lags behind other industries in cybersecurity preparedness despite advances in cybersecurity technologies. Technical safeguards to protect electronic health records must be combined with human behavioral interventions to promote a robust cybersecurity plan. Using data from the United States Department of Health and Human Services, we conducted an exploratory analysis of past data breaches in healthcare organizations from January 2015 to December 2020 to explore the extent to which human elements played a role in data security incidents. We found that a vast majority of health records were compromised due to poor human security. The mean number of records affected by a breach due to unintentional insider threats is more than twice that of breaches caused by malicious intent such as external cyberattacks and theft. Our findings also indicate that, on average, more patient records are compromised from falling for a phishing scam than any other reason. We argue that proper cybersecurity contingency plans in healthcare must include human behavioral interventions that go beyond technical controls.

Keywords: cybersecurity, healthcare breaches, human factors, insider threats

Introduction

The digitization of medical records has changed the landscape of healthcare systems worldwide. With the advent of the information age, paper-based healthcare records were gradually and systematically converted into digitized electronic health records (EHRs). In the last two decades, the push toward resource sharing in technology is revolutionizing the healthcare sector by providing an efficient way of sharing patient records between healthcare professionals. Compared to paper-based records, EHRs require less manpower, time, and physical storage. Caregivers and providers use EHRs to access care-related activities and provide evidence-based decision support and quality care.1 However, the ease of access to EHRs is accompanied by rising cybersecurity threats and challenges.

In the annual "Cost of Data Breach" report conducted by the Ponemon Institute, the 2020 study noted that each compromised record cost an average of $146 to the healthcare organization (HCO). That figure increases to $150 per compromised record where personal health information (PHI) was involved. According to the report, healthcare continues to be the industry suffering the highest cost of a data breach at $7.13 million when factoring in other costs such as incident response, lost business, and notification costs. Eighty percent of the breached organizations participating in the study reported that PHI was involved. The cost of healthcare breaches is expected to increase during the COVID-19 pandemic, as 76 percent of HCOs in the survey predicted that implementing an incident response strategy will be made much more difficult by the ubiquity of remote work during the pandemic.2 Most healthcare executives lack overall information security, employee security awareness, and incident response strategies.3 Breaches related to EHR can significantly affect HCOs, such as the accidental release of PHI to disruptions in clinical care.4, 5, 6 Disruptions and delays in patient care can result in patient death, and the impact on patient safety is likely to be underreported.7

Federal compliance laws such as Health Insurance Portability and Accountability Act (HIPAA) and the Health Information Technology for Economic and Clinical Health (HITECH) Act were enacted to require the adoption of electronic medical records and protect privacy and data security of PHI.8 As required by section 13402 (e) (4) of the HITECH Act, The United States Department of Health and Human Services (HHS) Office for Civil Rights (OCR) must post a list of breaches of unsecured protected health information affecting 500 or more individuals. The HIPAA Security Rule requires healthcare organizations and covered entities protect electronic personal health information from cybersecurity threats.9 It also imposes administrative, technical, and physical standards for safeguards that organizations must implement. These entities must implement data security safeguards to protect PHI, such as medical records and insurance information. This paper presents an exploratory analysis of past EHR breaches in the United States from 2015 to 2020. By exploring the factors that led to violations, executives and decision-makers in HCOs can apply lessons from these breaches in securing their organizations.

Research Question

We investigated the extent to which the lack of proper human security enabled data breaches in HCOs. Our research questions were thus:

To what extent did the lack of human security result in data security breaches in healthcare records?

On average, how does the lack of human security affect the number of records breached in a cybersecurity incident in healthcare?

We hypothesized that data breaches in healthcare caused by unintentional human factors, such as carelessness, negligence, and falling victim to phishing and ransomware, outnumber those caused by malicious intent. The following sections discuss our observational study results and help research determine solutions for incorporating human security into organizational policies. We argue that any security framework must emphasize securing the human in HCOs.

Classification Method

We conducted an exploratory study on the factors that play a role in EHR-related cybersecurity breaches. HCOs are required by law to notify the OCR after a breach compromising EHRs and PHIs. The OCR publishes details of these reported breaches beginning October 2009 and makes the dataset publicly available.10 It includes reports from HCOs that have suffered breaches that compromised 500 or more EHRs. Since the law requires HCOs to notify HHS in the event of a violation, we believe that this nationwide sample is sufficiently representative of the population of EHR-related breaches in healthcare, with some limitations. Our analysis used a methodology based on the Joanna Briggs Institute (JBI) approach, which provides an evidence-based process for qualitative research.11 We modified a similar method by Walker-Roberts et al.12 in our classification process. shows a graphical representation of the flow process, which describes how we identified and screened each entry in our dataset for inclusion in the exploratory analysis. Our criteria for inclusions were that entries:

Included valid and clear descriptions of the breach incident.

Occurred between January 1, 2015, and December 31, 2020, inclusive

Constituted a breach as defined under HIPAA. Entries alleging violations by entities not covered under HIPAA, or those determined by OCR to have not violated any HIPAA rules, were excluded.

We limited our analysis of the OCR dataset to the years spanning January 2015 to December 2020; after removing incomplete records and other entries that do not constitute a breach of PHI, the resulting set of data contained 1,485 security incidents. The data was then analyzed to determine the type of the cybersecurity breach. Entries were categorized based on the presence or absence of malicious intent. In 15 cases, we could not establish those criteria based on the incident descriptions provided in the original dataset. These cases were noted with an "Insufficient information" designation. Once the type of breach has been identified, we further classified each entry into the primary source or cause of the breach for analysis. describes in detail our classification methods for each entry.

Table 1

Source of Breach

Criteria for Inclusion

Example Entries

Carelessness/Negligence a

Generally used terms such as "inadvertently" and "accidentally"

Exhibited the common trait of unintentional violation of established policies

Phishing and ransomware a

Explicitly stated phishing or ransomware as the primary cause of the breach

Technical improper
disclosure a

A result of coding impropriety, software bugs, and insecure configurations that led to PHI exposure

Generally described as occurring due to technical errors or misconfigurations

Cyber-attack or external
hacking attempts b

Generally used terms such as "victim of a cyber-attack" and "hacking attempt" by external threat actors towards the organization

Unlike phishing or ransomware, there were no clear enabling actions by employees, vendors and other internal actors

Malicious insider b

Described direct actions by an employee or business associate that facilitated a cyber breach for personal gains or the enrichment of another individual/entity

"... an employee emailed the protected health information (PHI) of 950 individuals to other members of a research study without authorization."

"... an employee ... was arrested for fraud and the CE subsequently determined that she impermissibly accessed and used at least one patient's protected health information ... "

"... an employee improperly disclosed protected health information (PHI) with the intent to sell it to an unauthorized third party."

Theft

Described incidents such as burglary and physical theft by external actors

"... the covered entity (CE), reported that a computer was stolen that contained 27,113 patients' files of a newly acquired medical practice ..."

"... unknown persons broke into the covered entity's (CE) legal and audits offices, ransacked the offices and paper files, vandalized property, and started a fire that set off the building's sprinkler system, which caused water damage to many documents and workstations."

"... the covered entity (CE), reported that documents containing the protected health information (PHI) of approximately 616 individuals was stolen during looting due to civil unrest ..."

Results and Discussion

We studied 1,485 breach events occurring between January 2015 and December 2020, affecting 141,252,797 medical records. Of that number, 73.1 percent of all affected records resulted from breaches caused by unintentional factors, while 26.7 percent were caused by malicious factors. shows the resulting classification of the sources of EHR breaches in HCOs and their frequency of occurrence in the United States from January 2015 to December 2020. We found the most frequent reason for a cyber breach in HCOs is the result of carelessness and negligence (382 incidents), followed by theft (222 incidents), and falling victim to a phishing scam (221 incidents). Similarly, shows the total number of records affected by each breach categories based off our classification method.

Table 2

Type of Breach

Cause of Breach

Number of Records Affected

Percentage of Total Records Affected

Unintentional

Phishing

93,248,376

66.02%

Total

103,196,622

73.06%

Malicious

Malicious insider

5,199,447

3.68%

Total

36,768,848

26.03%

Insufficient information

287,327

0.20%

Total number of records

141,252,797

100.00%

Carelessness/Negligence

Existing literature on insider threats generally assumes that individuals who commit cybersecurity transgressions do so due to an ulterior motive that is typically accompanied by malicious intent or the desire to enrich themselves for financial or personal gain.13 However, our dataset analysis revealed that 382 incidents, or 26 percent of all human factor-based breaches, were due to an insider's carelessness, negligence, or apathy. In each of these cases, no malicious intent was visible in that there was no intent to access patient data, but a data breach occurred. Employees or business associates may partake in risky cybersecurity behaviors due to a high risk tolerance or the desire to be efficient or helpful.14 In some cases, employees may inadvertently circumvent established policies because they view those policies as cumbersome or unrelated to patient outcomes.15 This paper does not intend to define a framework for what constitutes an insider threat but rather to show that carelessness and negligence stemming from risky behaviors, lack of awareness, and apathy are essential domains of human security.

Of the 382 incidents stemming from carelessness or negligence, 212 (55.5 percent) were incidents whereby an employee or business associate erroneously mailed or emailed PHI to the wrong recipients. Some were caused by misalignment in the printing process or information mismatched with patient data. In other cases, PHI may have been mailed to the correct recipients but done so in a manner that unintentionally exposed the PHI in transit.

Misplaced hard drives or documents containing PHI lost in the mail or transit were described in 71 incidents. In most of these cases, the covered entities never recovered the lost records. According to the dataset, some of these losses were attributed to carelessness on behalf of an employee. We note that we did not include cases where external individuals deliberately stole PHI during a burglary; these entries were classified under the "Theft" source of breach category due to the clear presence of malicious intent. These cases are therefore distinguished from incidents where PHI was lost due to negligence or carelessness. In a further 59 incidents, PHI was unintentionally and improperly exposed by individuals who uploaded the data onto publicly accessible websites or databases without taking security steps such as encrypting or sanitizing the data beforehand.

These accidental transgressions led to tangible consequences for both the offending employee(s) and the organization. According to the HHS dataset, consequences included penalties ranging from reprimand and retraining of the individual(s) to the suspension or termination of employment, depending on the severity and impact of the risky behavior. In addition, under HIPAA, organizations face substantial fines for noncompliance with the HIPAA Privacy Rule. In a February 2022 update, the OCR noted that since the compliance date of the Privacy Rule in April 2003, it has imposed civil penalties totaling $131 million to organizations for non-compliance.16

Phishing/Ransomware

Falling victim to a phishing scam made up most of the number of EHRs affected in our dataset. There were 221 incidents directly attributed to phishing scams, and 119 reported breaches were related to ransomware. Together, they make up 40.7 percent of all non-malicious events in our chosen time range. Our analysis combined phishing and ransomware incidents because a cybersecurity victim's vector to produce that outcome is similar. Phishing, which is the act of tricking a user into disclosing confidential information17 through a legitimate-looking email or link, is the vehicle that delivers the ransomware payload.18

In a phishing attack, one compromised credential can lead to multiple subsequent attacks, as we saw in Anthem Inc. in 2015,19 whereby a targeted spear-phishing campaign opened the door to further parts of its network. During the attack, 78,800,000 affected records were attributed to Anthem's breach incident as corroborated by the OCR dataset. Anthem had disclosed that it had suffered a data breach that affected almost 80 million customers.20, 21 Anthem discovered that the attackers had managed to obtain several employees' credentials, possibly through a phishing attack in their investigation. Once the attackers had obtained the credentials, they ran several data queries between December 2014 and January 2015. The database credentials would then be trivial to access using the stolen credentials.22 Eventually, the attackers could access Anthem's enterprise data warehouse containing personally identifiable information (PII) and stole almost 80 million unique user records.23

Although the Anthem incident may seem like a statistical outlier, we argue that, on the contrary, it further underscores the gravity of falling victim to a phishing attack. One phishing incident led to the most significant cybersecurity breach in the healthcare industry. Victims of phishing, ransomware, and other social engineering attacks become a new vector or vehicle to launch more in-depth and large-scale attacks.24 Once in the system, the attackers ran queries and worked from there to gain higher-level access. The Anthem administrator who found the breach noticed that his password was used to run queries that he did not initiate.25 The ability for an administrator's password to be used in this manner points to a possible flaw in Anthem's data management policy; actions executed by elevated privilege accounts should always be accompanied by some form of additional verification or authentication beyond a simple password requirement. While it cannot be said definitively that the presence of an additional authentication factor would have prevented the breach, it would have been an extra layer of defense against the attack.

Malicious Insider

Malicious insiders refer to individuals with knowledge or access to internal systems or networks, who then commit cybersecurity crimes with the express intent of enriching themselves for financial, personal, or other gains.26, 27 As we noted earlier in this paper, the allure of economic gains from PHI on the black market may drive individuals to commit cybercrime. However, malicious insiders may have motives other than profit, such as disgruntled employees attempting to exact revenge for a perceived wrong or a sense of entitlement. Cybersecurity controls typically are designed to thwart external attacks, and there are few, if any, technical controls that specifically defend against internal threats.28 Insiders have a crucial advantage: They are generally knowledgeable about systems and processes in the organization and may have varying administrative access levels that external actors do not.

To illustrate this, we found that in our analysis of the OCR dataset, there were 217 incidents of malicious insiders, affecting a total of 55,199,447 records. In as many as 170 of these cases, employee(s) of the HCO accessed PHI without a legitimate business need.

Other Sources of Breach

Under the "Unintentional" type category, the source of a breach in a total of 32 incidents was something other than the abovementioned categories. Since there were relatively few of these, we combined and classified these entries as "Other" in our analysis. In three incidents, the breaches were caused by employees falling for a social engineering attack. Social engineering, which is an umbrella term that includes phishing and ransomware, describes a process whereby an attacker uses social interaction to deceive and obtain sensitive information from a victim.29, 30, 31 For instance, an attacker may pose as an authorized individual and trick a user into divulging credentials to an internal network. In the OCR dataset, due to the prevalence of healthcare breach incidents caused by falling victim to phishing or ransomware attacks, we distinguished these categories from the more broadly applicable social engineering category. Ten incidents in the "Other" category stemmed from a lack of a business associate agreement between a covered entity and its business associate. Under the HIPAA Rules, a business associate agreement must be executed to ensure that any business entity that establishes a relationship with a covered HCO will commit to safeguarding PHI. Other incidents include breaches due to miscellaneous policy violations (eight occurrences), unintentional physical exposure of PHI (five occurrences), easily guessed passwords (four), and natural disasters (two).

Across all incidents, the OCR dataset shows that from 2015 to 2020, the mean number of records affected by unintentional factors is 123,446, more than twice that of the mean caused by malicious factors. shows the mean number of records affected when considering the type of breach. A closer look at the subfactors shows that phishing and cyberattacks led to the highest mean number of records affected at 421,038 and 153,644 records. breaks those categories down further into its subcategories based off our classification method and shows the mean number of records affected by each subcategory.

Conclusion

As healthcare services evolve in technology and coverage, they aim to provide a variety of treatments in order to accommodate diverse patient demographics. This was especially noticeable with the influx of patients impacted by the COVID-19 global pandemic. Due to the volatile and unpredictable nature of the virus, healthcare providers were forced to find alternative means of treatment in order to adequately provide necessary services to their patients. This included an increase of services, which entailed the usage of tools such as technology through cloud-based data inference, surveys, COVID-19 screening symptom checklists, and virtual appointment services. Such alternative means of seeking treatment were designed to minimize risk of exposure to patients, healthcare providers, and workers. With the increase in telehealth services, many healthcare workers began to perform remote work during the pandemic.32 The Federal Bureau of Investigation (FBI) have reported that cyberattacks have increased almost 400 percent by the first few months of the pandemic.33 In addition, the rise of telehealth means that many remote employees are now using their personal computers and home networks to perform their jobs. HCOs with essential workers working on-site are also grappling with the necessity of "bring your own device" (BYOD) policies to maintain patient care and outcome pre-pandemic. Working remotely means that HCOs have to deal with significant amounts of data being sent over the network off-premises in remote locations. In addition to expanding the attack surface for cyber criminals to take advantage of,34, 35 these developments and decentralized resources also increase the risk of accidental exposure of PHI as telehealth signals a necessary paradigm shift in providing patient care.

Data breaches in healthcare are incredibly lucrative as pathways to identity theft on the black market. According to Verizon's latest data breach report, published in May 2021, 85 percent of all breaches involved a human element, and during the COVID-19 global pandemic, phishing continued to be one of the most commonly employed methods in a data security incident across all industries. The report also indicated that ransomware has jumped to third place in terms of the most frequently occurring source of breaches. Similarly, the Healthcare Information and Management Systems Society (HIMSS) released a survey report stating that phishing is the most common attack vector in healthcare.36 This finding is consistent with our discovery with the OCR dataset. We showed that, on average, more EHRs are compromised to a phishing scam (mean of 421,938 records affected) than any other reason. We also noted that in the time range of our analysis, carelessness, negligence, and phishing were the most frequently occurring sources of EHR breaches.

The discourse on data breaches and EHR exposure has changed from "if" to "when" an HCO will experience a data breach. Based on our observational study of the OCR dataset, threats involving human elements continue to be significant risk factors for EHR breaches. An organization's ability to train and impart information awareness to its employees' behaviors is paramount in the fight against cybersecurity attacks on HCOs. A survey conducted in Germany by PricewaterhouseCoopers (PwC) revealed that a staggering 87 percent of participants believe that better education for medical staff is crucial to an HCO's cybersecurity hygiene.37 Phishing as a security incident is not new, yet the fact that it remains one of the most common occurrences of data breaches suggests that it may not be taken as seriously as it should be. Organizational data security policies may not receive widespread compliance in an HCO because employees may not perceive the risk of poor cybersecurity hygiene. An effective information awareness and training program must do more than simply transfer knowledge about proper behavior in cybersecurity. Incorporating behavioral science into training programs to change deeply rooted online habits is crucial in combating human-influenced breaches such as carelessness and phishing.38, 39, 40 Technical safeguards should not be the only avenue to accomplish this goal; rather, it needs to be bolstered by the cyber vigilance of human elements.41 There is no one holy grail of countermeasures sufficient to prevent human risks that lead to cyberattacks. Each HCO must conduct its own risk assessment that accounts for resource constraints and the feasibility of such methods.42

Our analysis of the data breaches in healthcare as reported to HHS has identified several contributing factors. As we have observed, many of the cases we analyzed involved unintentional insider threats, and these cases lead to significant loss and exposure of EHR. This analysis was informative in specifying directions for future research and areas to focus on in mitigating cyber-attacks.

Author Biographies

Liu Hua Yeo (ude.hcime@oeyl) is a PhD candidate in cybersecurity at Eastern Michigan University. His research interests include risk factors in healthcare cybersecurity.

James Banfield (ude.hcime@dleifnabj) is an associate professor of cybersecurity at the School of Information Security and Applied Computing at Eastern Michigan University.

Notes

5. Jalali MS, Russell B, Razak S, Gordon WJ. "EARS to cyber incidents in health care." J Am Med Inform Assoc. 2019;26((1)):81-90. doi:10.1093/jamia/ocy148. [[ PMC free article] [ PubMed] Google Scholar]

9. Kosseff J. Cybersecurity Law. 2nd ed. John Wiley & Sons, Ltd; 2020. Cybersecurity Requirements for Specific Industries." In: [ Google Scholar]

13. Hadlington L. In: Psychological and Behavioral Examinations in Cyber Security. 1st ed. 201. The "Human Factor" In Cybersecurity: Exploring the Accidental Insider." pp. 46-63. IGI Global. [ Google Scholar]

Articles from Perspectives in Health Information Management are provided here courtesy of American Health Information Management Association

# data[].attributes.attachment_link.title
Human Factors in Electronic Health Records Cybersecurity Breach: An Exploratory Analysis

# data[].attributes.attachment_link.description
The healthcare sector continues to be the industry suffering one of the highest costs of a data security breach. Healthcare lags behind other industries in cybersecurity preparedness despite advances in cybersecurity technologies. Technical safeguards to protect electronic health records must be combined with human behavioral interventions to promote a robust cybersecurity plan.

# data[].attributes.attachment_caption
A layered approach to achieving a trustworthy health system environment

# data[].attributes.body
As depicted, there are seven layers of protection in this model and each is dependent upon layers below it. For example, layer 7 is dependent upon layers 1-6. Baker, D. (2021) stated, "All of the layers must work together to provide a trustworthy HIT environment for healthcare delivery."

Baker, D. (2021). Trustworthy Systems for Safe and Private Healthcare. Retrieved on March 30, 2023, from https://nursekey.com/trustworthy-systems-for-safe-and-private-healthcare/

# data[].attributes.headline
Systematic Review of Patient Portal Impacts

# data[].attributes.subject
Systematic Review of Patient Portal Impacts

# data[].attributes.body
The researchers determined patient portals can enhance the relationship between provider and patient, increase therapy adherence, improve awareness about health, and seem to favor health outcomes (Carini et al., 2021).

Carini E, Villani L, Pezzullo AM, Gentili A, Barbara A, Ricciardi W, Boccia S. The Impact of Digital Patient Portals on Health Outcomes, System Efficiency, and Patient Attitudes: Updated Systematic Literature Review. J Med Internet Res. 2021 Sep 8;23(9):e26189. doi: 10.2196/26189. PMID: 34494966; PMCID: PMC8459217.

# data[].attributes.attachment_link.readable_text
Monitoring Editor: Rita Kukafka

Reviewed by Sergio Pillon, Nancy Gordon, Iseult Wilson, and Feroze Kaliyadan

Elettra Carini, MD,#1 Leonardo Villani, MD,#1 Angelo Maria Pezzullo, MD, MSc, 1 Andrea Gentili, MD, 1 Andrea Barbara, MD, 2, 3 Walter Ricciardi, MD, MPH, 1 and Stefania Boccia, MSc, PhD 1, 4

Abstract

Background

Patient portals are becoming increasingly popular worldwide even though their impact on individual health and health system efficiency is still unclear.

Objective

The aim of this systematic review was to summarize evidence on the impact of patient portals on health outcomes and health care efficiency, and to examine user characteristics, attitudes, and satisfaction.

Methods

We searched the PubMed and Web of Science databases for articles published from January 1, 2013, to October 31, 2019. Eligible studies were primary studies reporting on the impact of patient portal adoption in relation to health outcomes, health care efficiency, and patient attitudes and satisfaction. We excluded studies where portals were not accessible for patients and pilot studies, with the exception of articles evaluating patient attitudes.

Results

Overall, 3456 records were screened, and 47 articles were included. Among them, 11 studies addressed health outcomes reporting positive results, such as better monitoring of health status, improved patient-doctor interaction, and improved quality of care. Fifteen studies evaluated the impact of digital patient portals on the utilization of health services with mixed results. Patient characteristics were described in 32 studies, and it was reported that the utilization rate usually increases with age and female gender. Finally, 30 studies described attitudes and defined the main barriers (concerns about privacy and data security, and lack of time) and facilitators (access to clinical data and laboratory results) to the use of a portal.

Conclusions

Evidence regarding health outcomes is generally favorable, and patient portals have the potential to enhance the doctor-patient relationship, improve health status awareness, and increase adherence to therapy. It is still unclear whether the use of patient portals improves health service utilization and efficiency.

Keywords: digital health, patient portal, health outcomes, patient satisfaction, patient attitudes, health service research, health care efficiency

Introduction

In recent years, electronic tools that allow patients to interact with health care professionals have considerably increased with consequences on the awareness of citizens about their own health [ 1]. The adoption of these technologies might represent an important measure to improve the quality and efficiency of health care services and is a key feature for the construction of a more equitable, effective, and safe health care system [ 2]. Indeed, the rapid growth and diffusion of digital health, including health information sources, such as electronic medical records (EMRs), has made online access to information by patients and health care professionals a crucial component of health care delivery [ 3].

In this context, patient portals are thought to allow patients secure access to health-related information and to communicate and share information with providers [ 4]. Besides guaranteeing protected access to EMRs, more advanced patient portals allow secure message exchange between health professionals and citizens, consultation of educational material adapted to patients' own characteristics, appointment scheduling, automatic renewal of medical prescriptions for chronic diseases, and facilitation of payments. Despite their potential benefits, several studies have proved underuse or inappropriate use of patient portals and their limited impact [ 5]. Furthermore, the majority of studies available on this topic have focused on users' characteristics and satisfaction, and few studies have considered the consequences on health outcomes [ 6- 8]. Patient portals are relatively new technologies with continuous updates. Several types are released every year, and this may explain the lack of research in this area [ 5].

A systematic literature review in 2013, which addressed the effect of patient portals on patient clinical care, reported that evidence was limited to evaluate whether patient portals had a positive, negative, or neutral impact on users' health [ 4]. Some of the most effective examples refer to patients with chronic diseases, such as diabetes, hypertension, and depression, specifically when the portal is included in a shared health care pathway [ 9- 13]. The effect of patient portals on health care utilization and efficiency, instead, is unclear due to the scarcity of studies examining the impact of patient portals on key indicators, such as inpatient hospitalizations, emergency department (ED) and outpatient visits, length of stay, and telephone contacts [ 14]. The aim of this systematic review was to update the study performed in 2013, by summarizing evidence on the impact of digital patient portals on patients' health outcomes, health care efficiency, and patients' attitudes and satisfaction.

Methods

Search Strategy

A search of relevant articles was performed in the PubMed and Web of Science databases using the query reported in Multimedia Appendix 1. The resulting records were entered in a dedicated work sheet to be subsequently screened according to the inclusion/exclusion criteria. Following the removal of duplicates, the selection was made by reading titles and abstracts, and then the full texts.

Inclusion/Exclusion Criteria

Eligible studies were primary studies reporting on the impact of patient portal adoption in relation with health outcomes, health care efficiency, and patients' attitudes and satisfaction. Articles included were published from January 1, 2013, to October 31, 2019, and written in English, Italian, Spanish, or French. We excluded studies describing portals that were not accessible for patients, as well as pilot studies, with the exception of studies evaluating patients' attitudes.

Selection Process and Data Extraction

Two authors screened the articles, and each reference retrieved was screened by two researchers independently, with any disagreement finally discussed and resolved by a third researcher, if necessary. The following information was extracted from the studies: first author name, publication year, study country, study design, study population, study setting, study duration and time, health information technology, study objective, main findings according to health outcomes, health care efficiency/utilization, patient characteristics, and patient attitudes and satisfaction. The systematic literature review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) 2009 checklist [ 15].

Results

Characteristics of the Included Studies

The database search, after duplicate removal, identified a total of 3456 records. According to the inclusion/exclusion criteria, the screening resulted in the inclusion of 47 full-text articles ( Figure 1 ).

Various patient portals have been described in the studies. Multimedia Appendix 2 provides details on the functionalities of the portals, and Multimedia Appendix 3 provides qualitative descriptions of the portals.

The population included in the studies was heterogeneous in terms of sample size (from 24 [ 52] to 2,171,325 patients [ 31]) and groups of included patients (eg, pediatric [ 32], older [ 31], oncology [ 52], and diabetic patients [ 60]).

The results were summarized in four categories, albeit the same study could belong to more than one category. In particular, 11 studies analyzed health outcomes and adherence, intended as a change in individual or population health, attributable to health-related interventions. Adherence is the degree to which a patient follows medical advice, especially drug compliance. Overall, 15 studies focused on health care efficiency/utilization (utilization of health care services), 32 studies referred to patient characteristics, and 30 studies analyzed attitudes and satisfaction.

Health Outcomes and Adherence

Eleven articles presented results on patient outcomes ( Table 1 ), such as prevention, diabetes, blood pressure control, asthma, mental health, and medication adherence.

Table 1

First author, year

Country

Health information technology

Study design

Sample size

Main findings

Ancker, 2019 [ 34]

United States

Blood glucose flow sheet (EpicCare and Weill Cornell Connect portal)

Observational, hypothesis testing

53 patients

Pregnant: average BMI dropped while average blood pressure increased significantly more in the 9 months among uploaders than nonuploaders.
Chronic disease patients: after 9 months, uploaders had significantly larger reductions in HbA 1c and BMI than nonuploaders. One subset of uploaders had low well-controlled HbA 1c values before and during PGHD a upload. Another uploader subset began to upload when their HbA 1c levels were elevated and experienced a decrease in HbA 1c levels followed by a plateau.

Fiks, 2015 [ 60]

United States

MyAsthma (clinical interface in MyChart)

RCT b

60 families of children

No significant differences in baseline control, quality of life, or parent activation between the two study arms ( P>.2 for all comparisons). Frequency of asthma flares improved in the intervention group over time by 2.0 points on a 25-point scale ( P=.02). Families in the intervention group had a marginally significant improvement in symptoms during periods without flares. A nonstatistically significant improvement in quality of life in terms of daytime symptoms and functional limitations was observed in the intervention group. There were no significant changes in parent activation.

Huang, 2019 [ 45]

United States

MyPennMedicine (branded version of Epic MyChart)

Observational, hypothesis testing

10,000 patients

Patients' preventive health behaviors were significantly associated with portal use. The proportions of annual flu vaccination, blood pressure checks, and lipid level screening were substantially higher in portal users compared with nonusers (OR c=1.58, 1.13, and 1.50, respectively; P<.001). The average composite prevention score was significantly higher among portal users compared with nonusers (mean difference=0.22; P<.001). The proportion of colorectal cancer screening between users and nonusers was statistically significant ( P<.001, OR very close to 1). No clinically meaningful difference between patient portal users and nonusers in chronic health outcomes.

Jhamb, 2015 [ 46]

United States

Free patient portal tethered to an ambulatory EHR d

Observational, hypothesis testing

1098 patients

In the fully adjusted model (controlling for hyperlipidemia, nephrolithiasis, history of kidney transplant, CCI e, proteinuria, eGFR f, number of nephrology and outpatient visits, and university affiliated PCP g), the association was not significant (OR 1.11, 95% CI 0.99-1.24).

Kipping, 2016 [ 35]

Canada

Ontario Shores HealthCheck Patient Portal

Observational, hypothesis testing

91 patients

The overall Mental Health Recovery Measure score increased from 70.4 (SD 23.6) at baseline to 81.7 (SD 25.1) at follow-up ( P=.01). Of the eight domains, seven increased from baseline to follow-up (overcoming stuckness, self-empowerment, basic functioning, overall well-being, new potentials, spirituality, and advocacy/enrichment; all P<.05. No change for learning and self-redefinition).

Lau, 2014 [ 36]

Canada

BCDiabetes.ca

Observational, hypothesis testing

1957 patients

Overall, 28 of 50 users had a follow-up HbA 1c ≤7%, whereas 22 of 50 did not (56% success rate).
Only 16 of 50 nonusers achieved a follow-up HbA 1c ≤7%, while 34 of 50 did not (32% success rate).
Users were significantly more likely to control their HbA 1c levels successfully than nonusers (McNemar test, P=.03). The HbA 1c level at the last follow-up was significantly lower for users compared to nonusers ( P=.02).

Manard, 2016 [ 37]

United States

Online patient portal

Observational, hypothesis testing

1571 patients

After adjusting for age, users were more likely to achieve BP h control (HR i 1.24, 95% CI 1.06-1.45). After adjustment for sociodemographics, portal use was no longer associated with BP control (HR 0.98, 95% CI 0.83-1.16).

In particular, a significant association was found between patients' preventive health behaviors and portal use [ 45]. Considering diabetes [ 34, 36] and blood pressure control [ 37], portal users were significantly more likely to control their HbA 1c levels successfully compared to nonusers.

An improved clinical condition over time was observed in the management of asthma in children, where the group using the patient portal reported better flare control compared to the control group [ 60].

Concerning mental health, patients using the portal had a positive impact related to the clinical condition in many domains [ 35], such as the reduction of drinking days [ 54]. However, concerning the health status, no marked short-term impact was described, and differences in patient-reported physical and mental health changes were minor [ 55].

Finally, higher medication adherence was described in portal users compared with nonusers, especially among pediatric patients with asthma and patients with rheumatic disorders [ 32, 57].

Efficiency/Utilization

Fifteen articles described the relationship between portal use and health care service efficiency and utilization ( Table 2 ). The use of a digital portal had an effect on the utilization of health care services in terms of the number of clinical visits, especially for asthmatic patients [ 32, 60], while no statistically significant changes in the number of primary care visits was reported in association with the use of secure messaging [ 38].

Table 2

First author, year

Country

Health information technology

Study design

Sample size

Main findings

Fiks, 2015 [ 60]

United States

MyAsthma (clinical interface in MyChart)

RCT a

60 families of children

The intervention group had a marginally significant reduction in the proportion of parents missing at least 1 day of work (reduction of 47%, P=.07). Families in the intervention group reported fewer ED b visits and hospitalizations for asthma over 6 months than the control group (3 vs 9 and 0 vs 2, respectively). Only two intervention families reported at least one ED visit (vs six control families), and no intervention families reported hospitalizations. Children in the intervention group had fewer visits with asthma specialists or primary care. Results were similar on stratifying by asthma severity.

Foster, 2019 [ 43]

United States

Epic MyChart

Observational, hypothesis testing

208,635 tests

ED visits: 80.56% (n=20,430) of patients had a single ED visit with laboratory testing, 16.04% (n=4069) had two or three ED visits, 3.16% (n=802) had four to 10 ED visits, and only 0.24% (n=60) had more than 10 ED visits. Activation rates were lower for those with only a single ED visit (7312/20,430, 35.79%) compared with either those with two to three ED visits (1770/4069, 43.50%; P<.001) or four or more ED visits (368/862, 42.7%; P<.001).

North, 2014 [ 38]

United States

Mayo Clinic Health System

Observational, hypothesis testing

2357 primary care patients

Primary care patients who sent at least one secure message or e-visit had a mean of 2.43 (SD 2.3) annual face-to-face visits before the first message and 2.47 (SD 2.8) after, with a nonsignificant difference ( P=.45). After adjustment for a first message surge in visits, no significant visit frequency differences were observed (mean, 2.35 annual visits per patient both before and after the first message; P=.93). Subgroup analysis showed no significant change in visit frequency for patients with higher message utilization or for those who had used the messaging feature longer.

Plate, 2019 [ 39]

United States

MyChart; Epic Systems Corporation

Observational, hypothesis testing

6426 patients

Active MyChart status was not associated with 90-day ED return ( P=.78) or readmission ( P=.51) based on univariable analysis. Similarly, during multivariable analysis controlling for age, gender, BMI, and ASA c category, active MyChart utilization was not significantly associated with 90-day ED visits (OR d 1.019, 95% CI 0.843-1.231; P=.85) or readmissions (OR 0.966, 95% CI 0.747-1.249; P=.79). Patients who sent secure messages within 90 days from surgery (2200 patients, 48% of active users) were not less likely to present to the ED ( P=.63) or be readmitted ( P=.59) within 90 days. For patients who sent two or more messages (1354 patients), provider or staff response rate <75% was significantly associated with 90-day readmission ( P=.004) with greater 90-day ED visits that neared statistical significance ( P=.07).

Riippa, 2015 [ 55]

Finland

Patient portal by The Finnish Medical Society, Duodecim

Interventional, other than RCT

876 patients

The effect on the cost of care was ambiguous; costs decreased by an average of €91 in the unadjusted model, but increased by €48 in the adjusted model. Due to the controversial result, the unadjusted analysis showed an 89% probability of cost-effectiveness with no willingness to pay for increased patient activation, whereas in the adjusted sample, the probability of the portal being more cost-effective than care as usual exceeded 50% at a willingness to pay €700 per clinically significant increase in the patient activation score. For doctor visits, portal access (n=80): 3.8 (SD 3.3) and control (n=57): 3.0 (SD 3.1) ( t=1.4; P=.18). For nurse visits, portal access (n=80): 3.5 (SD 2.6) and control (n=57): 4.1 (SD 2.5) ( t=−1.3; P=.18).

Tsai, 2019 [ 28]

United States

Epic's personal health record system

Descriptive, quantitative

109,200 patients

Active users had more outpatient and inpatient visits and fewer ER visits. Patients without a portal account had on average fewer outpatient visits per month (0.31 vs 0.89, P<.001) and fewer inpatient visits per month (0.007 vs 0.059, P<.001), but had more ER visits per month than patients who were active with the portal (0.047 vs 0.014, P<.001). The difference between no-show appointments was not significant.

Zhong, 2018 [ 42]

United States

MyUFHealth (also known as MyChart by Epic)

Observational, hypothesis testing

15,659 nonusers and 5494 users

At the time of adoption, the quarterly PCP f office visit RR g of users to nonusers was 1.33 (95% CI 1.27-1.39; P<.001). The RRs were between 0.94 and 0.99 up to four quarters after portal adoption ( P=.75, .10, .13, and .09, respectively), and it was significantly less than 1 at the seventh (RR 0.82, 95% CI 0.73-0.91; P<.001) and eighth (RR 0.80, 95% CI 0.70-0.90; P<.001) quarters post adoption. The no-show rate proxies in the user group were significantly lower than in the nonuser group. RRs were between 0.60 and 0.83 for eight out of 11 quarters, and for the remaining three quarters, differences were not significant ( P=.65, .29, and .44, respectively). Differences in cancellation rate proxies were not significant ( P>.05). Overall, appointment adherence improved after portal adoption.

Concerning emergency room (ER) visits, a decreasing trend was described [ 54, 60], and active portal users showed more outpatient and inpatient visits and fewer ER visits per month, compared with patients without an account [ 28]. The number of visits over 6 months for asthmatic patients was lower among users than among nonusers (3 vs 9) [ 60]. User status was not significantly associated with ED access in the study by Plate et al, and the use of secure messages showed a nonsignificant result [ 39].

A reduction in hospitalizations was described for asthma [ 60] and mental health conditions [ 54]. Different findings were observed in patients with acute myocardial infarction, congestive heart failure, or pneumonia. In these cases, the odds of 30-day readmission for active users was 66% higher than that for nonusers, while no significant difference was described between nonusers and light users [ 44]. Considering patients who underwent total hip or knee arthroplasty, there was a significant association between 90-day readmission and sending two or more secure messages [ 39].

Concerning clinicians' attitudes, the portal seemed to enable a new way of managing stable patients, facilitating clinical and cost-effective use of specialist nurses (improved two-way communication, and more optimal use of outpatient appointments and consultant time). The portal also facilitated a single rationalized pathway for stable patients, enabling access to information and proactive support [ 48].

Portal use by patients reduced missing appointments [ 35] and showed an improvement in appointment adherence after portal adoption [ 42]. However, no significant association between user/nonuser status and no-show appointments was found by one study [ 28]. The difference in cancellation rate proxies was also not significant between the user and nonuser groups [ 42].

Finally, concerning other aspects, only one study took into consideration the associated costs with ambiguous results. In this study, costs decreased in the unadjusted model but increased after adjusting for relevant variables [ 55]. An increase in HIV screening rates was described, but changes in the rates of HIV risk behaviors (eg, condom use) and modification of addiction treatments appeared nonsignificant in mental health patients [ 54]. Portal use had a positive effect on days of work lost due to asthma patients' issues [ 60]. Moreover, information services were positively affected by portal use, as it led to a reduced number of requests [ 35].

Patient Characteristics

Concerning patients' characteristics, 32 articles presented relevant results that were mainly related to demographic information, such as age, gender, education, and household status ( Table 3 ).

Table 3

First author, year

Country

Health information technology

Study design

Sample size

Main findings

Ancker, 2019 [ 34]

United States

Blood glucose flow sheet (EpicCare and Weill Cornell Connect portal)

Observational, hypothesis testing

53 patients

Pregnant patients: Uploaders were similar to the comparison group in terms of race, ethnicity, age, and socioeconomic status. Uploaders had more clinical encounters and portal logins before initial data upload, earlier establishment of patient portal accounts, and worse baseline blood pressure.
30 chronic disease patients: Uploaders were more likely to be Asian-American and younger, but the groups did not have other significant demographic differences.

Fiks, 2015 [ 60]

United States

MyAsthma (Clinical interface in MyChart)

RCT b

60 families of children

Parents of children with moderate or severe asthma used the portal more frequently (75% were frequent users vs 47% were parents whose child had mild persistent asthma).

Fiks, 2016 [ 32]

United States

MyAsthma

Descriptive, mixed methods

237 families

Users were more likely to have children aged 6-9 years ( P=.009), be White ( P<.001), be privately insured ( P<.001), have mild persistent or moderate or severe persistent asthma ( P=.002), be on an asthma controller medication ( P<.001), and be receiving a greater number of asthma medications at baseline on average than nonusers ( P<.001). Those with persistent asthma were twice as likely to use the portal versus those with intermittent asthma (2.37% vs 1.25% at hospital practices where these data were available; P<.001). Sustained portal users were more likely than one-time users to be Hispanic ( P=.02), have private insurance ( P=.02), and be from the Northeast ( P=.001). Sustained use parents had higher educational levels ( P=.002).
Positively associated with portal adoption in multivariable logistic regression: receipt of a controller medication at baseline (OR 2.0, 95% CI 1.5-2.7), private insurance (OR 2.0, 95% CI 1.3-3.1), lower child age (OR 1.4, 95% CI 1.1-1.9), and greater asthma severity (OR 1.9, 95% CI 1.2-3.0 for mild and OR 1.9, 95% CI 1.0-3.5 for moderate or severe persistent vs intermittent).

Foster, 2019 [ 43]

United States

Epic MyChart

Observational, hypothesis testing

208,635 tests

Females (5546/13,149, 42.18%) were significantly more likely to activate the portal than males (3897/12,212, 31.91%; P<.001). Activation rates were highest for Asian (262/451, 58.1%) and White individuals (8155/20,637, 39.52%) and lower for African American/Black (491/2254, 21.78%; P<.001 compared with White), Hispanic/Latino (333/1257, 26.49%; P<.001 compared with White), and other individuals (241/762, 31.6%; P<.001 compared with White). The activation rate for patients aged 18-70 years was 41.61% (7593/18,246). The overall pattern of radiologic image viewing with respect to age and gender showed similar trends to those described for laboratory testing.

Huang, 2019 [ 45]

United States

myPennMedicine (branded version of Epic MyChart)

Observational, hypothesis testing

10,000 patients

Users were more likely to be younger (63.46 years [users] vs 66.08 years [nonusers]; P<.001) and have higher income (US $74,172 [users] vs US $62,940 [nonusers]; P<.001) than nonusers. The percentage of White race was substantially higher among users (72.77% [4317/5932] [users] vs 52.58% [2139/4068] [nonusers]; P<.001). For users, the percentage of payments by commercial insurance was higher (60.99% [3618/5932] [users] vs 40.12% [1632/4068] [nonusers]; P<.001) and the percentage of payments by Medicare or Medicaid was lower (Medicare: 34.91% [2071/5932] [users] vs 48.72% [1982/4068] [nonusers]; P<.001; and Medicaid: 3.49% [207/5932] [users] vs 10.08% [410/4068] [nonusers]; P<.001). The difference in sex between users and nonusers was not statistically significant. No significant difference was found in any provider-level characteristic between the two groups.

Jhamb, 2015 [ 46]

United States

Free patient portal tethered to an ambulatory EHR c

Observational, hypothesis testing

1098 patients

Users were younger and more likely to be non-Black, be married, have private insurance, and have higher neighborhood median household income. Users were less likely to have diabetes, coronary artery disease, or congestive heart failure, but were more likely to have had a kidney transplant. Older age, Black race, unmarried status, Medicaid or Medicare insurance (vs private), and lower neighborhood median household income were associated with not using the portal.

Lau, 2014 [ 36]

Canada

BCDiabetes.ca

Observational, hypothesis testing

1957 patients

Users tended to be younger (mean difference of 4.28 years; P=.06), have lower baseline HbA 1c (mean difference of 0.89%; P<.01), and have higher baseline weight (mean difference of 7.53 kg; P=.06) than nonusers. There was no difference in gender or total follow-up time. Follow-up HbA 1c levels tended to be lower in users than nonusers (mean difference of 0.75%; P<.01), and users were significantly more likely to have HbA 1c of 7% at their last follow-up visit ( P=.01). No significant differences in LDL e and SBP f were observed between users and nonusers at initial visits and follow-up visits.

Manard, 2016 [ 37]

United States

Online patient portal

Observational, hypothesis testing

1571 patients

Users were significantly younger ( P<.001), more often White ( P<.001), and more often married ( P<.001) than nonusers. Users were significantly from upper-middle to the highest socioeconomic status compared with nonusers ( P<.001). Portal use was more common among FM g patients than GIM h patients ( P<.001), and users were more often high health care utilizers ( P<.02). Portal use was less common among current smokers ( P<.001). Users were more likely to have depression ( P<.01) and lower comorbidity scores ( P<.001).

Sun, 2019 [ 41]

United States

Epic's personal health record system

Observational, hypothesis testing

38,399 patients

Almost one-third of patients (n=12,615; 32.9%, 95% CI 32.38%-33.32%) had used the portal for a mean of 2.5 (SD 1.9) years prior to the study period. Portal use was higher on weekdays ( P<.001). An increase in portal use was observed in response to email reminders. A nonlinear relationship between age and portal use was observed and depended on several other predictors ( P<.05). Patients living in more rural areas with low income were at lower odds to use the portal ( P=.02), and this finding also applied to non‐Whites with low income ( P<.001). More chronic conditions and a higher initial HbA 1c value were associated with portal use ( P=.01).

Tsai, 2019 [ 28]

United States

Epic's personal health record system

Descriptive, quantitative

109,200 patients

Active portal users were on average older (49.45 vs 46.22 years) and frequently female (62.59% vs 54.91%). Both the differences in mean age ( P=.008) and gender ( P=.04) were significant. There was a bimodal peak in terms of active users, with active users more likely to be in their 30s and 60s. The difference among age groups was significant ( P<.001). Differences in racial composition, insurance, and language were not significant.

In terms of age, results were not homogeneous. Higher use in older patients (aged >50 years) was described in four studies [ 18, 20, 21, 33]. In a middle age and elderly cohort, the mean age of users was significantly lower [ 45]. On the other hand, younger patients were also the major users in disease/specialty-specific cohorts [ 32, 35, 46].

Most of the articles found a positive association between female gender and portal use [ 28, 35, 38, 42, 43, 58]. Only one study, conversely, found male gender to be a predictor of registering [ 27]. No statistically significant association between sex and user/nonuser status was found in two articles [ 36, 45].

In a cohort of patients with mental disorders, having schizophrenia or schizoaffective disorders was negatively associated with portal use [ 31]. Instead, depression was positively associated [ 37]. Moderate or severe asthma was more linked to portal use [ 32, 60]. Moreover, having a diagnosis of diabetes, hypertension, heart failure, or cardiovascular disease was a significant predictor of portal use [ 20, 40], with one exception [ 46]. On the other hand, there were contrasting results concerning the association between the number of comorbidities and portal use [ 27, 37, 41, 42, 44].

Studies conducted in the United States showed that having a private insurance was positively associated with portal use [ 18, 32, 39, 42, 45, 46, 58], with only one study reporting the absence of this association [ 28].

Other patient characteristics positively associated with portal use were being a member of a patient association [ 59] and being admitted to an academic medical center [ 44].

Attitudes and Satisfaction

Patient attitudes were evaluated in terms of perceived barriers and facilitators toward portal use. The overall satisfaction was also assessed, and it refers to the extent to which the patient is content about health care. Thirty articles addressed these topics ( Multimedia Appendix 4).

Discussion

Due to the considerable amount of literature published on the topic of patient portals, the aim of this review was to provide evidence and to gather information systematically. Similar to the review published in 2013 by Goldzweig et al [ 4], the outcomes were grouped into four aspects, namely, health outcomes and adherence, health care efficiency, patient characteristics, and attitudes and satisfaction, and showed nonunique results in terms of benefits brought by patient portals concerning patient experience and health. The functionalities available in the portals described by Goldzweig et al and our review were likewise the same.

Positive results were described relating to the enhancement of preventive behaviors [ 45], changes in chronic condition control with higher control of diabetes parameters [ 34, 36], and asthma flares [ 60]. However, conflicting results were described concerning blood pressure control [ 34, 37], mental health conditions [ 35, 54, 55], and medication adherence [ 32, 57]. A possible explanation of these results is that these particular studies involved patients (often with a low sample size) who utilized portals for a short period of time, preventing them from having a possible consistent outcome. Similarly, concerning health outcomes, Goldzweig et al found generally positive results, which documented improvement in patients' disease control and maintenance [ 4], even though many of the included studies in our review did not find a significant difference between portal users and nonusers. Adherence to therapy was always improved [ 4] similar to that in this review.

The number of clinical and ED visits [ 28, 32, 39, 54, 60] and hospital readmissions [ 39, 44, 54, 60] did not always decline in patients using portals. The number of missed appointments decreased [ 35, 42], but this finding was not always significant [ 28, 42]. The nondeclining trends of hospitalizations and outpatient visits as a consequence of portal use are concordant in the two studies, although different findings were described. Indeed, different from our review, in which we found more often a decrease in the utilization of in-person services, the review by Goldzweig et al reported that most of the studies found an increased number of outpatient visits and hospitalizations. As patient portals are normally designed to reduce inappropriate health care utilization, this might be partially explained by the fact that provider and patient adaptations to the patient portals have evolved over time.

The studies differed also in terms of the population included, with conflicting results in portal adoption and age, gender, ethnicity, kind of disease, and number of comorbidities. Generally, having a higher income [ 31, 37, 45, 46, 58] and having a private insurance were associated with increased portal use [ 18, 32, 39, 42, 45, 46, 58].

The main barriers faced in using the portals were (1) user-related issues, such as time constraints [ 57, 61], disinterest in managing the disease [ 26, 57], and limited digital knowledge [ 26, 51]; (2) clinician-related issues, such as age [ 33] and attitude toward the portal [ 33]; and (3) technology-related issues, such as limited internet access [ 25, 26, 51, 57] or ability to use technology [ 25, 26, 57] and security concerns [ 25, 48]. Privacy and security concerns, and ability to use technological appliances and systems were the most important barriers to utilization described in the studies retrieved by Goldzweig et al [ 4]. Indeed, other recent reviews on patient attitudes highlighted that privacy and security problems are the main barriers to the use of patient portals. These barriers are as evident in elderly patients (in whom there is also a limitation of portal use related to age) as in younger patients [ 62, 63]. Moreover, technical problems due to patient capacity and difficulties in using the portal also represent important barriers. Thus, correct and adequate information on safety issues and education on the technical use of the portal represent the best facilitators. Furthermore, engaging patients and making them realize that the portal represents a useful tool to support the management of their pathology (especially for chronic diseases) without replacing the doctor-patient relationship are important to encourage the use of digital portals [ 13, 62, 64]. In addition, technical improvements in the usability of portals could increase patient enrollment.

Security problems are complex issues that must be considered in any part of medical care. The use of information technologies in health care that can be accessed by multiple types of users (physician, patient, caregiver, and hospital administrative staff) represents the basis of the discussion about computer security [ 65]. Indeed, uncertainties about security of clinical data might hinder adoption of systems by both hospitals and patients [ 48]. Patients expressed concern about their privacy and the privacy of their family members, and asked for further information about confidentiality, as vulnerable data might be accessed by external providers, such as insurance providers, who are the main actors of health care access in many countries [ 47].

The facilitators retrieved were the prompt availability of health information that caused an increase in the awareness of the health status [ 19, 22, 32, 51, 60], improved communication with health care professionals [ 22, 48, 60], and the accuracy, timeliness, usefulness, and convenience of the functionalities included in the portal [ 22, 50]. In this latter argument, some functionalities of the portal were found to be more useful than others, including laboratory tests and imaging [ 23, 24, 29, 43, 52], medical notes [ 23, 26], messaging with providers [ 52], medication refill [ 52], and current medication list [ 29]. Generally and as observed by Goldzweig et al [ 4], patients declared being satisfied with the use of digital tools [ 8, 17, 21, 29, 49, 50, 59- 61].

Despite the considerable number of studies included, the high heterogeneity in terms of outcomes and described portal functionalities did not allow us to perform meta-analyses and to draw generalizable and strong conclusions concerning the utility of the unique features of the portals.

Technological and digital innovations in health care could contribute to achieving the health system goals of equity, efficiency, accessibility, quality, and sustainability, if they are purposefully designed and cost-effectively implemented. When designing a new patient portal or a new functionality, developers and providers should always consider to which health care need they are trying to respond and if other nondigital interventions may be more effective or as effective at a lower cost.

Moreover, the adoption of a new technology is a complex process, depending on the content and the context in which it is introduced. As an example, our review demonstrated that it is feasible to achieve better medication adherence in chronic disease patients through portal use, and highlighted the main facilitators (eg, prompt availability of reliable information and accessibility of communication with disease specialists) and barriers (eg, security and usability concerns, and limited digital knowledge) to portal use. Keeping in mind these contextual factors could ease the difficult task of identifying the best digital tool for a specific population.

Before designing or implementing a new tool, it can be useful to analyze the ideal conditions needed for the adaptation, transfer, absorption, up-scaling, and enhancement of digital technologies. By ideal conditions, we basically mean a situation where the new technology has demonstrated effectiveness in trials or pilots, the provider is committed to guarantee continuous improvement in user accessibility and usability, and the main barriers in the target population are given due consideration. In the absence of these conditions, satisfactory results may be difficult to reach or may take many years to be observed.

The benefits of digitalization cannot be taken for granted and the use of technology does not always lead to an improvement in patient care and health system performance; thus, there is a need for evidence, which is, to date, scarce. The identification of a set of main features with proven efficacy for a patient portal is a useful starting point for the development and implementation of patient-oriented portals. Further studies should be conducted in different aspects of digitalization in health care. None of the studies retrieved analyzed the cost-effectiveness of portal use. Similarly, none of the studies compared the portals to each other, which could be interesting to point out the best practices and features.

Even though a patient portal is not a new concept, its real utilization and implementation are still far from optimal, and it seem to be still considered a "future technology." It is important to adapt the portal functions to the needs and capacities of patients, in order to facilitate the use of this technology and improve its dissemination. In particular, overcoming ethnic and literacy barriers to portal use represents a fundamental goal to create more equitable, effective, and safe health care systems.

Abbreviations

ED

emergency department

EMR

electronic medical record

ER

emergency room

RCT

randomized controlled trial

Footnotes

Contributed by

Authors' Contributions: All authors contributed to the study conception and design. Material preparation and data collection were performed by EC, LV, AMP, AG, and AB. EC and LV screened the articles, with any disagreement finally discussed and resolved with AMP. The first draft of the manuscript was written by EC, LV, AMP, AG, and AB. WR and SB commented on the latest version of the manuscript. WR and SB supervised the study. All authors read and approved the final manuscript.

Conflicts of Interest: None declared.

Articles from Journal of Medical Internet Research are provided here courtesy of Gunther Eysenbach

# data[].attributes.attachment_link.title
The Impact of Digital Patient Portals on Health Outcomes, System Efficiency, and Patient Attitudes: Updated Systematic Literature Review

# data[].attributes.attachment_link.description
Monitoring Editor: Rita Kukafka Reviewed by Sergio Pillon, Nancy Gordon, Iseult Wilson, and Feroze Kaliyadan Elettra Carini, MD,# 1 Leonardo Villani, MD,# 1 Angelo Maria Pezzullo, MD, MSc, 1 Andrea Gentili, MD, 1 Andrea Barbara, MD, 2, 3 Walter Ricciardi, MD, MPH, 1 and Stefania Boccia, MSc, PhD 1, 4 Patient portals are becoming increasingly popular worldwide even though their impact on individual health and health system efficiency is still unclear.

# data[].attributes.headline
The Surprising Disadvantages of Patient Protals

# data[].attributes.subject
The Surprising Disadvantages of Patient Protals

# data[].attributes.body
By understanding the disadvantages of patient portals, we can better address them. Some things mentioned were trouble getting people to opt in and inability for others due to limited internet access or devices, security, more work for the physician, user confusion with use or information that is provided, and a learning curve.

# data[].attributes.attachment_link.title
The Surprising Disadvantages of Patient Portals

# data[].attributes.body
This short video briefly explains aspects of a patient portal. It could be used to give information to members of the health care team, organization, and even patients when implementing a patient portal or patient health care system at your currently facility.

# data[].attributes.body
As we all know, physical activity can lead to better health in a countless number of ways. Motivating patients can be trying, frustrating, and unsuccessful. The researchers of this study saw more behavior change in those who used a gaming app compared to those who used other interventions. Unfortunately, long-term effects were not notable. Does this mean a more challenging, more interactive, or more "addictive" game should be developed to keep users motivated?

# data[].attributes.attachment_link.readable_text
Review

1 Univ. Grenoble Alpes, SENS, 38000 Grenoble, France

2 National Research Institute for Agriculture, Food and Environment (INRAE), 63000 Clermont-Ferrand, France

3 Kiplin, 44200 Nantes, France

4 Department of Sport Medicine and Functional Exploration, University Hospital Clermont-Ferrand, Hospital G. Montpied, 63000 Clermont-Ferrand, France

5 Department of Biostatistics unit, Clermont-Ferrand University Hospital, 63000 Clermont-Ferrand, France

Background: Gamification refers to the use of game elements in nongame contexts. The use of gamification to change behaviors and promote physical activity (PA) is a promising avenue for tackling the global physical inactivity pandemic and the current prevalence of chronic diseases. However, there is no evidence of the effectiveness of gamified interventions with the existence of mixed results in the literature.

Objective: The aim of this systematic review and meta-analysis is to evaluate the effectiveness of gamified interventions and their health care potential by testing the generalizability and sustainability of their influence on PA and sedentary behavior.

Methods: A total of 5 electronic databases (PubMed, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials) were searched for randomized controlled trials published in English from 2010 to 2020. Eligibility criteria were based on the components of the participants, interventions, comparators, and outcomes framework. Studies were included when they used gamified interventions in daily life with an active or inactive control group and when they assessed a PA or sedentary behavior outcome. We conducted meta-analyses using a random-effects model approach. Sensitivity analyses, influence analyses, and publication bias analyses were performed to examine the robustness of our results.

Results: The main meta-analysis performed on 16 studies and 2407 participants revealed a small to medium summary effect of gamified interventions on PA behavior (Hedges g=0.42, 95% CI 0.14-0.69). No statistical difference among different subgroups (adults vs adolescents and healthy participants vs adults with chronic diseases) and no interaction effects with moderators such as age, gender, or BMI were found, suggesting good generalizability of gamified interventions to different user populations. The effect was statistically significant when gamified interventions were compared with inactive control groups, such as waiting lists (Hedges g=0.58, 95% CI 0.08-1.07), and active control groups that included a nongamified PA intervention (Hedges g=0.23, 95% CI 0.05-0.41). This suggests that gamified interventions are not only efficient in changing behavior but also more effective compared with other behavioral interventions. The long-term effect (measured with follow-up averaging 14 weeks after the end of the intervention) was weaker, with a very small to small effect (Hedges g=0.15, 95% CI 0.07-0.23).

Conclusions: This meta-analysis confirms that gamified interventions are promising for promoting PA in various populations. Additional analyses revealed that this effect persists after the follow-up period, suggesting that it is not just a novelty effect caused by the playful nature of gamification, and that gamified products appear effective compared with equivalent nongamified PA interventions. Future rigorous trials are required to confirm these findings.

J Med Internet Res 2022;24(1):e26779

doi:10.2196/26779

Keywords

Background

In this context, it is urgent to develop interventions that can effectively change PA. Therefore, digital health interventions constitute a new opportunity to take care of patients by involving them in their treatment in a dynamic and interactive way. Gamification is a promising avenue to capitalize on the efficacy of digital interventions. Gamification is defined as the use of game design elements in nongame contexts []. By integrating game mechanisms in interventions that are initially devoid of them, the purpose of gamification is to integrate into daily life the ingredients that make games enjoyable to motivate participants to engage in PA []. The use of motivational affordances created by gamification can influence psychological (eg, motivation, attitude, and enjoyment) and physical outcomes (eg, physical capacities) [] and therefore appears as a potentially powerful technique for behavior change.

By gamifying PA, participants are encouraged to move and walk to play, which tends to make their activities more enjoyable and playful []. Unlike serious games, which refer to the use of a full-fledged video game for educational or health purposes (ie, a video game in its entirety as opposed to selected elements or individual features of a game) [] and require a dedicated time, a location, and implementation [], gamification techniques are relatively open to varying situational modes of engagement [] and concern instead global PA in all aspects of daily life (eg, walking, running, or gardening). Gamification is made possible by mobile technologies and wearable devices that can track and collect daily activities in a continuous and web-based manner. This allows for intervening directly on the lifestyle of individuals without adding material or time constraints for the participants.

However, several literature reviews [,,] have reported inconsistent results concerning the use of gamification in behavioral interventions, with some studies demonstrating positive effects and other studies providing more mixed effects. These reviews also emphasized the lack of high-quality studies and highlighted the need for more rigorous trials to isolate the impact of gamification (ie, randomized controlled trials [RCTs]). Importantly, Koivisto and Hamari [] suggested that the effects of gamification could be smaller when using rigorous experimentation. In sum, these reviews indicate that there is no clear evidence of the effectiveness of gamified interventions. Nevertheless, no meta-analysis has been conducted yet. Quantifying the effect size of gamified interventions and identifying moderators of this effect would provide important information about the effectiveness of such interventions. Moreover, a meta-analysis appears as timely, as there are now enough RCTs to conduct such an analysis.

This Study

This study is the first to quantify the effects of gamified interventions on PA. Beyond the effect during or just after the intervention, we also seek to evaluate the long-term effects to determine the health relevance of these interventions. Indeed, we reasoned that to be considered effective, gamification must sustain its impacts over the long term and offer more than a short-term novelty effect []. The generalizability of gamification to different user populations is also a major issue because it would determine whether gamification can be introduced in health care settings with patients or it is more suited in prevention for healthy audiences.

The objectives of this systematic review and meta-analysis are to answer these research gaps by (1) evaluating the effect of gamified interventions on PA and SB, (2) assessing the long-term or sustained effects of gamified programs, and (3) evaluating the generalizability of gamification across different populations.

Design

This review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines []. Following recommendations to minimize bias and provide evidence of a priori analysis intentions [], the study was preregistered under the international prospective register of systematic reviews (PROSPERO; registration number: CRD42020186882) and on the Open Science Framework (OSF) []. Moreover, all materials and data are available on the OSF page of the project to facilitate reproducibility and transparency of this review [].

Search Strategy and Information Sources

We conducted a systematic literature search using five electronic databases: PubMed, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials. We combined alternate terms and keywords representing synonyms for the outcomes (PA or SB), intervention (gamification), device (eHealth), and trial (see Table S1 in [-] for an overview of the terms used). The search strategy was reviewed by an academic librarian. All databases were searched individually from January 1, 2010 (2010 being the date of the widespread adoption of the term gamification []), to December 31, 2020, and the research was restricted to English-language texts. The complete search equations for all databases are available in . In addition, we complemented our search with reference harvesting from the included studies and overview articles.

Eligibility Criteria

Studies were eligible for inclusion if they were RCTs and if they met other criteria based on the participants, interventions, comparators, and outcomes framework.

Participants

This review focused on the general population regardless of age, gender, or health status (ie, patients with chronic diseases were also included). We excluded studies involving participants with contraindications to PA or with diseases preventing them from engaging in PA or understanding the principles of the game (intellectual and cognitive impairments).

Intervention

Digital interventions targeting PA or SB and incorporating game elements and gamification techniques, such as points, levels, rewards, leaderboards, narratives, and teams, were of interest. We clearly distinguished between gamification and related constructs, such as serious games. Therefore, we excluded interventions based on active video games (ie, electronic games that allow players to physically play with the images on screen) that are more comparable with serious games than with gamified products.

Comparators

Studies that attempted to compare gamified interventions with control groups without gamification elements in a randomized design were integrated in the review. These groups could be either inactive (nonexposed control group, such as a waiting list) or active (another nongamified intervention).

Outcomes

In this review, we included studies assessing change in total PA or leisure PA (quantity in metabolic equivalent of task [MET] hour per week or MET minute per week or in duration, energy expenditure [METs], moderate to vigorous PA [MVPA], step count, walking time, and active minutes) and change in time spent in SB (total time, leisure time, work, time spent in front of the computer, and time spent in front of television). These outcomes were continuous data either objectively measured (through accelerometers, pedometers, and smartphones) or subjectively measured by self-reported questionnaires. Data measured objectively were always prioritized in the analyses over self-reported questionnaires, which are more susceptible to bias with a potential overestimation of PA [].

In addition, studies were excluded if they came from a review, commentary, or conference abstract; if they included data previously published in another study; if they were not randomized and controlled; if they were not written in English; and if they were published before 2010.

Screening

In total, 2 authors (AM and AC) independently screened the titles and abstracts resulting from the search. Full texts of the potential included studies were checked before inclusion. Disagreements were resolved by discussion or by consulting a third author (MD), and agreement was measured using the κ statistic. A complete list of excluded studies is available on the OSF page of the project.

Data Extraction

In the data collection process, AM and AC extracted data independently and were blinded to each other using a predetermined and tested template. Disagreements were resolved by discussion or consultation with a third author (MD). Extracted data included the results of each study and three types of potential moderators:

Population-level moderators to assess the generalizability of the intervention (population characteristics, age, gender, and pathology).

Intervention-level moderators to better understand gamification mechanisms (theoretical model used to develop the intervention, gamification elements, and modality of the intervention [eg, internet-based, smartphone app, and presence of social incentives]).

Outcome-level moderators (outcomes, measure of PA or SB, and device).

Risk of Bias Assessment

For each eligible study, 2 reviewers (AM and AC) assessed the risk of bias using the purpose-built Cochrane risk of bias tool (Table 8.5 in the Cochrane Handbook for Systematic Reviews of Interventions []), which evaluates 7 domains (sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, and other risk of bias). A judgment of the potential risk of bias was made from the extracted information and rated as high risk, low risk, or unclear if the related information was not available. These evaluations of bias are reported in the review and included in the analysis, and a measure of agreement with the κ statistic was calculated. After the full assessment, we decided not to present the item blinding of participants and personnel in the review because it was similar for all studies, which were rated as high risk, blinding being unfeasible for this kind of intervention.

Data Synthesis and Statistical Analyses

First, data were synthesized in a qualitative review assessing the key elements of the studies and highlighting intervention differences. This qualitative review integrates all studies that met the eligibility criteria, including those for which we were unable to extract the data.

For the quantitative analysis, means and SDs of continuous PA or SB outcomes from individual studies were compiled when available or estimated using the method by Hozo et al [] when median and IQR were reported. When the necessary data were not available in the original article, we first requested them from the authors. If data could not be obtained, we extracted them from the graphs when available. If this was not possible, we excluded the study from the quantitative analysis.

A global meta-analysis was conducted to obtain a summary effect. In addition, when sufficient data were available (ie, 4 studies or more reporting an outcome), we conducted different meta-analyses for each specific outcome (steps, MVPA, and time in SB) and for the follow-up effect. To address the nonindependence of data caused by study effect, random-effects models [] were preferred to the usual statistical tests. In addition, the Hartung-Knapp-Sidik-Jonkman method was used to reduce the production of false positives inherent to the DerSimonian-Laird method [] and to obtain more robust estimates of variance. Continuous outcomes were analyzed using standardized mean difference (SMD) to account for different measurement instruments or mean difference (MD) when the measurements were close enough. We computed Hedges g [] for effect sizes, which is similar to Cohen d but corrects for small sample bias, which are recurrent in the studies included. Thus, a Hedges g of 0.2 represents a small effect; 0.5, a moderate effect; and 0.8, a large effect []. We computed SMDs for outcome scores after the intervention (presented in the review) and change-from-baseline outcomes. Scores on postintervention effect sizes refer to treatment group results compared with the control group results after interventions. Change-from-baseline score effect sizes were calculated as a comparison between the treatment group pre-post effect size and control group pre-post effect size.

For studies that included multiple outcomes, we kept in the main analysis the primary outcome targeted in the initial article. If none of the PA outcomes reported by a study were the primary ones, we selected the one that was the most relevant from the perspective of the intervention and the original experiment. In designs with multiple time measurements, the assessment that was the most proximal to the end of the intervention was conserved. A time assessment had to be performed >2 weeks after the end of the intervention to be included in the follow-up analysis. When studies included multiple intervention groups with gamification features, they were combined into one group following the formulae recommended by the Cochrane Handbook []. Studies including multiple control groups could be integrated into different subgroup analyses if they compared their gamified intervention to both an active and an inactive control group.

Statistical heterogeneity was tested using forest plots and the I2 statistic, which is the most common metric for measuring the magnitude of between-study heterogeneity and is easily interpretable (0%-40% might not be important, 30%-60% may represent moderate heterogeneity, 50%-90% may represent substantial heterogeneity, and 75%-100% may represent considerable heterogeneity) []. We conducted different influential analyses to address between-study heterogeneity. We first explored the presence of outliers, defined as studies with CIs that do not overlap with the CI of the summary effect. We also performed leave-one-out analyses, which recalculated the summary effect several times, with 1 study omitted each time. Finally, we performed a Baujat plot [], which is a diagnostic plot to detect sources of heterogeneity in the meta-analysis by comparing the contribution of each trial in the pooled effect with the overall Cochran Q test for heterogeneity.

We applied different methods to detect publication bias (funnel plot, Egger regression test [], and Duval and Tweedie trim-and-fill procedure []). In addition, another approach to determine the evidential value of studies included in the analysis is to check the statistical power of individual studies. Therefore, we performed a sunset funnel plot [], which is a funnel plot variant that visualizes the statistical power of each study included in the meta-analysis based on the summary effect size.

Thus, sensitivity analyses were conducted to address studies with a high risk of bias or a strong heterogeneity in the sample or studies identified as outliers. Subgroup analyses were conducted to explore possible sources of heterogeneity and test for population differences. Therefore, we conducted tests for subgroup differences using a random-effects model. In addition, moderation analyses were performed to explore the impact of potential explanatory variables and moderators on the effect size with meta-regressions when sufficient data were available (ie, at least 10 studies for each explanatory variable []). The results were expressed as regression coefficient estimates, 95% CIs, and P values.

For crossover trials, we first checked whether carry-over or period effects were problematic in the original texts of studies. For cluster randomized trials, we checked if the influence of the different clusters was not too important, analyzing the values of the intraclass correlation coefficient in the studies. Then, in the absence of sufficient information in the published articles, we addressed these studies as traditional parallel trials. Nevertheless, this procedure increased the probability of a unit of analysis error. Therefore, we performed sensitivity analyses without clusters and crossover trials to test the robustness of our results.

A summary of the analytical procedure is available in (Figure S1). Analyses were performed on R (The R Project for Statistical Computing) using the dmetar package []. Risk of bias summary and risk of bias graphs were made via the robvis R package [].

Characteristics of Included Studies

We screened the titles and abstracts of 1626 articles, and 51 full-text articles were assessed for eligibility according to the inclusion criteria. Finally, 18 articles [-,,] were included in the qualitative analysis and 16 were included in the meta-analysis (). The κ value of agreement for the screening phase was 0.64 between the 2 authors, reflecting good agreement [].

‎
Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) flowchart of the literature search and screening process. PA: physical activity; RCT: randomized controlled trial.
View this figure

A total of 6 trials were conducted in Europe; 4 in the United States; 3 in Australia and New Zealand; 3 in Canada; and 2 in Asia. Studies were published between 2014 and 2020, with 39% (7/18) published after 2018.

Table 1. Characteristics of included studies.

Study

Participants

Intervention

Theory

PA a outcomes

Adolescents, n=224 (aged 12-14 years; 47% male participants)

The "StepSmart Challenge" was a web-based intervention that used gamification strategies to encourage and support PA behavior change; duration: 22 weeks

SDT c

Daily step count and MVPA d (min/day) objectively measured (Actigraph GT3x)

Adult workers in an automobile manufacture, n=144 (65% male participants)

"Healingo Fit" had the objective to promote low levels of PA using a tracking-based approach measuring PA with a Fitbit pedometer and a gamified intervention accessible by desktop and mobile devices; duration: 6 weeks

SCT e, TPB f, and health action process approach

Self-reported VPA g, MPA h, and minutes walked (min/week; IPAQ i)

Adolescents, n=35 (mean age 15.7 years, SD 1.2 years; 45% male participants; BMI 22.85)

"Zombies, run! 5K Training app" was a fully automated training program designed to improve fitness, combined with an immersing and fun story; duration: 8 weeks

SDT

MVPA, VPA, MPA, and LPA j and sedentary time (min/day) objectively measured (Actigraph GT3x) and self-reported PA (PAQ-A k)

Garde et al []

Adolescents, n=47 (mean age 10.3 years, SD 1.9 years; 34% male participants; BMI z- score 0.35)

"MobileKids Monster Manor" was a mobile exergame synchronized with an external activity monitor. The overall goal was to complete the story with PA and steps; duration: 1 week

SDT

Daily step count and active min/day objectively measured (Tractivity)

Adolescents, n=56 (mean age 11.3 years, SD 1.2 years; 62% male participants; BMI z- score 0.28)

"MobileKids Monster Manor" was a mobile exergame synchronized with an external activity monitor. The overall goal was to complete the story with PA and steps; duration: 1 week

SDT

Daily step count and active min/day objectively measured (Tractivity)

Garde et al []

Adolescents, n=37 (mean age 10.6 years, SD 0.5 years; 43% male participants; BMI z- score 0.21)

"MobileKids Monster Manor" was a mobile exergame synchronized with an external activity monitor. The overall goal was to complete the story with PA and steps; duration: 2 weeks

SDT

Daily step count and active min/day objectively measured (Tractivity)

Adults, n=20 (mean age 31 years, SD 3 years; 30% male participants; BMI 21.5)

Participants wore an activity monitor with computerized game functions, such as a story, a character, and objectives; duration: 6 weeks

- l

Daily step count and MVPA (metabolic equivalent of tasks hour/day) objectively measured (Lifecorder EX)

a PA: physical activity.

b The studies included in the meta-analysis.

c SDT: self-determination theory.

d MVPA: moderate to vigorous physical activity.

e SCT: sociocognitive theory.

f TPB: theory of planned behavior.

g VPA: vigorous physical activity.

h MPA: moderate physical activity.

i IPAQ: International Physical Activity Questionnaire.

j LPA: light physical activity.

k PAQ-A: Physical Activity Questionnaire for Adolescents.

l No theory mentioned.

Studies comparing gamified interventions with active control groups used a similar intervention without game elements (ie, an equivalent nongamified app [-] or a self-monitoring intervention with wearables or activity monitors [-,,]).

A total of 2 studies [,] were excluded from the meta-analysis and were only integrated in the qualitative review because we were unable to extract their results.

Risk of Bias

The 2 authors extracted the risk of bias data with a κ coefficient of 0.79, which is synonymous with excellent agreement []. [-] and present the authors' judgments about each risk of bias item presented as percentages across all included studies in the meta-analysis and an overview of the different biases for each study. Overall, 1 study [] was rated as high risk for sequence generation because assignments were based on recruitment order. Therefore, this study was also at a high risk for allocation concealment. A total of 3 studies [,,] were at high risk of bias for the blinding of outcome assessment item because they measured PA using only self-reports. In total, 5 studies [,,,,] were at high risk of bias for the incomplete outcome data item because they reported high dropout rates and did not include intention-to-threat analyses and 5 studies [,,,,] were rated at unclear risk for the selective outcome reporting item because they had not been preregistered or published in a protocol-study. Finally, 2 studies had other high risks of bias. The first one [] was a crossover trial conducted without a washout condition, and the other one [] was a cluster randomized trial with no control of clustering, no consideration of the clustering in the statistical analysis, and no test of baseline differences among groups.

Summary Effect

Overall, the SMD after the intervention for all PA outcomes (MVPA, daily step count, number of active minutes, and walking time) was a Hedges g of 0.43 (95% CI 0.03-0.82; I2=86%; ), representing a statistically significant small to medium effect. Similarly, we found a statistically significant SMD effect of a Hedges g of 0.38 (95% CI 0.07-0.69; I2=79%) for pre-post change scores.

Only 3 studies [,,] assessed sedentary time as an outcome. Owing to this small sample size, the meta-analysis was not performed on this outcome.

‎
Figure 2. Forest plot for the effect of gamification versus control on postintervention physical activity outcomes (moderate to vigorous physical activity, daily step count, number of active minutes, and walking time). Tau-square, chi-square, and I² measures of between-study heterogeneity [-]. IV: inverse variance.
View this figure

Outliers and Influential Analyses

In the first analysis, substantial statistical heterogeneity was observed. To address between-study heterogeneity, we first looked for the presence of outliers. A total of 3 studies were considered as outliers [,,], and after removing them, we still obtained a significant effect of a Hedges g of 0.34 (95% CI 0.17-0.51) with moderate heterogeneity ( I2=58%).

Leave-one-out analyses showed that sequential removal of each study did not have an important impact on the general effect size, with effect sizes ranging from a Hedges g of 0.33 (95% CI 0.00-0.66; I2=84%) to a Hedges g of 0.48 (95% CI 0.13-0.83; I2=78%; Figure S2 in ).

The Baujat plot (Figure S3 in ) shows that 4 studies explained more heterogeneity than the others, more specifically, the study by Corepal et al [] with a heterogeneity contribution of 40.13 and an effect size influence of 3.27.

Therefore, we excluded studies with a high or unclear risk of bias. After doing so, the effect was not significant (Hedges g=0.33, 95% CI -0.16 to 0.81; I2=78%).

The inclusion of crossover and cluster randomized trials in the meta-analysis may lead to biases. Thus, we excluded these designs from the analysis and obtained a statistically significant effect of g=0.49 (95% CI 0.05-0.92; I2=67%).

Finally, we decided to exclude articles by Corepal et al [] and Nishiwaki et al [] in the sensitivity analyses (which have been repeated afterward for each analysis) considering their influence on the pooled result, their contribution to overall heterogeneity, and their huge risk of bias (no control of clustering, no statistical consideration of clustering, and no test of baseline differences among groups in the study by Corepal et al [] and no washout period and very low power for the study by Nishiwaki et al []). After doing so, we obtained a statistically significant effect of a Hedges g of 0.42 (95% CI 0.14-0.69; I2=74%).

Subgroup Analyses

We found no statistical differences in the effects between studies with participants with chronic diseases or healthy participants (Cochran Q=0.73; P=.39), between adults and adolescents (Cochran Q=0.26; P=.61), between studies with objective (devices) or self-reported PA outcomes (Cochran Q=0.23; P=.63), between active or inactive control groups (Cochran Q=0.01; P=.92), and between short- and long-term interventions (less or more than 12 weeks; Cochran Q=0.60; P=.44).

When performing the sensitivity analysis, there was a statistically significant effect of intervention on PA in adults (Hedges g=0.36, 95% CI 0.03-0.69; I2=71%; Figure S4 in ), on healthy people (Hedges g=0.35, 95% CI 0.15-0.55; I2=63%; Figure S5 in ), when the PA measure was objective (Hedges g=0.45, 95% CI 0.08-0.82; I2=80%; Figure S6 in ), when the PA measure was self-reported (Hedges g=0.24, 95% CI 0.08-0.39; I2=0%; Figure S6 in ), and for short interventions of <12 weeks (equivalent to a 3-month program; Hedges g=0.44, 95% CI 0.19-0.69; I2=16%; Figure S7 in ).

Moreover, subgroup analyses allowed us to examine the effect of gamified interventions when compared with inactive control groups and active control groups. After sensitivity analyses, we found a statistically significant effect of gamified interventions, both when compared with inactive control groups (Hedges g=0.58, 95% CI 0.08-1.07; I2=81%; Figure S8 in ) and when compared with active control groups (Hedges g=0.23, 95% CI 0.05-0.41; I2=37%; Figure S8 in ).

Meta-Regressions

The age of participants ( β=.01, 95% CI -0.02 to 0.04; P=.39), their gender ( β=.01, 95% CI -0.01 to 0.02; P=.47), their BMI ( β=.04, 95% CI -0.16 to 0.09; P=.53), the duration of the intervention ( β=-.01, 95% CI -0.06 to 0.04; P=.74), or the number of game mechanics included in the intervention ( β=.01, 95% CI -0.17 to 0.19; P=.91) were not statistically significantly associated with an increase in PA.

Lack of data precluded further meta-regressions, such as comparisons of leisure PA, or test of moderators, such as the impact of social incentives or the theoretical model used to develop the intervention.

Publication Bias

First, an inspection of the funnel plot showed that the effect sizes of individual studies were relatively symmetrically distributed around the pooled effect size. This observation was supported by the Egger test of the intercept, which indicated no asymmetry in the funnel plot (b 0=1.38, 95% CI -0.83 to 4.77; P=.19). We then applied a bias-correction technique, the trim-and-fill method, which indicated that 3 studies were missing at the bottom left of the funnel plot to obtain a full symmetry. After imputing the effect sizes corresponding of these missing studies to obtain a totally symmetrical funnel plot (), the bias corrected summary effect was of a Hedges g of 0.24 (95% CI -0.24 to 0.73).

Finally, the sunset funnel plot () showed significant differences in power among studies, with some characterized by very low statistical power (7 studies under 45% power and 4 studies under 18%). The median power of all the tests was 63%.

‎
Figure 3. Funnel plot after trim-and-fill bias correction. A filled circle represents an included study, and an empty circle represents a missing study.
View this figure

‎
Figure 4. Power-enhanced funnel plot. White circles represent included studies. δ: true effect size; medpower: the median power of all tests; d33%: effect size needed for achieving 33% of median power; d66%: effect size needed for achieving 66% of median power; E: expected number of positive studies; O: observed number of positive studies; pTES: test of excess significance P value.
View this figure

Secondary Analyses

Follow-up

There was no statistically significant effect of gamified interventions on total PA (MVPA and daily step count) after follow-up periods with an SMD of a Hedges g of 0.09 (95% CI -0.07 to 0.26; I2=21%). When we performed the sensitivity analysis, gamification significantly increased PA (MVPA and daily step count) at follow-up (from 12 to 24 weeks after the end of the intervention; g=0.15, 95% CI 0.07-0.23; I2=0%; ).

Steps

We found no statistically significant effect of gamified interventions on step outcomes with an SMD of a Hedges g of 0.53 (95% CI -0.09 to 1.15; I2=89%), but a significant improvement in the number of daily steps with an MD of +1420.57 steps per day (95% CI 435.41-2405.73; I2=95%) was observed. When excluding the 2 studies in the sensitivity analysis, we obtained a statistically significant effect of gamification on daily steps of a Hedges g of 0.49 (95% CI 0.05-0.93; I2=75%; ) and a statistically significant MD of +1609.56 steps per day (95% CI 372.39-2846.73; I2=86%; ).

Moderate to Vigorous PA

There was no statistically significant effect of gamification on MVPA with an SMD of a Hedges g of 0.09 (95% CI -0.57 to 0.74; I2=93%). There was no statistically significant effect of a Hedges g of 0.31 (95% CI -0.19 to 0.80; I2=82%) in the sensitivity analysis ().

Summary of Findings (Grading of Recommendations Assessment, Development, and Evaluation)

The quality of evidence (grading of recommendations assessment, development, and evaluation, [GRADE]) in the included studies after sensitivity analyses for short-term PA, long-term PA, MVPA, steps, and daily steps was scored from high to low (). The quality was downgraded for some outcomes because of high heterogeneity, high risk of bias, or imprecision owing to large CIs. Summaries of the various meta-analysis conducted in this review both on postintervention scores and pre-post intervention change scores are presented in (Figures S9-S11).

Table 2. Summary of findings.

Outcome

Number of participants (number of studies)

Standardized mean difference or mean difference (95% CI)

Quality of evidence (grading of recommendations assessment, development, and evaluation)

General PA a

2197 (14)

0.42 (0.14 to 0.69)

Low b,c,d

General PA (in comparison with active control groups)

1485 (7)

0.23 (0.05 to 0.41)

High

Long-term PA (follow-up)

1306 (5)

0.15 (0.07 to 0.23)

High

MVPA e

739 (4)

0.31 (-0.19 to 0.80)

Low b,c,d

Steps

1438 (9)

0.49 (0.05 to 0.93)

Low b,c,d

Daily steps

1235 (7)

1609.56 (372.39 to 2846.73)

Moderate b,d

a PA: physical activity.

b Downgraded because of high heterogeneity.

c Downgraded because of risks of bias.

d Downgraded because of imprecision (large CIs).

e MVPA: moderate to vigorous physical activity.

Principal Findings

Summary Effect

This meta-analysis of RCTs, including 16 studies and 2407 participants, revealed a statistically significant effect of gamified interventions, on average, of 12 weeks on total PA (Hedges g=0.42, 95% CI 0.14-0.69 after sensitivity analyses). This effect was small to medium, suggesting the effectiveness of gamified interventions in promoting PA in both healthy participants and participants with chronic diseases. This significant effect was robust, as it persisted even after the different influence analyses were performed. Moreover, the effect was statistically significant both for objective measures of PA (Hedges g=0.45, 95% CI 0.08-0.82) and self-reported measures (Hedges g=0.24, 95% CI 0.08-0.39) after sensitivity analyses. Unsurprisingly, subgroup analyses revealed after sensitivity analyses that the effect of gamified interventions is greater when compared with inactive control groups (such as waiting lists) than when compared with active control groups benefiting from a nongamified intervention (Hedges g=0.58, 95% CI 0.08-1.07 vs Hedges g=0.23, 95% CI 0.05-0.41). Nevertheless, these effects were both statistically significant. This suggests that gamified interventions are not only efficient in changing behavior but also, to a lesser extent, effective compared with equivalent nongamified PA interventions (such as smartphone apps or self-monitoring interventions). These results are important considering the assets of gamification, which has the advantages of (1) reorganizing existing activity rather than adding additional demands to people's lives [], (2) being easily implemented in natural contexts, and (3) having a broad accessibility through technology and advancing sensors, permitting to address a large population.

Long-term Effect

When we analyzed the long-term effect of these interventions based on the follow-up measures of PA, carried out from 12 to 24 weeks (mean 14.4 weeks) after the end of the intervention, we found a statistically significant very small to small effect size of a Hedges g of 0.15 (95% CI 0.07-0.23) after sensitivity analyses. These results indicate that the effect of gamification persists after the end of the program, suggesting that it is not just a novelty effect due to the playful nature of gamification. However, this long-term effect was weaker and decreased with time after the end of the intervention.

Generalizability of Gamified Interventions

The absence of subgroup differences or effects of age, gender, and BMI on the pooled effect suggests a good generalizability of gamified interventions, which can be used for several types of populations. Thus, gamification may not only be efficient in young healthy individuals but can also target any kind of population regardless of their age or health status.

In sum, gamified interventions appear as an efficient tool to improve the PA of various populations, with moderate superiority over other similar interventions, such as mobile health monitoring apps, and a moderate sustainability of the effect after the intervention. Nevertheless, if many PA interventions increase PA levels in the short term, translating these temporary changes into long-term PA participation continues to be a challenge for PA research []. With that in mind, the potential of gamification for PA increases in the long term, even minimal, is particularly important and promising in the area of PA interventions.

Additional Findings

Effect of Gamification on the Step Count

If the overall effect of gamified interventions on PA is positive, they increase the step count more than MVPA. Indeed, after sensitivity analyses, on the one hand, the meta-analyses revealed a statistically significant effect (Hedges g=0.49, 95% CI 0.05-0.93) of gamification for steps outcomes, with a statistically significant improvement of 1609.56 steps per day (95% CI 372.39-2846.73) for participants benefiting from gamified intervention versus those in the control group. On the other hand, no statistically significant effect of gamified interventions on MVPA was found (Hedges g=0.31, 95% CI -0.19 to 0.80). This can be explained by the game metrics and mechanics of the interventions included in the review, which are mainly focused on the step count of participants. Few interventions directly targeted MVPA. In the included studies, only 2 interventions [,] integrated multi-PA intensity goals and mechanics, notably with physical exercises or running sessions in the game. In other words, participants played most of the time with their number of steps and had to generally walk more to make points and play the game. This results in an increase in walking time but not necessarily in more intense PA.

Duration of Intervention

Our meta-regression analysis did not find an association between the observed effect of gamification on PA and intervention length. However, although no statistically significant effect of gamification for an intervention length of ≥12 weeks was found (Hedges g=0.41, 95% CI -0.19 to 1.01), the meta-analysis revealed a statistically significant effect of gamified interventions of <12 weeks on global PA (Hedges g=0.44, 95% CI 0.19-0.69). According to a previous meta-analysis that reported significant positive effects of smartphone apps on PA only when used over a short-term period of <3 months [], these results suggest that a condensed intervention could benefit more than a longer one, which could become redundant, boring, and exhausting for participants in the long run.

Statistical Heterogeneity

The meta-analysis also revealed considerable statistical heterogeneity. This heterogeneity may be explained by differences in study quality, diversity of designs, and variations in study populations. Despite several subgroup analyses, we cannot rule out that these subgroups and characteristics may not explain all the variance of the interventions. Indeed, demographic data often do not fully explain the differences in the effectiveness of interventions [,], and more precise sociopsychological variables such as personality traits or motivational factors could explain this poorly understood significant heterogeneity. Moreover, the risk of bias analysis and the sunset funnel plot showed substantial differences in the quality of the included studies that can influence the heterogeneity. Finally, various trial designs were used (ie, parallel RCT, cluster RCT, and crossover RCT) that can also contribute to the overall statistical heterogeneity.

Better Understanding Gamification Mechanisms

This meta-analysis is informative regarding the effectiveness of gamified interventions. In view of the observed heterogeneity, the next step will be to investigate its causes from an interventional and theoretical perspective. Gamified interventions involve multiple interacting elements, and it is crucial to estimate the weight of each element in the behavior change process and how they interact with each other. Is it game mechanics, the implementation of behavior change techniques, or the presence of social interactions that make gamification effective? Unfortunately, the small number of studies included in the meta-analysis impeded us from conducting in-depth moderation analyses to answer this question. To better understand these relations, it is essential that both the development and assessment of gamified interventions be central, transparent, evidence-based, context-aware, and research-oriented []. Moreover, if theoretical psychological models are often mentioned in the introduction of articles included in the review (), few have investigated the psychological mechanisms of their interventions in the field [,,]. Future studies should explicitly discuss motivational theory and systematically test the effect of gamification on psychological outcomes known to be involved in behavior change (eg, self-efficacy, attitudes, and intention) to better understand its mechanisms. The consideration of personality traits and psychological variables to determine behavioral phenotypes [] is a promising way to evaluate participant's responses to the interventions.

Perspectives for Future Research and Implications for Practice

The findings from this meta-analysis allow us to draw and discuss future work concerning the gamification of PA and SB. First, future trials should be conducted with more adequately powered sample sizes and should be strictly multiple arms-RCTs to isolate the effects of gamification elements and better understand gamification mechanisms. Second, the long-term effects are currently the main challenge of health interventions. Thus, it is essential to investigate the evolution of the effects of gamified interventions over time. Therefore, there is a need for more long-term follow-up measurements. In addition, the potentialities of digital technologies and their capacity to collect a large amount of real-world data could be used to assess the evolution patterns of the effect, allowing the detailed identification of its sustainability and evolvement or even make forecasts via time series analyses. Third, to our knowledge, only 1 team of researchers worked on a gamified intervention targeting SB by introducing sedentary breaks as a gaming part []. Following this line of research, it could be interesting to develop gamified interventions affording participants to take more sedentary breaks. Finally, the cost-effectiveness ratio of gamified interventions may be better than that of many current interventions, considering the ease of implementation and generalizability of gamification. However, this assertion will have to be tested in future trials, including economic analyses.

In light of our results, gamified interventions appear to be a promising avenue to promote PA in different populations both in prevention in healthy people and in the treatment of chronic diseases. Gamified interventions have many benefits for participants with chronic diseases, such as empowerment of participants by improving their self-management skills, an effect across broad audiences enabling to target different types of pathologies, and an everyday life fit and easy implementation. Similar to other digital health processes, gamification makes it possible to address more patients, especially those who are isolated from health care facilities. Importantly, gamified interventions are especially pertinent during a health pandemic, such as the COVID-19 outbreak, in which PAs and social interactions are restricted because of lockdown or teleworking and where structured PA possibilities are limited both indoors and outdoors. Gamifying walking and daily activities is, in this context, a great way to improve PA and limit SB of individuals in addition to providing social interaction among players. In the meantime, the face-to-face management of chronic diseases is usually suspended during the pandemic, which underlines the importance of offering remote supervision of PA.

Nevertheless, in view of the weaker postintervention effect, this study suggests that a one-shot intervention is not sufficient. A more interesting design would be to address multiple gamification doses during or after the course of treatment to obtain a sustainable implementation of the PA behavior. This configuration would also provide an ideal duration of intervention to avoid exhausting the participants with gamified interventions for >12 weeks.

Strengths and Limitations

To our knowledge, this is the first meta-analysis to quantitatively evaluate the effects of gamification on PA. This review has several other strengths. First, we conducted a comprehensive search strategy using multiple databases in collaboration with an academic librarian. Second, all stages of the review (screening and data extraction) were independently realized by 2 reviewers. Finally, various novel publication bias analyses and influence analyses were conducted in parallel with different subgroup analyses and meta-regressions.

However, some limitations of this work must be mentioned. Overall, the meta-analysis included a small number of studies, and some articles were feasibility or pilot trials. Therefore, several trials included small sample sizes and were highly underpowered. Some studies were conducted with a high risk of bias. One of the main limitations of this work is the impossibility of demonstrating that the effect of gamified interventions is led by gamification itself given the lack of research examining this question. Finally, in the main analysis, we included diverse PA outcomes evaluating similar constructs but which are slightly different in practice. Moreover, not all included outcomes were objectively measured. As the field matures and new trials are published, an update of this work will be important to confirm these preliminary results.

Conclusions

To conclude, gamified interventions appear to be a promising avenue for promoting PA in various populations. Influencing primarily the number of daily steps of the participants, gamification is an interesting way to improve daily PA and appears more efficient than equivalent nongamified interventions, such as mobile health apps. However, if the effect of gamification persists during follow-up, suggesting that gamified interventions are more than a novelty effect, this effect decreases with time with a smaller long-term effect. The integration of gamification in more global health care interventions could be a way to address this limited sustainability. Future rigorous trials are required to explore these perspectives.

Acknowledgments

The authors would like to thank the librarian at Clermont Auvergne University (Nathalie Pinol) for assistance in developing the search equations. The work of AM is supported by a grant from the French National Association for Research and Technology (Cifre PhD thesis) and by the company Kiplin. None of the sponsors were involved in any other aspect of the project, such as the design of the project's protocol and analysis plan, collection, and analyses. The funders had no input on the interpretation or publication of the study results. The authors also want to thank Challenge 3 I-SITE Clermont Auvergne Project 20-25 for their grant.

Authors' Contributions

AM designed the study protocol and search strategy in consultation with other authors. AM and AC conducted the screening, data extraction, and risk of bias assessment. AM and BP developed the meta-analytic procedure, and AM performed the statistical analysis. The first draft of the manuscript was written by AM, and all authors commented on the previous versions of the manuscript. All authors read and approved the final manuscript.

Conflicts of Interest

None declared.
‎ ‎ ‎

Multimedia Appendix 3

Risk of bias graph: review authors' judgments about each risk of bias item presented as percentages across all included studies.
PDF File (Adobe PDF File), 59 KB

‎

GRADE: grading of recommendations assessment, development, and evaluation

MD: mean difference

MET: metabolic equivalent of task

MVPA: moderate to vigorous physical activity

OSF: Open Science Framework

PA: physical activity

PRISMA: Preferred Reporting Items for Systematic Reviews and Meta-Analyses

RCT: randomized controlled trial

SB: sedentary behavior

SMD: standardized mean difference

Edited by R Kukafka; submitted 25.12.20; peer-reviewed by C Maher, H Mehdizadeh, I Gupta, E Sadeghi-Demneh, H Pratomo, MDG Pimentel; comments to author 25.02.21; revised version received 31.03.21; accepted 19.10.21; published 04.01.22

©Alexandre Mazeas, Martine Duclos, Bruno Pereira, Aïna Chalabaev. Originally published in the Journal of Medical Internet Research (https://www.jmir.org), 04.01.2022.

This is an open-access article distributed under the terms of the Creative Commons Attribution License (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work, first published in the Journal of Medical Internet Research, is properly cited. The complete bibliographic information, a link to the original publication on https://www.jmir.org/, as well as this copyright and license information must be included.

Background: Gamification refers to the use of game elements in nongame contexts. The use of gamification to change behaviors and promote physical activity (PA) is a promising avenue for tackling the global physical inactivity pandemic and the current prevalence of chronic diseases. However, there is no evidence of the effectiveness of gamified interventions with the existence of mixed results in the literature.

Objective: The aim of this systematic review and meta-analysis is to evaluate the effectiveness of gamified interventions and their health care potential by testing the generalizability and sustainability of their influence on PA and sedentary behavior.

Methods: A total of 5 electronic databases (PubMed, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials) were searched for randomized controlled trials published in English from 2010 to 2020. Eligibility criteria were based on the components of the participants, interventions, comparators, and outcomes framework. Studies were included when they used gamified interventions in daily life with an active or inactive control group and when they assessed a PA or sedentary behavior outcome. We conducted meta-analyses using a random-effects model approach. Sensitivity analyses, influence analyses, and publication bias analyses were performed to examine the robustness of our results.

Results: The main meta-analysis performed on 16 studies and 2407 participants revealed a small to medium summary effect of gamified interventions on PA behavior (Hedges g=0.42, 95% CI 0.14-0.69). No statistical difference among different subgroups (adults vs adolescents and healthy participants vs adults with chronic diseases) and no interaction effects with moderators such as age, gender, or BMI were found, suggesting good generalizability of gamified interventions to different user populations. The effect was statistically significant when gamified interventions were compared with inactive control groups, such as waiting lists (Hedges g=0.58, 95% CI 0.08-1.07), and active control groups that included a nongamified PA intervention (Hedges g=0.23, 95% CI 0.05-0.41). This suggests that gamified interventions are not only efficient in changing behavior but also more effective compared with other behavioral interventions. The long-term effect (measured with follow-up averaging 14 weeks after the end of the intervention) was weaker, with a very small to small effect (Hedges g=0.15, 95% CI 0.07-0.23).

Conclusions: This meta-analysis confirms that gamified interventions are promising for promoting PA in various populations. Additional analyses revealed that this effect persists after the follow-up period, suggesting that it is not just a novelty effect caused by the playful nature of gamification, and that gamified products appear effective compared with equivalent nongamified PA interventions. Future rigorous trials are required to confirm these findings.

J Med Internet Res 2022;24(1):e26779

doi:10.2196/26779

Keywords

Background

In this context, it is urgent to develop interventions that can effectively change PA. Therefore, digital health interventions constitute a new opportunity to take care of patients by involving them in their treatment in a dynamic and interactive way. Gamification is a promising avenue to capitalize on the efficacy of digital interventions. Gamification is defined as the use of game design elements in nongame contexts []. By integrating game mechanisms in interventions that are initially devoid of them, the purpose of gamification is to integrate into daily life the ingredients that make games enjoyable to motivate participants to engage in PA []. The use of motivational affordances created by gamification can influence psychological (eg, motivation, attitude, and enjoyment) and physical outcomes (eg, physical capacities) [] and therefore appears as a potentially powerful technique for behavior change.

By gamifying PA, participants are encouraged to move and walk to play, which tends to make their activities more enjoyable and playful []. Unlike serious games, which refer to the use of a full-fledged video game for educational or health purposes (ie, a video game in its entirety as opposed to selected elements or individual features of a game) [] and require a dedicated time, a location, and implementation [], gamification techniques are relatively open to varying situational modes of engagement [] and concern instead global PA in all aspects of daily life (eg, walking, running, or gardening). Gamification is made possible by mobile technologies and wearable devices that can track and collect daily activities in a continuous and web-based manner. This allows for intervening directly on the lifestyle of individuals without adding material or time constraints for the participants.

However, several literature reviews [,,] have reported inconsistent results concerning the use of gamification in behavioral interventions, with some studies demonstrating positive effects and other studies providing more mixed effects. These reviews also emphasized the lack of high-quality studies and highlighted the need for more rigorous trials to isolate the impact of gamification (ie, randomized controlled trials [RCTs]). Importantly, Koivisto and Hamari [] suggested that the effects of gamification could be smaller when using rigorous experimentation. In sum, these reviews indicate that there is no clear evidence of the effectiveness of gamified interventions. Nevertheless, no meta-analysis has been conducted yet. Quantifying the effect size of gamified interventions and identifying moderators of this effect would provide important information about the effectiveness of such interventions. Moreover, a meta-analysis appears as timely, as there are now enough RCTs to conduct such an analysis.

This Study

This study is the first to quantify the effects of gamified interventions on PA. Beyond the effect during or just after the intervention, we also seek to evaluate the long-term effects to determine the health relevance of these interventions. Indeed, we reasoned that to be considered effective, gamification must sustain its impacts over the long term and offer more than a short-term novelty effect []. The generalizability of gamification to different user populations is also a major issue because it would determine whether gamification can be introduced in health care settings with patients or it is more suited in prevention for healthy audiences.

The objectives of this systematic review and meta-analysis are to answer these research gaps by (1) evaluating the effect of gamified interventions on PA and SB, (2) assessing the long-term or sustained effects of gamified programs, and (3) evaluating the generalizability of gamification across different populations.

Design

This review was conducted according to the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines []. Following recommendations to minimize bias and provide evidence of a priori analysis intentions [], the study was preregistered under the international prospective register of systematic reviews (PROSPERO; registration number: CRD42020186882) and on the Open Science Framework (OSF) []. Moreover, all materials and data are available on the OSF page of the project to facilitate reproducibility and transparency of this review [].

Search Strategy and Information Sources

We conducted a systematic literature search using five electronic databases: PubMed, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials. We combined alternate terms and keywords representing synonyms for the outcomes (PA or SB), intervention (gamification), device (eHealth), and trial (see Table S1 in [-] for an overview of the terms used). The search strategy was reviewed by an academic librarian. All databases were searched individually from January 1, 2010 (2010 being the date of the widespread adoption of the term gamification []), to December 31, 2020, and the research was restricted to English-language texts. The complete search equations for all databases are available in . In addition, we complemented our search with reference harvesting from the included studies and overview articles.

Eligibility Criteria

Studies were eligible for inclusion if they were RCTs and if they met other criteria based on the participants, interventions, comparators, and outcomes framework.

Participants

This review focused on the general population regardless of age, gender, or health status (ie, patients with chronic diseases were also included). We excluded studies involving participants with contraindications to PA or with diseases preventing them from engaging in PA or understanding the principles of the game (intellectual and cognitive impairments).

Intervention

Digital interventions targeting PA or SB and incorporating game elements and gamification techniques, such as points, levels, rewards, leaderboards, narratives, and teams, were of interest. We clearly distinguished between gamification and related constructs, such as serious games. Therefore, we excluded interventions based on active video games (ie, electronic games that allow players to physically play with the images on screen) that are more comparable with serious games than with gamified products.

Comparators

Studies that attempted to compare gamified interventions with control groups without gamification elements in a randomized design were integrated in the review. These groups could be either inactive (nonexposed control group, such as a waiting list) or active (another nongamified intervention).

Outcomes

In this review, we included studies assessing change in total PA or leisure PA (quantity in metabolic equivalent of task [MET] hour per week or MET minute per week or in duration, energy expenditure [METs], moderate to vigorous PA [MVPA], step count, walking time, and active minutes) and change in time spent in SB (total time, leisure time, work, time spent in front of the computer, and time spent in front of television). These outcomes were continuous data either objectively measured (through accelerometers, pedometers, and smartphones) or subjectively measured by self-reported questionnaires. Data measured objectively were always prioritized in the analyses over self-reported questionnaires, which are more susceptible to bias with a potential overestimation of PA [].

In addition, studies were excluded if they came from a review, commentary, or conference abstract; if they included data previously published in another study; if they were not randomized and controlled; if they were not written in English; and if they were published before 2010.

Screening

In total, 2 authors (AM and AC) independently screened the titles and abstracts resulting from the search. Full texts of the potential included studies were checked before inclusion. Disagreements were resolved by discussion or by consulting a third author (MD), and agreement was measured using the κ statistic. A complete list of excluded studies is available on the OSF page of the project.

Data Extraction

In the data collection process, AM and AC extracted data independently and were blinded to each other using a predetermined and tested template. Disagreements were resolved by discussion or consultation with a third author (MD). Extracted data included the results of each study and three types of potential moderators:

Population-level moderators to assess the generalizability of the intervention (population characteristics, age, gender, and pathology).

Intervention-level moderators to better understand gamification mechanisms (theoretical model used to develop the intervention, gamification elements, and modality of the intervention [eg, internet-based, smartphone app, and presence of social incentives]).

Outcome-level moderators (outcomes, measure of PA or SB, and device).

Risk of Bias Assessment

For each eligible study, 2 reviewers (AM and AC) assessed the risk of bias using the purpose-built Cochrane risk of bias tool (Table 8.5 in the Cochrane Handbook for Systematic Reviews of Interventions []), which evaluates 7 domains (sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective outcome reporting, and other risk of bias). A judgment of the potential risk of bias was made from the extracted information and rated as high risk, low risk, or unclear if the related information was not available. These evaluations of bias are reported in the review and included in the analysis, and a measure of agreement with the κ statistic was calculated. After the full assessment, we decided not to present the item blinding of participants and personnel in the review because it was similar for all studies, which were rated as high risk, blinding being unfeasible for this kind of intervention.

Data Synthesis and Statistical Analyses

First, data were synthesized in a qualitative review assessing the key elements of the studies and highlighting intervention differences. This qualitative review integrates all studies that met the eligibility criteria, including those for which we were unable to extract the data.

For the quantitative analysis, means and SDs of continuous PA or SB outcomes from individual studies were compiled when available or estimated using the method by Hozo et al [] when median and IQR were reported. When the necessary data were not available in the original article, we first requested them from the authors. If data could not be obtained, we extracted them from the graphs when available. If this was not possible, we excluded the study from the quantitative analysis.

A global meta-analysis was conducted to obtain a summary effect. In addition, when sufficient data were available (ie, 4 studies or more reporting an outcome), we conducted different meta-analyses for each specific outcome (steps, MVPA, and time in SB) and for the follow-up effect. To address the nonindependence of data caused by study effect, random-effects models [] were preferred to the usual statistical tests. In addition, the Hartung-Knapp-Sidik-Jonkman method was used to reduce the production of false positives inherent to the DerSimonian-Laird method [] and to obtain more robust estimates of variance. Continuous outcomes were analyzed using standardized mean difference (SMD) to account for different measurement instruments or mean difference (MD) when the measurements were close enough. We computed Hedges g [] for effect sizes, which is similar to Cohen d but corrects for small sample bias, which are recurrent in the studies included. Thus, a Hedges g of 0.2 represents a small effect; 0.5, a moderate effect; and 0.8, a large effect []. We computed SMDs for outcome scores after the intervention (presented in the review) and change-from-baseline outcomes. Scores on postintervention effect sizes refer to treatment group results compared with the control group results after interventions. Change-from-baseline score effect sizes were calculated as a comparison between the treatment group pre-post effect size and control group pre-post effect size.

For studies that included multiple outcomes, we kept in the main analysis the primary outcome targeted in the initial article. If none of the PA outcomes reported by a study were the primary ones, we selected the one that was the most relevant from the perspective of the intervention and the original experiment. In designs with multiple time measurements, the assessment that was the most proximal to the end of the intervention was conserved. A time assessment had to be performed >2 weeks after the end of the intervention to be included in the follow-up analysis. When studies included multiple intervention groups with gamification features, they were combined into one group following the formulae recommended by the Cochrane Handbook []. Studies including multiple control groups could be integrated into different subgroup analyses if they compared their gamified intervention to both an active and an inactive control group.

Statistical heterogeneity was tested using forest plots and the I2 statistic, which is the most common metric for measuring the magnitude of between-study heterogeneity and is easily interpretable (0%-40% might not be important, 30%-60% may represent moderate heterogeneity, 50%-90% may represent substantial heterogeneity, and 75%-100% may represent considerable heterogeneity) []. We conducted different influential analyses to address between-study heterogeneity. We first explored the presence of outliers, defined as studies with CIs that do not overlap with the CI of the summary effect. We also performed leave-one-out analyses, which recalculated the summary effect several times, with 1 study omitted each time. Finally, we performed a Baujat plot [], which is a diagnostic plot to detect sources of heterogeneity in the meta-analysis by comparing the contribution of each trial in the pooled effect with the overall Cochran Q test for heterogeneity.

We applied different methods to detect publication bias (funnel plot, Egger regression test [], and Duval and Tweedie trim-and-fill procedure []). In addition, another approach to determine the evidential value of studies included in the analysis is to check the statistical power of individual studies. Therefore, we performed a sunset funnel plot [], which is a funnel plot variant that visualizes the statistical power of each study included in the meta-analysis based on the summary effect size.

Thus, sensitivity analyses were conducted to address studies with a high risk of bias or a strong heterogeneity in the sample or studies identified as outliers. Subgroup analyses were conducted to explore possible sources of heterogeneity and test for population differences. Therefore, we conducted tests for subgroup differences using a random-effects model. In addition, moderation analyses were performed to explore the impact of potential explanatory variables and moderators on the effect size with meta-regressions when sufficient data were available (ie, at least 10 studies for each explanatory variable []). The results were expressed as regression coefficient estimates, 95% CIs, and P values.

For crossover trials, we first checked whether carry-over or period effects were problematic in the original texts of studies. For cluster randomized trials, we checked if the influence of the different clusters was not too important, analyzing the values of the intraclass correlation coefficient in the studies. Then, in the absence of sufficient information in the published articles, we addressed these studies as traditional parallel trials. Nevertheless, this procedure increased the probability of a unit of analysis error. Therefore, we performed sensitivity analyses without clusters and crossover trials to test the robustness of our results.

A summary of the analytical procedure is available in (Figure S1). Analyses were performed on R (The R Project for Statistical Computing) using the dmetar package []. Risk of bias summary and risk of bias graphs were made via the robvis R package [].

Characteristics of Included Studies

We screened the titles and abstracts of 1626 articles, and 51 full-text articles were assessed for eligibility according to the inclusion criteria. Finally, 18 articles [-,,] were included in the qualitative analysis and 16 were included in the meta-analysis (). The κ value of agreement for the screening phase was 0.64 between the 2 authors, reflecting good agreement [].

‎
Figure 1. PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis) flowchart of the literature search and screening process. PA: physical activity; RCT: randomized controlled trial.
View this figure

A total of 6 trials were conducted in Europe; 4 in the United States; 3 in Australia and New Zealand; 3 in Canada; and 2 in Asia. Studies were published between 2014 and 2020, with 39% (7/18) published after 2018.

Table 1. Characteristics of included studies.

Study

Participants

Intervention

Theory

PA a outcomes

Adolescents, n=224 (aged 12-14 years; 47% male participants)

The "StepSmart Challenge" was a web-based intervention that used gamification strategies to encourage and support PA behavior change; duration: 22 weeks

SDT c

Daily step count and MVPA d (min/day) objectively measured (Actigraph GT3x)

Adult workers in an automobile manufacture, n=144 (65% male participants)

"Healingo Fit" had the objective to promote low levels of PA using a tracking-based approach measuring PA with a Fitbit pedometer and a gamified intervention accessible by desktop and mobile devices; duration: 6 weeks

SCT e, TPB f, and health action process approach

Self-reported VPA g, MPA h, and minutes walked (min/week; IPAQ i)

Adolescents, n=35 (mean age 15.7 years, SD 1.2 years; 45% male participants; BMI 22.85)

"Zombies, run! 5K Training app" was a fully automated training program designed to improve fitness, combined with an immersing and fun story; duration: 8 weeks

SDT

MVPA, VPA, MPA, and LPA j and sedentary time (min/day) objectively measured (Actigraph GT3x) and self-reported PA (PAQ-A k)

Garde et al []

Adolescents, n=47 (mean age 10.3 years, SD 1.9 years; 34% male participants; BMI z- score 0.35)

"MobileKids Monster Manor" was a mobile exergame synchronized with an external activity monitor. The overall goal was to complete the story with PA and steps; duration: 1 week

SDT

Daily step count and active min/day objectively measured (Tractivity)

Adolescents, n=56 (mean age 11.3 years, SD 1.2 years; 62% male participants; BMI z- score 0.28)

"MobileKids Monster Manor" was a mobile exergame synchronized with an external activity monitor. The overall goal was to complete the story with PA and steps; duration: 1 week

SDT

Daily step count and active min/day objectively measured (Tractivity)

Garde et al []

Adolescents, n=37 (mean age 10.6 years, SD 0.5 years; 43% male participants; BMI z- score 0.21)

"MobileKids Monster Manor" was a mobile exergame synchronized with an external activity monitor. The overall goal was to complete the story with PA and steps; duration: 2 weeks

SDT

Daily step count and active min/day objectively measured (Tractivity)

Adults, n=20 (mean age 31 years, SD 3 years; 30% male participants; BMI 21.5)

Participants wore an activity monitor with computerized game functions, such as a story, a character, and objectives; duration: 6 weeks

- l

Daily step count and MVPA (metabolic equivalent of tasks hour/day) objectively measured (Lifecorder EX)

a PA: physical activity.

b The studies included in the meta-analysis.

c SDT: self-determination theory.

d MVPA: moderate to vigorous physical activity.

e SCT: sociocognitive theory.

f TPB: theory of planned behavior.

g VPA: vigorous physical activity.

h MPA: moderate physical activity.

i IPAQ: International Physical Activity Questionnaire.

j LPA: light physical activity.

k PAQ-A: Physical Activity Questionnaire for Adolescents.

l No theory mentioned.

Studies comparing gamified interventions with active control groups used a similar intervention without game elements (ie, an equivalent nongamified app [-] or a self-monitoring intervention with wearables or activity monitors [-,,]).

A total of 2 studies [,] were excluded from the meta-analysis and were only integrated in the qualitative review because we were unable to extract their results.

Risk of Bias

The 2 authors extracted the risk of bias data with a κ coefficient of 0.79, which is synonymous with excellent agreement []. [-] and present the authors' judgments about each risk of bias item presented as percentages across all included studies in the meta-analysis and an overview of the different biases for each study. Overall, 1 study [] was rated as high risk for sequence generation because assignments were based on recruitment order. Therefore, this study was also at a high risk for allocation concealment. A total of 3 studies [,,] were at high risk of bias for the blinding of outcome assessment item because they measured PA using only self-reports. In total, 5 studies [,,,,] were at high risk of bias for the incomplete outcome data item because they reported high dropout rates and did not include intention-to-threat analyses and 5 studies [,,,,] were rated at unclear risk for the selective outcome reporting item because they had not been preregistered or published in a protocol-study. Finally, 2 studies had other high risks of bias. The first one [] was a crossover trial conducted without a washout condition, and the other one [] was a cluster randomized trial with no control of clustering, no consideration of the clustering in the statistical analysis, and no test of baseline differences among groups.

Summary Effect

Overall, the SMD after the intervention for all PA outcomes (MVPA, daily step count, number of active minutes, and walking time) was a Hedges g of 0.43 (95% CI 0.03-0.82; I2=86%; ), representing a statistically significant small to medium effect. Similarly, we found a statistically significant SMD effect of a Hedges g of 0.38 (95% CI 0.07-0.69; I2=79%) for pre-post change scores.

Only 3 studies [,,] assessed sedentary time as an outcome. Owing to this small sample size, the meta-analysis was not performed on this outcome.

‎
Figure 2. Forest plot for the effect of gamification versus control on postintervention physical activity outcomes (moderate to vigorous physical activity, daily step count, number of active minutes, and walking time). Tau-square, chi-square, and I² measures of between-study heterogeneity [-]. IV: inverse variance.
View this figure

Outliers and Influential Analyses

In the first analysis, substantial statistical heterogeneity was observed. To address between-study heterogeneity, we first looked for the presence of outliers. A total of 3 studies were considered as outliers [,,], and after removing them, we still obtained a significant effect of a Hedges g of 0.34 (95% CI 0.17-0.51) with moderate heterogeneity ( I2=58%).

Leave-one-out analyses showed that sequential removal of each study did not have an important impact on the general effect size, with effect sizes ranging from a Hedges g of 0.33 (95% CI 0.00-0.66; I2=84%) to a Hedges g of 0.48 (95% CI 0.13-0.83; I2=78%; Figure S2 in ).

The Baujat plot (Figure S3 in ) shows that 4 studies explained more heterogeneity than the others, more specifically, the study by Corepal et al [] with a heterogeneity contribution of 40.13 and an effect size influence of 3.27.

Therefore, we excluded studies with a high or unclear risk of bias. After doing so, the effect was not significant (Hedges g=0.33, 95% CI -0.16 to 0.81; I2=78%).

The inclusion of crossover and cluster randomized trials in the meta-analysis may lead to biases. Thus, we excluded these designs from the analysis and obtained a statistically significant effect of g=0.49 (95% CI 0.05-0.92; I2=67%).

Finally, we decided to exclude articles by Corepal et al [] and Nishiwaki et al [] in the sensitivity analyses (which have been repeated afterward for each analysis) considering their influence on the pooled result, their contribution to overall heterogeneity, and their huge risk of bias (no control of clustering, no statistical consideration of clustering, and no test of baseline differences among groups in the study by Corepal et al [] and no washout period and very low power for the study by Nishiwaki et al []). After doing so, we obtained a statistically significant effect of a Hedges g of 0.42 (95% CI 0.14-0.69; I2=74%).

Subgroup Analyses

We found no statistical differences in the effects between studies with participants with chronic diseases or healthy participants (Cochran Q=0.73; P=.39), between adults and adolescents (Cochran Q=0.26; P=.61), between studies with objective (devices) or self-reported PA outcomes (Cochran Q=0.23; P=.63), between active or inactive control groups (Cochran Q=0.01; P=.92), and between short- and long-term interventions (less or more than 12 weeks; Cochran Q=0.60; P=.44).

When performing the sensitivity analysis, there was a statistically significant effect of intervention on PA in adults (Hedges g=0.36, 95% CI 0.03-0.69; I2=71%; Figure S4 in ), on healthy people (Hedges g=0.35, 95% CI 0.15-0.55; I2=63%; Figure S5 in ), when the PA measure was objective (Hedges g=0.45, 95% CI 0.08-0.82; I2=80%; Figure S6 in ), when the PA measure was self-reported (Hedges g=0.24, 95% CI 0.08-0.39; I2=0%; Figure S6 in ), and for short interventions of <12 weeks (equivalent to a 3-month program; Hedges g=0.44, 95% CI 0.19-0.69; I2=16%; Figure S7 in ).

Moreover, subgroup analyses allowed us to examine the effect of gamified interventions when compared with inactive control groups and active control groups. After sensitivity analyses, we found a statistically significant effect of gamified interventions, both when compared with inactive control groups (Hedges g=0.58, 95% CI 0.08-1.07; I2=81%; Figure S8 in ) and when compared with active control groups (Hedges g=0.23, 95% CI 0.05-0.41; I2=37%; Figure S8 in ).

Meta-Regressions

The age of participants ( β=.01, 95% CI -0.02 to 0.04; P=.39), their gender ( β=.01, 95% CI -0.01 to 0.02; P=.47), their BMI ( β=.04, 95% CI -0.16 to 0.09; P=.53), the duration of the intervention ( β=-.01, 95% CI -0.06 to 0.04; P=.74), or the number of game mechanics included in the intervention ( β=.01, 95% CI -0.17 to 0.19; P=.91) were not statistically significantly associated with an increase in PA.

Lack of data precluded further meta-regressions, such as comparisons of leisure PA, or test of moderators, such as the impact of social incentives or the theoretical model used to develop the intervention.

Publication Bias

First, an inspection of the funnel plot showed that the effect sizes of individual studies were relatively symmetrically distributed around the pooled effect size. This observation was supported by the Egger test of the intercept, which indicated no asymmetry in the funnel plot (b 0=1.38, 95% CI -0.83 to 4.77; P=.19). We then applied a bias-correction technique, the trim-and-fill method, which indicated that 3 studies were missing at the bottom left of the funnel plot to obtain a full symmetry. After imputing the effect sizes corresponding of these missing studies to obtain a totally symmetrical funnel plot (), the bias corrected summary effect was of a Hedges g of 0.24 (95% CI -0.24 to 0.73).

Finally, the sunset funnel plot () showed significant differences in power among studies, with some characterized by very low statistical power (7 studies under 45% power and 4 studies under 18%). The median power of all the tests was 63%.

‎
Figure 3. Funnel plot after trim-and-fill bias correction. A filled circle represents an included study, and an empty circle represents a missing study.
View this figure

‎
Figure 4. Power-enhanced funnel plot. White circles represent included studies. δ: true effect size; medpower: the median power of all tests; d33%: effect size needed for achieving 33% of median power; d66%: effect size needed for achieving 66% of median power; E: expected number of positive studies; O: observed number of positive studies; pTES: test of excess significance P value.
View this figure

Secondary Analyses

Follow-up

There was no statistically significant effect of gamified interventions on total PA (MVPA and daily step count) after follow-up periods with an SMD of a Hedges g of 0.09 (95% CI -0.07 to 0.26; I2=21%). When we performed the sensitivity analysis, gamification significantly increased PA (MVPA and daily step count) at follow-up (from 12 to 24 weeks after the end of the intervention; g=0.15, 95% CI 0.07-0.23; I2=0%; ).

Steps

We found no statistically significant effect of gamified interventions on step outcomes with an SMD of a Hedges g of 0.53 (95% CI -0.09 to 1.15; I2=89%), but a significant improvement in the number of daily steps with an MD of +1420.57 steps per day (95% CI 435.41-2405.73; I2=95%) was observed. When excluding the 2 studies in the sensitivity analysis, we obtained a statistically significant effect of gamification on daily steps of a Hedges g of 0.49 (95% CI 0.05-0.93; I2=75%; ) and a statistically significant MD of +1609.56 steps per day (95% CI 372.39-2846.73; I2=86%; ).

Moderate to Vigorous PA

There was no statistically significant effect of gamification on MVPA with an SMD of a Hedges g of 0.09 (95% CI -0.57 to 0.74; I2=93%). There was no statistically significant effect of a Hedges g of 0.31 (95% CI -0.19 to 0.80; I2=82%) in the sensitivity analysis ().

Summary of Findings (Grading of Recommendations Assessment, Development, and Evaluation)

The quality of evidence (grading of recommendations assessment, development, and evaluation, [GRADE]) in the included studies after sensitivity analyses for short-term PA, long-term PA, MVPA, steps, and daily steps was scored from high to low (). The quality was downgraded for some outcomes because of high heterogeneity, high risk of bias, or imprecision owing to large CIs. Summaries of the various meta-analysis conducted in this review both on postintervention scores and pre-post intervention change scores are presented in (Figures S9-S11).

Table 2. Summary of findings.

Outcome

Number of participants (number of studies)

Standardized mean difference or mean difference (95% CI)

Quality of evidence (grading of recommendations assessment, development, and evaluation)

General PA a

2197 (14)

0.42 (0.14 to 0.69)

Low b,c,d

General PA (in comparison with active control groups)

1485 (7)

0.23 (0.05 to 0.41)

High

Long-term PA (follow-up)

1306 (5)

0.15 (0.07 to 0.23)

High

MVPA e

739 (4)

0.31 (-0.19 to 0.80)

Low b,c,d

Steps

1438 (9)

0.49 (0.05 to 0.93)

Low b,c,d

Daily steps

1235 (7)

1609.56 (372.39 to 2846.73)

Moderate b,d

a PA: physical activity.

b Downgraded because of high heterogeneity.

c Downgraded because of risks of bias.

d Downgraded because of imprecision (large CIs).

e MVPA: moderate to vigorous physical activity.

Principal Findings

Summary Effect

This meta-analysis of RCTs, including 16 studies and 2407 participants, revealed a statistically significant effect of gamified interventions, on average, of 12 weeks on total PA (Hedges g=0.42, 95% CI 0.14-0.69 after sensitivity analyses). This effect was small to medium, suggesting the effectiveness of gamified interventions in promoting PA in both healthy participants and participants with chronic diseases. This significant effect was robust, as it persisted even after the different influence analyses were performed. Moreover, the effect was statistically significant both for objective measures of PA (Hedges g=0.45, 95% CI 0.08-0.82) and self-reported measures (Hedges g=0.24, 95% CI 0.08-0.39) after sensitivity analyses. Unsurprisingly, subgroup analyses revealed after sensitivity analyses that the effect of gamified interventions is greater when compared with inactive control groups (such as waiting lists) than when compared with active control groups benefiting from a nongamified intervention (Hedges g=0.58, 95% CI 0.08-1.07 vs Hedges g=0.23, 95% CI 0.05-0.41). Nevertheless, these effects were both statistically significant. This suggests that gamified interventions are not only efficient in changing behavior but also, to a lesser extent, effective compared with equivalent nongamified PA interventions (such as smartphone apps or self-monitoring interventions). These results are important considering the assets of gamification, which has the advantages of (1) reorganizing existing activity rather than adding additional demands to people's lives [], (2) being easily implemented in natural contexts, and (3) having a broad accessibility through technology and advancing sensors, permitting to address a large population.

Long-term Effect

When we analyzed the long-term effect of these interventions based on the follow-up measures of PA, carried out from 12 to 24 weeks (mean 14.4 weeks) after the end of the intervention, we found a statistically significant very small to small effect size of a Hedges g of 0.15 (95% CI 0.07-0.23) after sensitivity analyses. These results indicate that the effect of gamification persists after the end of the program, suggesting that it is not just a novelty effect due to the playful nature of gamification. However, this long-term effect was weaker and decreased with time after the end of the intervention.

Generalizability of Gamified Interventions

The absence of subgroup differences or effects of age, gender, and BMI on the pooled effect suggests a good generalizability of gamified interventions, which can be used for several types of populations. Thus, gamification may not only be efficient in young healthy individuals but can also target any kind of population regardless of their age or health status.

In sum, gamified interventions appear as an efficient tool to improve the PA of various populations, with moderate superiority over other similar interventions, such as mobile health monitoring apps, and a moderate sustainability of the effect after the intervention. Nevertheless, if many PA interventions increase PA levels in the short term, translating these temporary changes into long-term PA participation continues to be a challenge for PA research []. With that in mind, the potential of gamification for PA increases in the long term, even minimal, is particularly important and promising in the area of PA interventions.

Additional Findings

Effect of Gamification on the Step Count

If the overall effect of gamified interventions on PA is positive, they increase the step count more than MVPA. Indeed, after sensitivity analyses, on the one hand, the meta-analyses revealed a statistically significant effect (Hedges g=0.49, 95% CI 0.05-0.93) of gamification for steps outcomes, with a statistically significant improvement of 1609.56 steps per day (95% CI 372.39-2846.73) for participants benefiting from gamified intervention versus those in the control group. On the other hand, no statistically significant effect of gamified interventions on MVPA was found (Hedges g=0.31, 95% CI -0.19 to 0.80). This can be explained by the game metrics and mechanics of the interventions included in the review, which are mainly focused on the step count of participants. Few interventions directly targeted MVPA. In the included studies, only 2 interventions [,] integrated multi-PA intensity goals and mechanics, notably with physical exercises or running sessions in the game. In other words, participants played most of the time with their number of steps and had to generally walk more to make points and play the game. This results in an increase in walking time but not necessarily in more intense PA.

Duration of Intervention

Our meta-regression analysis did not find an association between the observed effect of gamification on PA and intervention length. However, although no statistically significant effect of gamification for an intervention length of ≥12 weeks was found (Hedges g=0.41, 95% CI -0.19 to 1.01), the meta-analysis revealed a statistically significant effect of gamified interventions of <12 weeks on global PA (Hedges g=0.44, 95% CI 0.19-0.69). According to a previous meta-analysis that reported significant positive effects of smartphone apps on PA only when used over a short-term period of <3 months [], these results suggest that a condensed intervention could benefit more than a longer one, which could become redundant, boring, and exhausting for participants in the long run.

Statistical Heterogeneity

The meta-analysis also revealed considerable statistical heterogeneity. This heterogeneity may be explained by differences in study quality, diversity of designs, and variations in study populations. Despite several subgroup analyses, we cannot rule out that these subgroups and characteristics may not explain all the variance of the interventions. Indeed, demographic data often do not fully explain the differences in the effectiveness of interventions [,], and more precise sociopsychological variables such as personality traits or motivational factors could explain this poorly understood significant heterogeneity. Moreover, the risk of bias analysis and the sunset funnel plot showed substantial differences in the quality of the included studies that can influence the heterogeneity. Finally, various trial designs were used (ie, parallel RCT, cluster RCT, and crossover RCT) that can also contribute to the overall statistical heterogeneity.

Better Understanding Gamification Mechanisms

This meta-analysis is informative regarding the effectiveness of gamified interventions. In view of the observed heterogeneity, the next step will be to investigate its causes from an interventional and theoretical perspective. Gamified interventions involve multiple interacting elements, and it is crucial to estimate the weight of each element in the behavior change process and how they interact with each other. Is it game mechanics, the implementation of behavior change techniques, or the presence of social interactions that make gamification effective? Unfortunately, the small number of studies included in the meta-analysis impeded us from conducting in-depth moderation analyses to answer this question. To better understand these relations, it is essential that both the development and assessment of gamified interventions be central, transparent, evidence-based, context-aware, and research-oriented []. Moreover, if theoretical psychological models are often mentioned in the introduction of articles included in the review (), few have investigated the psychological mechanisms of their interventions in the field [,,]. Future studies should explicitly discuss motivational theory and systematically test the effect of gamification on psychological outcomes known to be involved in behavior change (eg, self-efficacy, attitudes, and intention) to better understand its mechanisms. The consideration of personality traits and psychological variables to determine behavioral phenotypes [] is a promising way to evaluate participant's responses to the interventions.

Perspectives for Future Research and Implications for Practice

The findings from this meta-analysis allow us to draw and discuss future work concerning the gamification of PA and SB. First, future trials should be conducted with more adequately powered sample sizes and should be strictly multiple arms-RCTs to isolate the effects of gamification elements and better understand gamification mechanisms. Second, the long-term effects are currently the main challenge of health interventions. Thus, it is essential to investigate the evolution of the effects of gamified interventions over time. Therefore, there is a need for more long-term follow-up measurements. In addition, the potentialities of digital technologies and their capacity to collect a large amount of real-world data could be used to assess the evolution patterns of the effect, allowing the detailed identification of its sustainability and evolvement or even make forecasts via time series analyses. Third, to our knowledge, only 1 team of researchers worked on a gamified intervention targeting SB by introducing sedentary breaks as a gaming part []. Following this line of research, it could be interesting to develop gamified interventions affording participants to take more sedentary breaks. Finally, the cost-effectiveness ratio of gamified interventions may be better than that of many current interventions, considering the ease of implementation and generalizability of gamification. However, this assertion will have to be tested in future trials, including economic analyses.

In light of our results, gamified interventions appear to be a promising avenue to promote PA in different populations both in prevention in healthy people and in the treatment of chronic diseases. Gamified interventions have many benefits for participants with chronic diseases, such as empowerment of participants by improving their self-management skills, an effect across broad audiences enabling to target different types of pathologies, and an everyday life fit and easy implementation. Similar to other digital health processes, gamification makes it possible to address more patients, especially those who are isolated from health care facilities. Importantly, gamified interventions are especially pertinent during a health pandemic, such as the COVID-19 outbreak, in which PAs and social interactions are restricted because of lockdown or teleworking and where structured PA possibilities are limited both indoors and outdoors. Gamifying walking and daily activities is, in this context, a great way to improve PA and limit SB of individuals in addition to providing social interaction among players. In the meantime, the face-to-face management of chronic diseases is usually suspended during the pandemic, which underlines the importance of offering remote supervision of PA.

Nevertheless, in view of the weaker postintervention effect, this study suggests that a one-shot intervention is not sufficient. A more interesting design would be to address multiple gamification doses during or after the course of treatment to obtain a sustainable implementation of the PA behavior. This configuration would also provide an ideal duration of intervention to avoid exhausting the participants with gamified interventions for >12 weeks.

Strengths and Limitations

To our knowledge, this is the first meta-analysis to quantitatively evaluate the effects of gamification on PA. This review has several other strengths. First, we conducted a comprehensive search strategy using multiple databases in collaboration with an academic librarian. Second, all stages of the review (screening and data extraction) were independently realized by 2 reviewers. Finally, various novel publication bias analyses and influence analyses were conducted in parallel with different subgroup analyses and meta-regressions.

However, some limitations of this work must be mentioned. Overall, the meta-analysis included a small number of studies, and some articles were feasibility or pilot trials. Therefore, several trials included small sample sizes and were highly underpowered. Some studies were conducted with a high risk of bias. One of the main limitations of this work is the impossibility of demonstrating that the effect of gamified interventions is led by gamification itself given the lack of research examining this question. Finally, in the main analysis, we included diverse PA outcomes evaluating similar constructs but which are slightly different in practice. Moreover, not all included outcomes were objectively measured. As the field matures and new trials are published, an update of this work will be important to confirm these preliminary results.

Conclusions

To conclude, gamified interventions appear to be a promising avenue for promoting PA in various populations. Influencing primarily the number of daily steps of the participants, gamification is an interesting way to improve daily PA and appears more efficient than equivalent nongamified interventions, such as mobile health apps. However, if the effect of gamification persists during follow-up, suggesting that gamified interventions are more than a novelty effect, this effect decreases with time with a smaller long-term effect. The integration of gamification in more global health care interventions could be a way to address this limited sustainability. Future rigorous trials are required to explore these perspectives.

# data[].attributes.attachment_link.title
Evaluating the Effectiveness of Gamification on Physical Activity: Systematic Review and Meta-analysis of Randomized Controlled Trials

# data[].attributes.attachment_link.description
Background: Gamification refers to the use of game elements in nongame contexts. The use of gamification to change behaviors and promote physical activity (PA) is a promising avenue for tackling the global physical inactivity pandemic and the current prevalence of chronic diseases.

# data[].attributes.attachment_caption
 The Happify App helps users improve their emotional well-being.

# data[].attributes.headline
Telehealth Helping Parents with Autistic Children

# data[].attributes.subject
Telehealth Helping Parents with Autistic Children

# data[].attributes.body
The researchers saw many benefits of this program as stated in their abstract - "The findings showed that participation in the program promotes parents' empowerment and reduce parental stress and a general improvement in the parents' ability to stimulate children's learning. Moreover, the study reveals an effect of parents' age in mediating the relationship among the parents' ability, stress, and empowerment levels. These results suggest that an early intervention telehealth program may help parents become aware of how to benefit from interactions with their children, promote their empowerment, and reduce parenting stress" (Gentile et al., 2022).

Gentile, M., Messineo, L., La Guardia, D., Arrigo, M., Città, G., Ayala, A., Cusimano, G., Martines, P., Mendolia, G., & Allegra, M. (2022). A Parent-Mediated Telehealth Program for Children with Autism Spectrum Disorder: Promoting Parents’ Ability to Stimulate the Children’s Learning, Reduce Parenting Stress, and Boost Their Sense of Parenting Empowerment. Journal of Autism & Developmental Disorders, 52(12), 5285–5300. https://doi-org.ezproxy.simmons.edu/10.1007/s10803-022-05482-6

# data[].attributes.attachment_link.readable_text
Working on a manuscript?

Avoid the most common mistakes and prepare your manuscript for journal editors.

Learn more

Introduction

Within the field of health care, the use of information and communication technologies (ICT)-in various ways-has become increasingly widespread. The introduction of ICT in health care has contributed to the conception of new terms referring to the different kinds of interventions. These interventions directly or indirectly benefit the people affected by short or long-term health problems or diseases. Among these interventions, "telehealth" has had considerable success. In the literature, "telehealth" is defined as the practice of continuously and/or remotely delivering health-related services through electronic technologies (Marchi et al., 2021)-treatment services for different diseases, training services for health professionals, exchange of health-related information services, online instruction services for interventionists, connection services for caregivers and remote experts (Symon, 2001; Neely et al., 2017).

Caregivers, especially parents, are one of the groups that has benefited the most from the introduction of telehealth in healthcare. Telehealth has come to benefit caregivers in the following ways: (a) learning to reflect on situations; (b) gaining confidence to try new resolution strategies; (c) learning to accept and deal with the condition of the people they are assisting more consciously (Wallisch et al., 2019).

As the use of telehealth in the care of people with autism spectrum disorder (ASD) is characterized by replacing or complementing the existing healthcare and training practices for autistic individuals and their caregivers, it has numerous advantages (Ingersoll & Berger, 2015). The research demonstrated the efficacy of using telehealth for providing various services to individuals with ASD and their families in different areas and fields of treatment (Boisvert et al., 2010; Knutsen et al., 2016; Lindgren et al., 2016). As has been highlighted in the critical review of the last decade, telehealth methods permit the provision of early intervention, communication and language therapy, and diagnostic assessment (Ellison et al., 2021). In a systematic review, the current trends in telehealth applications to deliver social communications interventions were investigated (Simacek et al., 2020).

The authors have found some of the studies that highlight the potential benefits of using parent-mediated telehealth interventions to improve social communication skills, such as language skills and conversational functions. Furthermore, Simacek et al. ( 2020) have shown that, in the use of telehealth for the healthcare of autistic subjects, the ability to combine synchronous and asynchronous telehealth sessions is crucial. On the one hand, synchronous telehealth enables one to set up "live" interventions within a home context. It allows a live connection between a "tele-provider" (clinician, professional, educator) and those who are the beneficiaries of the intervention (individuals with ASD, caregivers).

On the other hand, asynchronous telehealth allows the caregivers of individuals with ASD to let a "tele-provider" manage the collection and transmission of audio/video data, who can view the data at suitable times and respond to the caregivers' needs in the best possible manner.

In the specific context of parent-mediated telehealth pathways, many studies have highlighted the effectiveness of telehealth as a tool to improve not only autistic individuals but also their parents (Solomon & Soares, 2020; Heitzman-Powell et al., 2014). For nearly 20 years, the role and involvement of parents have been officially and repeatedly recognized as a key component in the early treatment of ASD and in improving the quality of life of families of individuals with ASD (Council, 2001; Keen et al., 2010; Tonge et al., 2006).

There are studies that have implemented telemedicine to provide training to parents and improve their engagement. These studies have found the children of these parents to show acceptability and improvement in social communication skills (Bearss et al., 2017; Ingersoll & Berger, 2015; Ingersoll et al., 2016; Kuravackel et al., 2017; Pickard et al., 2016; Vismara et al., 2013, 2018; Wainer & Ingersoll, 2014). Over time, parent coaching and the monitoring, implementation, and evaluation of parent intervention have been accredited as the best evidence-based practice (Ferguson et al., 2018).

Various reviews have examined the use of telehealth in creating, providing, and managing educational pathways for caregivers and parents of children with ASD (Boisvert et al., 2010; Wainer & Ingersoll, 2014). Within this context, while some studies have investigated parental knowledge acquisition using telehealth (Peter et al., 2014), certain reviews have focused on the role of telehealth in parents' learning and skill acquisition during therapeutic interventions (Hamad et al., 2010). Further, some studies that addressed both the issues (Pickard et al., 2016) found the parents engaging in educational and coaching pathways delivered through telehealth to show enhanced knowledge and intervention skills and be aware agents of therapeutic interventions (Parsons et al., 2017).

According to Baggett et al. ( 2009), this phenomenon occurs because telehealth allows the integration of the principles of instructional design and adult learning. The primary effect of this integration is the increase in parents' understanding, management, and implementation of effective interventions for their children with ASD (Vismara et al., 2018). These interventions are more effective if the entire parent-mediated intervention and parent training pathways are therapist-assisted.

According to Ingersoll et al. ( 2016), therapist-assisted programs seem to be more effective than self-directed programs. Parents can improve in many ways. Their perception of their children with ASD increases positively because online interactive communication facilitates a close connection between parents and healthcare professionals (Hermaszewska & Sin, 2021) and enables parents to focus on important psychoeducation principles. Therefore, the educational use of telehealth pathways in parent-mediated interventions for individuals with ASD has an incremental effect on parental competencies (knowledge and intervention skills) and empowerment and decreases the effect on parental stress (Ingersoll et al., 2016; Hermaszewska & Sin, 2021; Kuhn & Carter, 2006).

Many studies have shown the importance of empowerment. Programs that focus on the parent-child relationship and involve parents learning new skills have been found to have the largest effects on child behavior and parental behavior and skills (Kaminski et al., 2008) and the greatest effects on reducing parental stress (Kuhn & Carter, 2006). In the specific case of empowerment, the educational dimension inherent in distance learning processes delivered via parent-mediated telehealth pathways has been explicitly highlighted by a few studies (Allegra et al., 2019; Ingersoll et al., 2016; Wallisch et al., 2019). While engaged in the learning experience, parents learn to reflect on (and discuss) the modalities and strategies of action that have helped them. They are more likely to discuss how they have found solutions to problematic situations involving both professionals and their peers. More aware of their own behavior and that of their children, they gain the necessary confidence to try out new coping strategies. They also gain a more positive perception of their family situation (Wallisch et al., 2019).

Regarding parental stress, teaching parents how to intervene in different stressful situations appropriately and conscientiously makes them aware of the tools that are required to deal with these situations and improves their psychological well-being. The contribution of telehealth as a coaching tool has taken on a central role in recent years (as suggested by Ingersoll et al. ( 2016)). A study compared the two approaches adopted for conducting a telehealth-based parent-mediated intervention for young children with ASD (one self-directed and one therapist-assisted). Here, the constant and more easily accessible support offered by the telehealth tool allowed to improve parental self-efficacy and reduce parental stress in the cases of both approaches (Ingersoll et al., 2016). Telehealth (in general) and telehealth-based and parent-mediated interventions in ASD (specifically) have multiple effects on parents. The various components are reciprocally connected and can potentially improve parents' skills (intervention skills and knowledge) and level of empowerment and decrease their stress levels.

To the best of our knowledge, there are only a few studies that have investigated in detail how empowerment and stress are related to the improvement in parents' skills in a telehealth context for children with ASD (Ingersoll et al., 2016). Thus, this study aims to examine this relationship based on the following research questions/hypotheses.

This study examines the effects of the ATHENA telehealth program on the parents of children with ASD (Allegra et al., 2019). ATHENA is a training-and-care telehealth program that provides therapist-assisted and parent-mediated interventions for children with ASD. It is a 6-month program designed to provide remote assistance for parent-mediated early intervention services. It started in the outpatient clinic and quickly transitioned to in the child's home with both synchronous and asynchronous sessions as the intervention program progressed. Parent coaching and the implementation, monitoring, and evaluation of the parent-child-relationship play a key role. To be precise, it focuses on the changes in parents' levels of empowerment, stress levels, and their ability to stimulate their children's learning during treatment sessions. We hypothesize there to be an improvement in the parents' teaching ability, a decrease in their stress levels, and an increase in their sense of empowerment during the 6-month program.

Further, we examine if and how the parents' abilities may be related to their levels of stress and empowerment. In particular, the aim is to verify if the expected improvements in skills correspond to a decrease in stress and an increase in empowerment. Finally, to improve the educational model and make it more effective in various family conditions, we test if the parents' age and the initial level of their children's observed symptoms impact these relationships.

The rest of the paper is organized as follows. Section ' Methods' describes the measures used in this study, the study sample, and the intervention structure, both in terms of the services made available to parents and the scheduling of activities. Section ' Results' illustrates the results, first regarding the effects on the individual dimensions and then examining their relations. Finally, Section ' Discussion and Conclusion' reports the significant findings of the study.

Methods

The ATHENA program is a 6-month program designed to provide remote assistance for parent-mediated early intervention services. Over this period, the parent moves from being a passive observer of the therapist-child session to becoming the child's primary teaching partner while sessions with the therapist are phased out.

In this section, we present the measures employed, the digital environment designed to support the telehealth program, the sample description, and the intervention structure in terms of services and its time development.

Measures

Autism Diagnostic Observation Schedule-2

The second edition of the Autism Diagnostic Observation Schedule (ADOS-2) (Lord et al., 2012) was used during the diagnostic evaluation. The ADOS-2 is a play and activity based semi-structured observation of social behaviors, communication, and restricted/repetitive behaviors. It consists of activities that elicit behaviors related to the diagnosis of ASD. It comprises five different modules, each of which is appropriate for a specific child based on their chronological age and expressive language level. Calibrated severity scores (Esler et al., 2015; Gotham et al., 2009) were used to measure ASD severity at the beginning and end of treatment. The ADOS calibrated severity scores measure ASD severity based on the ADOS raw totals, with relative independence from verbal level and age.

Autism Diagnostic Interview, Revised

The Autism Diagnostic Interview, Revised (ADI-R) (Lord et al., 1994) was used during the diagnostic evaluation. The new ADI-R algorithms for toddlers and young preschoolers were used (Kim & Lord, 2011). The ADI-R is a semi-structured parent or caregiver interview about developmental history and current functioning. It has 93 items. It focuses on three functional domains: (a) language/communication, (b) reciprocal social interactions, and (c) restricted, repetitive, and stereotyped behaviors and interests.

Vineland Adaptive Behavior Scale

The Vineland Adaptive Behavior Scale (VABS) (Sparrow et al., 2005) is an assessment tool used to diagnose and assess the special needs of individuals. It gathers information pertaining to adaptive behaviors in the areas of socialization, communication, daily living, and basic motor skills via semi-structured interviews conducted with the children's parents or caregivers.

Parenting Stress Index, Fourth Edition

The level of perceived parental stress was assessed using the shortened form of the Parenting Stress Index, Fourth Edition (PSI-4) (Abidin, 2012). It is a test formulated to explore the identification of stressors in parent-child relationships. It measures the specific problematic characteristics of the relationship between children (aged between 1 month and 12 years) and their parents. These problematic characteristics can alter the normal development of children, resulting in them facing issues such as emotional and/or behavioral disorders, and parents may risk perceiving their role in a dysfunctional manner. The instrument is based on the assumption that parental stress can originate from multiple sources and results from a combination of personal characteristics and inherent features of the parental role.

The test provides information that can be used to develop specific interventions and evaluate the outcome of the treatments. Parents respond to the questions on a scale ranging from 1 (strongly disagree) to 5 (strongly agree). The shortened form of PSI-4 is composed of 36 items, which are divided into three subscales. The Parental Distress subscale (12 items) explores the level of distress that parents feel in relation to their specific parental role. The Parent-Child Dysfunctional Interaction subscale (12 items) is focused on the parents' expectations of their interactions with their children. The Difficult Child subscale (12 items) addresses the fundamental characteristics of children's behavior that make them easy or difficult to handle.

Family Empowerment Scale

The parents' sense of empowerment was assessed by using the Family Empowerment Scale (FES) (Koren et al., 1992). FES is a widely self-administered survey that evaluates the parents' sense of empowerment at three levels: family, service system, and community. It was developed to measure these components of empowerment in families that raise children with emotional, behavioral, or mental disabilities. It is a 34-item rating scale (1 = very untrue; 5 = very true). It is used in many contexts and instances of different childhood disabilities such as ASD (e.g. Weiss et al., 2013). A high score in each subscale reflects a considerable sense of empowerment in three areas. Before administering it, we translated the FES into Italian with the permission of the authors. We used only the family subscale of the survey because it was the type of empowerment that was the most likely to improve during the treatment program. It investigates the parents' sense of self-efficacy in managing daily concerns regarding taking care of children with ASD and consists of 12 items.

Observation Tool

To evaluate the parents' ability to stimulate their children's learning during the recorded video sessions, an observational tool like the Early Start Denver Model (ESDM) Parent Fidelity Tool (Rogers & Dawson, 2010) was created.

Ten types of parental behaviors were selected, and these were to be scored by the therapists on a 10-point Likert scale. These ten types of behaviors were as follows:

Item 1: ability to manage children's emotional states and activation levels.

Item 2: use of positive emotions.

Item 3: ability to motivate children.

Item 4: ability to create joint activities.

Item 5: sensitivity toward children's communicative signals.

Item 6: ability to create different opportunities for children to practice their communication goals.

Item 7: use of language that is appropriate to the level of children.

Item 8: to possess a large repertoire of games.

Item 9: to offer frequent learning opportunities.

Item 10: to provide clear instructions.

The therapists performed video evaluations through a dedicated web interface that allowed them to review and annotate the video sessions (Fig. 1). The annotation summary and the ability to directly access and review therapy highlights were designed to aid the assessment task.

ATHENA Digital Environment

A telehealth delivery model meant to provide early intervention services from a distance was developed. Figure 2 presents the ATHENA digital environment. This architecture evolved during the planning phase of the intervention methodology to meet the following needs:

provide telehealth services;

monitor the parents' activity and stimulate empowerment;

record the therapists' treatment sessions in the outpatient clinic;

provide the therapists with the tools necessary to monitor the children's progress.

The Application Server acts as the operating center where parents, therapists, health professionals, and all administrators can access ATHENA digital services but each with their own set of services and privileges. Two services were developed for parents: a telehealth system and a training system. An overview of successful telehealth systems (Chi & Demiris, 2015; Weinstein et al., 2014) has shown that many of these systems use mobile devices with simple, intuitive interfaces. Systems characterized by simplicity are preferred over those with many functions and customizing possibilities.

Based on these considerations, a mobile app named "ATHENA App" was created. It has a dashboard that opens with a simple click. ATHENA uses a simple "family-friendly" technology. The only instrument that the families needed to operate was a tablet, which was given to them at the beginning of the program. The families used the tablets to record videos and communicate with the therapists during the sessions.

ATHENA App

The ATHENA App was developed for tablets equipped with a Bluetooth headset to allow parents to interact in real-time with therapists as well as perform asynchronous activities. The decision to use a tablet was dictated by the need to provide families with a technologically friendly solution that could be used easily and simultaneously offers all the technical requirements necessary to conduct the training-and-care program. The tablet provided a simple and efficient audio-video system, allowing the parents to interact with therapists in synchronous sessions or record asynchronous sessions. Additionally, the possibility of using a mobile internet connection makes the system considerably flexible, easily transportable, adaptable to any house setting, and usable even by families who do not have an internet connection at home.

The system complies with privacy regulations. Any communication between the app and the server is conducted by a secure protocol (HTTPS), which is encrypted using an Secure Sockets Layer (SSL) certificate issued by the certification authority of the Gestione Ampliamento Rete Ricerca consortium (GARR) - Trans European Research and Education Networking Association (TERENA). Further, the audio/video communication is conducted through the Web Real-Time Communication (WebRTC) protocol.

Once they are logged in, parents can access the activity control dashboard (Fig. 3). The app presents a list of objectives that the therapists have chosen for the parents to work on with their children. The dashboard was designed to be considerably intuitive via the proposal of a few buttons/choices to make the interaction and activation of functions easy and swift.

To activate a new session, parents simply had to touch the "New session" button. Subsequently, a connection with the ATHENA server will start, and a video audio acquisition interface will pop up with two panels concerning the video streams of the children and therapists, respectively (Fig. 4). Therapists' video streams will be remotely activated by a professional only if they deem it appropriate and useful for the session. Parents can start the audio/video streaming of sessions by touching the "Start" button. The app also offers a "silent mode" function, which can be enabled by simply tapping the button at the top right of the interface. This darkens the tablet screen and displays only information about the elapsed time, avoiding all forms of distraction to children.

During the session, the app provides a two-way audio channel between the parents and therapists. Through the Bluetooth headset, parents can listen to their therapists' suggestions silently, and therapists can listen to the environment audio and interactions of parents and their children. At the end of the session, a small 5-question questionnaire evaluating the activity performed is proposed. Once the questionnaire is sent, the app returns to the dashboard page. Parents can consult all the work sessions conducted and view the filled questionnaire and recorded video by tapping on the "Sessions" tab.

Participants

The study participants comprised a total of 27 families of children with ASD or a pattern of development showing risks of developing ASD.

The sample was made up of 6 female and 21 male toddlers. Their ages ranged from 27 to 53 months. All children met the ADOS-2 cut-off for Autism Spectrum or Autism on Module 1 or a range of concerns for at least the mild/moderate criteria on the Toddler Module.

They also met the ADI-R toddler algorithm range of concern for at least the mild/moderate or satisfied criteria for an ASD classification on the normal algorithm. The mean mental age was 24.0 months (\(SD=4.76\)). The mean child age at intake was 36.0 months (\(SD=6.41\)).

Of the 27 selected families, 23 completed the entire intervention program. Thus, the analyses requiring a pre-post assessment were performed for the 23 families who completed the program. However, while evaluating the parental skill growth, the technique used (i.e., linear mixed models) allowed us to perform the analysis of the whole sample of enrolled families. The parents involved in the activities were mothers in 20 of the cases and fathers in three of the cases. The mean age of the parents was 35.78 years (\(SD=7.17\)).

Procedure

The ATHENA program supports parents by offering synchronous and asynchronous counseling services through two systems: the telehealth system and the training system. Figure 5 presents the ATHENA intervention structure and shows how the different services are made available as the intervention progresses. Further, Table 1 illustrates the number and types of treatment sessions held per week.

Diagnostic Evaluation and Enrollment

The first phase is the diagnostic evaluation aimed toward recruitment. Families were recruited from a public autism center in southern Italy. During the diagnostic evaluation, the second edition of the ADOS-2, the ADI-R, and the VABS were used.

The criteria for including families in the study program were positive results for ASD or possible ASD on both ADOS-2 and ADI-R and the mental age of the child being at least 18 months as evaluated by the VABS. The other criteria included the availability of one parent of each child for participation in all the activities and internet connection at home. Families with no internet connection were provided internet data connection for the entire duration of the treatment program by the research institute involved in the research study.

Overview Meeting and Pre-tests

Prior to the treatment's start, the parents attended an overview meeting, which was immediately followed by an assessment session. During the first phase of the meeting, the complete treatment program was illustrated. The parents received a written document explaining the entire treatment program and their rights. They gave their written informed consent and then completed the measures, including the shortened form of the PSI-4 and the FES. Families were provided a tablet to allow them to record all the home therapies/training sessions with the therapists and participate in all three types of therapy sessions (synchronous, asynchronous, and feedback sessions). All the therapy/training sessions were recorded in the outpatient clinic as well as at home.

Video Baseline

Before starting the intervention program, parents recorded three 15-20 min videos of them interacting with their children in the natural home environment.

Video Lessons

The ATHENA program provided access to the video lessons on various topics of raising a child with ASD. This included selected chapters from the book "An Early Start for Your Child with Autism: Using Everyday Activities to Help Kids Connect, Communicate, and Learn" (Rogers et al., 2012)-a recommended reading material for the parents participating in the program.

Outpatient Clinic Sessions

During the first month, sessions were held twice a week for an hour in the outpatient clinic. The parents were present in the room as the therapists worked with the children. The focus of these first sessions was to assess the children's abilities and develop their teaching goals. The therapy room had the necessary play materials for the sessions and a system for recording the videos on a local server (Outpatient Clinic Server). As the parents observed the sessions, they start learning how to interact with their children through the demonstrations. The recordings were stored in the local server and then synchronized with the central server using a remote synchronization link.

Home and Synchronous Sessions

After the first month, the intervention transitioned to consultative service, focusing on transferring knowledge and skills to the parents who could promote their children's development and learning. The therapists provided the parents teaching objectives and strategies that they were expected to use with their children, as well as instructions to create daily play sessions. Teaching moments were encouraged during normal family activities such as mealtimes, bathing, and playing with siblings. In the second month, the therapists went to the homes of respective families once a week and worked directly with the children, continuing to model teaching strategies for the parents. When the "Synchronous" sessions were introduced, the therapist watched the ongoing sessions once a week through videoconferencing in real-time and offered suggestions to parents as they interacted with their children. Both types of sessions lasted for an hour.

Synchronous, Asynchronous and Feedback Sessions

In the third month of the project, synchronous and asynchronous sessions took place alternately. Synchronous therapies are sessions in which the therapists provide supervision to parents from a remote location to help them learn new strategies on how to teach their children. Asynchronous sessions are sessions in which the parents work alone with their children for 30 min and record the sessions. During the week, the therapists remotely watched the asynchronous video that the parents had previously recorded. By using the video annotation tools (Fig. 1), the therapist could take notes and highlight the behaviors and interactions of the child-parent pair. Subsequently, the therapist would give precise and detailed comments to the parents in the feedback sessions. Starting from the fourth month, the model included only asynchronous and feedback sessions.

Support Group

The ATHENA program offered additional support to parents in monthly group meetings with a psychotherapist-an expert in ASD-to discuss the strategies and principles presented in the video lessons, the concerns about their children's progress in the project, or any other issue concerning their children on which they would like help. Both the parents-not just the parent participating in the program activities-were invited to participate. Additional help was provided via the possibility to discuss different topics with an ASD expert using the chat option provided by the messaging service.

Final Evaluation

ADOS-2 and VABS were repeated at the end of the 6-month period. In this period, the parents attended an exit session, during which they provided feedback about the program and completed the same measures administered at the beginning-PSI-4 and FES.

Statistical Analysis

All the statistical analyses were performed using the version 3.6.1 of the R software (R Core Team, 2018). We estimated these models using the lme4 package (version 1.1.21) (Bates et al., 2015). Further, we described in detail how we performed the analysis of the three main aspects. First, we analyzed the effect of the ATHENA program on the ASD symptoms observed, the parents' sense of empowerment in their families, and the parents' stress. Subsequently, we analyzed the trend in the parents' performance during the ATHENA program. Finally, we analyzed the interaction between the expected improvement in parents' skills and the possible effect on stress and empowerment.

Single Dimension Analysys

To analyze the effect of the ATHENA program, three paired-samples t tests were conducted to compare (1) the number of ASD symptoms observed (which was measured by ADOS-2), (2) the parents' sense of empowerment in their families measured by FES, and (3) the sum of parents' stress (ST) measured by PSI-4, both before and after the intervention.

Evaluation of Videos

The videos were analyzed to prove the positive effect of the intervention on the parents' ability to stimulate their children's learning. The measure used is the observation tool described above. The observations of the outcome variables considered in this analysis are not independent as we repeated the measurements on the same subject. To deal with this condition, we relied on mixed linear models, which allowed us to reproduce the nested model and define the specific variance term for the intra-subject analysis. Let \(Q_{p}\) be the level of quality observed by the therapist for parent p. The step variable is calculated as the difference in days between the day of the therapy linked to the observation and that of the first therapy session. First, we wanted to verify the effect of the step on the quality of the parent intervention observed by the therapist. To this end, we compared the following models:

Model 1 is a classical random intercept used with a fixed mean model. In Model 2, the variable item is added to Model 1 to verify the fixed effect of the different items of the observation tool. The aim of Model 3 is to consider the fixed effect of the step variable. Finally, Model 4-a correlated random intercept and slope model-is added to verify the random effect of the step variable. Multiple fit indices were considered to compare models such as Akaike information criterion (AIC), Bayesian information criterion (BIC), and chi-square (\(\chi ^2\)).

Analysis on the Interaction Among the Different Dimension

One of the principal aims of this work has been to verify the relationship between the parents' ability to stimulate their children's learning and some of the measures generally used as a signal of the difficulties encountered by parents, such as the levels of stress and empowerment.

In particular, our goal is to verify if there is a relationship between the expected improvement in the parents' skills induced by the training intervention on the one hand and the decrease in stress and boosting of empowerment on the other hand.

To this end, we first performed a simple correlation analysis between the \(\Delta\) of the variables. The next step was to test if the variables such as the parents' age or the initial level of the children's observed symptoms played a role in the relationship between the improvement in parents' ability and that of their stress and empowerment levels.

To this end, we conducted the analysis of the linear models described by Eqs. 5 and 6.

Results

Single Dimension Analysys

The paired-samples' t test was conducted to compare the quantity of ASD symptoms observed, which was measured using ADOS-2 calibrated severity scores before (\(M=7.260\), \(SD=2.027\)) and after the therapy (\(M=6.040\), \(SD=1.827\)). Further, \(t(22)=-2.74\), \(p=0.006\), \(d=0.630\) suggest a decrease in the symptoms observed. Thus, based on the quantity of ASD symptoms, the therapy is observed to have a medium positive effect on children (Fig. 6).

Similarly, the paired-samples t test was conducted to compare the parents' sense of empowerment in families measured by FES before (\(M = 45.0\), \(SD = 4.09\)) and after therapy (\(M = 47.826\), \(SD = 4.745\)). Here, \(t(22) = 2.25\), \(p = 0.017\), \(d = 0.638\) indicate an increased sense of empowerment; thus, the intervention had a medium positive effect on the parents' sense of empowerment (Fig. 7).

Finally, the paired-samples t test was conducted to compare the sum of parents' stress (ST), which was measured by PSI-4 before (\(M = 92.348\), \(SD = 13.58\)) and after the therapy (\(M = 81.0\), \(SD = 18.703\)). Here, \(t(22) = -4.16\), \(p = 0.0002\), \(d = 0.655\) indicate a decrease in stress levels; thus, the therapy had a discrete positive effect on the sum of parents' stress (Fig. 8).

Evaluation of Videos

While Table 2 presents the regression coefficients of the linear mixed models, Table 3 presents the ANOVA comparison of these models.

The analysis of the models highlights a significant fixed effect of the item and step variables. Model 4 accounts for the individual differences in the parental improvement process and presents the best fit for the data while confirming the significance of the item and step variables' fixed effects. In particular, the coefficient of the step variable \(\beta _{2}=0.0105\) (\(p = 0.0001 < 0.001^{****}\)) is positive, which indicates a significant improvement in the parents' ability to stimulate their children's learning over time.

The analysis of the overall effect on the individual parent of the step variable, obtained by adding the random effects to the fixed effect, shows that parents generally recorded the improvement in their ability. A slight worsening was noted only in four cases.

Analysis on the Interaction Found Among the Different Dimension

The correlation analysis did not reveal a statistically significant correlation between improvement in parents' ability and total stress variation \(\beta =-0.1896\); \(t(21)=-0.89\) (\(p = 0.4\)). Also between improvement in parents' ability and family empowerment variation, the correlation analysis did not reveal a statistically significant correlation \(\beta =-0.1063\); \(t(21)=-0.49\) (\(p = 0.6\)).

\(\Delta\)Total Stress vs \(\Delta\)Ability of the parents in relation to their age and the initial ASD symptoms observed

\(\Delta\)Family Empowerment vs \(\Delta\)Ability of the parents in relation to their age and the initial ASD symptoms observed

Nevertheless, Figs. 9 and 10 show that the parents' age seemed to have impacted those relationships. This evidence has been confirmed by the results of the linear models described by equations 5 and 6 (see Table 4).

In particular, Model 5 (estimated using ordinary least squares) explained the significant and substantial proportion of the variance observed (\(R^2 = 0.60\), \(F(5, 17) = 5.04\), \(p = 0.005\), \(adj.R^2 = 0.48\)). Within this model, the effect of \(\Delta Ability\) was significantly negative (\(\beta = -42.58, p < 0.01\)), the effect of parent age was significantly negative (\(\beta = -1.28, p < 0.01\)), and the interaction effect of age on \(\Delta Ability\) was significantly positive (\(\beta = 1.34, p < 0.001\)).

Finally, Model 6 (estimated using OLS) explained the not significant and substantial proportion of variance (\(R^2 = 0.33\), \(F(5, 17) = 1.69\), \(p = 0.191\), \(adj.R^2 = 0.14\)). Within this model, only the interaction effect of age on \(\Delta Ability\) was significantly negative (\(\beta = -0.46, p < 0.05\)).

Discussion and Conclusion

This study aimed to investigate the effectiveness of the training-and-care telehealth program called ATHENA in promoting parents' ability to stimulate their children's learning, reduce parenting stress, and improve their sense of parenting empowerment.

The ATHENA program was based on a naturalistic developmental behavioral intervention. It involved a total of 27 parents of children with ASD. It was conducted over 6 months.

The principal aim of the program was to help parents become more active in their teaching role with their children via the provision of parent training, psychological support, and coaching. It took place primarily in the homes of children via the use of interactive social contexts such as play and daily routines.

Overall, the results of this study show a significant improvement in the quality of parental intervention as judged by the therapist, an improvement in parents' sense of empowerment, and a reduction in parental stress.

The improvement in the parents' abilities to stimulate their children's learning was measured via the therapist's observation of the videos of the parents' asynchronous sessions. The video analysis supported the positive effect of the program intervention on the parents' behavior while managing the interaction sessions with their children.

Furthermore, this study showed an increase in empowerment. In particular, the statistical analysis showed the intervention to have a medium positive effect on the parents' sense of empowerment.

Finally, following the parents' participation in the ATHENA program, we observed that the parents reported a reduction in the parental stress related to managing the daily concerns of raising children with ASD. A positive effect of the intervention program on the sum of parents' stress, measured by the PSI-4, was estimated. This result contributes to the debate in the literature about the efficacy of parent-mediated interventions on parental stress reduction. There have been studies on non-ASD clinical populations that have found a positive association between parental interventions and reduced stress and an increased locus of control (Moreland et al., 2016).

A recent study demonstrated that the participation of parents in a parent training program positively affected both the child outcomes-such as disruptive behaviors-and parental factors such as parental competence and reduction in parental stress (Iadarola et al., 2018).

The positive results of this study may be ascribed to the benefit provided by the telehealth approaches in implementing a naturalistic developmental approach.

The parents introduced the intervention through play and everyday family routines, which affected the improvement of child outcomes as well as their abilities as co-therapists. These results are in agreement with previous research (Kaminski et al., 2008). As reported in a meta-analytic review of the aspects associated with parent efficacy, programs that focus on the inter parent-child interaction and involve parents learning new skills with their children have been shown to have a significant effect on both child behavior and parenting skills (Kaminski et al., 2008).

Regarding the relationship between the improvement in parenting skills and its effects on stress and empowerment levels, the data call for the need to have further investigation. Indeed, while it is not possible to present a significant correlation between these variations, further analysis conducted on the role of parental age and the initial level of the observed symptoms has brought out the effect of parental age. In particular, data have shown that the initial hypothesis of the positive effect of skill enhancement on the levels of stress and empowerment has been verified among younger parents. Conversely, in older parents, an opposite effect has been witnessed.

While it is entirely plausible that younger people may benefit from these educational approaches, perhaps because they are more open to using technology, the older parents' negative effect is difficult to explain (AARP, 2016).

Further studies are needed to investigate this phenomenon in greater depth. They should (a) increase the sample size and (b) introduce new measures, such as technology acceptance, within the theoretical framework of the Unified Theory of Acceptance and Use of Technology, which could consistently explain how age is a moderating variable for the behavioral intention to use technology such as Venkatesh et al. ( 2003).

The strength of this study lies in the use of telehealth to provide the intervention program. Teaching parents in individual encounters, such as for the first month at the outpatient clinic, 2 months at home, and 3 months from a distance (synchronously and asynchronously), led to an improvement in their empowerment, a reduction in their stress levels, and significant positive effects on their social behaviors with their children. This study confirms the literature results pertaining to the effectiveness of the use of telemedicine as a service delivery method for ASD (Knutsen et al., 2016).

Telehealth programs can reduce costs and increase the availability of services, which is especially important in geographical areas that experience economic stress and/or lack quality services for ASD. The results have brought out the possible importance of providing a naturalistic developmental intervention to support the development of children and parents in their parenting roles.

Limitations

This study has a few limitations. First, the study lacks a control group. Further research should consider the estimation of the differences in outcomes between a group of parents who have their children exposed to a parent-mediated intervention program and an equivalent parent group exposed to traditional children intervention. Another limitation of this study is the use of a relatively small sample of parents and their children. Future research should recruit larger samples of parents and their children to gain insights into the wider context.

Funding

This study was supported by the Azienda Sanitaria Provinciale of Trapani.

Author information

Authors and Affiliations

Istituto per le Tecnologie Didattiche, Consiglio Nazionale delle Ricerche, via Ugo la Malfa, 153, 90146, Palermo, Italy

Manuel Gentile, Linda Messineo, Dario La Guardia, Marco Arrigo, Giuseppe Città, Antonia Ayala, Gaspare Cusimano & Mario Allegra

Azienda Sanitaria Provinciale di Trapani, Cittadella della Salute, viale della Provincia, 2, 91016, Erice Casa-Santa, Trapani, Italy

Pio Martines & Giovanna Mendolia

Contributions

Conceptualization: MG, LM, DLG, MA (Marco Arrigo), AA, GC (Gaspare Cusimano), MA (Mario Allegra) Data curation: MG, LM, AA Formal Analysis: MG Funding acquisition: GC (Gaspare Cusimano), MA (Mario Allegra) Investigation: MG, LM, DLG, MA (Marco Arrigo), AA, GC (Gaspare Cusimano), MA (Mario Allegra) Methodology: MG, GC (Gaspare Cusimano), MA (Mario Allegra) Project administration: GC (Gaspare Cusimano), MA (Mario Allegra) Resources: PM, GM Software: DLG, MA (Marco Arrigo) Supervision: MA (Mario Allegra) Writing-original draft: MG, LM, DLG, MA (Marco Arrigo), GC (Giuseppe Città), AA Writing-review & editing: MG, LM, GC (Giuseppe Città), MA (Mario Allegra).

Corresponding author

Correspondence to Linda Messineo.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Ethics Approval

This study was performed in line with the principles of the Declaration of Helsinki. Approval was obtained from the ethics committee of Azienda Ospedaliera Universitaria Policlinico Paolo Giaccone of Palermo.

Consent to Participate

Written informed consent was obtained from the parents.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Reprints and Permissions

About this article

Cite this article

Gentile, M., Messineo, L., La Guardia, D. et al. A Parent-Mediated Telehealth Program for Children with Autism Spectrum Disorder. J Autism Dev Disord 52, 5285-5300 (2022). https://doi.org/10.1007/s10803-022-05482-6
Download citation

Accepted:

Published:

Issue Date:

DOI: https://doi.org/10.1007/s10803-022-05482-6

Get shareable link

Keywords

Telehealth

Autism spectrum disorder

Parent training

Parental empowerment

Parental stress

# data[].attributes.attachment_link.title
A Parent-Mediated Telehealth Program for Children with Autism Spectrum Disorder - Journal of Autism and Developmental Disorders

# data[].attributes.attachment_link.description
This study investigates the effectiveness of a 6-month parent-mediated early intervention telehealth program for children with an autism spectrum disorder. The participants comprised a total of 27 parents. The findings showed that participation in the program promotes parents' empowerment and reduce parental stress and a general improvement in the parents' ability to stimulate children's learning.

# data[].attributes.body
This program send "targeted life cycle based" SMS messages to pregnant and new mothers, male partners, and caregivers. Messages included information for how they can keep themselves and their new babies healthy during the first 1,000 days. This short video is a member of the team talking about the impact this program has had. As we become FNPs, this is something we could utilize telehealth for, especially in rural communities without convenient access to in-person care.

# data[].attributes.attachment_link.title
FamilyConnect: A story by a Village Health Team (VHT) member

# data[].attributes.body
This video provides education about accessing personal information using a third party, such as a smart phone. Once again, a real-life scenario allows someone to better understand their right to access personal information via a third party.

# data[].attributes.attachment_link.authors[].name
Office of the National Coordinator for Health IT

# data[].attributes.attachment_link.title
Video 3 - Your Health Information, Your Rights

# data[].attributes.body
By watching this video, someone can learn about fees, forms, and time frames associated with getting a copy of this own medical records. Using a real-life scenario likely will enhance someone's understanding while viewing this video

# data[].attributes.attachment_link.authors[].name
Office of the National Coordinator for Health IT

# data[].attributes.attachment_link.title
Video 2  - Your Health Information, Your Rights

# data[].attributes.body
This video provides education about what information is included in EHRs. It continues to explain how to access, share, and use this information to become advocates and liaisons for someone's own health care.

# data[].attributes.attachment_link.authors[].name
Office of the National Coordinator for Health IT

# data[].attributes.attachment_link.title
Video 1 -  Your Health Information, Your Rights

# data[].attributes.headline
Patient Access Information for Individuals

# data[].attributes.subject
Patient Access Information for Individuals

# data[].attributes.body
As health information exchanges evolve and allow patients more access to their EHR, providers need to ensure they understand key concepts about this access. This website provides an overview, three videos (linked below), and various online resources to help them learn how to navigate their health records. A possible idea would be to place a television in the waiting room and include these videos on a video real; other pertinent information such as information about HIPPA policies, a preventative health measure that changes monthly, and flu vaccination during the season.

# data[].attributes.attachment_link.readable_text
This guidance remains in effect only to the extent that it is consistent with the court's order in Ciox Health, LLC v. Azar, No. 18-cv-0040 (D.D.C. January 23, 2020), which may be found at https://ecf.dcd.uscourts.gov/cgi-bin/show_public_doc?2018cv0040-51. More information about the order is available at https://www.hhs.gov/hipaa/court-order-right-of-access/index.html. Any provision within this guidance that has been vacated by the Ciox Health decision is rescinded.
Easy access to your health records puts you in control of decisions regarding your health and well-being. You can monitor your health conditions better, understand and stay on track with treatment plans, and find and fix errors in your record. The resources and videos below will help you understand your rights under the Health Insurance Portability and Accountability Act (HIPAA) and will show you how to work with your providers to get, check, and use your health information.

# data[].attributes.attachment_link.title
Patient Access Information for Individuals: Get it, Check it, Use it!

# data[].attributes.attachment_link.description
This guidance remains in effect only to the extent that it is consistent with the court's order in Ciox Health, LLC v. Azar, No. 18-cv-0040 (D.D.C. January 23, 2020), which may be found at https://ecf.dcd.uscourts.gov/cgi-bin/show_public_doc?2018cv0040-51 . More information about the order is available at https://www.hhs.gov/hipaa/court-order-right-of-access/index.html .

# data[].attributes.body
This article gives an overview of health information exchange with features in Maryland. Understanding regulations in your home state and staying up-to-date on new House Bills and state requirements changes are imperative for practicing practitioners.

# data[].attributes.attachment_link.readable_text
Health information exchange allows authorized users to securely and electronically access and share patient health information for clinical, quality improvement, and public health purposes. Organizations providing HIE services (HIEs) support efforts to improve safety and efficiency in health care by making the right information available in the right place at the right time. HIEs range from regional, public entities (including State-Designated entities) and developers of health information technology (health IT) (e.g., electronic health record vendors) certified by the Office of the National Coordinator for Health Information Technology

Information for Health Care Consumers

Consumers benefit from understanding the electronic exchange of their health information among treating providers. The MHCC is helping raise consumer awareness and understanding of HIEs and their role in electronic data exchange. For more information, visit MHCC's HIE Consumer web page.

Nationally, there is a coordinated focus to improve access, exchange, and use of electronic health information (see the 2020-2005 Federal Health IT Strategic Plan). Federal programs and policies aiming to advance electronic health information exchange and interoperability include:

Information Blocking

Beginning in April 2021, health care providers and HIEs are prohibited from information blocking under federal regulations. Information blocking consists of certain actions that could interfere with electronic health information access or exchange by authorized users. Certain exceptions apply. More information is available here.

Centers for Medicare & Medicaid Services (CMS) Interoperability and Patient Access Rule

As of July 2021, CMS is enforcing requirements that aim to improve patient access to their health information. Under the rule, CMS-regulated payers must implement patient access and provider directory APIs, and payer to payer data exchange. More information is available here.
Trusted Exchange Framework and Common Agreement (TEFCA)
Released in January 2022, TEFCA outlines a set of principles, terms, and conditions for nationwide electronic health information exchange. A voluntary network based on the Common Agreement offers a centralized model for standardized electronic health information exchange. More information is available here.

State Regulations

Maryland law (2011) requires MHCC to adopt regulations governing the privacy and security of protected health information (PHI) obtained or released through an HIE. COMAR 10.25.18, Health Information Exchanges: Privacy and Security of Protected Health Information (regulations) builds upon protections established by the Health Insurance Portability and Accountability Act of 1996 (HIPAA), as amended by the Health Information Technology for Economic and Clinical Health Act (HITECH) in 2009. The regulations aim to ensure privacy and security of PHI while improving access to clinical records by treating providers and supporting public health goals. The MHCC is currently aligning the regulations with leading national policies and new HIE requirements legislated in 2021 and 2022 by the Maryland General Assembly.

HIE Policy Board

The MHCC engages stakeholders with a diversity of perspectives to advise staff on policies for statewide health information exchange. The HIE Policy Board informs development of COMAR 10.25.18 relating to private and secure data exchange; consumer control over the use of and access to their health information, to the extent technically feasible; and maximizing the benefit of HIE for clinical and public health purposes.

In 2021 and 2022, the HIE Policy Board convened to deliberate on proposed draft updates to COMAR 10.25.18, aiming to align the regulations with evolving federal and state polices. For more information, visit MHCC's HIE Policy Board web page.

State-Designated HIE
Maryland law (2009) charged MHCC and the Health Services Cost Review Commission with designating a statewide HIE, a process that occurs every three years. The State-Designated HIE is responsible for building and maintaining technical infrastructure and an efficient and effective data management strategy that can support the secure statewide exchange of electronic health information. The Chesapeake Regional Information System for our Patients (CRISP) was competitively selected to serve in this role in 2009 and has been chosen at each designation cycle based on their performance. CRISP offers a variety of tools and services to meet the needs of health care facilities, providers, patients, and State agencies.

For more information about CRISP services, click here. For historical information about HIE planning in Maryland, click here.
Quarterly Reports
CRISP reports quarterly to MHCC information on participating organizations and their utilization of HIE services. To view the most recent quarterly report, click here.

HIE Registration
Entities operating in Maryland that meet the definition of an HIE are required to register with MHCC annually, including EHR developers whose systems offer data exchange capabilities.

For a printable list of registered HIEs, click here.

For more information, including how to register or renew registration, click here.
Peer-to-Peer Learning Health Data Utility Roundtables
In July 2021, MHCC convened state leaders and HIE representatives to discuss opportunities, challenges, and solutions to advancing electronic health information exchange. The outcome of these discussions highlighted the need and future direction for HIEs to function more broadly as a data utility to support providers, consumers, public health agencies, and payers. A resulting series of national roundtable discussions (January - April 2022) with states and HIEs was held in collaboration with Civitas Networks for Health and the Association for Health IT State Leaders

The roundtables centered on establishing and maintaining health data utilities (HDU), an advanced HIE model, providing services (regionally, statewide, or across state lines) for clinical and public health purposes. More information about the national vision for HDUs is available here. Roundtable participants included HIEs, state agencies, regional health improvement collaboratives, and other stakeholders involved in public-private health data partnerships.

Meeting Materials

January 31, 2022: Main Session Recording, Breakout #1 Recording, Breakout #2 Recording, Presentation, and Participating Organizations

February 28, 2022: Main Session Recording, Breakout #1 Recording, Breakout #2 Recording, Presentation, and Participating Organizations

March 28, 2022: Main Session Recording, Breakout Session Recording, and Participating Organizations

April 25, 2022: Recording, Presentation, and Participating Organizations
Resources
Information on national HIE efforts and web-based interoperability training:
Office of the National Coordinator for Health Information Technology (ONC)
Information about HIE benefits and the federal vision for achieving interoperability.
Centers for Medicare and Medicaid Services (CMS)
Information about how CMS engage states to align their Medicaid programs to support HIE.
Heath Information and Management Systems Society (HIMSS)
Resource library that supports electronic information exchange to advance interoperability.
ONC Interoperability Training Courses
A five-part series of training on interoperability as it relates to the following key areas: basics training, transitions of care, lab exchange, patient and family engagement, and public health.
Contact
For more information, contact Christine Karayinopulos at christine.karayinopulos@maryland.gov.

# data[].attributes.attachment_link.keywords[]
chesapeak regional information system for our patients

# data[].attributes.attachment_link.description
Health information exchange allows authorized users to securely and electronically access and share patient health information for clinical, quality improvement, and public health purposes. Organizations providing HIE services (HIEs) support efforts to improve safety and efficiency in health care by making the right information available in the right place at the right time.

# data[].attributes.attachment_caption
BMC Medical Research Methodology - BioMed Central

# data[].attributes.body
Similarly to the previous post, this another model depicting how clinical decision support systems work.

A simplistic example would be:
1 - Ordering Medication (Trigger)
2 - Creatine level & current medications (Input Data)
3 - CDS gives options (Intervention)
4 - Change Dose (Action Step)

# data[].attributes.attachment_caption
Journal of the American Academy of Child & Adolescent Psychiatry

# data[].attributes.headline
Clinical Decision Support System Approach

# data[].attributes.subject
Clinical Decision Support System Approach

# data[].attributes.body
Even though this infographic was made to show an approach to enhancing mental health services, its fairly simple design gives us a visual for how clinical decision support systems work.

# data[].attributes.body
This short video discusses how EMR & EHR differ. It is simplistic enough to share with patients to help them better understand the pros and cons of storing their health care information.

# data[].attributes.attachment_link.title
EMR vs EHR Systems: Their Differences, Uses, and Advantages

# data[].attributes.body
Interestingly, only 20% of physicians, surveyed for this study, believed that quality of care, patient-centeredness of care, or care they provide themselves would improve with the meaningful use of the EHR.

# data[].attributes.attachment_link.readable_text
Abstract

Background  There is continuing interest in how physicians are responding to the meaningful use of the electronic health record (EHR) incentive program. However, little research has been done on physician beliefs about the meaningful use of the EHR.

Objective  This study aims to conduct a follow-up study of physician beliefs about the meaningful use of the EHR.

Methods  Online survey of physicians at two academic medical centers (AMCs) in the northeast who were participating in the meaningful use of the EHR incentive program and were using an internally developed EHR was conducted.

Results  Of the 2,033 physicians surveyed, 1,075 completed the survey for an overall response rate of 52.9%. Only one-fifth (20.5%) of the physicians agreed or strongly agreed that meaningful use of the EHR would help them improve quality of care, and only a quarter (25.2%) agreed or strongly agreed that the meaningful use of the EHR would improve the care that their organization delivers. Physician satisfaction with the outpatient EHR was the strongest predictor of self-efficacy with achieving stage 2 of the meaningful use of the EHR incentive program (odds ratio: 2.10, 95% confidence interval: 1.61, 2.75, p  < 0.001). Physicians reported more negative beliefs in stage 2 than stage 1 across all belief items. For example, 28.1% agreed or strongly agreed that the meaningful use of the EHR would decrease medical errors in stage 2 as compared with 35.9% in stage 1 ( p  < 0.001).

Conclusion  Only one-fifth of the physicians in our study believed that the meaningful use of the EHR would improve quality of care, patient-centeredness of care, or the care they personally provide. Primary care physicians expressed more negative beliefs about the meaningful use of the EHR in stage 2 than in stage 1. These findings show that physicians continue to express negative beliefs about the meaningful use of the EHR. These ongoing negative beliefs are concerning for both implementation and policy.

Keywords: electronic health records, physicians, beliefs, meaningful use, self-efficacy

Background and Significance

The objectives of the Medicare and Medicaid Electronic Health Record (EHR) Incentive Program, also widely referred to as the meaningful use of the EHR incentive program, are to increase the adoption of EHRs and the meaningful use of EHRs to improve delivery of care, decrease medical errors, improve efficiency of care, and enhance patient-centeredness of care. 1 2 In the context of the meaningful use of the EHR incentive program, there has been growing interest among clinical informatics researchers in evaluating the impact of EHR use. This evaluation can be objective, for example, through outcomes such as quality measures. 3 4

Another approach is to focus on perceptions and beliefs about the meaningful use of the EHR. One qualitative study explored perceptions of academic and private physicians about the impact of EHR use on workflow and patient care. 5 The study did not find differences in perceptions between these two groups but both had negative perceptions of the impact of EHR on workflow and patient care. The same researchers reported that super-user physicians (physicians who were product champions) had similar negative perceptions. 6 Weeks et al compared perceptions of the impact of EHR use in two groups of physicians users (active vs. inactive) of the meaningful use of the EHR incentive program. 7 They found that even in the active, meaningful user group a majority of physicians disagreed about the impact of meaningful use on reducing care disparities (78% disagreed) and improving the accuracy of patient information (70% disagreed). In a previous study, we found that a majority of physicians had negative beliefs about the meaningful use of the EHR. 8 For example, only 23 and 27% of physicians in our study agreed or strongly agreed that the meaningful use of the EHR would help them improve the care they personally deliver and improve quality of care, respectively. We also found negative physician beliefs about a specific functionality of the EHR, the provision of the after visit summary (AVS). 9 In this study, 79.2% of primary care physicians and 75.2% of specialists reported that generating and providing the AVS had an adverse effect on physician workload.

Objective

The objective of this study was to assess physician beliefs about the meaningful use of the EHR. To achieve this objective, we conducted a follow-up survey using items which we had developed in a previous study. 8 A secondary objective was to assess physician self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program and the predictors of self-efficacy.

Methods

Context

The context for this study is stage 2 of the meaningful use of the EHR incentive program, introduced in the Health Information Technology for Economic and Clinical Health (HITECH) provision of the American Recovery & Reinvestment Act of 2009. 1 2 10 11 12 The meaningful use incentive program was originally designed to take place over three stages from 2011 through 2016. 2 Stage 1 focused on the collection of discrete or coded health information. Stage 2 was intended to see the leveraging of those data for computerized decision support. Stage 3 anticipated actual improvement in clinical care, based on the use of data in the EHR. For such achievement, maintained over the three-stage life of the incentive program, eligible professionals (EPs) could receive up to $44,000 (Medicare) or $63,750 (Medicaid). Furthermore, EPs who failed to achieve meaningful use would see a reduction of their CMS (Centers for Medicare & Medicaid Services) payments beginning at 1% of total payments, with penalties increasing by 1% per year afterward.

This staged approach to the meaningful use of the EHR incentive program has been considered analogous to the concept of an escalator. 1 With each stage, the notion has been that providers would move upward toward the final goal of achieving improvement in quality, safety, and efficiency of care. 2 At the same time, proponents of the program acknowledged the need for calibrating the speed of the escalator to account for technological and resource limitations in the real world. 1

Participants

The participants in this study were physicians at two academic medical centers (AMCs) in the northeast who were participating in the meaningful use of the EHR incentive program. With respect to the AMCs, AMC1 is a 777-bed medical center with 57,000 admissions per year and AMC2 is a 907-bed medical center with 48,000 admissions per year. AMC1 has 101 affiliated outpatient practices with 170 primary care physicians, 1,423 specialists, and 769,000 outpatient visits a year. AMC2 has 175-affiliated outpatient practices, with 259 primary care physicians, 1,737 specialists, and 1.5 million outpatient visits a year. At both AMCs, a common home-grown outpatient EHR was in use when this study was conducted. The home-grown EHR was an internally developed, web-based, fully functioning EHR that included notes from primary care and subspecialty clinics; hospital discharge summaries; International Classification of Diseases, Ninth Revision-coded problem lists; health maintenance lists; medication prescribing; coded allergies; laboratory and radiographic results; and result management. The adoption and use rate of the outpatient EHR by physicians was 100% across both AMCs as it is a requirement for affiliation with the AMCs.

Survey Instrument

In a previous study, we described the development of our survey instrument. 8 We adopted the same belief items from that study for this survey. We conceptualized beliefs as outcome expectancies, and as a result, our belief items focus on the meaningful use of the EHR for decreasing medical errors, increasing efficiency of care, and improving patient-centeredness and quality of care. The belief items were captured as five-point Likert scales ranging from strongly disagree to strongly agree ( ).

Table 1

Meaningful use will improve the quality of care I deliver

Meaningful use will improve the patient-centeredness of care I provide

Meaningful use will not increase the efficiency of care I provide

Meaningful use will not lead to improved patient outcomes

Meaningful use will decrease medical errors

Overall, I think meaningful use as being measured will help me use the EHR to improve the care I personally deliver

Overall, I think meaningful use as being measured will help me use the EHR to improve the care the AMC delivers

In addition to beliefs, we assessed self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program. Self-efficacy was captured as the degree of confidence physicians have in achieving stage 2 and measured on a Likert scale (1 = not at all confident to 5 = extremely confident). We also gathered data on factors identified in previous studies as influencing the adoption and meaningful use of the EHR 8 9 13 14 15 16 : physician age, gender, race, specialty (primary care, medical specialty, and surgical specialty), number of outpatients seen per week, number of outpatient hours worked per week, practice size, and satisfaction with the outpatient EHR. We recorded physician age into two categories: Younger than 55 years and 55 years and older. We recorded satisfaction with the outpatient EHR into two categories: Satisfied (defined as satisfied or very satisfied) and other (defined as very dissatisfied or dissatisfied or somewhat dissatisfied or somewhat satisfied).

Data Collection

As described above in the section on participants, our subjects were physicians who were EPs at two AMCs participating in the meaningful use incentive program. We administered the survey using the tool "Survey Monkey." We sent the initial survey followed by three reminders. To enhance the response rate, we offered an Amazon tablet to 10 randomly drawn respondents.

Statistical Analysis

Because all variables of interest in our analysis are categorical, descriptive statistics are presented using percentages. We conducted bivariate analyses using the Pearson's chi-square test. A p value of less than 0.05 was considered statistically significant. We employed logistic regression analysis to assess significant correlates of self-efficacy, our main outcome of interest. Our explanatory variables were all dichotomous (e.g., other vs. agree/strongly agree). In the model, we only included variables that were significant in our bivariate analyses. We entered all variables into the model rather than choose a forward or backward selection approach. The statistical significance of the covariates in the logistic regression model was evaluated using Wald's p values. 17 Finally, we analyzed whether beliefs about the meaningful use of the EHR changed from stage 1 to stage 2 of the meaningful use of the EHR incentive program. We conducted pairwise comparisons of the responses to the belief items from our survey on the meaningful use of the EHR conducted in 2012 with respect to stage 1 with those from this survey conducted in 2014 with respect to stage 2. All analyses were completed using SPSS (IBM Corp.) for Windows 23.0.

Institutional Review Board Approval

Approval for the study was obtained from the Partners HealthCare Institutional Review Board.

Results

Response Rates

Of the 2,033 physicians at both AMCs participating in the meaningful use of the EHR incentive program, 1,075 completed the survey for an overall response rate of 52.9%. There was no difference in age between responders and nonresponders: The mean age of survey responders was 50.66 years as compared with 50.74 years for nonresponders. Female physicians were more likely to respond to the survey: 55.9% were responders as compared with 44.1% of non-responders ( p  = 0.02). shows the demographic and practice characteristics of respondents.

Table 2

Beliefs about the Meaningful Use of the EHR

Only one-fifth of respondents agreed or strongly agreed that meaningful use of the EHR would improve patient-centered care (20.9%, n  = 223) and quality of care (20.5%, n  = 217) ( ). Only a quarter of respondents (25.2%, n  = 268) agreed or strongly agreed that meaningfully using the EHR will improve the care that their AMC delivers. Of the respondents, 62.9% (673) agreed or strongly agreed that the meaningful use of the EHR would not increase the efficiency of the care they provide.

Table 3

Of the respondents, only 29% (311 of the 1,075 respondents) provided comments in an open-ended comment field. The comments were overwhelmingly negative and reflected the negative beliefs about the meaningful use of the EHR reported above. A common theme was physician concern about the amount of data entry required for the meaningful use of the EHR. As one physician commented: "Time consuming goals, not linked to improving outcomes for patients. Creates lots of work, mainly tapping on the computer while we should be talking to and examining patients." Another physician noted: "Meaningful use is focused on checking boxes that do not improve care and interfere with my ability to spend time talking with my patients. It puts the physician in the role of data entry technician." provides a sampling of the comments across different physician specialties.

Table 4

PCP

Meaningful use demands computer-centered rather than patient-centered care. The first stage was doable. I think the next will require many burdensome workarounds as I try to prove to others the patient communication and follow-up I was doing anyway and now have time to do less well

PCP

Constantly demands workarounds. Alternatively, the scripted attestations that I have talked to my patients about hypertension, etc. It has increased the amount of time involved with computer and administration, is clunky in its implementation, and reduced the time available for patient care. If this were a drug, it would never have cleared phase 1 trials. It could be worse, and I am sure it will be

PCP

As is often the case, too much time and effort are being dedicated to checking off boxes to fulfill meaningful use criteria, even if not directly beneficial to patient care (and at times not even appropriate for specific patients). More time chipping away at our 15 min visit. The result; less time to actually see the patient and care for the patient. A tale that is becoming as old as time

Medical specialty

Meaningful use has only reduced the efficiency with which providers can see their patients. Forcing us to check off certain boxes and fulfill the various criteria of meaningful use does not leave any time to treat the patients. For example, forcing physicians to enter 30% of radiology orders does nothing to improve patient care and shifts the workload from nurses and admin staff to the already overworked physicians

Medical specialty

It is ridiculous to ask us to care for very complex patients and provide them with the personal attention they need, including looking at them while we talk, and expect us to fill out all these required fields and check all these boxes. Because I think it is more important to look at the patient than at the computer screen, I spend on average 30 min in addition to direct time with each patient just to complete my note and checkboxes. This is terribly inefficient

Medical specialty

Meaningful use 1 had many features that are useful for actual patient care: e-prescribing, and maintaining a problem list being the most useful. Meaningful use 2 seems to address the administrative issue and not clinical ones, I have honestly not found the elements very helpful, and some take longer and make no difference in terms of article use (like the end of the visit, where a form still needs to be printed). I have found that people sometimes enter misleading or incorrect information to meet the goals, for example, they would not enter the correct diagnosis when it is a rare one, but something similar and this can lead to errors in management because it is easy to copy. What would work better is a way to pull all needed information for a prior authorization for MRIs, sleep studies, and medications-this would actually improve efficiency, meaningful use 2 does not

Surgical specialty

While I am a strong early-adopter and very tech savvy, the implementation of MU is adding many hours to my very full schedule. Too much of it is adding time without adding value. Much of it may help a primary care practice, but not a surgical practice

Surgical specialty

The way meaningful use is being rolled out is quite incompetent; the metrics fall way short of improving patient care. Instead, it is a ton of administrative work that detracts from patient care. It is like asking the pilot to scrub the floor rather than flying the plane

Surgical specialty

Meaningful use is a one-size fits all top-down mandate from Washington with punitive measures if you do not comply. Small pieces of it are helpful, but most of MU is a workflow killer that does not provide any patient benefit. It turns the physician into a data entry machine

With respect to demographic characteristics, physician age was not associated with beliefs about the meaningful use of the EHR ( ). Physician gender was also not associated with beliefs except the belief that meaningful use will not improve patient outcomes. Physician race was associated with beliefs that the meaningful use of the EHR will improve quality of care, the care the physician personally delivers, and the care the AMC delivers. Non-Caucasian physicians had more positive beliefs than Caucasian physicians. For example, 29.1% of non-Caucasian physicians believed that meaningful use would improve the care they personally deliver compared with 19.3% of Caucasian physicians ( p  = 0.003).

Table 5

With respect to practice characteristics, physician specialty was strongly associated with the beliefs that the meaningful use of the EHR will decrease medical errors, will not improve patient outcomes, and will improve the quality of care ( ). Overall, 36.6% of primary care respondents agreed or strongly agreed that the meaningful use of the EHR would decrease medical errors compared with 31.2% of medical specialists and 19% of surgical specialists ( p  = 0.001). Also, 48% of surgical specialists agreed or strongly agreed that meaningful use would not improve patient outcomes compared with 37.0% of medical specialists and 31.5% of primary care physicians ( p  = 0.005).

The number of outpatient hours worked per week was not associated with the beliefs except for the belief that the meaningful use of the EHR will not increase the efficiency of care ( ). The number of outpatients seen per week was significantly associated with all the beliefs except the belief that the meaningful use of the EHR will improve patient-centered care. Respondents who saw 30 or fewer outpatients per week had more positive beliefs than respondents who saw more than 30 outpatients per week. For example, 23.4% of respondents who saw 30 or fewer outpatients per week agreed or strongly agreed that the meaningful use of the EHR would improve quality of care compared with 17.5% of respondents who saw more than 30 outpatients per week ( p  = 0.02). Practice size was not associated with beliefs except for the belief that the meaningful use of the EHR will not improve patient outcomes. Satisfaction with outpatient EHR was significantly associated with all the beliefs: Physicians in the satisfied group had more positive beliefs about the meaningful use of the EHR.

Self-Efficacy

Self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program (defined as moderately confident to extremely confident) was not related to demographic variables (gender, age, and race) ( ). In the case of practice characteristics, number of outpatients seen per week, number of outpatient hours worked per week, and practice size were not associated with self-efficacy. Physician specialty was associated with self-efficacy: 68.1% of primary care physicians reported self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program compared with 56.5% of medical specialists and 44.6% of surgical specialists ( p  < 0.001). Self-efficacy was also significantly related to satisfaction with the outpatient EHR. More than two-thirds of respondents (67.5%) who were satisfied with the outpatient EHR reported self-efficacy for achieving stage 2 meaningful use compared with 49.1% in the other group ( p  < 0.001). Self-efficacy was significantly associated with all the belief items except for the belief that the meaningful use of the EHR will improve patient-centered care. Across all belief items, more positive beliefs were associated with higher self-efficacy for achieving stage 2 meaningful use. The multivariate logistic regression model found three items to be significant correlates of self-efficacy: Satisfaction with the outpatient EHR (odds ratio [OR]: 2.10, 95% confidence interval [CI]: 1.61, 2.75, p  < 0.001), physician specialty (OR: 1.75, 95% CI: 1.24, 2.47, p  < 0.001), and the belief that the meaningful use of the EHR will decrease medical errors (OR: 1.53, 95% CI: 1.04, 2.24, p  = 0.029).

Table 6

Change in Beliefs and Self-Efficacy from Stage 1 to Stage 2

We assessed change in beliefs and self-efficacy from stage 1 to stage 2 of the meaningful use of the EHR incentive program; 518 physicians completed both stage 1 and stage 2 meaningful use surveys. Physicians overall reported more negative beliefs in stage 2 as compared with stage 1 across all belief items ( ). For example, 28.1% agreed or strongly agreed that the meaningful use of the EHR would decrease medical errors in stage 2 as compared with 35.9% in stage 1 ( p  < 0.001). Primary care physicians reported more negative beliefs in stage 2 as compared with stage 1 ( ). For example, 63% of primary care physicians agreed or strongly agreed that meaningful use of the EHR would not increase the efficiency of care in stage 2 compared with 46.2% in stage 1 ( p  < 0.001). However, self-efficacy for achieving meaningful use did not change from stage 1 to stage 2.

Table 7

Discussion

Only a fifth of the physicians responding to our survey agreed or strongly agreed that the meaningful use of the EHR would improve patient-centered care and the quality of care. A majority of physicians, 62.9%, agreed or strongly agreed that the meaningful use of the EHR would not increase the efficiency of the care they provide. These findings show that physicians continue to express negative beliefs about the meaningful use of the EHR. 5 6 7 8

Also, the beliefs in this follow-up study were even more negative than those found with respect to stage 1 of the meaningful use of the EHR incentive program. 8 The greater negative beliefs about the meaningful use of the EHR in stage 2 as compared with stage 1 holds true across all the seven beliefs that we examined in our study. One factor that may play a role in these greater negative beliefs is that stage 2 sets more advanced goals than stage 1 as envisioned by the escalator concept of the stages of meaningful use 1 : it was always expected that stage 2 would be harder to meet than stage 1, and physicians were expected to meet more requirements in stage 2 than stage 1. At the same time, physicians receive about 50% of the incentive for achieving stage 2 that they received in stage 1. 18 Also, whether or not the physicians actually receive the incentives personally depends on their departments and hospitals, and varies considerably across institutions.

Among sociodemographic characteristics, we generally did not find gender and age to be associated with beliefs. As in the case of stage 1 our study did not find a difference between physicians 55 years and older and those younger than 55 in either beliefs about the meaningful use of the EHR or self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program although studies have reported that younger physicians are more likely adopt EHRs and be ready for meaningful use. 14 16

The number of hours worked per week was not associated with beliefs but the number of outpatients seen per week was significantly associated with beliefs about the meaning of the EHR for improving the efficiency of care, patient outcomes, quality of care, and the care the physician and the AMC delivers. Physicians who saw 30 or fewer outpatients per week had significantly greater positive beliefs in these areas than physicians who saw more than 30 outpatients per week. This suggests that physician workload may play an important role in influencing attitudes and beliefs about the meaningful use of the EHR. Several studies have proposed initiatives, such as workflow help and practice redesign to support physician workload. 7 16 19 The NCQA (National Committee for Quality Assurance) patient-centered medical home certification was one of the strongest predictors of progress toward meaningful use in one study. 16 More specifically, a group health study on meaningful use reported that the patient-centered medical home was used to reduce the number of patients seen by primary care physicians. 19 Such initiatives which affect physician workload are likely to play an important role in influencing positive beliefs about the meaningful use of the EHR among primary care physicians. At the same time, similar initiatives need to be developed for medical and surgical specialists who have expressed concerns about participation in the meaningful use program and whether EHRs can be meaningfully used in their areas. 8 9

We found physician specialty was associated with some beliefs but not others. Physician specialty was associated with beliefs that the meaningful use of the EHR will improve quality of care, decrease medical errors, and improve patient outcomes. Primary care physicians had more positive beliefs in these areas than specialists as we and others have reported. 8 20 However, physician specialty was not associated with beliefs that the meaningful use of the EHR will improve the efficiency of care, patient-centered care or the care the physician and organization deliver. Moreover, our study found that the gap between primary care physicians and specialists decreased with respect to these beliefs as primary care physicians reported greater negative beliefs about the meaningful use of the EHR in response to stage 2. As previously noted, and as the comments in illustrate, the greater number of requirements in stage 2 could be one contributing factor to these greater negative beliefs. Another factor that could influence the greater negative beliefs among primary care physicians in stage 2 are the criteria related to patient engagement whose achievement falls to a large extent on such physicians. In a study examining physician perceptions and beliefs about one of these criteria, the provision of the AVS to patients, we found that a majority of primary care physicians (79%) reported a negative effect of the provision of the AVS on workload. 9

Finally, physician satisfaction with the outpatient EHR was significantly associated with all the beliefs, with physicians in the satisfied group reporting more positive beliefs about the meaningful use of the EHR. Physician satisfaction with the outpatient EHR also emerged as the most significant predictor of self-efficacy in our multivariate model. Physicians who were satisfied with the EHR were two times more likely to have self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program. Physician satisfaction with the EHR appears to be an important factor in beliefs and self-efficacy at a global level as found in our studies as well as with respect to the specific functionality of the EHR. Makam et al 21 found variations in satisfaction with key EHR functionality and concluded that dissatisfaction with and suboptimal use of such key functionality might in part explain the lack of association between EHR use and quality of care.

Limitations

This study was conducted in the setting of two AMCs in one region. Thus, the study findings may not apply to other settings, including different practice types or regions. However, many of the physicians evaluated do practice in the community setting. Our study focused on beliefs captured as outcome expectancies, but it is troubling that the beliefs were not more positive. Other types of beliefs could also be examined, including beliefs about controllability and normative beliefs. 22 Another limitation of the study was that the response rate of the survey was just over half (52.9%) and the physicians who responded to our survey may have stronger negative beliefs than those who chose not to respond to the survey. Finally, the study also did not include factors, such as physician expectations of EHRs, which may also impact beliefs and self-efficacy 23 and did not assess outcomes which may or may not be correlated with beliefs.

Conclusion

We conducted a follow-up survey of physician beliefs about the meaningful use of the EHR. We found that physicians continue to have negative beliefs about the meaningful use of the EHR in very important domains including efficiency, patient-centeredness, and quality of care, and that these negative beliefs have increased from stage 1 to stage 2 of the meaningful use of the EHR incentive program. These findings potentially represent a barrier to the meaningful use of the EHR, and more broadly to improving quality, safety, and efficiency with EHRs. If negative beliefs about the meaningful use of the EHR persist and grow, the escalator approach to implementing the meaningful use of the EHR incentive program as well as the speed of the escalator will be impacted. One study found that active users of the meaningful use program are dropping out of the program frustrated by the effort required compared with the incentive offered for achieving meaningful use. 7

With respect to the role of physician specialty in meaningful use, two findings emerged from this follow-up study that can be of relevance to practitioners. The gap between primary care physicians and specialists in beliefs about the meaningful use of the EHR decreased as primary care physician beliefs grew more negative from stage 1 to stage 2. It is possible that this finding is specific to the physicians in our study. However, we suspect that other health care settings may see the development of more negative beliefs among primary care physicians given the increasing requirements of meaningful use from one stage to the next. At the same time, primary care physicians had greater self-efficacy than specialists for achieving stage 2 meaningful use. This is not surprising, as primary care physicians have been using EHRs much more intensively and longer than specialists, who are finding meaningful use to be much more disruptive to existing workflows. 24 To enhance self-efficacy among specialists, interventions, such as feedback on individual performance and interactive continuing medical education (CME) that focuses on skill development could be deployed. 25

This study, and others we have conducted, 8 9 found physician satisfaction with the EHR to be an important factor in beliefs about the meaningful use of the EHR and self-efficacy for achieving meaningful use. This suggests the importance of enhancing physician satisfaction with the EHR for successful implementation of the meaningful use of the EHR incentive program. Physicians can play an important role in this process by suggesting improvements and enhancements in the key functionality of the EHR. 21 Finally, we suggest the inclusion of physician experience with EHRs, such as physician satisfaction with the use of EHRs, in evaluations of meaningful use similar to a proposal for the inclusion of patient experience in meaningful use. 19

Clinical Relevance Statement

Given that few physicians had positive beliefs about the meaningful use of the EHR, and that negative beliefs may be growing, we suggest that organizations should devote strategies and resources to promote positive beliefs about the meaningful use of the EHR among all physicians (and not just specialists). As the growing literature on this subject suggests, there does not appear to be one fix to the problem. Instead, a range of strategies will need to be adopted such as practice and workflow redesign, enhancement of EHR functionality as needed, training, collaborative learning sessions and practice improvement advice, and audit and feedback of individual performance around the criteria for becoming a meaningful user of the EHR. Physicians can provide valuable input into the suite of strategies that best fit practices and providers, reduce physician workload and increase the effectiveness of care and the care the physician delivers. 26 27

Multiple Choice Questions

The escalator concept of meaningful use refers to:

The three stages of the meaningful use of the EHR incentive program

Providing incentives to physicians to participate in the meaningful use of the EHR incentive program

A staged approach toward achieving the final goals of the Meaningful use of the EHR incentive program

Differential beliefs about meaningful use of the EHR among primary care physicians and specialists

Correct Answer: The correct answer is C. A staged approach to achieving the goals of the meaningful use of the EHR incentive program has been considered analogous to the concept of an escalator. 1 With each stage, the notion has been that providers would move upward toward the final goal of achieving improvement in quality, safety, and efficiency of care. 2

In this study the strongest predictor of self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program was:

Physician specialty

Practice size

Number of outpatients seen per week

Physician satisfaction with outpatient EHR

Correct Answer: The correct answer is D. The multivariate logistic regression model found three items to be significant predictors of self-efficacy for achieving stage 2 of the meaningful use of the EHR incentive program, of which the strongest predictor was physician satisfaction with the outpatient EHR: physician satisfaction with the outpatient EHR (OR: 2.10, 95% CI: 1.61, 2.75, p  < 0.001), physician specialty (OR: 1.75, 95% CI: 1.24, 2.47, p  < 0.001), and the belief that the meaningful use of the EHR will decrease medical errors (OR: 1.53, 95% CI: 1.04, 2.24, p  = 0.029).

Protection of Human and Animal Subjects

The study was performed in compliance with the World Medical Association Declaration of Helsinki on ethical principles for medical research involving human subjects, and was reviewed by the Partners HealthCare institutional review board.

Footnotes

Conflict of Interest None.

References

11. Abbett S K, Bates D W, Kachalia A.The meaningful use regulations in information technology: what do they mean for quality improvement in hospitals? Jt Comm J Qual Patient Saf 201137 07333-336., 289 [[ PubMed] Google Scholar]

17. Hosmer D W, Lemeshow S. Hoboken, NJ: John Wiley & Sons, Inc.; 2000. Special Topics. [ Google Scholar]

# data[].attributes.attachment_link.title
Physician Beliefs about the Meaningful Use of the Electronic Health Record: A Follow-Up Study

# data[].attributes.attachment_link.description
Background  There is continuing interest in how physicians are responding to the meaningful use of the electronic health record (EHR) incentive program. However, little research has been done on physician beliefs about the meaningful use of the EHR. Objective  This study aims to conduct a follow-up study of physician beliefs about the meaningful use of the EHR.

# data[].attributes.body
This is a similar image for anyone to reference when they want to differentiate the objectives between the three stages. It is important to remember each subsequent stage builds upon the prior one(s).

# data[].attributes.body
It is interesting that stage 1 incentives have the highest payments, but I would imagine the goal is to encourage everyone to at least utilize the stage 1 objectives. Possibly over time, and once the majority have implemented stage 1, the annual incentive payments for stage 2 and 3 will increase to encourage more to adopt those objectives as well.

# data[].attributes.body
As something that's namely used for research and development, I did not think about applying it to our daily work. But this article talks about the benefits of sharing data, information, and knowledge during nursing handoff to reduce adverse patient events due to miscommunication. As future NPs, we could consider utilizing aspects of this model to give hand off reports to other providers if we need to refer our patients for emergent or specialty care. The article mentions that more research is needed to investigate other attributes of staff and their relationship with what is provided in hand off report - those include years in practice, specialty area, and education level.

# data[].attributes.attachment_link.readable_text
Galatzan, Benjamin J. PhD, RN; Carrington, Jane M. PhD, RN, FAAN

Author Information

Author Affiliations: University of Alabama at Birmingham School of Nursing (Dr Galatzan); and College of Nursing Health Professions, University of Florida, Gainesville (Dr Carrington).

This work was supported by Renz Doctoral Award, College of Nursing, The University of Arizona.

The authors have disclosed that they have no significant relationships with, or financial interest in, any commercial companies pertaining to this article.

Corresponding author: Benjamin J. Galatzan, PhD, RN, University of Alabama at Birmingham School of Nursing, 1701 University Blvd, Birmingham, AL 35294 ().

# data[].attributes.attachment_link.title
Communicating Data, Information, and Knowledge in the... : CIN: Computers, Informatics, Nursing

# data[].attributes.attachment_link.description
he aim of this study was to examine the content of the nurse-to-nurse change-of-shift handoff communication in terms of data, information, and knowledge for both bedside and nonbedside handoffs of a patient who has experienced a clinical event. The setting was an urban medical center on a medical-surgical floor.

# data[].attributes.body
For our future practice, here is a free template to help us conduct a company analysis of strengths, weaknesses, opportunities, and threats.

# data[].attributes.attachment_link.readable_text
If you've ever worked in a corporate office environment, you may have come across the term "SWOT analysis." This has nothing to do with evaluating militarized law enforcement response units, and everything to do with taking a long, hard look at your company.

Conducting a SWOT analysis is a powerful way to evaluate your company or project, whether you're two people or 500 people. In this article, you'll learn: what a SWOT analysis is, see some SWOT analysis examples, and learn tips and strategies for conducting a comprehensive SWOT analysis of your own. You'll also see how you can use the data a SWOT exercise yields to improve your internal processes and workflows, and get a free, editable SWOT analysis template.

The Complete Guide to SWOT Analysis:
Table of contents
What is a SWOT analysis?

A SWOT analysis is a technique used to determine and define your Strengths, Weaknesses, Opportunities, and Threats - SWOT.

SWOT analyses can be applied to an entire company or organization, or individual projects within a single department. Most commonly, SWOT analyses are used at the organizational level to determine how closely a business is aligned with its growth trajectories and success benchmarks, but they can also be used to ascertain how well a particular project - such as an online advertising campaign - is performing according to initial projections.

Whatever you choose to call them, SWOT analyses are often presented as a grid-like matrix with four distinct quadrants - one representing each individual element. This presentation offers several benefits, such as identifying which elements are internal versus external, and displaying a wide range of data in an easy-to-read, predominantly visual format.

Breaking down the SWOT analysis definition

We know that SWOT stands for Strengths, Weaknesses, Opportunities, and Threats - but what does each of these elements mean? Let's take a look at each element individually.

Strengths

The first element of a SWOT analysis is Strengths.

Things your company does well

Qualities that separate you from your competitors

Internal resources such as skilled, knowledgeable staff

Tangible assets such as intellectual property, capital, proprietary technologies, etc.

As you've probably guessed, this element addresses things that your company or project does especially well. This could be something intangible, such as your company's brand attributes, or something more easily defined such as the unique selling proposition of a particular product line. It could also be your people, your literal human resources: strong leadership, or a great engineering team.

Weaknesses

Once you've figured out your strengths, it's time to turn that critical self-awareness on your weaknesses.

Things your company lacks

Things your competitors do better than you

Resource limitations

Unclear unique selling proposition

What's holding your business or project back? This element can include organizational challenges like a shortage of skilled people and financial or budgetary limitations.

This element of a SWOT analysis may also include weaknesses in relation to other companies in your industry, such as the lack of a clearly defined USP in a crowded market.

Opportunities

Next up is Opportunities.

Underserved markets for specific products

Few competitors in your area

Emerging needs for your products or services

Press/media coverage of your company

Can't keep up with the volume of leads being generated by your marketing team? That's an opportunity. Is your company developing an innovative new idea that will open up new markets or demographics? That's another opportunity.

In short, this element of a SWOT analysis covers everything you could do to improve sales, grow as a company, or advance your organization's mission.

Threats

The final element of a SWOT analysis is Threats - everything that poses a risk to either your company itself or its likelihood of success or growth.

Emerging competitors

Changing regulatory environment

Negative press/media coverage

Changing customer attitudes toward your company

This could include things like emerging competitors, changes in regulatory law, financial risks, and virtually everything else that could potentially jeopardize the future of your company or project.

SWOT analysis internal and external factors

The four elements above are common to all SWOT analyses. However, many companies further compartmentalize these elements into two distinct subgroups: Internal and External.

Internal factors

Typically, Strengths and Weaknesses are considered internal factors, in that they are the result of organizational decisions under the control of your company or team. A high churn rate, for example, would be categorized as a weakness, but improving a high churn rate is still within your control, making it an internal factor.

External factors

Similarly, emerging competitors would be categorized as a threat in a SWOT analysis, but since there's very little you can do about this, this makes it an external factor. This is why you may have seen SWOT analyses referred to as Internal-External Analyses or IE matrices.

Subcategorizing your four primary elements into Internal and External factors isn't necessarily critical to the success of your SWOT analysis, but it can be helpful in determining your next move or evaluating the degree of control you have over a given problem or opportunity.

Now that we know what each of the elements of a SWOT analysis means, let's take a look at how to go about creating and conducting a SWOT analysis.

How to do a SWOT analysis

You can get the full experience in our video below, and this entire post is dedicated to answering that question, but for simplicity's sake, here's how to do a SWOT analysis:

Gather your team together-ideally bring candy.

Set up your quadrants-on a whiteboard or projector (perhaps using our template).

Start with strengths-ask the below list of questions.

Follow suit with weaknesses, opportunities, and threats.

Organize the information collected into a neat and tidy document.

Send out to the team with notes.

Organize a second meeting to come up with action items and owners.

SWOT analysis questions

Like feature-benefit matrices, there are several ways to conduct a SWOT analysis. However, regardless of how you choose to structure your analysis, we need to start by asking a series of questions. Here is a breakdown of the questions you should seek to answer when performing your SWOT analysis.

Strengths questions

Let's take our first element, Strengths, for example. To determine what your strengths are as an organization, you could begin by asking some of the following questions:

What do your customers love about your company or product(s)?

What does your company do better than other companies in your industry?

What are your most positive brand attributes?

What's your unique selling proposition?

What resources do you have at your disposal that your competitors do not?

By answering these questions, you'll be in great shape to start identifying and listing your organization's strengths.
Positive brand attributes associated with WordStream, as identified by our customers
Weakness questions

We can use the same principle to determine your company's weaknesses:

What do your customers dislike about your company or product(s)?

What problems or complaints are often mentioned in your negative reviews?

Why do your customers cancel or churn?

What could your company do better?

What are your most negative brand attributes?

What are the biggest obstacles/challenges in your current sales funnel?

What resources do your competitors have that you do not?

You may find that determining the strengths and weaknesses of your organization or project is considerably easier or takes less time than figuring out the opportunities and threats facing your company. This is because, as we said earlier, these are internal factors. External factors, on the other hand, may require more effort and rely upon more data, as these are often beyond your immediate sphere of influence.

Opportunities questions

Identifying opportunities and threats may require you to conduct in-depth competitive intelligence research about what your competitors are up to, or the examination of wider economic or business trends that could have an impact on your company. That's not to say that opportunities and threats cannot be internal, however; you may discover opportunities and threats based solely on the strengths and weaknesses of your company. Some possible questions you could ask to identify potential opportunities might include:

How can we improve our sales/customer onboarding/customer support processes?

What kind of messaging resonates with our customers?

How can we further engage our most vocal brand advocates?

Are we allocating departmental resources effectively?

Is there budget, tools, or other resources that we're not leveraging to full capacity?

Which advertising channels exceeded our expectations - and why?

Threat questions

When it comes to threats, you could certainly begin by asking a series of questions like those above. However, it's often quite easy to come up with a list of potential threats facing your business or project without posing questions beforehand. This could include "branded" threats such as emerging or established competitors, broader threats such as changing regulatory environments and market volatility, or even internal threats such as high staff turnover that could threaten or derail current growth.

What is PEST analysis?

While we're on the topic of internal versus external factors, I wanted to mention a tangential but entirely separate type of analysis closely relevant to SWOT analyses, known as a PEST analysis.

Earlier, I mentioned that external factors such as changing regulatory policies and market volatility could be considered threats in a standard SWOT analysis. However, despite their importance, challenges like this are often highly nuanced and driven by dozens or hundreds of individual factors. This can place them beyond the scope or intent of a typical SWOT analysis. This is why many companies also conduct PEST analyses.

This type of analysis is not what an exterminator does upon arriving at a roach-infested tenement. Rather, a PEST analysis functions very similarly to a SWOT analysis, only they're concerned with four external factors:

Pros of PEST analysis

One of the main reasons it's worth looking at PEST analyses is because many of the factors that could end up in a PEST matrix could also be relevant to the Opportunities and Threats in our SWOT analysis. The kind of political and economic turmoil we've seen in the United States during the past year, for example, could very well pose legitimate and serious threats to many businesses (as well as some opportunities), but these kinds of obstacles tend to be much more complicated than the opportunities and threats you'd see in most SWOT analyses, given their broader scale and often-complex underlying factors.
Image via Dr. Jean-Paul Rodrigue/Hofstra University
Cons of PEST analysis

Obstacles identified in a typical PEST analysis also tend to be on much longer timeframes - it's a lot easier and quicker to try and overcome internal challenges like high staff turnover than it is to wait and see if the economy picks up (or if the bubble will burst again). That's why many larger companies conduct both SWOT and PEST analyses simultaneously - the SWOT analysis provides them with more immediate, potentially actionable roadmaps, whereas PEST analyses can be highly valuable when it comes to formulating longer-term plans and business strategies.

Benefits of SWOT analysis for small businesses

If you're a marketer or small-business owner, you might be wondering if SWOT analyses are practical or even feasible for smaller companies and organizations. Although there is definitely a resource overhead involved in the creation of a SWOT analysis, there are many benefits in doing so, even for the smallest of companies.

Get a bird's eye view: For one, conducting a comprehensive SWOT analysis provides a unique opportunity to gain greater insight into how your business operates. It's all too easy to get lost in the weeds of the day-to-day workings of your company, and conducting a SWOT analysis allows you to take a broader, bird's eye view of your business and the position it occupies in your industry.

Improve specific campaigns and projects. Another benefit of SWOT analyses is that this technique can be applied to a wide range of scenarios, not just as an overview of your business. You could use SWOT analyses to evaluate the potential strengths and weaknesses of a forthcoming advertising campaign, a planned content project, or even whether your company should be represented at a trade show or industry event.

Here's an example of a project SWOT analysis:

Develop tangible roadmaps. Obviously, it almost goes without saying that conducting a SWOT analysis allows you to identify what your company does well, where it could improve, and the opportunities and threats facing your business. However, conducting a SWOT analysis provides you with the opportunity to not only identify these factors, but also develop and implement tangible roadmaps and timelines for potential solutions. This can be beneficial in the creation of budgetary plans, identifying hiring needs and other mid- to long-term strategic planning.

A full SWOT analysis example

So, now we know what each element of a SWOT analysis is concerned with and the kinds of exploratory questions we can ask to get the ball rolling, it's time to actually get to work and create your SWOT analysis.

To illustrate how it works, we'll create our own SWOT analysis example: a family-owned restaurant, with a single location, operating in an urban area.

Here's the SWOT analysis example based on our fictional restaurant:

As you can see, this matrix format allows you to quickly and easily identify the various elements you've included in your analysis.

Strengths examples

Excellent, well-trafficked location

Good reputation in local community

Seasonal menu, locally sourced.

Weakness examples

Higher costs than comparable chain restaurants

Single location means limited reach

Modest advertising budget

Not currently using food delivery apps/technology

Opportunity examples

Growing interest in/support for locally sourced ingredients

Seasonal menu keeps things fresh and interesting

Potential growth via food delivery apps/technology

Threat examples

Intensifying competition from established chain restaurants

Uncertain economic environment

Rising cost of ingredients

SWOT analysis example: Acting on your results

So, you've finally got your hands on a completed SWOT matrix. You've identified internal strengths and weaknesses, as well as external opportunities and threats. You've begun to see your company in a whole new light.

Now what?

Ideally, there are two stages of action you should take upon completing a SWOT analysis. First, you should attempt to match your strengths with your opportunities. Next, you should try to convert weaknesses into strengths. Let's take a look how this works.

1. Harness your strengths

One of the best things about the strengths you identified in your SWOT analysis is that you're already doing them.

In our example above, the restaurant's location, reputation, and seasonal menu are all strengths. This tells the fictitious company that it should continue to experiment with its popular seasonal menu. It also tells the company it should continue to develop and nurture the strong relationships with its regular customers that have strengthened the restaurant's reputation in the community.

Essentially, acting upon your business' strengths consists of "do more of what you're already good at."

2. Shore up your weaknesses

Acting on the weaknesses you identified in your SWOT analysis is a little trickier, not least because you have to be honest enough with yourself about your weaknesses in the first place.

Going back to our example, some of these weaknesses are very challenging to act upon. Going up against the considerable purchasing power of rival chain restaurants can be very difficult for smaller, family owned businesses. The restaurant is also struggling with its limited reach, the restrictions of a modest advertising budget, and is also failing to leverage the potential to increase sales by allowing customers to order food online through delivery apps like Foodler or GrubHub.

However, that's not to say all hope is lost. It might be harder for our example business to compete with a chain, but there are plenty of other ways small companies can be more competitive - such as by developing strong, meaningful relationships with customers, which was not only one of the company's strengths, but also something chain restaurants simply cannot offer.

3. Seize opportunities

The Opportunities section of your SWOT analysis is by far the most actionable, and that's by design. By identifying opportunities by evaluating your organization's strengths, you should have a ready-made list of targets to aim for.

In the example above, increasing consumer appetites for ethically produced, locally grown ingredients is a major opportunity. However, our restaurateurs cannot rest on their laurels - there's still work to be done. In this example, this may involve investing in technical expertise to take advantage of the opportunities presented by food delivery apps, or sourcing locally grown produce more aggressively in an attempt to reduce costs.

It's also important to avoid hubris or complacency in your opportunities. Even if you have an iron-clad advantage over every other business in your industry, failing to devote sufficient time, money, or personnel resources in maintaining that advantage may result in you missing out on these opportunities over time.

Every business' opportunities will differ, but it's vital that you create a clearly defined roadmap for capitalizing upon the opportunities you've identified, whether they be internal or external.

4. Mitigate threats

Anticipating and mitigating the threats identified in your SWOT analysis may be the most difficult challenge you'll face in this scenario, primarily because threats are typically external factors; there's only so much you can do to mitigate the potential damage of factors beyond your control.

Every threat, and the appropriate reaction to that threat, is different. Regardless of the specific threats you've identified in your SWOT analysis, responding to and monitoring those threats should be among your very top priorities, irrespective of the degree of control you have over those threats.

In the example above, all three threats are particularly challenging. To compete with the prices of its chain competitors, our restaurateurs may be forced to either compromise on their values to secure cheaper ingredients, or willingly cut into their profit margins to remain competitive. Similarly, economic uncertainty is virtually impossible to fully mitigate, making it a persistent threat to the stability of our example restaurant business.

In some SWOT analyses, there may be some overlap between your opportunities and threats. For example, in the analysis above, the popularity of locally sourced ingredients was identified as an opportunity, and heightened competition was identified as a threat. In this example, highlighting the restaurant's relationships with local farmers - further reinforcing the restaurant's commitment to the local community and regional economy - may be an effective way for our restaurateurs to overcome the threat posed by the increasingly desperate chain restaurants vying for their customers.

When compiling the results of your SWOT analysis, be sure to look for areas of crossover like this and see if it's possible to seize an opportunity and reduce a threat at the same time.

Free SWOT Analysis template

Ready to put it all into practice for your own business? Here's our free, editable, SWOT analysis template.

It's a Google Sheet so you can add and remove rows, edit and answer the questions, paste into a slide deck, and more.

We've also thrown a blank one in there too:

# data[].attributes.attachment_link.title
How to Do a SWOT Analysis (Examples & Free Template!)

# data[].attributes.attachment_link.description
What is a SWOT analysis? With this guide, learn how to do a SWOT analysis with a real small business example and an editable, free SWOT analysis template.

# data[].attributes.body
This is an infographic depicting the DIKW Hierarchy in terms of time, context, and understanding. Understanding the hierarchal steps, how one leads to the next, and the transformation of understanding through each allows individuals and institutions to utilize this concept more efficiently for policy and protocol reform. In my experience, more emphasis should be placed on reflection.

# data[].attributes.body
This is an interesting article discussing some challenges of using EHRs for research. There is a significant amount of evidence about why EHRs are beneficial, but looking at the downfalls are how informatics can continue to improve for the better.

# data[].attributes.attachment_link.title
Electronic health records to facilitate clinical research - Clinical Research in Cardiology

# data[].attributes.attachment_link.description
Electronic health records (EHRs) provide opportunities to enhance patient care, embed performance measures in clinical practice, and facilitate clinical research. Concerns have been raised about the increasing recruitment challenges in trials, burdensome and obtrusive data collection, and uncertain…

# data[].attributes.attachment_caption
Advancing Nursing Informatics: From Learner to Leader 

# data[].attributes.body
This link shows some great infographics from a HIMMS powerpoint presentation discussing informatics' role, practice setting, & functional areas. The ending mentions HIMM Analytics and it's role in "enabling better health through information technology".

# data[].attributes.body
This Webinar discusses the 2021 essentials for education & practice, the "new model for nursing education", and implementation. The domains of competence, standardization of core competencies, ideas for how education programs can better prepare nursing graduates, importance of community-based education, and how schools can implement the new Essentials were some of the topics discussed.

# data[].attributes.attachment_link.title
American Association of Colleges of Nursing: The Essentials

# data[].attributes.attachment_caption
The Evolution of TIGER Competencies & Informatics Resources

# data[].attributes.body
This next graphic shows a more in depth structure of SNOMED CT to better understand how relationships define and connect concepts.

# data[].attributes.body
The logical model organizes concepts used to represent & descriptions used to refer to clinical meanings, as well as associations between concepts.

# data[].attributes.body
Wow! Suggested from research, eliminating behavior-related risk factors could prevent 80% of all heart disease, diabetes, and stroke & 40% of US cancer. Digital Health can help shape patient behaviors along with other solutions to help healthcare improve.

# data[].attributes.attachment_link.title
Digital Health Approaches for Improved Population Health Outcomes: Time for a Disease Vulnerability Matrix for Individuals and Communities?

# data[].attributes.attachment_link.readable_text
About 25% of patients used telehealth last year, far exceeding the 5% who accessed care this way before the pandemic. And as the budding field evolves, that share is expected to continue to rise in the coming years.

During an AMA Telehealth Immersion Program webinar, experts shared their views and perspectives on how a system propelled to the forefront of medicine evolved in 2022 and where it is headed in 2023.

Telehealth usage spiked to 70% in 2020, but that happened in a U.S. health care system that wasn't built or optimized for such an influx of patients.

The usability wasn't there and, culturally, patients and physicians who weren't fully ready to move to virtual care were happy to go back to in-person visits, internist Eyal Zimlichman, MD, said during the webinar. He is the chief innovation and transformation officer at Sheba Medical Center, Israel's largest hospital, and previously served as an adviser to the U.S. Department of Health and Human Services Office of the National Coordinator for Health Information Technology.

So, the nation's health care system will experience an evolution-not a revolution-of telehealth, he said.

"We are going to see a slow rise from the 20% to 25% we are currently [seeing], moving up maybe toward that 70% again, but in a more natural course where the usability is improving and where the culture is slowly getting there," Dr. Zimlichman said. "It's going to take another five years or so ... to see that go up to the 50% to 70% rate that we've seen" during the early part of the pandemic.

Supporting telehealth is an essential component of the AMA Recovery Plan for America's Physicians.

Telehealth is critical to the future of health care, which is why the AMA continues to lead the charge to aggressively expand telehealth policy, research and resources to ensure physician practice sustainability and fair payment.

5 predictions for 2023

Dr. Zimlichman was joined by panelists Lisa Lavin, founder and CEO of Ōmcare Inc., a digital health care company offering a customizable home health platform for organizations, and Tory Cenaj, founder and publisher of Telehealth and Medicine Today.

Subscribe to AMA Update

Get videos with expert opinions from the AMA on the most important health care topics affecting physicians, residents, medical students and patients-delivered to your inbox.

Here are their predictions for telehealth in 2023.

Telehealth will continue to evolve into something that's easier to use. Expect to see more effort and money put into the usability of telehealth solutions. Patients and physicians have had a taste of how technology can revolutionize health care, but the technology needs to continue to improve to expand the number of people who use telehealth on a regular basis.

The field will see steps toward aggregated, all-in-one technology. Look for some "eyebrow-raising" joint ventures between technology and health care professionals that are going to help take steps toward more seamless technology.

Chronic disease management will see strides. Medicine has been focused on pharma when it comes to managing chronic illness. But it is much more complicated than just pills and that's where digital health tools will come into play. Expect to see more wearable technology and sensors, along with chat boxes and symptom-management tools to manage an area that is one of the biggest buckets of cost and morbidity and mortality in health care.

Government funding and regulation for telehealth will continue to be a hot topic. Telehealth will only continue to grow if physicians and other health care providers are being paid properly for the services. Look for this, along with regulatory issues such as licensing and treating patients across state lines to get more attention in 2023.

More capital will flow to innovators in the digital health care arena. Investors have capital to deploy into health care and a number of health care companies are sitting on large capital reserves. Expect more capital to go to innovators in virtual care and digital health solutions.

Learn more about the AMA's new " Blueprint for Optimizing Digitally Enabled Care " (PDF), which is designed to help bridge the digital health chasm as the pace of digital health progress in medicine does not yet match the technology's potential.

The Telehealth Immersion Program is part of the AMA STEPS Forward™ Innovation Academy, which enables physicians to learn from peers and experts and discover ways to implement and optimize time-saving practice innovation strategies.

# data[].attributes.attachment_link.title
What to expect in telehealth in 2023? Here are 5 predictions

# data[].attributes.attachment_link.description
Telemedicine use is likely to see another upswing, say experts. Learn what else they are forecasting for this year.

# data[].attributes.body
Advances in image processing allows image features to be analyzed. Improvements in precision, efficiency, and consistency have been noted compared to the previous manual or semi-automatic with manual correction analysis techniques used to measure segmentation of regions of interest.

# data[].attributes.body
Fundamental Health Informatics Principles
- intersection of information science, computer science, & health care
- collection, retrieval, storage, analyzation, & use of data
- used to increase productivity & decrease costs of healthcare; improve patient outcomes & safety

Key Players Involved: patients (the focus), government (CMS), clinicians (care treatment plans), researchers (to find new patterns/trends), public health (health policy development), vendors (developments of new innovations), education, health plans (cost reduction), clinics & hospitals (process & quality improvement)

Future of Healthcare IT (video)
- Radiology: now looking at digital pathology & genomics; EMR; wearables from blood test
- Cardiology: bring together blood & genetic information; electronic cardiogram from watch
- Oncology: MRI image to calculate lesion & determine how much is malignant

Knowledge in Digital Age: levels of data (1-4), "data warehouse" combining EHRs from various vendors, algorithms used for classification, banking (numbers) & information (account balances), health IT ecosystems to sort patients by characteristics to determine trends & develop health policy, a lot of ambiguity in healthcare currently

Digital Healthcare (video)
- Gap in efforts and outcomes -> higher costs, poor consumer experience, lack of coordination
- 360 degree strategy = remote care models + clinical data integration + analytics + social media
- mobile apps to connect consumers w/ providers, personalize & monitor wellness goals, community interactions, & payment management