# data[].attributes.body
Source: National Center for Biotechnology Information (NCBI)

Author: Institute of Medicine (IOM)

Date Published: 2020

# data[].attributes.attachment_link.readable_text
For more than 30 years, clinicians, health services researchers, and others have been investigating the use of advanced telecommunications and information technologies to improve health care. At the intersection of many of these efforts lies telemedicine-a combination of innovative and mainstream technologies. As defined here, telemedicine is the use of electronic information and communications technologies to provide and support health care when distance separates the participants.

Telemedicine has a variety of applications in patient care, education, research, administration, and public health. Some uses such as emergency calls to 911 numbers using ordinary telephones are so commonplace that they are often overlooked as examples of distance medicine. Other applications such as telesurgery involve exotic technologies and procedures that are still in the experimental stage. The use of interactive video for such varied purposes as psychiatric consultations and home monitoring of patients attracts much attention and news coverage, although such applications are far from routine in everyday medical practice.

For many decisionmakers, the case for new or continued investment in telemedicine remains incomplete, particularly given the competition for resources in an era of budgetary retrenchment in health care and government. Most clinical applications of telemedicine have not been subjected to systematic comparative studies that assess their effects on the quality, accessibility, or cost of health care. Although telemedicine is hardly unique among health care services in lacking evidence of its effectiveness, the increasing demand for such evidence by health plans, patients, clinicians, and policymakers challenges advocates of clinical telemedicine to undertake more and better evaluations of its practicality, value, and affordability.

In response to the scarcity of sound evaluations, the National Library of Medicine ( NLM) asked the Institute of Medicine ( IOM) to develop a broad framework for evaluating clinical telemedicine. This report, developed by a 15-member committee of the IOM, presents that framework, which focuses on telemedicine's effects on the quality, accessibility, cost, and acceptability of health care. The objective is to encourage evaluations that will guide policymakers, reassure patients and clinicians, inform health plan managers, and help those who have invested in telemedicine to identify shortcomings and improve their programs. This report is aimed primarily at these policymakers, clinicians, patients, and managers, but it is also intended to provide context and support for researchers with an interest in evaluating information and communications technologies.

Telemedicine Past And Present

Historically, access concerns have driven much of the work to develop clinical telemedicine. Early applications often focused on remote populations scattered across mountainous areas, islands, open plains, and arctic regions where medical specialists and sometimes primary care practitioners were not easily reached. Most of the telemedicine projects from the 1960s through the early 1980s failed, however, to survive the end of grant funding or trial financing. Telecommunications costs tended to be high, and the technologies were awkward to use. Few projects appeared to be guided by a business plan or an appreciation of the project features and results necessary for a sustainable program.

Recently, another wave of interest in telemedicine has prompted a range of new activities. Costs have dropped for many of the information and communications technologies supporting telemedicine, and the developing National Information Infrastructure ( NII) is making these technologies more commonplace and more easily used.

Teleradiology appears to be the most common application, in part because Medicare and other payers reimburse for radiology consultations without demanding the face-to-face relationship required for most other consultations.

With the nation's health care system undergoing profound changes and experiencing relentless financial pressures, telemedicine is being investigated for its utility in urban as well as rural settings. To the extent that telemedicine offers a mechanism for centralizing specialists and supporting primary care clinicians, managed care plans may find certain applications efficient and attractive in the cities and suburbs where their patients are concentrated. Some academic medical centers and other organizations, faced with reduced revenues and even exclusion from local managed care networks, are exploring telemedicine as they seek to develop new regional, national, and international markets for their highly specialized clinicians. In these contexts, telemedicine has the potential to radically reshape health care in both positive and negative ways and to fundamentally alter the personal face-to-face relationship that has been the model for medical care for generations.

Despite recent growth, obstacles to widespread use of clinical telemedicine persist. For example, although many groups are working to develop hardware and software standards, it remains frustrating and difficult to put together systems in which the components operate predictably and smoothly together, work in different settings without extensive adaptation, and accommodate replacement components. Technical systems still may be poorly adapted to the human infrastructure of health care, that is, the work environment, needs, and preferences of clinicians, patients, and other decisionmakers. Moreover, sustainable telemedicine programs require attention to organizational business objectives and strategic plans that is not always evident in current applications.

In a period characterized by increased competition, structural realignments, and surpluses of some categories of health professionals, clinicians may see telemedicine as an economic threat. Even though interstate telemedicine is not a priority for many users or potential users, jurisdictional issues relating to professional licensure and medical liability are generating considerable controversy. As computer-based patient information systems and databases have proliferated, the relative weakness of state and federal policies to protect the privacy and confidentiality of personal medical information has stimulated legislative reform proposals but no action to date.

Challenges In Evaluating Clinical Telemedicine

Major challenges confront those evaluating clinical applications of telemedicine. These difficulties also characterize many other applications of advanced technologies, and, thus, they are not unique to telemedicine. Nonetheless, the combination of challenges is formidable. They include

the rapid advance of information and telecommunications technologies, which exposes systematic and often expensive evaluations to obsolescence as key hardware and software components of telemedicine applications move from state of the art to outmoded;

a complex and often unwieldy technical infrastructure, which may yield disappointing evaluations until it becomes more ubiquitous and user-friendly;

a diverse and sometimes dazzling array of telemedicine technologies and uses that may distract managers and evaluators from the task of identifying practical, affordable, and sustainable ways to achieve defined quality, access, or cost objectives; and

the unusual level of cooperation that medicine at a distance often demands of independent institutions and individuals whose reluctance to participate may preclude the kinds of comparisons and the volume of cases needed for strong evaluations.

In addition, several more general challenges may complicate evaluations of clinical telemedicine. One is the restructuring of the nation's health care delivery system, which has brought with it shifts in institutional missions and priorities related to patient care, education, and research. A second is the growth of investor-owned enterprises that are not much inclined to allocate resources for purposes such as clinical research that do not add to corporate profits. At the state and federal level, policymakers are cutting budgets and may be reluctant to shift even modest resources from the core activities of grant programs to support evaluations of their actual consequences.

Fortunately, a number of government and private organizations have recognized the need for more systematic evaluation of telemedicine. This report draws on this work as well as on the contributions of individual researchers who are also working to improve the methods and strengthen the evidence base for telemedicine.

A Framework For Evaluation

In most respects, better evaluations of clinical telemedicine will depend on careful attention to evaluation concepts and methods that form the well-established foundation of health services research and evaluation research generally. The framework presented in this report has four components: basic principles, a careful planning process, key evaluation elements, and fundamental evaluation questions. The principles that guided the development of the framework call for telemedicine evaluations to be

treated as an integral part of program design, implementation, and redesign;

viewed as a cumulative and forward-looking process for building useful knowledge for decisionmakers rather than as an isolated research exercise;

designed to compare the benefits and costs of telemedicine with those of current practice; and

focused on identifying practical and economical ways to achieve desired results rather than investigating the most exciting or advanced telemedicine options.

In conjunction with these principles, the evaluation framework developed by this study ( Box S.1) constitutes a base for strengthening individual evaluations of telemedicine and encouraging the coordination of evaluation strategies across projects and organizations, when possible. The framework highlights the importance of both delineating how technical, clinical, and administrative processes are intended to work and determining how they actually are implemented. This is crucial if evaluators who find disappointing or unexpected results are (a) to distinguish the failure of an application from the failure of an application to be implemented as intended and (b) to provide guidance to decisionmakers considering whether to adopt, substantially redesign, or discontinue telemedicine programs.

The fast pace of change and other uncertainties surrounding telemedicine applications argue strongly for an evaluation plan to include sensitivity analyses that explore to what extent conclusions may change if values of key variables or assumptions change. Such analyses are appropriately keyed to a business plan that explicitly states how the evaluation will provide information to help decisionmakers determine whether a telemedicine application is useful, consistent with their goals and objectives, and sustainable beyond the evaluation phase.

To build both on this framework and on past initiatives, the committee encourages federal agencies to strengthen provisions for evaluating demonstration projects and other telemedicine activities and to support innovative research strategies and methods development. Given the relative sparsity of evaluations of telemedicine, the committee also urges those sponsoring and funding a number of different projects to consider how their project evaluations might be designed to reinforce and supplement each other despite differences in the objectives, applications, and other characteristics of the projects. The efforts of the federal Joint Working Group on Telemedicine are constructive steps in this direction.

In the private sector, the committee likewise encourages organizations considering telemedicine to build evaluation into their program plans. Decisionmakers can also demand from vendors more complete and relevant documentation of costs and promised benefits.

Finally, because the evaluation literature in telemedicine is weighted toward nonexperimental studies, the report particularly encourages researchers and funding organizations to look beyond nonexperimental designs to more rigorous experimental and quasi-experimental designs. The latter attempts to control some important threats to validity through statistical adjustments and other means when random assignment of participants, homogeneous populations, or strict treatment protocols are not feasible. Sophisticated computer-based patient information systems are gradually making such designs more practical and robust. Peer-reviewed publications can also play a role by moving toward standards for systematic reporting of evaluation methods and results.

Basic Evaluation Questions

Clinical applications of telemedicine are marked by diversity. They differ in the medical problems addressed, the evidence base for decisionmaking, the personnel and settings of care involved, the diagnostic and therapeutic strategies employed, and the organizational and cost implications of these strategies. Given the large number of possible quality, access, cost, and acceptability measures for different clinical applications of telemedicine and the difficulty of stipulating many of them in abstract form, this study did not focus on application-specific measures and criteria.

Instead, to guide the selection of evaluation criteria or measures for particular evaluation projects, it proposed broadly relevant questions about the quality, accessibility, cost, and acceptability of telemedicine services. Quality is the degree to which health care services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge. Access refers to the timely receipt of appropriate care (or, more informally, the right care at the right time without undue burden). The cost of care is the economic value of resource use associated with the pursuit of defined objectives or outcomes. Acceptability refers to the degree to which patients, clinicians, or others are satisfied with a service or willing to use it. In some telemedicine evaluations, patient satisfaction data appear to be the only patient-level data collected, a focus that the committee considers too limiting.

Box S.2 presents the basic categories of evaluation questions identified by the committee, and the appendix to this summary lists more specific questions in each category. Although the questions present the concepts of quality, access, cost, and acceptability in sequence, their interactions and interrelationships also warrant evaluation. More generally, the questions should be considered in the context of the overall evaluation framework. That is, relevant patient and organizational characteristics should be identified and considered as they might affect results. The actual as well as the planned technical and clinical processes should be recorded. The fit between the project objectives and results and the sponsoring organization's purposes or strategic plan also needs to be factored into the plan for analysis and the interpretation of results.

For some evaluation results, the findings will strongly suggest certain decisions. For example, if a telemedicine application is more costly than the alternative and performs less well (e.g., produces fewer health benefits), it should not be adopted. Likewise, if the application is more costly and performs as well, it should not be adopted. In contrast, if the telemedicine application is less costly but performs better than the alternative or if it is less costly and performs as well, it should be considered. Results are sometimes more equivocal and decisions more difficult. For example, if a telemedicine application is more costly and performs better than the alternative, are the benefits gained worth the extra costs? If an alternative is less costly and performs less well, are the savings worth the health benefits foregone?

Some telemedicine evaluations will focus less on individual patients than on populations, including but not limited to those enrolled in managed care plans. Analyses may consider outcomes for an entire patient population or may concentrate on outcomes for the least healthy or most vulnerable groups in a population (e.g., elderly individuals, migrant workers). In addition, because telemedicine programs may also serve educational and administrative as well as clinical objectives, evaluations may reasonably seek to assess program effects in these areas. Broader community effects may also be considered. For example, although improved access to health care for rural populations has been an important objective of many telemedicine projects, policymakers may also be interested in the effects of telemedicine on the survival of rural health care providers and the implications of such effects on the economic health of rural areas including their ability to attract or maintain business, educational, and other resources.

Conclusion

Special challenges notwithstanding, more rigorous and systematic evaluation is as necessary for telemedicine as it is for other health care technologies. Decisionmakers still do not have good enough information comparing the effects of telemedicine applications to alternative health care strategies. They also lack good analyses of the infrastructure implications and financial requirements for sustaining telemedicine past an initial "test of concept" period.

Although individual research approaches will vary, the evaluation and implementation of telemedicine projects will benefit by the more consistent adoption of sound evaluation principles and methods. They will also benefit from the lessons learned in implementing computer-based patient records and integrated patient information systems, an undertaking that remains dauntingly difficult, even after 25 years of groundwork. These difficulties suggest the importance of persistence and realism for those working to demonstrate telemedicine's promise.

For some applications of telemedicine, more rigorous evaluations will make claims of their value more credible and will encourage their more widespread use. For other applications, better evaluation may discourage adoption, at least until technologies or infrastructures improve or other circumstances change. This is to be expected. The purpose of evaluation-and the purpose of this report-is not to endorse telemedicine but to endorse the development and use of good information for decisionmaking. The evaluation framework presented here is offered in that spirit.

Appendix. Questions About The Quality, Accessibility, Cost, And Acceptability Of Telemedicine

# data[].attributes.attachment_link.description
For more than 30 years, clinicians, health services researchers, and others have been investigating the use of advanced telecommunications and information technologies to improve health care. At the intersection of many of these efforts lies telemedicine-a combination of innovative and mainstream technologies.

# data[].attributes.body
Telemedicine is a healthcare technology that uses electronic information and communications technologies to provide and support healthcare services when distance separates the participants. It has been in use for over three decades, with various applications in patient care, education, research, administration, and public health. Telemedicine offers several advantages, including reduced travel costs and time, decreased risk of communicable disease transmission, and increased convenience. It also facilitates seamless transitions of care and connects isolated patient groups with healthcare providers.

# data[].attributes.body
Telemedicine is depicted as a multifaceted tool that enhances various aspects of patient care and healthcare management. Here’s a brief summary of the features of telemedicine for healthcare:

Chronic Health Management: Offers care for chronic conditions post-hospitalization, hyper-specialty care, and team-based medicine.

Care Throughput: Enables remote follow-up visits, more flexible practices, and effective chronic disease care.

Prescription Compliance: Focuses on virtual second opinions, mental health services, and lifestyle support.

Store-and-Forward: Supports functional and alternative medicine, recovery after natural disasters, and eldercare.

# data[].attributes.body
PROS

Reduced Travel Costs and Time: Telemedicine saves both patients and healthcare providers time and money by eliminating the need for physical visits.

Increased Convenience: Patients can receive medical care from the comfort of their own homes, reducing the need for lengthy commutes and wait times.

Improved Accessibility: Telemedicine bridges the gap between healthcare providers and isolated patient groups, enhancing access to medical care.

Enhanced Patient Engagement: Telemedicine allows patients to track their vital signs and medication adherence from home, improving their overall engagement in their healthcare.

CONS

Accessibility Barriers: Not all patients have access to the necessary devices and networks, creating a barrier to telemedicine adoption.

Technical Difficulties: Network connection disruptions and other technical issues can hinder the effectiveness of telemedicine.

Limited Accessibility of Imaging Technology: Telemedicine may not be able to fully replace in-person physical examinations due to limitations in imaging technology.

# data[].attributes.body
Telemedicine has the potential to revolutionize healthcare by providing greater accessibility and convenience. While there are some limitations and technical challenges, the benefits of telemedicine far outweigh the drawbacks. It is essential to continue improving and expanding telemedicine services to ensure that they are accessible to all patients who need them.

# data[].attributes.body
Top five key Strength

Learner: I have a great desire to learn and want to continuously improve. The process of learning, rather than the outcome, excite me.

Achiever: I work hard and possess a great deal of stamina. I also take immense satisfaction in being busy and productive.

Responsibility: I take psychological ownership of what I say I will do. I am committed to stable values such as honesty and loyalty.

Discipline: I enjoy routine and structure. My world is best described by the order I create.

Strategic: I create alternative ways to proceed, when faced with any given scenario, I can quickly spot the relevant patterns and issues.

My Dream Team

Louis: Learner

Important for conducting research and analysis to identify new market opportunities for the technology.

Alice: Archiver

Ensures that team results are high quality on time and timeless.

Randall: Responsibility

Execute all task assign by team diligently, unscheduled and to the highest standard.

Daniel: Discipline

Set clear guidelines and policies and provide regular feedback to the team.

Sheila: Strategic

Ability to formulate strategies to archive goals and monitor progress of the goals and provide recommendation on archiving the goals.

# data[].attributes.body
Institute of Medicine (IOM). (2015). Telemedicine: A Guide to Evaluating the Quality of Health Care. National Center for Biotechnology Information (NCBI). https://www.ncbi.nlm.nih.gov/books/NBK45437/

Haleem, A., & Javaid, M. (2023). Telemedicine for healthcare: Capabilities, features, barriers, and applications. The Egyptian Journal of Internal Medicine, 35(1), 49. https://ejim.springeropen.com/articles/10.1186/s43162-023-00234-z

Abid Haleem, aDepartment of Mechanical Engineering, Jamia Millia Islamia, New Delhi, India. (2023). Telemedicine for healthcare: Capabilities, features, barriers, and applications. The Egyptian Journal of Internal Medicine, 35(1), 49. https://ejim.springeropen.com/articles/10.1186/s43162-023-00234-z