Chapter  24:  Telemedicine for the Medicare Population: Evidence Report/Technology Assessment Number 24

Store-and-Forward Telemedicine

The use of store-and-forward telemedicine services in Medicare-eligible patient populations was examined, asking the following questions comparing telemedicine to face-to-face medical encounters:

Self-Monitoring/Testing Telemedicine

The use of self-monitoring/testing telemedicine services in Medicare-eligible patient populations was examined, asking the following questions relative to face-to-face encounters:

Clinician-Interactive Telemedicine

The use of clinician-interactive telemedicine services in Medicare-eligible patient populations was examined, asking the following questions comparing telemedicine to face-to-face encounters:

General Observations

Through the EPC team's review of the general literature, they identified 455 telemedicine programs, of which 362 are in the United States. Among U.S. programs, 111 are located at academic medical centers and 68 are in hospital-based health care networks; 80 are federal, military, or Department of Veterans Affairs medical centers. Over 30 medical specialties are represented. Many programs include more than one activity. The most common telemedicine activities are consultations or second opinions (290), diagnostic test interpretation (169), chronic disease management (130), posthospitalization or postoperative followup (102), emergency room triage (95), and "visits" by a specialist (78). About 50 programs provide services in patients' homes.

Many diverse populations are served by telemedicine. More programs serve rural patients than any other group. Of the 455 programs catalogued in the general literature review, approximately 120 (26 percent) provide health care to rural populations. Telemedicine also serves a large number of veterans and elderly. The numbers of telemedicine encounters increased steadily throughout the 1990s, with significantly more consults in 1997 and 1998 than in previous years. A total of 177 articles were determined to potentially have evidence for the efficacy of one of the three study areas, and were included in the systematic review. After exclusion criteria were applied, there were 15 articles that assessed store-and-forward telemedicine, 14 articles that evaluated self-monitoring/testing, and 48 articles that assessed clinician-interactive services. A total of nine randomized controlled trials were identified, one in store-and-forward, six in self-monitoring/testing, and two in clinician-interactive services.

Specific Results

Individual studies were assessed for evidence based on criteria applicable to the study question. Each question was analyzed from the framework of Medicare-eligible patient populations, relative to face-to-face medical encounters. Studies were too heterogeneous and their quality varied too much to undertake any quantitative aggregate analysis, i.e., meta-analysis.

Store-and-Forward Telemedicine

While store-and-forward telemedicine programs operate in many clinical domains, studies assessing the efficacy are lacking for most of them. In addition, for settings where studies have been done, the quality of the evidence is of insufficient quality to judge the efficacy of store-and-forward telemedicine. Teledermatology is the most-studied clinical specialty in store-and-forward telemedicine; its diagnostic accuracy and patient management decisions being made are comparable to those of in-person clinical encounters. It may improve access to care and have adequate patient acceptance.

Self-Monitoring/Testing

Self-monitoring/testing telemedicine is used less frequently than the other two types of telemedicine studied in this report. It is most commonly used for management of chronic diseases or specific conditions, such as heart disease, diabetes mellitus, or asthma. As with store-and-forward telemedicine, there are programs operating in many clinical domains where there is little evidence from peer-reviewed studies to support its use. Some studies show it results in comparable outcomes, improves access, increases satisfaction with care delivered, and may be cost effective.

Clinician-Interactive Telemedicine Services

Clinician-interactive telemedicine is used in more heterogeneous clinical specialties than store-and-forward. As with store-and-forward, however, few studies assess efficacy. Furthermore, for many settings where studies have been done, the evidence for efficacy is of insufficient quality to judge how well clinician-interactive telemedicine works. Unlike store-and-forward telemedicine, several studies showed interactive teledermatology to be inferior to in-person consultation in making diagnostic and appropriate management decisions, though many of these were done with older technology, unlike the store-and-forward studies. In several other clinical specialties, clinician-interactive telemedicine shows comparable diagnostic accuracy, and in emergency medicine one randomized controlled trial shows it to have comparable health outcomes. Some studies demonstrate improved access to care, patient and provider satisfaction, and reduced costs of care, though most have problematic designs.

The Evidence-based Approach

An evidence-based report focuses attention on the strength and limits of evidence from published studies about the effectiveness of a clinical intervention. The development of an evidence report begins with a careful formulation of the problem In this phase, a preliminary review of the literature and input from experts, stakeholders, and patients can be used toidentify the patient populations, interventions, health outcomes, and harms. These parameters are summarized in an analytic framework, or causal pathway, which is used in turn to generate a list of key questions to examinein a systematic review of the published literature.

An evidence-based report emphasizes the quality of the evidence, giving the most weight to studies that meet high methodologic standards in order to reduce the likelihood of biased results. It emphasizes studies that measure health outcomes instead of intermediate outcomes. For example, a study that measured the effect of a telediabetes program on patients' functional status and quality of life would be given more weight than one that reported only changes in health care utilization or laboratory markers of disease. An evidence-based report also emphasizes studies that reflect clinical efficacy in unselected patients and community practice settings. Finally, an evidence-based report considers the net benefit, after a thorough effort to assess both the benefits and the harms of aservice or technology.

In the context of developing clinical guidelines, evidence reports are useful because they define the limits of the evidence, clarifying when assertions about the value of the intervention are based on strong evidence from clinical studies. The quality of the evidence on effectiveness is a key component, but not the only component, of decision-making on coverage decisions. Medicare coverage determinations are also based on whether a service has been determined to be safe, medically appropriate, and provided in accordance with recognized standards of medical practice.²⁰ Other considerations are its expected cost and its potential for overutilization and abuse.

The Physician Payment Review Commission proposed evaluating telemedicine serviceson several dimensions: the value the services offer compared with traditional methods of performing the same service, access for underserved populations, increased efficiency in servicedelivery, improved quality of care through integrated approaches, strengthening capabilities for emergency services, and the potential for escalating Medicarecosts by stimulating overutilization.²¹ Additional criteria include acceptability to physicians, patients, or others²² and cost-effectiveness.²³

Key Questions About the Efficacy of Telemedicine Services

Figure 1. Analytic framework for each study area

   Figure 1. Analytic framework for each study area

Figure 1

Electronic bibliographic databases

A search string designed to find any publications about telemedicine was used to search MEDLINE, EMBASE, CINAHL, and HealthSTAR

(Appendix B). The search strategy initially resulted in 3,422 citations, and monthly updates of the same strategy through January 2000 yielded 171 additional citations.

Published reports

We searched through telemedicine reports and compilations such as the "Quebec Report" from the Conseil d'Evaluation des Technologies de la Sante du Quebec,²⁴ the Telemedicine Strategic Healthcare Group report²⁵ the International Society for Telemedicine Conference 1999 summary,²⁶ the Australian National Telehealth Committee reports,²⁷ and the Telemedicine Sourcebook 1998. ²⁸ These reports provided valuable perspectives on a wide variety of telehealth projects. We also assessed three systematic reviews (different in scope from this report) from theInternational Network of Agencies for Health Technology (INHATA),²⁹ the Cochrane Database of Systematic Reviews,³⁰ and the Agencia d'Avaluacio de Technologia Medica.³¹

Internet program information

We also searched the Internet for telemedicine program information. Examples of sources obtained through this approach are the Telemedicine Information Exchange (http://tie.telemed.org), the Canadian Initiatives on Networking Clearing House (http://strategis.ic.gc.ca), and the Midwest Rural Telemedicine Consortium (http://www.mrtc-iowa.org).

Reference lists

We identified 178 additional articles from the reference lists of included reports and articles.

Contacts with experts

We contacted known telemedicine experts to find additional resources to identify and describe telemedicine programs. Through one such contact, the project purchased the Association of Telemedicine Service Providers' database⁴ of self-reported information about current telemedicine programs in 1997 and 1998. The information in this database was obtained by surveying administrators of these telemedicine programs. These administrators (n = 551) were identified by reviewing vendor press releases, notices of grant awards, telemedicine literature, and ATSP membership records. New and prior respondents were surveyed, and respondents received a complimentary copy of the 1998 report for participating.³² The electronic version of this database has been updated by an annual survey and by interim telephone validation of information gleaned from key informants, the literature, and the Internet. A total of 141 programs were described in the 1998 ATSP database. (From 1998 report. 1999 report not yet available.)

Inventory of Programs

We developed an inventory of telemedicine programs worldwide. A program was defined as a set of actions within a location where telemedicine was centered and organized. Within each program were one or more activities (e.g., one program might have activities in teleradiology and teledermatology). The inventory of programs was constructed using the ATSP database⁴, articles from the general-literature review, and other sources of information as listed above. We used information only from written descriptions of programs in print publications or on Web sites; we did not attempt to contact programs personally. For each article, we attempted to obtain information about nine characteristics of the program: geographic location; clinical applications; payment sources and methods; quality standards; data systems for tracking utilization; telecommunication links and equipment; system costs; utilization; and other issues. We also tried to determine whether a program used store-and-forward telemedicine, either purely or in combination with interactive telemedicine.

The abstracted data of programs were entered intoa relational database for aggregation and interpretation. We used many of the same database fields and enumerated values within fields as the ATSP database, such as clinical specialty and clinical activity.⁴ The database was constructed using Microsoft Access^TM software and initially populated with the ATSP 1998 data. Information about other domestic and international programs identified in the literature review was added as it was acquired. Appendix C lists the variables in this database.

Systematic Review

Included articles were categorized by the key question(s) they addressed, by WRH and SGS (using consensus to break ties), and distributed to members of the research team. For each key question, data from each study were abstracted by a single reviewer, using paper or electronic abstraction forms, and entered into an electronic database. For all studies, we recorded the setting (e.g., academic center, community hospital, government hospital), clinical activities, type of equipment used, year(s) in which data were collected, features of the design of the study (e.g., controlled trial, case series), aspects of recruitment (volunteers, invitation, consecutive or non-consecutive patients), type of control (if applicable), and measures used to assess the effectiveness of the intervention. Additional information was abstracted depending on the key question that was addressed. For studies of diagnostic performance, we recorded the test used as the reference standard and the reported sensitivity and specificity of the telemedicine diagnosis. Additional variables that were abstracted for studies of the economic impact of telemedicine activities included the type of economic study, types of comparisons, data sources, cost unit, discount rate, sensitivity analysis parameters, program expansion capability, and generalizability of the program.

Access to Care

In appraising studies of access to care, we employed the Institute of Medicine⁷ (IOM) model of access to care, which incorporates four types of indicators: barriers (structural, financial, and personal); utilization; mediators (treatment effectiveness, provider quality, and patient adherence); and outcomes, including health status (e.g., mortality, well-being, or functionality) and equity of services among various populations. The IOM strongly recommended that studies of access to care measure both utilization and outcomes.

Our preliminary review of the telemedicine studies that addressed access showed that the majority relied on utilization indicators alone. A few used indicators of reduced barriers to care. Most studies measured access as 1) increased opportunity to obtain a service locally, and 2) reduced amount of time for seeking and/or obtaining care.

Satisfaction Studies

Forty-five studies were identified by two reviewers (WRH and SES) as containing data on patient and/or provider satisfaction. These studies were divided into three categories: 1) studies dealing with patient satisfaction; 2) studies concerning provider satisfaction; and 3) studies that addressed both patient and provider satisfaction.

After examining the articles on satisfaction, we found that there was insufficient information in 13 studies to evaluate the strength of the evidence presented. For example, there was either no description of the sample or sample size, no description about how satisfaction information was collected, or no description how the data were analyzed. However, these papers did contain reports of patient and/or provider satisfaction that could be considered anecdotal, and may lend support to the general sense of the levels of satisfaction among patients and providers. Eleven of the studies were of various clinician-interactive services; one involved store-and-forward telemedicine and one was about self-monitoring/testing.

Economic Analyses

The economic evaluation of the telemedicine applications from the literature review included both cost studies and cost-consequence studies. Economic evaluation is a more encompassing term than cost-effectiveness analysis. The term cost-effectiveness analysis refers to a particular class of economic assessments in which costs are compared to measures of effectiveness (such as life-years). Economic assessments encompass other study designs, including program-cost analyses and cost-of-illness studies. Program-cost analyses review only the cost of implementation and maintenance of a particular application. The other designs (including cost-effectiveness studies) are cost-consequence studies, which compare the costs to other consequences, including other economic consequences (as the costs averted by the application), technical consequences (as images of the same quality), or health outcomes (as length of stay or life-years).

Table 5. Summary of six principles for economic evaluations (more...)

Table 5. Summary of six principles for economic evaluations³⁶

Principle

Description

Perspective stated Benefit described Costs included Discounting included Sensitivity analyses Cost-effectiveness ratios stated

Whose costs and consequences are considered? What are non-economic consequences of program? Describe intervention costs, morbidity or side effect costs, averted costs and induced costs? Are future costs and consequences adjusted for timing? For values that are uncertain (e.g. assumed), are analyses performed using alternative values? Are alternatives compared in a way that allows decisions on prioritization to be made?

Key Question 1a. To what extentdoes store-and-forward telemedicine have the same components as a traditionalclinical encounter?

Store-and-forward telemedicine encounters are being done in many specialties, but differ from face-to-face consultations.

What types of encounters are being done?

Table 8. Clinical specialties of store-and-forward telemedicine (more...)

Table 8. Clinical specialties of store-and-forward telemedicine programs

Clinical specialty	Number of programs
Radiology	39
Cardiology	20
Dermatology	19
Orthopedics	14
Mental Health	13
Ophthalmology	11
Pathology	9
Neurology	7
Pulmonary Care	5
Nutrition	4
Endocrinology	4
Gastroenterology	4
Emergency/Triage	3
Oncology/Hematology	3
Specialty Surgery	3
Primary Care	2
General Surgery	2
Infectious Disease	2
Neuroradiology	2
Pediatrics	2
Otolaryngology	2
Nuclear Medicine	1
Plastic Surgery	1
Nephrology	1
Oral/Maxillofacial Surgery	1
Obstetrics/Gynecology	1
Genetic Counseling	1
Urology	1
Rehabilitation Counseling	1
Rheumatology	1
Internal Medicine	1
Speech Pathology	1
Dentistry	1
Other	1

Table 9. Clinical activities of store-and-forward telemedicine (more...)

Table 9. Clinical activities of store-and-forward telemedicine programs

Clinical activity	Number of programs
Specialist Consults or Second Opinions	31
Diagnostic Test Interpretation	26
Emergency Room/Triage	8
Specialist Visits	6
Chronic Disease Management	4
Post-Hospital/Post-Operative Follow-Up	3
Primary Care	2
Physiological Monitoring	2
Home Health	2
Rehabilitation	1
Nutrition Consultation	1
Nursing Home Care	1

In 1997, about 12,000 encounters were done using store-and-forward applications.⁵ We identified 186 activities within 117 programs use store-and-forward telemedicine. A total of 70 activities use store-and-forward exclusively, while the remainder use it in association with other technologies, such as video conferencing. The most common clinical specialties using store-and-forward technology are listedin Table 8, with the most common activities shown in Table 9.

What type of information is transmitted?

Table 10. Types of information transmitted from store-and-forward (more...)

Table 10. Types of information transmitted from store-and-forward telemedicine programs

Program	Reference	Specialty	Clinical Activity	Data Provided
Georgetown University Med Center/ISIS, Washington. D.C.	Tohme, 1997,42 Anonymous, 1997,131 Levine, 1998132	Urology	Specialty consult	Images, documented history and physical exam.
Georgetown University Med Center/ISIS, Washington. D.C.	Tohme, 1997,42 Anonymous, 1997,131 Levine, 1998132	Nephrology	Specialist clinic	Dialysis data, stethoscope recording of heartbeat, images,
University of Houston Eye Institute Teleophthalomology, Houston, TX	Wheeler, 1997133	Ophthalmology	Specialty consult	Images of eye, video clips of eye
Kaiser Permanente Teleophthalmology, Oakland, CA	Wheeler, 1997133	Ophthalmology	Specialist clinic	Images of retina from Canon non-mydriatic fundus camera
Whiteman Airforce Base, Knob Noster, MO	Anonymous, 1998134	Multiple, especially orthopedics, dermatology	Specialty consult	Images, documented history and physical exam.
Carmelitos Preventive Eye Care Center, Los Angeles, CA	McCormack, 1998135	Ophthalmology	Specialty consult	Images of eye
Sipovo, Bosnia - Royal Hospital Haslar Telemedicine Link, Gosport, United Kingdom	Vassallo, 1998136	Dermatology, plastic surgery, orthopedics, orology, ophthalmology, maxillo-facial surgery	Specialty consult	Digital photos, photos of x-rays, documented history and physical exam.
Massachusetts General Hospital Saudi Arabia Telemedicine, Boston, MA	Richardson, 1996137	All specialties, especially orthopedics, neurosurgery, neurology, and cardiology	Specialty consult	Images, labs, documented history and physical exam
St. Louis Veterans Affairs Medical Center, St. Louis, MO	FTD, 1998134	Nuclear medicine	Image reading	Images

Store-and-forward telemedicine typically provides the consultant with a report on the patient's history and physical examination, either from the original medical record or summarized on a form, along with digitized photographs, radiographs, or other images. Table 10 shows the characteristics of programs from which the details could be abstracted from published reports.

Key Question 1b. Does store-and-forward telemedicine result in comparable diagnosis and appropriateness of recommendations for management?

We examined peer-reviewed studies toanswer two questions:

• How accurate is diagnostic interpretation of transmitted data in a store-and-forward teleconsultation?

• How do recommendations made after teleconsultation compare in quality to those made after face-to-face consultation?

The best evidence for the efficacy of store-and-forward telemedicine for diagnostic accuracy and appropriate management decisions comes from teledermatology, where several studies compare it to in-person examinations. There is some evidencethat it achieves comparable efficacy in otolaryngology, outpatient medicine, ophthalmology, and wound healing.

Key Question 1c. Does the availability of store-and-forward telemedicine provide comparable access tocare?

The only studies to assess patient access issues for store-and-forward telemedicine came from teledermatology, where some evidence suggests that better access to dermatology consultations is possible using this technology.

One teledermatology study demonstrated increased utilization after a store-and-forward teledermatology service was deployed in rural areas.⁵⁸ Before installation of the teledermatology system, the baseline rate of consultation was 1.8 percent of all skin problems seen by the local primary care physician. At 4-6 months after installation, the rate of teledermatology consultation jumped to 9.6 percent of all skin problems. Even the rate of in-person dermatology consultation increased to 3.6 percent. However, at 10-12 months, the consultation rate leveled off to 2.8 percent for teledermatology and 0.2 percent for in-person consultations, which together still add up to a higher rate than the baseline.

Another study⁵⁹ addressed the effect of telemedicine on dermatology consultation utilization, interpreted as appointment triage decisions (i.e., urgent, routine, or never), but the quality of this study was very limited. Triage decisions made from a referral letter were compared with triage decisions made 13 months later from skin-condition images obtained near the time of the letter for the same patients (although the length of time between referral letter and image photography was not reported). Some flaws were no descriptions of the sample selection procedures or sample characteristics; no reported inter-rater reliability between the two consultant raters; no definitions of the triage-rating measures; use of different rating scales for each set of triage assessments that were compared; a discrepancy in the reported outcome rate (stated differently in the text and the table); and no reported significance levels. Since the investigator did not indicate which triage decisions would be seen by a consultant, it was also not possible to estimate the effect of telemedicine on access to a dermatologist.

Key Question 2. What are the potentialadverse effects of store-and-forward telemedicine?

Potential adverse effects of store-and-forward telemedicine include the consequences of technical problems, of false-positive or false-negative interpretation of diagnostic information, and of overutilization. While these issues are frequently mentioned in editorial and news items, they have received little or no attention in research studies.

Store-and-forward teleconsultation may change the intensity of work performed by the referring physician. The referring physician's office has to arrange for the patient's medical history, physical examination report, and any telemedicine imaging to be collected and transferred to the teleconsultant. Collecting this information for store-and-forward telemedicine encounters has been reported in several studies to be more time-consuming than in-person consults for both referring and consulting physicians.^55,60,61

Key Question 3. Does store-and-forward telemedicine lead to comparable health outcomes?

We found no studies that assessed health outcomes using store-and-forward telemedicine. One study assessed "clinical outcome," which was defined as the need to have a follow-up appointment with a hospital-based specialist.⁶² Store-and-forward consultations were found to require more follow-up visits (69%) thaninteractive teledermatology consultation (46%) or in-person consultation (45%). No statistical analysis of the differences was performed.

Key Question 4. Does store-and-forward telemedicine lead to comparable patientor clinician satisfaction with care?

Measures of patient or provider satisfaction with store-and-forward telemedicine were incorporated into four studies. Two studies of provider satisfaction were from excludeddomains -- telepathology⁶³ and teleradiology.⁶⁴ However, we found no information by which to evaluate provider satisfaction with the store-and-forward telemedicine applications of most interest to this report.

As with provider satisfaction, there was not enough reported evidence to determine whether patients are as satisfied with store-and-forward telemedicine as they are with face-to-face encounters with specialists. Two studies that reported patient satisfaction concerned teledermatology. Zelickson and Homan,⁵⁰ in a study of teledermatology in a nursing homesetting, asked patients and/or their guardians to rate their satisfaction with the service. Only 23 percent of those surveyed returned the questionnaires. Harrisonet al.,⁶⁵ in a study in a dermatology clinic, also received alow return rate (24 percent) on their satisfaction questionnaire. Although both studies reported high rates of patient/guardian satisfaction with teledermatology, the low response rate limits the inferences that can be drawn from the data.

Key Question 5. Does store-and-forward telemedicine lead to comparable costs of care?

Key Question 6. Is store-and-forward telemedicine cost-effective?

We identified no studies of the marginal cost-effectiveness of store-and-forward telemedicine.

Summary

Data from economic evaluations to support store-and-forward telemedicine are poor because the studies provide an incomplete picture of costs. Rather than compare the total costs for each approach, they compare the costs of setting up and maintaining a telemedicine application to only some of the other health care costs for a given episode of care. Thus, the studies of telemedicine applications provide, at best, comparisons of only one component of averted costs to intervention costs and, at worst, report only intervention costs. Cost-effectiveness has yet to be demonstrated in this technology.

Key Question 1a. What are the characteristics of self-monitoring/testing telemedicine in terms of patients included, services provided, equipment used, and information transmitted?

As with all reported telemedicine programs, most self-monitoring/testing applications in the world are located in the United States (see Appendix F). These programs are scattered around the country, with a slightly higher concentration in California. Of the 21 programs identified, nine are in the Department of Veterans Affairs health system, 17 operate in medical centers, two are outside the United States and four are from independent agencies; nine provide home health services.

The most common situation where self-monitoring/testing is used is in managing a disease or specific condition, particularly heart disease, diabetes mellitus, or asthma. At the San Diego VA Medical Center, for example, on-call cardiologists can view electrocardiograms and monitor pacemakers from patients' homes.²⁸ Patients use portable electrocardiogram machines to record and send cardiac-function data through the telephone for interpretation. Some patients also use electronic stethoscopes to record, monitor, and transmit heart and lung sounds. If the physician detects a problem, he/she then calls the patient and recommends a course of action. Such a telecommunication service is relatively inexpensive because it uses analog lines, or "plain old telephone service" (POTS).

The holistic support and care of at-risk patients are other important uses for self-monitoring/testing programs. Many older patients, terminally ill patients, and those with more than one chronic-disease risk factor are asked by professionals to assume increasing responsibility for their own care. With self-monitoring/testing telemedicine, patients can have periodic linkage with a professional nurse, physician, or another type of provider in a way that does not require the provider to spend travel time and does not over burden the patient with hosting responsibilities. Such contacts allow at-home monitoring by the patient or caregiver, which promotes joint responsibility for health maintenance. These contacts can be used for patient assessment, education, and counseling to prevent functional decline, nutritional emergencies, and medication lapses. The Kansas University Medical Center Home Health Pilot Project, for example, uses video-conferencing to interact with patients in their homes.^{67, 40} The nurse can assess the patient's skin condition, medication supply, insulinself-injection technique, and mental status. During a video conference call, the nurse can also reinforce instructions about how to check for skin breaks and prevent skin breakdown, and can do some planning with the patient and family to prevent anticipated problems in the home environment. In this way, a comprehensive case-management visit can be accomplished and reports can be distributed to various medical specialists and the primary care physician.

Key Question 1b. Does self-monitoring/testing telemedicine result in comparable diagnosis and appropriateness of recommendations for management?

A study of telemedicine wound assessment,⁵⁷ discussed above in the context of store-and-forward applications, can also be viewed as a self-monitoring/testing application. This study showed that skin-wound diagnosis is reliable when using digital photography. It also shows potential for teaching patients and family caregivers this mechanism of self-monitoring/testing.

Key Question 1c. Does the availability of self-monitoring/testing telemedicine provide comparable access to care?

A qualitative study of home monitoring for nursing management⁷² attempted to interpret utilization and outcomes. The intervention was a visit in which patients at home used a two-way video phone, a blood pressure and pulse monitor, an electronic stethoscope, a call button, and an optional emergency-response system to report their condition to a nurse at a central station. All subjects (n = 12 disabled or chronically ill persons) had on-demand, face-to-face electronic access with the nurse; traditionally, a call from the patient had prompted an urgent (i.e., same day or within 2 hours) home visit from the nurse. However, this change was not actually measured in the study. The authors provided only case descriptions, and reported that eight subjects (67 percent) had more home care without added in-person visits and that four subjects (33 percent) avoided some days in a hospital or nursing home, or avoided clinic visits. The findings suggest that patients with certain types of conditions and living arrangements may benefit from self-monitoring/testing via a telemedicine technology. However, because it was uncontrolled, the study does not support the premise that improved access to care, or reduced utilization of more intensive services, was due to the telemedicine intervention.

Key Question 2. What are the potential adverse effects of self-monitoring/testing telemedicine?

We found no studies that assessed the potential adverse effects of self-monitoring/testing applications. That is, no studies reported on technical problems or incomplete or inaccurate information nor whether the availability of self-monitoring/testing applications led to over-utilization or increased the cost of care.

Key Question 3. Does self-monitoring/testing telemedicine result in comparable health outcomes?

We identified three randomized controlled trials that assessed the study population for improvement in the intermediate health outcomes of HgbA1c level in patients using upload of data from glucose monitoring machines compared with paper diaries assessed by the clinician at a periodic office visit. All of the studies were of short duration (no longer than a few months) and used relatively small numbers of patients. The largest study (n = 42), which also reported the most detail, showed that both the experimental and control groups improved, with no statistically significant difference between them, though the small sample size may have lacked statistical power to show significance for the difference which did occur infavor of the experimental group.⁷³

Two other studies were smaller and reported their results in less detail. One study demonstrated a small magnitude but statistically significant benefit⁷⁴ The other compared a home system for glycemic monitoring to a paper-diary system.⁷⁵ Although it was a controlled trial, it was limited by incomplete reporting of data, including the magnitude of the difference between the experimental and control groups.

Another study of intermediate outcomes was a randomized controlled trial by Friedman etal. assessing the effect of a computer-controlled, automated telephone system versus usual office-based care on adherence and blood pressure control in older hypertensive patients (n = 267).⁷⁶ Mean antihypertensive medication adherenceimproved 17.7 percent for telephone system users and 11.7 percent for controls (p = .03). Mean diastolic blood pressure decreased 5.2 mm Hg in users compared to a 0.8 mm Hg drop in controls (p = .02). There was also a positive relationship between medication adherence and blood pressure reduction. Although the improvements were modest, the quality of this study indicates that automated reporting is a valuable technique for reducing cardiac risk.

Two other studies described the use of videophones for patient communication with health care providers. In a study in Japan, 16 elderly patients who were provided with videophones were compared to 16 matched patients who used regular home health services.⁷⁸ After 3 months, the videophone group significantly exceeded the control group in activities of daily living, communications, and social cognition, as measured with the Functional Independence Measure.⁷⁹ The videophone intervention was a supplement to (not a substitute for) regular home health services. The second study,⁷² a qualitative report of 12 cases, was described earlier in the section on self-monitoring/testing and access to care.

In another study,⁸⁰ a randomized trial of 20 patients, a home-exercise program with transtelephonic exercise monitoring and a hospital-based program improved cardiac function by a similar amount. However, the study had such low statistical power that it would not be likely to detectany difference between the two interventions in efficacy or in complications even if they existed.

Key Question 4. Does self-monitoring/testing telemedicine result in comparable patient or provider satisfaction with care?

Key Question 5. Does self-monitoring/testing result in comparable costs of care? Is self-monitoring/testing telemedicine cost-effective?

This paper⁷² provided virtually no detail on assessed costs, which were based on estimates of care-intensity savings.

Key Question 6. Does self-monitoring/testing result in comparable costs of care? Is self-monitoring/testing telemedicine cost-effective?

One study reported cost-effectiveness data. In a controlled trial of hypertension management,⁷⁶ subjects randomized to the group receiving a weekly telephone call showed greater increases in medication adherence and greater decreases in mean diastolic blood pressure than the control group, who received usual care. The cost component of the economic evaluation was based on estimated costs of the system and not on actual patient experience. Few details of the cost estimate were provided and no sensitivity analyses were performed. Also, since no data on alternative regimens were provided, it is difficult to assess the economic impact of this intervention compared to other treatments or ways to improve compliance. Despiteits limitations, this study was the only example of a true cost-effectiveness study that we identified.

Summary

Key Question 1a. Which clinical services are or might be provided by clinician-interactive telemedicine?

Table 13. Clinical specialties of clinician-interactive telemedicine (more...)

Table 13. Clinical specialties of clinician-interactive telemedicine programs

Clinical specialty	Number of programs
Cardiology	67
Mental Health	66
Dermatology	51
Radiology	51
Emergency/Triage	41
Neurology	36
Orthopedics	29
Pulmonary Care	27
Pediatrics	24
Internal Medicine	22
Pathology	18
Oncology/Hematology	18
General Surgery	15
Gastroenterology	13
Ophthalmology	12
Obstetrics/Gynecology	11
Nephrology	11
Infectious Disease	11
Rheumatology	10
Nutrition	7
Primary Care	7
Otolaryngology	6
Home Health Nurse	5
Speech Pathology	5
Dentistry	5
Urology	4
Endocrinology	4
Specialty Surgery	3
Geriatrics	3
Rehab Nursing	2
Burn Treatment	2
Nuclear Medicine	2
Pediatric Cardiology	2
Podiatry	2
Plastic Surgery	2
Physical Therapy	2
Rehabilitation Counseling	2
Oral/Maxillofacial Surgery	1
Pharmacy	1
Nurse Clinician	1
Neonatology	1
Gerontology	1
Electrodiagnosis	1
Dialysis	1
Diabetic Instruction	1
Audiology	1
Pain Management	1
Other	1

Table 14. Clinical activities of clinician-interactive telemedicine (more...)

Table 14. Clinical activities of clinician-interactive telemedicine programs

Clinical activity	Number of programs
Specialist Consults or Second Opinions	166
Chronic Disease Management	93
Diagnostic Test Interpretation	91
Post-Hospital/Post-Operative Follow up	70
Emergency Room/Triage	67
Specialist Visits	59
Home Health	20
Physiological Monitoring	19
Nursing Home Care	8
Primary Care	7
Psychiatric Exams	4
Patient Education	4
Legal/Judicial Proceedings	4
Nutrition Consultation	3
Rehabilitation	2
Medical Supervision	1
Initial Assessment/Evaluation	1
Hospice Care	1
Trauma	1
Grand Rounds	1
Social Service	1
Medication Management	1

Key Question 1b. Does clinician-interactive telemedicine result in comparable diagnosis and appropriateness of recommendations for management?

We assessed the evidence for the diagnostic and/or management decision capability of telemedicine systems on a per-specialty basis. The most represented specialty was dermatology, with no other specialties having more than two studies.

The remaining studies had more serious methodologic limitations. Three used the same dermatologist for the teledermatology and in-person consultation.^86-88 Two studies assessing the agreement of detecting malignancies had insufficient sample size to estimate the false-negative rate of the teledermatology examination, and did not employ a gold standard (e.g., biopsy) to determine whether cancer was present.^{89, 90} Although most of the other teledermatology studies had adequate patient sample sizes, all of the studies were also limited by the small number of dermatologists taking part. Only four listed the number of participating dermatologists, and none of these had more than six dermatologists.

• Chest pain assessment in the emergency department^{91, 92}

• Availability of teleradiology in the emergency department^{93, 94}

• Parkinson's Disease assessment⁹⁵

• Ophthalmology services for some patients (e.g., general eye and HIV patients) but not others (e.g., diabetics)^{41, 56}

• Pulmonary history and physical assessment⁹⁶

• Dental evaluation⁹⁷

• Otolaryngology assessment⁹⁸

• Psychometric testing⁹⁹

• Urology assessment¹⁰⁰

Two studies focused on the effect of telemedicine on management decisions. In one of these, a remote cardiologist reviewed the history and electrocardiograms of patients with suspected myocardial infarction, then advised a senior house officer whether or not touse thrombolytic therapy. Eight patients who would have received thrombolytic therapy did not, and four who would not have, did.⁹¹ In the other study, remote urologists assessed 14 real and 18 simulated patients with urolithiasis and recommended medical or surgical intervention.¹⁰⁰ The urologistin the academic center using telemedicine advised a change in plan of care for half of the real and 17 percent of the simulated patients of the local urologist. In both studies, there commendations of the remote specialist was the gold standard, and there was no independent verification of their assessments nor any follow-up to determine whether their recommendations proved beneficial to the affected patients.

Key Question 1c. Does the availability of clinician-interactive telemedicine provide comparable access to care?

As noted above, some teleconsultations (CPT codes 99241-99245, 99251-99255, 99261-99263, and 99271-99275) in rural HPSAs have been covered under Medicare since 1998, but the impact of this coverage is not yet clear. One reason for this uncertainty is that no formal effort has been made to measure the impact of this coverage on access.

All of these studies showed improved access to care by allowing more local care for patients. In two observational, controlled studies with similar sample sizesand similar telemedicine interventions that transmitted image and other data to neurosurgeons for evaluation and triage assistance, local care (i.e., fewer distant-care transfers) increased by 33 percent¹⁰³ and 21 percent¹⁰² respectively. In a comparison study, telemedicine increased the availability of ultrasound service in a small, remote town by 36 percent.¹⁰⁷ None of these studies used randomly selected or randomly assigned samples, none specified the sampling inclusion criteria, and almost none supplied details about the subjects' medical conditions. Thus, the improvements could possibly be attributed to differences in the samples rather than to the intervention.

Other findings regarding access were based on descriptive studies. Bailes¹⁰¹ found that in 100 consecutive contacts with neurosurgeons for assistance with emergency triage decisions, there were fewer transfers for spinal injury (33 percent), stroke (25 percent), intra-cranial hemorrhage (23 percent), suspected tumor (21 percent), and various other conditions (62 percent). The implied comparison was the pre-telemedicine situation when all patients with these conditions were transported to a distant hospital. Since the study did not use a matched sample, and few details were provided about the triage decisions or decision-makers, the evidence is weak.

Several studies utilized poor measures or data-collection processes to determine access change. In two,^{104, 106} improved geographic access was measured with physician ratings of the potential for transfer, rather than actual transfer. In another,¹⁰⁵ primary care military physicians entered patient-evacuation impressions in a log and were later interviewed about changes in their initial plans, but no steps were taken to make the data-gathering process more objective.

The other type of finding regarding access to care with telemedicine was quicker service, and four studies measured access time. A study in Hong Kong¹⁰² showed that telemedicine reduced the mean transport time required for emergency care from 80 to 72 minutes, but the difference was not significant. Bailes¹⁰¹ reported that direct admission from helicopter to operating room was achieved for 20 percent of the experimental subjects (n = 25) and for none of the control subjects, when examination and image data could be sent ahead electronically. In another example of access based on timing, Srikanthan⁹¹ reduced the evening care limitations by offering after-hours access to a cardiologist, who could review transmitted data at home. In the sample of 112 patients, 15 percent received "more appropriate" treatment after hours. Although these studies show modest positive results, they have several design and reporting limitations, including the lack of randomized samples, unreported sample inclusion and exclusion criteria, unreported reliability and validity, and unreported significance. One study¹⁰⁷ is also limited by considerable missing data (39 percent in the control group, 14 percent in the experimental group).

Key Question 2. What are the potential adverse effects of clinician-interactive telemedicine?

As with the other study areas, we found few studies that explicitly assessed the potential adverse effects of clinician-interactive applications. That is, no studies reported on technical problems or incomplete or in accurate information or on whether the availability of clinician-interactive applications led to over-utilization or increased cost of care. The adverse effects of increased burden on the referring clinician reported for store-and-forward telemedicine are likely to apply here as well.

Key Question 3. Does clinician-interactive telemedicine result in comparable health outcomes?

One randomized trial assessed outcomes in patients entering an emergency department who were randomized to in-person or telemedicine care.¹⁰⁸ There were no differences in the need for additional follow-up care or return to the emergency department, showing that telemedicine was as effective as regular care in this setting.

Another randomized controlled trial assessed "clinicaloutcome," which was defined as the need to have a follow-up appointment with a hospital-based specialist.¹⁰⁹ Interactive teledermatology consultation (46 percent) was found to have the same rate of need for follow-up care as in-person consultation (45 percent). No statistical analysis of the differences was performed.

The remaining studies lacked matched control groups^{102, 110-112} and had a very small sample size, with the resultant lack of statistical power.¹¹¹ Consequently, there is little evidence that telemedicine for clinician-interactive services improves or does not improve the outcomes of clinical care.

Key Question 4. Does clinician-interactive telemedicine result in comparable patient or clinician satisfaction with care?

Nineteen studies reported on patient satisfaction with clinician-interactive services. Overall, patients, parents of patients, and families of patients were satisfied with the services. While the quality of these studies was poor, there was no contrary evidence in the literature. All 19 studies were based on convenience samples, and 13 were uncontrolled. In a randomized trial,¹⁰⁸ there were no significant differences between the telemedicine and control groups on positive patient-physician interaction, patient-nurse interaction, or overall satisfaction rate. Patients in both groups were generally satisfied with their care. Of the other studies that included control groups, four had patients serving as their own controls, undergoing both teleconsultations and face-to-face consultations.^{88, 113, 114} One of these¹¹³ reported on a videoconferencing system for oncology patients living a substantial distance from their outpatient clinic site. Those same patients served as their own controls at a later, face-to-face visit. While the authors reported that the patients were generally equally satisfied, the patients stated that they were more comfortable saying what they needed to say in-person. In addition, after the in-person consultation, patients were reported as less inclined to want to use the teleconsulting service.

Of the 10 provider satisfaction studies, nine were case series with convenience samples and no control groups. Sample sizes ranged from one referring physician and one consultant¹¹⁵ to five dermatologists and 27 referring physicians.⁸⁸ In several studies, the sample size was unspecified.^{66, 116-118} Provider satisfaction was determined through the use of printed questionnaires, interviews, and focus groups, with no one method predominating. No study reported reliability or validity information on any of the data-collection tools.

In general, providers were satisfied with their teleconsultation experiences. The providers felt, overall, that videoconferencing provided them with enough information of good quality to constitute a satisfactory alternative to face-to-face consultations. Studies in the psychiatry domain found that participating physicians expressed overall satisfaction with the technology, but, given a preference, would chose face-to-face assessments. Despite the limitations of the evidence usedto demonstrate provider satisfaction with telemedicine, there was no conflicting evidence in the literature indicating any provider dissatisfaction.

Key Question 5. Does clinician-interactive telemedicine result in comparable costs of care?

These studies fall into two categories: patient visit replacement and decision support. In visit replacement, the telemedicine application is designed to avoid an office visit by transmitting information from the patient's location to the physician's location. Decision support involves transmission of information designed to implement an immediate decision on patient care (typically whether or not the patient needs emergency transport from a regional medical center to a central medical center such a tertiary care center).

Nine studies assessed visit replacement. The medical specialties included otolaryngology,¹¹⁹ dermatology,^{120, 61, 62, 109, 121} psychiatry,¹¹⁶ oncology,¹²² and general medicine⁷². Three studies appear to come from the same randomized controlled trial of real-timedermatology visits conducted in Northern Ireland.^{61, 62, 109} One study reports that the total societal costs for a real-time teledermatology visit are greater than for a conventional in-person consultation, but greater volume and other efficiencies might reverse this difference. For the rest of the papers, the typical study in this group compared the cost of the set-up and maintenance of a telemedicine application to the costs of transporting either the patients from a remote site to a central site or the specialists from a central site to a remote site. The number of patient visits required to break even (travel costs saved would equal total costs of the telemedicine application) was usually estimated. All of these studies concluded that, given some adequate volume of patients, the telemedicine application would cost less than transporting patients or physicians. A tenth study by Cameron¹²³ presented a general simulation-model approach to economic evaluations and used a video teleconsultation service as an example.

These studies leave important questions unanswered. They do not report comparisons of the total costs of an episode of care or even many components of episodes of care other than transportation of patients for emergency care. They also ignore potential adverse events or necessary follow-up visits. Studies that provide data to answer these questions will allow meaningful assessment of costs of care and cost-effectiveness of telemedicine applications.

Decision support was described in six studies covering neurology and neurosurgery,^{124, 125} radiology,^{103, 126} and emergency medicine.^101,127 In mostcases, the decision was whether or not a patient should be transported from a remote site to a central site. Based on reviews of patients who received telemedicine support (up to 200 patients), other averted or induced costs were ignored. These studies report how many patient transports need to be averted to offset the costs of the telemedicine program. They pose similar problems to those for the visit replacement category above, and several have potential biases. As with the previous category, the only costs considered beyond the costs of the telemedicine application were patient transport costs.

Key Question 6. Is clinician-interactive telemedicine cost-effective?

We identified no studies of the marginal cost-effectiveness of clinician-interactive telemedicine.

Summary

As can be seen, there tends to be good correlation between the existence of programs in given specialties and studies assessing the evidence for those specialties. Unfortunately, however, the methodologic quality of those studies makes the beneficial evidence weak at best. For several services, there are a number of Class II-B and III-B studies for some key questions, indicating that telemedicine technologies show promise in methodologically less rigorous studies. The table also shows that diagnosis and management studies tend to be of higher quality than outcomes, satisfaction, or cost studies.

Store-and-Forward Telemedicine

• A prospective trial should be done to compare the performance of store-and-forward teledermatology versus interactive teledermatology. The study design should compare the two telemedicine modalities to in-person examination and assess inter-observer variation among all of the modalities - i.e., compare one person's store-and-forward or in-person diagnostic assessment with that of another usingthe same modality.

• Within the population of Medicare beneficiaries, aquasi-experimental design in which assignment to conventional or teledermatologic services was made by geographic region or by self-selection could provide useful information about costs and, in particular, about the effect of telemedicine coverage on acceptance, satisfaction of patients and providers, and total utilization. Morbidity from dermatologic diseases would be an important measure of outcome in such a study. Even a very large trial, however, would be unlikely to show an effect on mortality from invasive dermatologic malignancies, but an effect on the stage at which cancers were detected should be measured.

Self-Monitoring/Testing Telemedicine

• Because it targets high-volume, serious conditions -- such as diabetes, congestive heart failure, and management of anticoagulation therapy -- an effective self-monitoring/testing application could offer asubstantial benefit. There are sufficient data from small trials and from observational trials to merit a more definitive assessment of efficacy in large randomized trials.

• We propose a randomized controlled trial of telemedicine monitoring of patients with congestive heart failure. Congestive heart failure is the only major heart disease for which mortality is increasing, and it will likely continue to do so as the American population ages. Many self-monitoring/testing interventions can be done for patients with congestive heart failure. These patients generate much data, such as weight, fluid input and output, and description of symptoms (e.g., orthopnea and pedal edema). They may benefit from frequent communication with, as well as observation by, their health care providers. Moreover, morbidity from congestive heart failure, including admission to the hospital for exacerbations, has been associated with poor access to care and substantial reductions in quality of life. The major outcome measures would include mortality, functional status assessment, and hospitalization requirements. The cost-effectiveness evaluation should include the costs of the monitoring, as well as all physician office visits, emergency room visits, and hospitalizations related to the heart failure. The incremental cost per life year of quality-adjusted life years for the telemedicine application should be estimated.

Clinician-Interactive Services

• Well-done, registry-based, observational studies are needed to assess how telemedicine affects management decisions and how acceptable it is to broad populations of physicians and patients. As shown in Table 12 on pages 53 and 54, programs exist in many domains but little evidence exists to demonstrate efficacy for telemedicine. The acceptability and applicability of many of these services have not been evaluated. Once observational studies have provided strong evidence of effectiveness, randomized trials might focus on patient outcomes. Several clinical areas, including psychiatry, neurology (for Parkinson's disease and stroke care), otolaryngology, orthopedics, and others, merit additional research of this kind.

• We propose a randomized controlled trial to assess the benefit of telepsychiatry. While a substantial proportion of patients with depression and other psychiatric disorders can be managed in primary care, the frequency with which specialty care is needed is higher than for several other common conditions. Individuals with non-psychotic disorders severe enough to warrant a psychiatrist, such as those with severe depression or anxiety disorders, would be randomized to receive their care in-person or via telemedicine. Patients would receive care via one or the other modality, and outcome measures would assess clinical improvement (assessed by a blinded third party), as well as patient satisfaction. Costs of the telepsychiatry service should include the costs of the application and costs associated with all physician and emergency room visits, medications, and hospitalizations for the specified condition. The cost-effectiveness can compare the cost per patient without serious exacerbation of the disorder. Alternatively, total cost of care may be an adequate measure, since higher total costs are likely to be driven by either hospitalizations or intense physician treatment of a worsening of the disorder.

References

Health Care Financing Administration (HCFA). Common Procedure Coding System (HCPCS). [Free Full text in PMC]

American Medical Association. Common Procedural Terminology (CPT) 1999 Edition. [Free Full text in PMC]

Broad search strategy

1. exp telemedicine/

2. telemedicine.mp.

3. telehealth.tw.

4. remote consultation$.mp.

5. 1 or 2 or 3 or 4

6. limit 5 to english language

7. 5 not 6

8. limit 7 to abstracts

9. 6 or 8

Supplemental search strategy

1. exp telemedicine/

2. telemedicine.mp.

3. telehealth.tw.

4. remote consultation$.mp.

5. 1 or 2 or 3 or 4

6. exp home care services/

7. home nursing/

8. 6 or 7

9. exp therapy, computer-assisted/

10. exp computers/

11. exp computer communication networks/

12. exp medical informatics/

13. exp telecommunications/

14. exp monitoring, physiologic/

15. monitor$.tw.

16. blood glucose self-monitoring/

17. self-examination/

18. self exam$.tw.

19. self monitor$.tw.

20. self test$.tw.

21. 14 or 15 or 16 or 17 or 18 or 19 or 20

22. tele$.tw.

23. (remote or offsite or distance).tw. tw=abstract, title

24. rural population/

25. rural health services/

26. hospitals, rural/

27. rural.tw.

28. 22 or 23 or 24 or 25 or 26 or 27

29. 21 and 28

30. 9 or 10 or 11 or 12 or 13 or 29

31. 8 and 30

32. 31 not 5

33. limit 32 to english language

34. 32 not 33

35. limit 34 to abstracts

36. 33 or 35

General Design Considerations

Technical Feasibility and Diagnostic Accuracy

Technical feasibility refers to comparison of images or other telemedicine assessments to similar assessments performed in a more traditional manner. Diagnostic accuracy takes technical feasibility one step further and evaluates decisions that clinicians arrive at using telemedicine assessments rather than traditional assessments. For this discussion, diagnostic accuracy may also include treatment plans made using information from assessments and, ultimately, patient outcomes (such as hospital length of stay or adverse experiences). Because study design approaches are similar for these two links in the analytic framework except for potential comparisons of patient outcomes following diagnoses, we begin by discussing both of these areas and then discuss patient outcomes related to diagnostic accuracy.

Two criteria are important for a well-designed study of technical feasibility.⁵ The first is whether or not the range of study patients covers the range of patients for whom the telemedicine application is intended. If this criterion is not met, there is a potential for selection bias. For example, a teleradiology application may be used to assist in making a decision whether or not to transport a patient from a regional hospital to a tertiary medical center. If there are patients whose condition is sufficiently severe that they would be transported without the teleradiology, then the application has not demonstrated technical feasibility in these more severe patients. Similarly, if this application is used to make preliminary diagnoses, then its use provides no evidence of diagnostic efficacy in more severe patients.

The second criterion is whether or not there is an independent, blinded comparison with a reference standard. The reference standard for a telemedicine application is the traditional application for which the telemedicine application is intended to substitute. Thus, a well-designed study should ensure that there is an independent and, if possible, blinded comparison of the two assessments. For example, having the same radiologist review the teletransmitted image and the original image does not constitute independent assessment and should be avoided. For technical feasibility, both independence and blinding are preferable.

Two other criteria are also useful for a well-designed study of technical feasibility or diagnostic accuracy. First, the results of the test being performed should not influence the decision to use the reference standard. In most telemedicine applications, this is unlikely to occur, since a telemedicine image will usually be compared to an already existing image. Secondly, the details of the test should allow another researcher to replicate the results. Thus, if there are criteria used to make specific diagnoses, these criteria should be listed completely to allow others to replicate the research.

Clinical Outcomes

The usual process of evaluation begins with a demonstration of efficacy. That is, in a carefully controlled situation, can a telemedicine application improve health, access, satisfaction, or so forth? Alternatively, if the telemedicine application is designed to be equivalent to conventional care, this equivalence should be demonstrated. A study designed to show equivalence requires adequate statistical power to demonstrate with a sufficient degree of assurance that the difference between two alternatives is less than a pre-specified target. Studies that have inadequate power (i.e., in which the sample size is too small) cannot be used to provide evidence of the equivalence of two alternative applications. An underpowered study provides little more evidence than does an assumption of equivalence.

Once the efficacy of a telemedicine application has been demonstrated, the next step is to demonstrate the effectiveness of the application. That is, once the application has been shown to work under carefully selected and controlled conditions, will the application work "in the real world" in a broad range of patients under a broad range of conditions? Suppose, for example, that the primary care providers in a phase III randomized efficacy trial of a tele-endoscopy consult are carefully selected based on prior training and interest in endoscopy. While this selected group may provide evidence that a tele-endoscopy consult can work, unselected primary care providers may lack the interest and training to perform as well. In this case, the tele-endoscopy consult may not be effective in the broader group of primary care providers.

To evaluate outcomes, all patients in an effectiveness trial should be accounted for. If some subjects in an assessment failed to complete a survey evaluation, are lost to followup, or are otherwise unaccounted for, the conclusions of the trial could change, since these subjects may have responded differently than those who did respond or stayed in the study. If subjects are randomized within a trial, then intent-to-treat analyses should be performed. That is, the primary analysis should be based on the group to which the subject was randomly assigned. If 20% of subjects assigned to use a telemedicine application chose instead to travel for more traditional care, these subjects (and their associated costs) should be attributed to the telemedicine application. Lack of compliance may be a feature of the telemedicine application in regular usage and thus should be included. An analysis that classifies subjects who are treated may also be performed, but as a secondary analysis. If there is a difference between the two analyses, further research may be needed to reduce the non-compliance and improve the outcomes of the telemedicine application. Similarity of the comparison groups at the start of the study should be assessed and, where differences occur, adjustment should be made in the analyses to account for such differences. Also, the treatment received by each subject (other than the telemedicine or traditional application) should be as similar as possible. For example, if the subject receiving a teleconsult is given an opportunity for an additional face-to-face meeting with a specialist, then a subject using the traditional approach should have the same option. Finally, while it may not be possible to blind a subject to the type of application (telemedicine or traditional), the person performing an assessment on such a subject should be blinded as to the subject's group assignment.

Patient Satisfaction

Many principles that apply to health-related quality of life⁴ are also applicable to patient satisfaction. The first principle in considering a satisfaction study is whether or not the investigators measure aspects of satisfaction that are important to the patient. For example, communication may be very important to the patient having a teleconsult, while physician expertise may be of less importance. If the satisfaction assessment focuses on physician expertise but does not address communication, then it misses an important component. A second consideration is whether or not the satisfaction instrument has been validated. If the instrument does not measure satisfaction as intended, the result will not be useful. To the extent that there are existing validated instruments that evaluate patient satisfaction, telemedicine evaluations should employ such instruments. Evaluating patient satisfaction with a telemedicine application should focus on satisfaction, not merely on the technology. Thus, a satisfaction tool already in use in a traditional application should be applied to the telemedicine application. It may be appropriate to add additional questions to explore specific aspects of the telemedicine application, but it is also important to assess satisfaction compared to a traditional application that is not based on the technology. This is analogous to including both generic and disease-specific quality-of-life tools in a quality-of-life evaluation.

Economic Evaluation

The literature contains several good reviews of economic evaluations.^1-3,6 For example, the text by Gold et al.¹ is a comprehensive review of economic evaluations, and Drummond et al² have recently updated another good overview. The paper by Udvarhelyi³ is a shorter primer of economic evaluations and was used to assess the quality of the economic evaluations reviewed in this report. For this appendix, the 10-item checklist by Drummond et al. (Box 3.1) is used as a guideline to planning economic evaluations. The principles of Gold et al. and Udvarhelyi are all similar.

References

References

References

1.

Rosenthal BE. Telemedicine Sourcebook 1998: A resource guide for the practical application of telemedicine in the managed care marketplace. New York: Faulkner & Gray, Inc.; 1997.

2.

Telemed Virtual Reality 1998. 3:(3). [Free Full text in PMC].

3.

Johnston B, Wheeler L, Deuser J. Kaiser Permanente Medical Center's pilot Tele-Home Health Project Telemed Today 1997. 5:(4):16–7.

4.

Cox R, York ML. Rural telemedicine - the electronic signal flows in both directions Best practices & benchmarking in healthcare: a practical journal for clinical & management applications 1997. 2:(1):28–32.

5.

Lindberg CC. Implementation of in-home telemedicine in rural kansas: answering an elderly patient's needs J Am Med Inform Assoc 1997. 4:(1):14–7. [Free Full text in PMC] [PubMed].

6.

Finkelstein J, Hripcsak G, Cabrera MR. Patients' acceptance of Internet-based home asthma telemonitoring. Proceedings/AMIA Annual Symposium. 1998: 336–40.

7.

Finkelstein J, Hripcsak G, Cabrera M. Telematic system for monitoring of asthma severity in patients' homes Medinfo 1998. 9:(Pt 1):272–6. [PubMed].

8.

Patel UH, Babbs CF. A computer-based, automated, telephonic system to monitor patient progress in the home setting J Med Syst 1992. 16:(2-3):101–12. [PubMed].

9.

Miller PZ. Home monitoring for congestive heart failure patients Caring 1995. 14:(8):53–4.

10.

Slipy SM. Boston hospital using seven telemedicine systems within facility Health Manag Technol 1995. 16:(12):–.

11.

Crist TM, Kaufman SB, Crampton KR. Home telemedicine: a home health care agency strategy for maximizing resources Home Health Care Manag Pract 1996. 8:(4):1–9.

12.

Allen A, Roman L, Cox R, et al. Home health visits using a cable television network: user satisfaction J Telemed Telecare 1996. 2:(Suppl 1):92–4. [PubMed].

13.

Crawford D. Georgia Medical Center delivers home health care. Telemed Virtual Reality. 1997; 2(11)

14.

Ricci MA, Knight SJ, Nutter B, et al. Desktop telemedicine in vascular surgery: some preliminary findings Telemed J 1998. 4:(4):279–85. [PubMed].