Figure 1. Decision process for including and excluding citations
 |
|
|
The Agency for Healthcare Research and Quality (AHRQ), through its Evidence-based Practice Centers (EPCs), sponsors the development of evidence reports and technology assessments to assist public- and private-sector organizations in their efforts to improve the quality of health care in the United States. The reports and assessments provide organizations with comprehensive, science-based information on common, costly medical conditions and new health care technologies. The EPCs systematically review the relevant scientific literature on topics assigned to them by AHRQ and conduct additional analyses when appropriate prior to developing their reports and assessments.
To bring the broadest range of experts into the development of evidence reports and health technology assessments, AHRQ encourages the EPCs to form partnerships and enter into collaborations with other medical and research organizations. The EPCs work with these partner organizations to ensure that the evidence reports and technology assessments they produce will become building blocks for health care quality improvement projects throughout the Nation. The reports undergo peer review prior to their release.
AHRQ expects that the EPC evidence reports and technology assessments will inform individual health plans, providers, and purchasers as well as the health care system as a whole by providing important information to help improve health care quality.
We welcome written comments on this evidence report. They may be sent to: Director, Center for Practice and Technology Assessment, Agency for Healthcare Research and Quality, 6010 Executive Blvd., Suite 300, Rockville, MD 20852.
| Acting Director | John M. Eisenberg, M.D. |
| Center for Practice and Technology | Director |
| Assessment | Agency for Healthcare Research and Quality |
| Agency for Healthcare Research and Quality |
| The authors of this report are responsible for its content. Statements in the report should not be construed as endorsement by the Agency for Healthcare Research and Quality or the U.S. Department of Health and Human Services of a particular drug, device, test, treatment, or other clinical service. |
The goal of this report is to extend our original evidence report on the efficacy of telemedicine by extending the assessment to the pediatrics and obstetrics populations along with those receiving home telemedicine where the health care provider was involved in an indirect manner.As with the initial report, which covered telemedicine for the Medicare population, we assessed telemedicine services that substitute for face-to-face medical diagnosis and treatment and focused on three distinct telemedicine study areas -- store-and-forward, self-monitoring/testing, and clinician-interactive services.
We conducted a search in the peer-reviewed literature for studies assessing the efficacy and cost of telemedicine in the study areas and designated populations. The search focused on peer-reviewed articles in the MEDLINE, CINAHL, and HealthSTAR databases. We also identified relevant articles through hand searching and reference lists in key papers.
The inclusion criteria were that the study addressed one of the designated patient populations, was relevant to at least one of the three study areas, addressed at least one key question in the analytic framework for that study area, and contained reported results. We excluded articles that assessed clinical services that did not historically require face-to-face encounters (e.g., radiology or pathology diagnosis).
Included articles were categorized by the key question(s) they addressed. For each study area, we constructed a summary table of the results and strength of the evidence for each key question.
We identified a total of 28 studies that met inclusion criteria. In the new clinical areas, we found few studies in store-and-forward telemedicine. There is some evidence that diagnosis and clinical management decisions are improved by store-and-forward telemedicine in the areas of pediatric dental screening, pediatric ophthalmology, and neonatalogy. In self-monitoring/testing telemedicine for the areas of pediatrics and obstetrics there is evidence that access to care can be improved when patients and families have the opportunity to receive telehealth care at home rather than in-person care in a clinic or hospital. In the study area of clinician-indirect home telemedicine, there is evidence that clinical outcomes are improved for patients with Human Immunodeficiency Virus (HIV) infection and Alzheimer Disease.
There is some evidence that this form of telemedicine provides comparable health outcomes relative to face-to-face care, but the study sample sizes were usually small, as were the treatment effects. There is also some evidence for the efficacy of clinician-interactive telemedicine, but the studies do not clearly define which technologies provide benefit or cost-efficiency. Some promising areas for diagnosis included emergency medicine, psychiatry, and cardiology. Most of the studies measuring access to care provide evidence that it is improved. Clinician-interactive telemedicine was the only area for which any cost studies were found. The three cost studies did not adequately demonstrate that telemedicine reduces costs of care (except when comparing only selected costs). No study addressed cost-effectiveness.
Our conclusions echo the original report: Existing telemedicine programs demonstrate that the technology can be made operational, but most of the studies assessing the efficacy or cost are insufficient to permit definitive statements about the evidence supporting (or not supporting) the benefits of telemedicine. Future studies should focus on the use of telemedicine in conditions where burden of illness and/or barriers to access for care are significant. Use of recent innovations in the design of randomized controlled trials for emerging technologies would lead to higher quality studies. Journals publishing telemedicine evaluation studies must set high standards for methodologic quality so that evidence reports need not rely on studies with marginal methodologies.
This document is in the public domain and may be used and reprinted without permission except those copyrighted materials noted for which further reproduction is prohibited without the specific permission of copyright holders.
Hersh WR, Wallace JA, Patterson PK, et al. Telemedicine for the Medicare Population: Pediatric, obstetric, and clinician-indirect home interventions. Evidence Report/Technology Assessment No. 24, Supplement (Prepared by Oregon Health Sciences University, Portland, OR under Contract No. 290-97-0018). AHRQ Publication No. 01-E060. Rockville (MD): Agency for Healthcare Research and Quality. August 2001.
This report is a supplement to an earlier evidence report, Telemedicine for the Medicare Population, which was intended to help policymakers weigh the evidence relevant to coverage of telemedicine services under Medicare. That report focused on telemedicine programs and clinical settings that had been used with or were likely to be applied to Medicare beneficiaries. While we prepared that report, it became apparent that there are also telemedicine studies among non-Medicare beneficiaries -- e.g., children and pregnant women -- that could inform policymakers and provide more comprehensive evidence of the state of the science regarding telemedicine applications. In addition, the first evidence report only partially included a class of telemedicine applications (called self-monitoring/testing telemedicine) in which the beneficiary used a home computer or modem-driven telephone system to either report information or access information and support from Internet resources and indirectly interact with a clinician. Self-monitoring/testing applications in the first report required direct interaction with a clinician.
The goal of this report is to systematically review the evidence in the clinical areas of pediatric and obstetric telemedicine as well as home-based telemedicine where there is indirect involvement of the health care professional. (In this report, we will refer to the latter as clinician-indirect home telemedicine.) Specifically, the report summarizes scientific evidence on the diagnostic accuracy, access, clinical outcomes, satisfaction, and cost-effectiveness of services provided by telemedicine technologies for these patient groups. It also identifies gaps in the evidence and makes recommendations for evaluating telemedicine services for these populations in the future. The evidence is clustered according to three categories of telemedicine service defined in our original report: store-and-forward, self-monitoring/testing, and clinician-interactive services.
The three clinical practice areas reviewed in this report are defined as follows. The term pediatric applies to any telemedicine study in which the sample consisted wholly or partially of persons aged 18 or younger, including studies with neonatal samples. The term obstetric applies to any telemedicine study in which the sample consisted entirely of women seeking pregnancy-related care. The term clinician-indirect home telemedicine applies to home-based telemedicine (called self-monitoring/testing in our original report) where a telemedicine application used in the home has only indirect involvement by the health care professional. Interactive home telemedicine was applied in this report to all patient populations.
The key questions that served as a guide for reviewing the literature in the evaluation of pediatric, obstetric, and clinician-indirect home telemedicine applications were derived by consensus among the evidence-review team based on the analytic framework established for the original evidence report. For the current report, the questions were applied to studies in all three practice areas as a whole group within each of the three categories of telemedicine services: store-and-forward; self-monitoring/testing; and clinician-interactive. The specific key questions were:
Does telemedicine result in comparable diagnosis and appropriateness of recommendations for management?
Does the availability of telemedicine provide comparable access to care?
Does telemedicine result in comparable health outcomes?
Does telemedicine result in comparable patient or clinician satisfaction with care?
Does telemedicine result in comparable costs of care and/or cost-effectiveness?
We searched for peer-reviewed literature using several bibliographic databases. In addition, we conducted hand searches of leading telemedicine journals and identified key papers from the reference lists of journal articles. For our original evidence report on telemedicine for the Medicare population, we designed a search to find any publications about telemedicine and used it to search the MEDLINE, CINAHL, and HealthSTAR databases for all years the databases were available. Through this process, we captured studies of pediatric, obstetric, and clinician-indirect home telemedicine; however, they were excluded from the original report since they were outside its scope. For this supplemental report, we reviewed our original search results and identified studies relevant to this report. We identified additional studies from the reference lists of included papers and from hand searching two peer-reviewed telemedicine publications, the Journal of Telemedicine and Telecare and Telemedicine Journal.
We critically appraised the included studies for each study area and key question and discussed the strengths and limitations of the most important studies at weekly meetings of the research team. We also developed recommendations for research to address telemedicine knowledge gaps. To match these gaps with the capabilities of specific research methods, we classified the telemedicine services according to the type of evidence that would be needed to determine whether the specific goals of covering such services had been met. We emphasized the relationship between the type and level of evidence found in the systematic review of effectiveness and the types of studies that might be funded to address the gaps in knowledge in this growing field of research.
We identified a total of 28 eligible studies. In the new clinical areas, we found few studies in store-and-forward telemedicine. There is some evidence of comparable diagnosis and management decisions made using store-and-forward telemedicine from the areas of pediatric dental screening, pediatric ophthalmology, and neonatalogy.
In self-monitoring/testing telemedicine for the areas of pediatrics, obstetrics, and clinician-indirect home telemedicine, there is evidence that access to care can be improved when patients and families have the opportunity to receive telehealth care at home rather than in-person care in a clinic or hospital. Access is particularly enhanced when the telehealth system enables timely communication between patients or families and care providers that allows self-management and necessary adjustments that may prevent hospitalization. There is some evidence that this form of telemedicine improves health outcomes, but the study sample sizes are usually small, and even when they are not, the treatment effects are small.
There is also some evidence for the efficacy of clinician-interactive telemedicine, but the studies do not clearly define which technologies provide benefit or cost-efficiency. Some promising areas for diagnosis include emergency medicine, psychiatry, and cardiology. Most of the studies measuring access to care provide evidence that it is improved. Although none of these studies were randomized controlled trials, they provide some evidence of access improvement over prior conditions. Clinician-interactive telemedicine was the only area for which any cost studies were found. The three cost studies did not adequately demonstrate that telemedicine reduces costs of care (except comparing only selected costs). No study addressed cost-effectiveness.
This supplemental report covering the areas of pediatrics, obstetrics, and indirect-clinician home telemedicine echoes the findings of our initial report for the Medicare domain, which is that while the use of telemedicine is small but growing, the evidence for its efficacy is incomplete. Many of the studies are small and/or methodologically limited, so it cannot be determined whether telemedicine is efficacious. Future studies should focus on the use of telemedicine in conditions where burden of illness and/or barriers to access for care are significant. Use of recent innovations in the design of randomized controlled trials for emerging technologies would lead to higher quality studies. Journals publishing telemedicine evaluation studies must set high standards for methodologic quality so that evidence reports need not rely on studies with marginal methodologies.
This report is a supplement to an earlier evidence report, Telemedicine for the Medicare Population,1 which was intended to help policymakers weigh the evidence relevant to coverage of telemedicine services under Medicare. That report focused on telemedicine programs and clinical settings that had been used with or were likely to be applied to Medicare beneficiaries. As we prepared that report, it became apparent that there are also telemedicine studies among non-Medicare beneficiaries -- e.g., children and pregnant women -- that could inform policymakers and provide more comprehensive evidence of the state of the science regarding telemedicine applications. In addition, the first evidence report only partially included a class of telemedicine applications in which the beneficiary used a home computer or modem-driven telephone system to either report information or access information and support from Internet resources and indirectly interact with a clinician. Self-monitoring/testing applications in the first report required direct interaction with a clinician
The goal of this report is to systematically review the evidence in the clinical areas of pediatric and obstetric telemedicine as well as home-based telemedicine where there is indirect involvement of the health care professional. (In this report, we will refer to the latter as clinician-indirect home telemedicine.) Specifically, the report summarizes scientific evidence on the diagnostic accuracy, access, clinical outcomes, satisfaction, and cost-effectiveness of services provided by telemedicine technologies for these patient groups. It also identifies gaps in the evidence and makes recommendations for evaluating telemedicine services for these populations in the future.
For the purpose of continuity, the report format is similar to that of the prior telemedicine evidence report. This chapter contains definitions of the telemedicine services covered by this report and our key questions. Chapter 2 describes the methods used in the evidence review process. Chapter 3 presents the results for studies that met the inclusion criteria for this evidence review. The evidence is clustered according to three categories of telemedicine service defined in our original report: store-and-forward, self-monitoring/testing, and clinician-interactive services. Chapter 4 provides conclusions and recommendations for future research.
Telemedicine is the use of telecommunications technology for medical diagnostic, monitoring, and therapeutic purposes when distance separates the participants. Some descriptions use the broader term telehealth to indicate care beyond that provided by medical doctors -- for example, its use for patient-to-patient or caregiver-to-caregiver communication. Other descriptions use narrower terms focused on medical specialties, such as telenursing, telecardiology, or telepsychiatry. A telemedicine encounter is the event where clinical services are provided using telemedicine. The narrower term teleconsultation is used when a traditional specialist medical consultation is performed using telemedicine.
The three clinical practice areas reviewed in this report are defined as follows. The term pediatric applies to any telemedicine study in which the sample consisted wholly or partially of persons aged 18 or younger, including studies with neonatal samples. The term obstetric applies to any telemedicine study in which the sample consisted entirely of women seeking pregnancy-related care. The term clinician-indirect home telemedicine applies to home-based telemedicine (called self-monitoring/testing in our original report) where a telemedicine application used in the home has only indirect involvement by the health care professional. This form of telemedicine was studied for all patient populations in this report. This was different from other study areas in this report, which were assessed only for the pediatric and obstetric populations.
As with the previous report, we reviewed three categories of telemedicine services: store-and-forward, self-monitoring/testing, and clinician-interactive services. Likewise, we also excluded services where telemedicine was not a substitute for face-to-face patient care (e.g., radiology and pathology) and where the telemedicine system made exclusive use of audio-only telephone services.
Store-and-forward telemedicine services collect medical data, store them, and then forward them to be interpreted later. This type of service provides the ability to capture and store electronic still or moving images of patients, as well as audio and text data. A store-and-forward system eliminates the need to have the patient and clinician available at the same time. It is usually employed as a clinical medical consultation (as opposed to an office or hospital visit from a medical specialist). The key question associated with store-and-forward use is, Are store-and-forward teleconsultations an acceptable alternative to real-time consultations?
Self-monitoring/testing telemedicine services enable clinicians and others to monitor physiologic measurements, test results, images, and sounds, usually collected in a patient's residence or a nursing facility. Post-acute-care patients, those with chronic illnesses, and patients with conditions that limit their functionality often require close monitoring and followup. These patients may also be taking medications that require testing and/or titration of dosage. Telemedicine systems use a variety of strategies to accomplish this monitoring while reducing the need for face-to-face visits with clinicians that may be inconvenient and costly for the patient and family. For example, some technologies allow patients to directly upload monitoring data to a health care system whereby it can be made available to several clinicians on the patient's health care team. Others make use of higher bandwidth telephone or cable television infrastructures to apply two-way interactive video, audio, and medical diagnostic instrumentation. The close monitoring afforded by these approaches may allow better health care through early detection of problems or more precise dosing of medications and biologic agents, potentially reducing costs. These types of systems also support self-determination and empowerment of patients that can also enhance clinician-patient collaboration.
Clinician-interactive telemedicine services are real-time, clinician-patient interactions that conventionally require face-to-face encounters between a patient and a clinician. Examples of this category include office visits, hospital visits, and consultations, and these may include examinations and procedures conducted during the course of the visits.
shows the decision process for including or excluding studies in this report. All studies passing through these inclusion criteria still had to meet the methodologic inclusion criteria outlined in Chapter 2. Figure 2 shows another view of the coverage of this supplemental report versus the original one, with the pediatric and obstetric areas added to the three telemedicine study areas plus the broadening of self-monitoring/testing telemedicine to include clinician-indirect home telemedicine.The key questions that served as a guide for reviewing the literature in the evaluation of pediatric, obstetric, and clinician-indirect home telemedicine applications were derived by consensus among the evidence-review team based on the analytic framework established for the original evidence report. For the current report, the questions were applied to studies in all three practice areas as a whole group within each of the three categories of telemedicine services: store-and-forward; self-monitoring/testing; and clinician-interactive.
The key underlying question was, To what extent does the telemedicine service delivery have the same components as a traditional clinical encounter for that type of service? These components might include a clinical history, which could be comprehensive or problem-focused; an examination; and clinical decisionmaking, which encompasses analyzing clinical data, initiating or validating a diagnosis, and generating or reviewing a management plan. The specific study questions were:
Does telemedicine result in comparable diagnosis and appropriateness of recommendations for management?
Does the availability of telemedicine provide comparable access to care?
Does telemedicine result in comparable health outcomes?
Does telemedicine result in comparable patient or clinician satisfaction with care?
Does telemedicine result in comparable costs of care and/or cost-effectiveness?
To assess the diagnostic accuracy and effect of these services on management decisions, we looked at the evidence to determine if the telemedicine mode of delivery results in comparable diagnosis and/or appropriate recommendations for management of the patient.
The most powerful evidence for the safety and efficacy of telemedicine applications would directly link their use to comparable health outcomes such as functional status, quality of life, or mortality. In the absence of direct evidence, the effectiveness of telemedicine might be inferred from evidence about improved diagnosis, coordination, or management decisions. To make this inference, evidence should be available that these particular intermediate outcomes are reasonable indicators or proxies for actual health outcomes. For a teledermatology application among children, for example, evidence should be available that more timely or accurate diagnosis of specific skin conditions reduces morbidity or mortality.
Other types of evidence were also sought. One was improved access to care, interpreted as improved convenience, quicker time to urgent or emergent care, or a similar level of care provided in a less intense environment. Another was satisfaction with the telemedicine services, according to patients and/or clinicians. A final type of review of the study findings was to determine whether telemedicine contributes to cost reduction in providing the health services, or whether it can provide added health benefits at a reasonable marginal cost.
This chapter describes the methods the research team used to identify and analyze the telemedicine literature for the supplemental domains of pediatrics, obstetrics, and clinician-indirect home telemedicine in the three study areas and to summarize the scientific evidence for effectiveness and cost.
We searched for peer-reviewed literature using several bibliographic databases. In addition, we conducted hand searches of leading telemedicine journals and identified key papers from the reference lists of journal articles.
For our original evidence report on telemedicine for the Medicare population, we designed a search to find any publications about telemedicine and used it to search the MEDLINE, CINAHL, and HealthSTAR databases through January 2000 (Appendix A). Through this process, we captured studies of pediatric, obstetric, and clinician-indirect home telemedicine; however, they were excluded from the original report since they were outside its scope. For this supplemental report, we reviewed our original search results and identified studies relevant to this report. In addition, we ran the same searches on MEDLINE, CINAHL, and HealthSTAR covering November 1999 to December 2000, which produced 1,035 citations not previously identified. We identified 41 additional studies from the reference lists of included papers, and 58 separate articles from hand searching two peer-reviewed telemedicine publications, the Journal of Telemedicine and Telecare and Telemedicine Journal, to make a total of 1,172 studies for our initial review.
The results of the literature search and selection of articles for inclusion are shown in Figure 3
. For the 1,172 papers identified by the search, reference lists, or hand searches, two teams of two reviewers (WRH and JAW, PKP and JAW) conducted independent screening reviews of all citation titles and abstracts retrieved. The reviewers read citation abstracts (or titles only, if the abstract was not available) to make inclusion decisions for subsequent full-text review. A third reviewer (either WRH or PKP), who was blinded to previous reviewers' results, made "tiebreaker" decisions for citations about which the pairs disagreed.
The inclusion criteria for this review were 1) that the study be relevant to at least one of the three study areas; 2) that it address at least one key question in the analytic framework for that study area; and 3) that it contain reported results (i.e., "data"). Studies that assessed a clinical service that did not require face-to-face encounters (e.g., radiology or pathology diagnosis) were excluded.
For the store-and-forward study area, we included studies that used store-and-forward techniques as well as studies that used systems that could be easily adaptable to store-and-forward. We excluded reports of telephone care programs. We also excluded studies of services that provided medical advice directly to the public -- e.g., Internet-based models with no direct or indirect telecommunications contact with a medical provider.
For each key question, data from each included study were abstracted by an assigned single reviewer, using paper or electronic abstraction forms, and entered into an electronic database. The information abstracted for each study varied by the key question addressed. For studies of diagnostic performance, we recorded the reference standard and whatever measure in the paper reported its efficacy. Additional variables that were abstracted for studies of the economic impact of telemedicine activities included the type of economic study, types of comparisons, data sources, cost unit, discount rate, sensitivity analysis parameters, program expansion capability, and generalizability of the program.
| Study Class | Characteristic | |
|---|---|---|
| Strength of evidence | ||
| I | Properly designed random controlled trials Random controlled trials that contain design flaws preventing specification of Class I Properly designed trials with control groups not randomized Multi-center or population-based longitudinal (cohort) study Case control studies Descriptive studies (uncontrolled case series) Clinical experience Expert opinion Case reports | |
| II | Random controlled trials that contain design flaws preventing specification of Class I Properly designed trials with control groups not randomized Multi-center or population-based longitudinal (cohort) study Case control studies | |
| III | Descriptive studies (uncontrolled case series) Clinical experience Expert opinion Case reports | |
| Direction of effect | ||
| A | Strong positive effect | |
| B | Weak positive effect | |
| C | Conflicting evidence for effect | |
| D | Negative effect (evidence that the technology is inferior or ineffective) |
| Study Class | Characteristic |
|---|---|
| I | Case series of consecutive patients from relevant population of individuals who would use telemedicine; using an objective gold standard with blinded interpretation of results; with inter-observer analysis |
| II | Case series of patients from relevant population of individuals who would use telemedicine; using an objective gold standard |
| III | Case series not from relevant population or not using appropriate methodology for diagnostic test evaluation |
We applied the following additional criteria to studies addressing access to care and economic analyses.
In appraising studies of access to care, we employed the Institute of Medicine (IOM) model of access to care,5 which incorporates four types of indicators: barriers (structural, financial, and personal); utilization; mediators (treatment effectiveness, provider quality, and patient adherence); and outcomes, including health status (e.g., mortality, well-being, or functionality) and equity of services among various populations. The IOM strongly recommended that studies of access to care measure both utilization and outcomes. Also, when access to an improved level of care was the experimental condition, compared with the control group that did not have the same level of access, we considered the experimental group as having 100 percent improved access. Although improved access was inherent in the study design in these instances, we interpreted it as demonstrating improved access feasibility.
The economic evaluation of the telemedicine applications from the literature review included both cost studies and cost-consequence studies. Economic evaluation is a more encompassing term than cost-effectiveness analysis, which refers to a particular class of economic assessments in which costs are compared to measures of effectiveness (such as life-years). Economic assessments encompass other study designs, including program-cost analyses and cost-of-illness studies. Program-cost analyses review only the cost of implementation and maintenance of a particular application. The other designs (including cost-effectiveness studies) are cost-consequence studies, which compare the costs to other consequences, including other economic consequences (as the costs averted by the application), technical consequences (as images of the same quality), or health outcomes (as length of stay or life-years).
Since the literature was limited, studies were not excluded based on reporting costs versus charges. Perspective and study design were summarized for studies evaluated based on how they were described in the articles. Economic evaluations of populations specific to these clinical domains were included -- e.g., pediatric cardiography. We excluded economic analyses of telemedicine applications in prisons because the costs of transporting prisoners are not relevant to these clinical domains. We also excluded reports that claimed to report program costs but which had incomplete data or only a single summary cost, such as only the total cost to set up a telemedicine application.
| Principle | Description |
|---|---|
| Perspective stated | Whose costs and consequences are considered? |
| Benefit described | What are non-economic consequences of program? |
| Costs included | Describe intervention costs, morbidity or side effect costs, averted costs and induced costs? |
| Discounting included | Are future costs and consequences adjusted for timing? |
| Sensitivity analyses | For values that are uncertain (e.g. assumed), are analyses performed using alternative values? |
| Cost-effectiveness ratios stated | Are alternatives compared in a way that allows decisions on prioritization to be made? |
We reported our data synthesis in a summary that represents the state of knowledge for telemedicine in practice for these specific three study areas and three clinical domains. For the synthesis, we began with the evidence found in a systematic review of telemedicine research. Results of the systematic review are presented in Evidence Tables 1-10. The reviewer for each key question constructed separate evidence tables for each of the three study areas (store-and-forward, self-monitoring/testing, and clinician-interactive). In general, the evidence tables include author/date, key research question(s), study design/level, population, sample/selection, measures, results, quality rating, and limitations.
We also developed recommendations for research to address telemedicine knowledge gaps. To match these gaps with the capabilities of specific research methods, we classified the telemedicine services according to the type of evidence that would be needed to determine whether the specific goals of covering such services had been met. We emphasized the relationship between the type and level of evidence found in the systematic review of effectiveness and the types of studies that might be funded to address the gaps in knowledge in this growing field of research.
This chapter presents the results of the literature review followed by identification and evaluation of the evidence. The results are organized into three major sections by study area -- store-and-forward, self-monitoring/testing, and clinician-interactive services. Each of these sections then covers the five key questions as they relate to that study area. Finally, within each question, the clinical domains (pediatrics, obstetrics, and clinician-indirect home telemedicine) are discussed.
Through the abstract review process, we selected 122 studies as possibly eligible. We then retrieved the full-text articles for these studies. For papers excluded from this systematic review, the reviewers were asked to judge their relevance for inclusion in an update to the original telemedicine evidence review -- that is, telemedicine interventions relevant to a Medicare population. This designation was catalogued to provide a method to update the original report at a later date.
In addition, papers that had been identified but excluded from the original report (e.g., wrong subject population or clinical domain) were re-reviewed for inclusion in the supplemental report. Fourteen papers identified from the original search strategy and time period were found to have applicable evidence for the supplemental report. These studies were added to the 122 for a total of 136 papers retrieved for full-text review.
| Diagnosis and management | Access | Outcomes | Satisfaction | Cost | |
|---|---|---|---|---|---|
| Store-and-forward | 3 | 1 | 0 | 0 | 0 |
| Self-monitoring/testing | 0 | 11 | 9 | 1 | 0 |
| Clinician-interactive | 4 | 11 | 1 | 2 | 3 |
| TOTALS | 7 | 23 | 10 | 3 | 3 |
A second study compared diagnostic and management agreement in pediatric ophthalmology, assessing 19 eyes in 10 patients. Agreement was high for diagnosis (89-95 percent) but lower for management plans (42 percent).9 A limitation of this study was the lack of measurement of agreement in examiners using the same modality. A third study, in neonatology, found that neonatologists having access to chest radiographs via telemedicine had a higher rate of accuracy in identifying findings than the in-person general pediatrician.10 A problem with this study was present in all three, which was the small number of physicians used.
The dental study by Patterson,8 described above, demonstrated 100 percent improved access to experts in dental screening for school children and validated that telehealth screening compared favorably with the traditional method of visual inspection by dental hygenists or assistants. Although telehealth did not reduce travel, it did improve access to a higher level of dental expertise.
No studies assessed health outcomes for store-and-forward telemedicine in pediatrics or obstetrics.
No studies addressed patient or clinician satisfaction with store-and-forward telemedicine in pediatrics or obstetrics.
No studies of store-and-forward telemedicine addressed issues of cost or cost-effectiveness in pediatrics or obstetrics.
No studies assessed diagnosis and/or management in self-monitoring/testing telemedicine in pediatrics, obstetrics, or clinician-indirect home telemedicine.
The three pediatric studies that evaluated self-monitoring/testing telehealth interventions provided evidence of improved access. In Japan, a "near-television quality" home digital videophone system was used to provide respiratory specialist communication with families of 10 children at home.11 The system was deployed between a pediatric intensive care unit, patient homes, and technical (ventilation) advisors for the purpose of providing specialty care support directly to patients. The impact of this observational study on access to care could be interpreted from findings of fewer clinic visits and hospital days required by patients following installation of home videophones, thereby achieving longer periods of comparable care that did not require family travel. A randomized controlled trial (RCT) of children with diabetes compared those who used an electronic glycemic data transmittal system at home with those who did not.12 Subjects in both groups had clinic visits every three months, so there was no opportunity to compare family travel requirements. However, experimental subjects had improved access to nursing guidance for insulin adjustment and other alterations following each data transmittal between clinic visits. Another RCT evaluated a system of Internet communication between a neonatal intensive care unit (NICU) and patient homes.13 Access was comparable between the study groups, as determined by the frequency of family telephone calls and visits to the NICU. However, when the whole episode of neonatal care was taken into account, access was improved for experimental families, since 20 percent of control patients needed additional hospitalization following NICU discharge versus zero percent for the experimental group.
In obstetrics, two controlled studies14, 15 evaluated electronic monitoring systems designed to help diabetic patients maintain good glycemic control and reduce the incidence and severity of complications associated with pregnancy. Both studies had relatively small samples, but they provided evidence of greater access via telehealth with more timely metabolic management advice. In one, an RCT,14 both groups regularly monitored their blood glucose at home, but experimental subjects had 24-hour access to electronic insulin adjustment decisionmaking support, while control subjects relied on this type of help from a nurse who visited approximately weekly. Although experimental subjects saw the nurse only every 2-4 weeks, they were more satisfied with their access to professional help. The other trial15 was not an RCT, and staff saw patients in the clinic rather than at home. Both studies found that telehealth subjects needed less frequent contact with clinicians than control subjects. Thus, evidence of improved access was based on increased guidance availability (24-hour) for experimental subjects14 and increased home care and decreased clinic care for experimental subjects in both studies.
Three other obstetrics studies, all RCTs, provided positive evidence of improved access with telehealth self-monitoring systems at home. An RCT by Cartwright et al.16 compared blood pressure monitoring at home versus in the hospital and demonstrated that home monitoring improved access because it reduced travel and other barriers associated with hospital admission. An RCT by Dawson and colleagues17 compared fetal monitoring at home versus in a hospital or clinic for high-risk patients, and both experimental and control subjects had access to visits at home and in the clinic. Telehealth subjects demonstrated they had improved access because they received more care at home and less care in clinic or hospital than conventional care subjects did. A multi-center RCT of uterine contraction monitoring in Hungary18 also demonstrated that telehealth has the potential of improved access over traditional care because it can be done at home instead of in the clinic or hospital, although few details were provided.
Three RCTs provided evidence of increased access to care via the use of clinician-indirect home telemedicine.19-21 These studies evaluated informational and support programs that utilized a keyboard and computer network and were developed for use by caregivers of persons with Alzheimer Disease, persons living with AIDS, and persons who were HIV-positive. Evidence of improved access to care was strong in the Flatley-Brennan studies,19, 20 since access was the independent (experimental) variable that was assessed. In addition, the frequency of system use was also reported, with experimental subjects using it an average of twice per week over a one-year period. In the Gustafson et al. study,21 evidence of access was comparable, since there were no differences in the use of outpatient care and emergency care for experimental and control groups. The study showed evidence of improved access because experimental subjects made more phone calls to providers during the experiment, there were fewer hospitalizations, and they had fewer hospital days per subject.
Three of the five studies addressing access from obstetrics also looked at health outcomes. Both studies of home diabetic monitoring found improvements in blood sugar values but not in HgbA1C, although each had very small sample sizes and probable inadequate statistical power to detect a difference.14, 15 The study of hypertension found that an at-home blood pressure monitoring system resulted in comparable levels of blood pressure, anxiety, and gestational age of delivery as hospital-based monitoring.16
The three clinician-indirect home telemedicine studies demonstrated a variety of improved health outcomes, such as improved patient decision-making confidence, 19, 20 and decreased hospitalizations. 21 Each system provided a variety of functions, including online information and support groups, the latter with some monitoring by a health care professional. All the studies showed improvement in the health outcomes they set out to measure.
We identified no studies of self-monitoring/testing telemedicine that addressed issues of cost or cost-effectiveness in the subject areas for this report.
One of the cardiology studies assessed pediatric stethoscopy. A total of 116 patients were assessed for diagnostic agreement among cardiologists using acoustic, electronic, and tele-electronic stethoscopes, with the acoustic considered the gold standard.24 The kappa scores for agreement with the acoustic stethoscope and the others were 0.64-0.65, while the sensitivity and specificity in the need for echocardiogram or followup were 88 percent and 97 percent respectively.
In the second cardiology study, 10 neonates suspected of having congenital heart disease were first examined via a telemedicine link and then examined in followup by a pediatric cardiologist.25 Of nine patients who were actually assessed, the correct diagnosis was obtainable via the telemedicine link.
Non-experimental designs were also used to demonstrate improved access to care in pediatric cardiology studies. Three25, 26, 30 demonstrated improved access to local screening for neonatal abnormal heart conditions by experts, and improved access to local treatment rather than transfer to a distant tertiary care center. Although the determination of negated transfer was based on unvalidated diagnostic differences between consultant and local provider, rather than on validated diagnostic differences, the comparison of experimental and control subjects, or pre-post sample incidence rates, these findings nevertheless provided positive evidence of access improvement.
In one study of cardiology telemedicine consultation for newborns and older children,27 patient transfers were prevented for 28 percent of a non-emergency sample. Although some echograms were repeated (19 percent) and diagnostic agreement was 81 percent, there was no discussion of how transfer judgments were made or validated. Thus, it was difficult to determine whether telehealth was the predictive factor for local care and improved access.
Another pediatric study demonstrated the use of telehealth in schools for medical consultation.31 Although access was not a central question in this study, the total number of consultations represented 100 percent improved access to medical expertise from inner-city school sites, since there were no consultations prior to the intervention.
Three teleconsultation studies in pediatrics demonstrated improved access to specialist care in remote geographic areas.32-34 Although these were not controlled studies, they demonstrated that the telehealth innovation provided 100 percent more local care than the prior method of travelling to a tertiary care center, and two studies measured the number of travel miles saved per patient. In Minnesota, patients from 15 sites participated in a study of burn specialist followup care via telemedicine.34 Evidence of 100 percent improved access to care is based on findings that nearly 700 miles of travel were saved per patient. Another type of need that has traditionally been met in large medical centers is for consultation regarding rare genetic disorders and neurological conditions. In Georgia, provision of this type of distance consultation resulted in saved travel of 89 miles per patient. Finally, a qualitative demonstration study showed that adult patients and parents of pediatric patients who were willing to try telehealth for otolaryngology consultation felt they had access to more information with image visualization and more opportunity for informed communication with physicians than they did prior to telehealth.
In obstetrics, two papers met the inclusion criteria for this telemedicine mode. Researchers in Australia,35 where distances between tertiary care and primary care are great, evaluated a system of real-time ultrasound transmission for diagnosis and assessment. Maternal-fetal medicine subspecialists provided consultation and a few patients (12 percent) would have been transferred a great distance (1,500 kilometers) for specialty care if it were not for telemedicine. Although this was a non-controlled pilot study with a small sample size, it provided modest evidence of improved access with the intervention. In Taiwan, where emergency department physicians provided consultation to remote islanders, teleconsultation also improved access (32 percent of the whole sample) with reduced transfers to tertiary care.36 However, the rate of improved access for the few obstetric patients was not reported.
The same two studies were also the only ones identified that addressed clinician attitudes or satisfaction with this modality. In response to an unvalidated questionnaire, child psychiatrists indicated a significant preference for face-to-face interviews.23 However, videoconferencing was viewed as an "acceptable" alternative and did not interfere with diagnosis. This study is compromised by a small sample size (n=5) and the fact that the psychiatrists served as their own controls. Attitudes of pediatric subspecialists toward telemedicine were reported in the second study.33 On an unvalidated survey, 70 percent reported positive attitudes toward telemedicine regarding its feasibility in providing patient consultations and improving access for patients; 44 percent felt it was cost-effective, and 48 percent time-effective. An increase in experience with telemedicine was correlated with more positive responses (r=.3); (p<.05). However, only 14 of the 56 respondents actually participated in the telemedicine sessions, and there are no specific satisfaction data available for this group.
The Finley et al. study27 assessing real-time echocardiography for regional hospitals in Canada found that for over 2 years, the costs of emergency room and clinic visits avoided by those using the telemedicine application would have exceeded the cost of the telemedicine application. These authors compared diagnosis by the telemedicine application and diagnosis by clinic visits for 26 cases and found no "important discrepancies" in diagnosis. However, the lack of clinical outcomes (the episode of care or all care over a specified time period) and the small number of cases compared (with low statistical power to detect a difference in diagnostic accuracy) make it difficult to draw conclusions.
The previously described study by Sable et al.30 demonstrated that reduction in the number of helicopter transports could offset the costs associated with a telemedicine application. They also report that charges for echocardiography interpretation and for inpatient care of transported patients were greater during the period when telemedicine was used than during a previous period of equal duration. However, since the number of patients and severity of conditions are not reported and since no data on patients not transported are provided, interpretation of these results is problematic.
This supplemental report covering the areas of pediatrics, obstetrics, and indirect-clinician home telemedicine echoes the findings of our initial report for the Medicare domain, which is that while the use of telemedicine is small but growing, the evidence for its efficacy is incomplete. Many of the studies are small and/or methodologically limited, so it cannot be determined whether telemedicine is efficacious.
In the new clinical areas, we found few studies in store-and-forward telemedicine. There is modest evidence of benefit in diagnosis and management decisions from the areas of pediatric dental screening, pediatric ophthalmology, and neonatalogy.
In self-monitoring/testing telemedicine for the areas of pediatrics, obstetrics, and clinician-indirect home telemedicine, there is evidence that access to care can be improved when patients and families have the opportunity to receive telehealth care at home rather than in-person care in a clinic or hospital. Access is particularly enhanced when the telehealth system enables timely communication between patients or families and care providers that allows self-management and necessary adjustments that may prevent hospitalization. There is some evidence that this form of telemedicine improves health outcomes, but the study sample sizes are usually small, and even when they are not, the treatment effects are small.
There is also some evidence for the efficacy of clinician-interactive telemedicine, but the studies do not clearly define which technologies provide benefit or cost-efficiency. Some promising areas for diagnosis included emergency medicine, psychiatry, and cardiology. Most of the studies measuring access to care provide evidence that it is improved. Although none of these studies were randomized controlled trials, they provide moderately strong evidence of access improvement over prior conditions. Clinician-interactive telemedicine was the only area for which any cost studies were found. The three cost studies did not adequately demonstrate that telemedicine reduces costs of care (except comparing only selected costs). No study addressed cost-effectiveness.
In interpreting these results, it is important to recognize the possibility of publication bias, whereby studies with negative results may be less likely to be published. Because of the heterogeneous studies in the data we have uncovered, we cannot assess whether such a bias is present. We do know that many programs have not published efficacy data, but we cannot determine whether this is due to failure to collect data or failure to publish it.
Numerous gaps remain in the research on the efficacy of telemedicine. The problem is not that studies have strong evidence against efficacy, but rather that their methodologies preclude definitive statements. Many of them have small sample sizes that limit statistical power. Others are done in settings that may not generalize to real clinical settings. Most include convenience samples of patients rather than target populations that might benefit most from improved access to health services, such as those who are indigent and/or have complex chronic diseases.
While these gaps underscore the importance of scientific validation, they do not diminish the case for developing telemedicine. It is important that these gaps be closed by research funded by objective third parties, such as the National Institutes of Health (NIH) and the Agency for Healthcare Research and Quality (AHRQ).
In the peer-reviewed literature as well as other media reports, we found a preponderance of observational studies evaluating active telemedicine programs. Active programs demonstrate that the technology can be made operational and, in some cases, may be clinically and economically viable. The longevity of these programs, however, is not clear, and many may fail to survive beyond initial funding or enthusiasm. Lack of information about the viability of these programs limits the generalizability of these studies reviewed for this supplemental report.
Another gap in research is that many studies are not focused on the patients who stand to gain the most from the availability of telemedicine services. It may well be that telemedicine studies in the somewhat artificial experimental conditions set up in outpatient clinics, emergency rooms, and nursing homes would have different results in actual underserved rural or inner city locations. This is because the research goals of most evaluations are often limited to proving the feasibility of implementing new technology. As McLaren and Ball wrote in 1995,
(Telemedicine's) driving force has been developments in communications technology, and as new communications systems are developed health applications are proposed such as supporting the delivery of primary health care to geographically remote areas or regions underserved through the maldistribution of professional expertise. Despite rapid technological advances, evaluations of such systems have been largely superficial, and more thorough evaluations have failed to show significant advantages for more advanced and expensive technology over older technology such as the telephone. Methods for evaluating the impact of particular technologies on the health care system need to be developed and clearer benefits shown in terms of improved standards of care.37
This pattern is not unique to telemedicine. Frequently, when new technologies are introduced into practice, they are introduced "without a clear idea of which patients will benefit most, what the balance of benefits and harms is, and what value for money technologies offer."38 To close these information gaps, specific hypotheses concerning a target population, methods of implementation, and effects on access to care, resource use, and health outcomes must be the driving force in research.
These gaps in information limit the ability of policymakers to make informed judgments about telemedicine coverage. This is particularly problematic if the decision to provide reimbursement for a particular service depends on having high-quality evidence that the benefits outweigh the harms and that the service is cost-effective. For many services, we found no reliable evidence about clinical effectiveness, harms, or cost-effectiveness.
Prioritizing research needs is an important, early step in evaluating telemedicine. Some criteria for prioritizing effectiveness research are listed below. Programs with high research priority should:
Address common, serious clinical conditions that require frequent contact with the health care system and often place a heavy burden on patients or their parents and caregivers that limits the effectiveness of care. In some specialties, such as pediatrics and obstetrics, covered in this report, there are fewer practitioners in rural areas (proportionally to internal medicine or family medicine), so telemedicine may be able to provide additional primary care.
Address clinical procedures and circumstances for which components of care can be performed remotely. Examples include inexpensive home otoscopes that allow assessment of ear pain in children, home fetal monitoring in pregnant women, store-and-forward teleradiology and telepathology, home health care delivered by a parent or nurse, frequent review of laboratory parameters, and revision of management (e.g., medication use) in patients with chronic conditions.
Aim to reduce medical errors in diagnosis or management.
Extend the capacity of the health care system to provide care to populations for whom barriers to access have been shown to affect indicators of health outcome and quality of care.
Promote other policy goals, such as strengthening rural health care by keeping care local.
Priority-setting for effectiveness studies should also consider the degree of uncertainty about benefits, harms, and costs, and the likelihood that the research will have an impact.
Also in future research, economic evaluations need to focus on episodes of diseases or conditions rather than a single interaction with the health care system. In addition, such evaluations need to focus on societal and patient perspectives. They also need to focus on meaningful outcomes (such as quality-adjusted life-years), should include adverse events, and should incorporate indirect costs associated with travel (that may be reduced with telemedicine applications). These recommendations are described in more detail in the original report.1
The need for randomized controlled trials to establish the effectiveness of new technologies is widely accepted.39 Well-done trials can ensure early adoption of techniques that are clearly cost-effective and early detection of problems that need to be corrected. Large, inclusive, observational studies that measure the effect of providing a service on population-based measures of utilization, access, and outcomes are also important because they provide information about how a particular service performs in actual practice. These designs, properly reviewed by experienced investigators in study sections, offer a higher potential to provide high-quality information about effectiveness than do demonstration projects.
There are differing opinions about the role and timing of randomized controlled trials in evaluating the efficacy of telemedicine. Some caution against starting trials too early, before they have had time to detect and address system problems or an effective implementation strategy. Others note that, by the time a randomized trial is designed and funded, not to mention completed, the targeted intervention may already have changed substantially.
While designing studies of emerging technologies is not simple, it can be accomplished. In fact, evaluating the clinical impact of a telemedicine service beginning with the first patients it is used for has several advantages. First, starting a registry or trial when technology is still changing "ensures maximum use of information after it has stabilized."40 Second, when technical performance and operator skill are important component of effectiveness, they are part of the intervention and should be studied. Such a design prevents selective reporting of only the best results of a given application, a bias that is a common flaw in existing studies. Over time, a registry containing all clinicians who provide these services, and all patients who use them, can provide important data about the evolution of the technique and about the "learning curve" for clinicians who might want to adopt the service in their practices.
The fact that telemedicine is evolving thus heightens the need to assess its impact systematically. Telemedicine is an ideal topic for innovative experimental designs that are adapted to rapidly changing technologies. Recently, techniques to adapt randomized trial design to rapidly changing interventions have been proposed. For example, a "tracker trial" is guided by flexible protocols and offers the opportunity to add or drop arms as clinically available options for delivering a service emerge and others become obsolete.40 The use of this approach, or other innovative approaches to studying technologies that change rapidly, should be incorporated into future research about telemedicine.
Along with telecommunication technology itself, the characteristics of the patients studied and the judgment of an individual physician, often a specialist, are major components of the clinical intervention in telemedicine. The results of an evaluation will be more generalizable to other patients and to clinicians in other settings if the characteristics of the patient population are defined and the judgments of the study physicians are consistent and reproducible. Most studies have focused on recruitment of participating clinicians rather than on identifying a target patient population, selected because of the potential to improve outcomes via better access to care.
From the viewpoint of informatics, then, there is a need for basic research to inform the implementation of telemedicine programs. This research should:
Refine the target population for telemedicine services. This research should:
- Address the burden of disease attributable to poor access to specialty care.
- Evaluate barriers to access that coincide with (and might affect the impact of telemedicine services on) distance and mobility barriers.
- Identify groups that are most likely to benefit from specific telemedicine services. Clinical studies should examine differences in effectiveness and cost-effectiveness based on the circumstances of the use of telemedicine. The research should examine the relationship between the effectiveness of telemedicine services and characteristics,
- Establish valid measures of access that allow relevant indicators to be quantified, e.g. hours or days saved by patients, additional encounters made possible with telemedicine, miles saved with telemedicine per encounter. Health needs, and barriers to access of rural poor, inner-city poor, and other populations.
Refine clinical interventions prompted by telemedicine services. This research should:
- Develop and validate protocols, including computer-assisted decision tools, to use self-monitoring and testing information effectively in reducing preventable hospitalizations and improving functional outcomes.
- Examine strategies for organizing telemedicine services in a way that reduces the burden on participating practitioners. Specifically, we recommend developing a request for proposals to explore ways to implement telemedicine services in the context of community-based primary care practices. This can be best accomplished by conducting this research in association with primary care practice networks. In the past, most telemedicine research involving a referring primary care physician has used small convenience samples of clinicians. Primary care practice networks are the "laboratory" for conducting research about practice management, for designing research that minimizes the disruption to the flow of practice, and for examining the context and effects of innovative services in community-based practice settings.
Develop or adapt standardized tools to measure the effectiveness and harms of telemedicine services.
Explore different mechanisms for delivery and payment for telemedicine services, assessing their impact on utilization in a target population of patients.
Editors of scientific journals might also play a role in improving the quality of studies that evaluate telemedicine technology. The telemedicine literature we reviewed, whether in core or clinical specialty journals, is riddled with studies that describe creative uses of technology but feature a low-quality evaluation study. We recommend that journal editors decline to publish these low-quality evaluations and instead let the technologists publish their technology descriptions and allow their technology-assessment collaborators to carry out appropriate evaluations.
A large number of gaps remain in both efficacy and effectiveness research concerning access, satisfaction, quality of care, cost, and cost-effectiveness of telemedicine applications. The body of current knowledge surveyed in this supplemental review provides sufficient evidence that new, better-designed studies need to be undertaken. With the availability of telemedicine programs throughout the United States expanding rapidly, the demand for the technology has already begun to grow, even without compelling evidence to demonstrate its efficacy or cost-effectiveness. This leaves the policymakers with the difficult question of how to deal with this growing field. Given that demand, it seems equally compelling that some significant resources should be expended proposing, funding, designing, implementing, and reporting on this intriguing question.
exp telemedicine/
telemedicine.mp.
telehealth.tw.
remote consultation$.mp.
1 or 2 or 3 or 4
limit 5 to english language
5 not 6
limit 7 to abstracts
6 or 8
exp telemedicine/
telemedicine.mp.
telehealth.tw.
remote consultation$.mp.
1 or 2 or 3 or 4
exp home care services/
home nursing/
6 or 7
exp therapy, computer-assisted/
exp computers/
exp computer communication networks/
exp medical informatics/
exp telecommunications/
exp monitoring, physiologic/
monitor$.tw.
blood glucose self-monitoring/
self-examination/
self exam$.tw.
self monitor$.tw.
self test$.tw.
14 or 15 or 16 or 17 or 18 or 19 or 20
tele$.tw.
(remote or offsite or distance).tw. tw=abstract, title
rural population/
rural health services/
hospitals, rural/
rural.tw.
22 or 23 or 24 or 25 or 26 or 27
21 and 28
9 or 10 or 11 or 12 or 13 or 29
8 and 30
31 not 5
limit 32 to english language
32 not 33
limit 34 to abstracts
33 or 35
exp computer communication networks
patient participation
exp consumer satisfaction
delivery of health care
exp home care services
exp home nursing
house calls/or housecalls.mp
2 or 3 or 4 or 5 or 6 or 7
1 and 8
limit 9 to english language
Principal Investigator:
William R. Hersh, MD
Associate Professor and Chief
Division of Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
OHSU EPC Director:
Mark Helfand, MD, MPH
Associate Professor of Internal Medicine and Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
OHSU EPC Administrator:
Kathryn Pyle Krages, AMLS, MA
Division of Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
Research Coordinator:
James A. Wallace, BA
Division of Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
Co-investigator:
Merwyn R. Greenlick, PhD
Professor and Chair Emeritus
Department of Public Health and Preventive Medicine
Oregon Health Sciences University
Portland, OR
Co-investigator:
Patricia K. Patterson, RN, PhD
Assistant Professor of Nursing and Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
Co-investigator:
Dale F. Kraemer, PhD
Assistant Professor
Division of Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
Co-investigator:
W. Paul Nichol, MD
Clinical Associate Professor of Medicine
University of Washington
Seattle, WA
Librarian:
Patty Davies, MS
Oregon Health Sciences University Library
Portland, OR
Technical Writer/Editor:
Gary Miranda, MA
Portland, OR
Administrative Assistant:
Susan Wingenfeld, BA
Division of Medical Informatics & Outcomes Research
Oregon Health Sciences University
Portland, OR
Mark Segal, PhD
Vice President, Business Development for Governments & Associations
Medical Present Value, Inc.
Oak Park, IL
Peter Tarczey-Hornoch, MD
Assistant Professor
Department of Pediatrics
University of Washington
Seattle, WA
Hussein Noorani, MSc
Research Officer
Canadian Coordinating Office for Health Technology Assessment
Ottawa, Ontario
Canada
Penny Jennett, MD
Professor, Faculty of Medicine
University of Calgary Health Sciences Center
Calgary, Alberta
Canada
Susan Zollo, MA
Director, Telemedicine Resource Center
University of Iowa
Iowa City, IA
David Lairson, PhD
Professor of Management & Policy Sciences
University of Texas - Houston
Houston, TX
| Reviewer | Review Section | General and Specific Comments | Disposition |
|---|---|---|---|
| Zollo | C. Methods | Missing critical lit: Citation for Scholz | Article does not address any of our key questions; it is about telemedicine usage. |
| E. Conclusions and Future Research | Long comments not included in e file | No specific suggestions for change in these comments. | |
| Segal | No specific comments | ||
| Lairson | A. Gen Comments | The phrases "modest evidence" and "moderately strong" are used to characterize the evidence. It is not entirely clear what this means and therefore should be defined or more specific language substituted. | These phrases have been replaced with "some evidence." |
| C. Methods | Telematics Application Program, European Commission see the web site http:158.169.50.95:10080/telematics/projectguide/healthcare.html for a list of ongoing projects in Europe. | This site lists ongoing projects but not evaluation studies. | |
| D. Results | See page specific comments | ||
| E. Conclusions and Future Research | Future studies should measure indirect cost consequences associated with telemedicine services. The technology has the potential to save large amounts of patient and family time and this should be valued and included in evaluations done form the societal and patient perspectives. | We agree and have noted this in the report. | |
| Jennett | A. Gen Comments | Excellent report. Important addition to the literature, thorough lit search, well written and researched. Good flow and easily understood. Pertinent information provided. | |
| Tarczy-Hornoch | A. Gen Comments | Please send copies. | |
| A. Gen Comments | The NLM telemedicine projects are under-represented, does not appear that all the data presented at recent NLM evaluate in Telemedicine symposium was incorporated. | The studies presented at the NLM meeting, which the PI attended and have the proceedings for, are mostly preliminary. The short papers in the proceedings have not been peer-reviewed. | |
| E. Conclusions and Future Research | See page 6A and 6B also | ||
| Noorani | C. Methods | The Cochrane Library, Issue 4, Oxford Update software (2000). A recent Cochrane review by Correl R, et al on "telemedicine versus face-to-face care: effects on professional practice and health care outcomes". Five (of 7) trials reviews are concerned with the provision of home care or patient self-monitoring of chronic disease. Two of these studies (Cartwright, Marrero) are included in your report. | The other studies in the Cochrane review were not pertinent to this patient population, and were cited in the original Medicare report. |
| D. Results | Considering in titles of labels ?: "clinician-indirect" as opposed to "patient-indirect". Table 7 | The title of the table has been changed to use "clinician-indirect." | |
| E. Conclusions and Future Research | Well-written sections on "gaps in research and future directions |
| Reviewer | Page | Line | Comments | Disposition |
|---|---|---|---|---|
| 43 | 8-14 | Discussion about primary care networks. I'm not sure I would agree with this. The resistance to telemedicine is coming from the consultants because they are already way too busy with clinical practice, teaching, research, and publishing to add another area of responsibility to their schedules. PC practitioners and patient and families benefit the most from telemedicine so it is not as difficult to elicit their participation. If you have a PC network as a test bed, who does the consulting? Consultants also need a way to integrate telemedicine into their schedules. | We disagree. Although we acknowledge the resistance from consultants, primary care networks would still be an excellent place to study telemedicine. | |
| Zollo | ||||
| 10 | 22 | It's difficult to assess cost effectiveness when there is only minimal reimbursement | We disagree. Reimbursement and cost-effectiveness are two separate issues. | |
| 18 | 1-7 | As stated previously, it is difficult to assess where the articles included in this study are representative of the published literature w/o being able to review the search strategies used. Many telemedicine articles are being published in non-telemedicine journals, some with a more rigorous peer-review process. | The search strategy was in Appendix I, but this reviewer did not see it. No other reviewers had any comments on it. | |
| 21 | 9-13 | Did costs include both start-up and ongoing costs? | Yes. | |
| 42 | 9-11 | "Most studies focused on recruitment of clinicians rather than identify a target patient population..." That's because it is so difficult to recruit consultants to participate and w/out them, you have no teleconsultation process. | True. | |
| Segal | No comments | |||
| 10 | 29 | Is vague. Which health outcomes were improved? | The specific improved outcomes are now listed. | |
| Lairson | ||||
| 3 | 7 | What does "modest evidence" mean? | We changed this to "some evidence." | |
| 10 | 10 | What does "modest evidence" mean? | We changed this to "some evidence." | |
| 10 | 18 | What does "moderately strong evidence" mean? | We changed this to "some evidence." | |
| 39 | 13 | Remove "ness" from cost-effectiveness | Done. | |
| 42 | Add a section on economic evaluation, e.g. incorporate indirect cost savings in economic evaluations done from societal and patient perspectives. | We have added some comments on this. | ||
| Title page | Evidence-based should be Evidence-Based | Fixed. | ||
| Jennett | ||||
| 2 | search strategy | Insert "years" which were searched. Was the TIE searched? | We added the years searched. We did not search TIE, whose peer-reviewed evaluation articles are similar to those in the databases we did search. | |
| 2 | 15 | Add review of progress reports and conference papers from NLM Telemedicine initiative | The studies presented at the NLM meeting, which the PI attended and have the proceedings for, are mostly preliminary. The short papers in the proceedings have not been peer-reviewed. | |
| Tarczy-Hornoch | ||||
| 4 | 2 | Consider also especially in Pediatrics and Obstetric interactions of benefit/cost savings for conditions that are very frequent, (I.e. a cheaper home store/forward tele otoscope for otitis media as a screening tool to decrease visits to ER and clinic for rule out otitis) | We use these as examples in the future research section. | |
| 9 | 12 | See p2, l 15 comment | The studies presented at the NLM meeting, which the PI attended and have the proceedings for, are mostly preliminary. The short papers in the proceedings have not been peer-reviewed. | |
| 11 | 6 | See p 4, l 2 comments. Also consider funding only studies w/adequate statistical power and encouraging multi-center studies) | We use these as examples in the future research section. | |
| 36 | Same comments as above apply to conclusions section and priorities for future research | Agree. | ||
| 39 | 20 | Extremely common but less serious a possible target for pediatrics | We mention this in future research. | |
| 40 | 5 | In peds and OB total # of practitioners are smaller than IM (ditto for subspecialties). Thus another high yield area would be those subspecialties that are a) capable of delivery telemedicine services and b) more than 200 miles from most common sites of peds and ob care | We mention this in future research. | |
| 6 | 28,32,37 | Spelling corrections (some other sections also require careful editing (e.g.. Table 7) | ||
| Noorani | ||||
| 9 | 11 | Journal of Telemedicine and TELECARE, NOW Telemedicine journal and e-health | We prefer to use the names that these journals had when we hand-searched them. | |
| 9 | 12 | Journal of Telemedicine and TELECARE, NOW Telemedicine journal and e-health | We prefer to use the names that these journals had when we hand-searched them. | |
| Reviewer | General and Specific Comments | Disposition |
|---|---|---|
| Zollo | Would like to see search strategy | Search strategy was in Appendix I in her copy but she missed it. |
| Segal | No specific comments | |
| Jennett | No specific comments | |
| Tarczy-Hornoch | No specific comments | |
| Lairson | No specific comments | |
| Noorani | Check Table 7 | Legend fixed per general comments. |
Free Full text in PMC]Free Full text in PMC].Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC]
[PubMed].Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC]Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC]Free Full text in PMC]Free Full text in PMC].Free Full text in PMC].Free Full text in PMC]Free Full text in PMC]
[PubMed].Free Full text in PMC]
[PubMed].Free Full text in PMC]Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC].Free Full text in PMC].Free Full text in PMC].Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC]Free Full text in PMC].Free Full text in PMC]
[PubMed].Free Full text in PMC]
[PubMed].Free Full text in PMC]
[PubMed].The two papers by Rendina et al. are considered one study.
