Home > DARE Reviews > Are we measuring the right end-points?...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. York (UK): Centre for Reviews and Dissemination (UK); 1995-.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet].

Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review

V Sintchenko, F Magrabi, and S Tipper.

Review published: 2007.

CRD summary

This review found that electronic decision support systems improve prescribing and treatment outcome for patients with acute illness and are less effective in primary care. The review suffered from a number of limitations, including a restricted literature search, failure to assess study quality and a questionable approach to analysis. These findings are therefore unlikely to be reliable.

Authors' objectives

To determine the effects of computerised decision support systems on patient outcomes.

Searching

MEDLINE/PubMed and DARE were searched from 1994 to January 2006; the search terms were reported. The reference lists of included studies were screened. The review was restricted to studies reported in English.

Study selection

Randomised controlled trials (RCTs), conducted in a clinical setting, that randomised patients, encounters, practitioners or practices to an electronic decision support system (EDSS) intervention, and that reported objective or surrogate measures of patient outcomes or measurable variables with an indirect or unestablished connection to the targeted clinical outcome, were eligible for inclusion. An EDSS was defined as any system that provided electronic patient-specific information to support clinical decisions. Studies in which patients measured or assessed the outcomes were excluded.

The included studies, which were conducted in the USA, UK, Canada and Norway, enrolled patients with acute illnesses or acute exacerbation of chronic disease in a hospital setting, or enrolled patients presenting to primary care practitioners with severe chronic diseases. The studies assessed treatment decisions and diagnostic or planning decisions. The former included antibiotic prescribing, drug-dosing of anticoagulants, management of diabetes, and therapy for cardiovascular events and asthma; the latter related to diagnosis and management of asthma, chronic heart disease and hypertension, treatment of diabetes and depression, and diagnostic decisions for patients with respiratory failure. Most treatment decision support systems provided alerts and reminders for prescribing decisions, assisted with drug-dosing, or generated alerts during provider order entry. Diagnostic decisions were supported by EDSS that were designed to provide point-of-care access to evidence-based guidelines, focus attention on specific investigations, or assess the risk of complications of chronic conditions. The included studies assessed a variety of different outcomes, ranging from mortality and length of hospital stay to rates of optimal decision-making

The authors did not state how the papers were selected for the review, or how many reviewers performed the selection.

Assessment of study quality

The authors did not state that they assessed validity.

Data extraction

Measures of clinical performance and patient outcomes were classified as positive or negative according to whether a statistically significant change in patient outcome or targeted clinical practice was reported.

Three reviewers independently evaluated and characterised each study, with any disagreements resolved through consensus.

Methods of synthesis

Clinical decision tasks were classified into diagnosis, treatment or plan; system functionality was classified as consultative or critiquing; and patients were classified as having acute or chronic conditions. The proportion of studies reporting positive findings within each of these categories was calculated.

Results of the review

Twenty-four RCTs (n=75 245) were included.

Thirteen studies showed positive results and eleven showed negative results. Only one of the studies reporting a positive result found an improvement in clinical outcomes; all of the others reported improvements in surrogate outcomes. All 8 systems that targeted clinical decisions relating to acute disease or acute exacerbations of disease showed improved patient outcomes, compared with 5 (38%) of the 16 systems that focused on chronic conditions. Critiquing systems and consultative systems showed an impact in 71% and 47% of studies, respectively.

Authors' conclusions

EDSS improves prescribing practices and treatment outcomes for patients with acute illnesses but are less effective in chronic conditions in primary care.

CRD commentary

The review addressed a broad question with defined inclusion criteria, although these were difficult to follow. The literature search was limited to MEDLINE and DARE, although it is unclear why DARE was searched given that only RCTs were sought. The review was restricted to studies reported in English and no attempts were made to locate unpublished studies. It is therefore likely that relevant studies might have been missed and the review may be subject to language and publication bias. Appropriate steps were taken to minimise bias and error in the extraction of data, but it is unclear whether such steps were also taken for the inclusion assessment. In addition, since study quality was not assessed, the reliability of the included studies is unclear. Some relevant study details were tabulated but further information, in particular in relation to the participants, would have been helpful. The decision not to pool the results appears appropriate given the heterogeneity between the studies. However, the appropriateness of classifying studies as positive or negative and then calculating the proportion of positive findings is questionable. Such a method does not consider the clinical significance of any observed differences and it is unclear how studies that reported both positive and negative findings were classified. The limitations highlighted mean that the authors' conclusions are unlikely to be reliable and should be interpreted with extreme caution.

Implications of the review for practice and research

Practice: The authors did not state any implications for practice.

Research: the authors state that further research is needed to quantify the range of benefits of EDSS ,and to explore new measurement metrics and enhance the appropriate clinical use of electronic decision support.

Funding

National Institute of Clinical Studies, Melbourne, Australia (fellowship).

Bibliographic details

Sintchenko V, Magrabi F, Tipper S. Are we measuring the right end-points? Variables that affect the impact of computerised decision support on patient outcomes: a systematic review Medical Informatics and the Internet in Medicine 2007; 32(3): 225-240. [PubMed: 17701828]

Indexing Status

Subject indexing assigned by NLM

MeSH

Acute Disease /therapy; Chronic Disease /therapy; Decision Support Systems, Clinical /standards; Diagnosis, Computer-Assisted; Endpoint Determination; Humans; Outcome Assessment (Health Care); Randomized Controlled Trials as Topic

AccessionNumber

12008009419

Database entry date

01/12/2008

Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.

CRD has determined that this article meets the DARE scientific quality criteria for a systematic review.

Copyright © 2014 University of York.

PMID: 17701828

Download

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...