Home > DARE Reviews > Computerized clinical decision support...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. York (UK): Centre for Reviews and Dissemination (UK); 1995-.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet].

Computerized clinical decision support systems for acute care management: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes

N Sahota, R Lloyd, A Ramakrishna, JA Mackay, JC Prorok, L Weise-Kelly, T Navarro, NL Wilczynski, and RB Haynes. CCDSS Systematic Review Team.

Review published: 2011.

CRD summary

The review concluded that most computerised clinical decision support systems improved process of care, but patient outcomes were rarely assessed. Systems have not matured sufficiently to enable application in acute care settings. Wide variability in studies and limitations surrounding the determination of study significance and the use of vote counting methods mean the conclusions cannot be considered reliable.

Authors' objectives

To assess the effects of computerised clinical decision support systems on process of care and patients outcomes for acute medical care.

Searching

MEDLINE, EMBASE, Cochrane Database of Systematic Reviews, DARE, ACP Journal Club, Inspec and other unnamed evidence-based medicine review databases were searched between 2004 and January 2010 without language restrictions. Search terms were reported in Haynes 2010 (see Other Publications of Related Interest). Studies in the reference list of the authors’ previous systematic review were reassessed.

Study selection

Randomised controlled trials (RCTs) that compared the effects of patients care with clinical decision support systems versus no clinical decision support systems in acute medical care (defined as episodic health conditions with potential for cure or stabilisation in less than six months) were eligible for inclusion. Eligible studies were required to assess effects in healthcare professionals, provide patient-specific information in terms of management options or probabilities and measure clinical performance (a measure of process of care) or patient outcomes. Studies were excluded if they reported feedback on groups of patients without individual assessment, or only provided computer aided instruction.

Most included studies were conducted in the USA and included over 121 different clinics at over 106 sites. Computerised clinical decision support systems were mainly standalone, but some were integrated with other systems and were categorised as treatment management assistants (alerts and reminders), management systems (guidelines and algorithms), diagnostic assistants and medication dosing assistants. The most common delivery method was computer; a range of other methods were also employed. Over 75% of system users were physicians, the rest represented a range of health-related professionals. Patient outcomes varied considerably. Treatment comparisons included usual care, standard decision-making, questionnaires and various diagnostic assessment tools.

Two reviewers independently screened studies for inclusion, with disagreements resolved through consensus or referral to a third reviewer.

Assessment of study quality

Study quality was assessed using a modified version of the Jadad scale, which measured five sources of bias and awarded a maximum score of 10.

The authors did not state how many reviewers performed the quality assessment.

Data extraction

Two reviewers extracted statistical data on the primary, main or most relevant, or available outcome. A series of decision rules were applied to determine which outcomes were evaluated and how that translated into a judgment of whether the study was considered to have been statistically significantly positive (+) or negative (-), or no effect (0). Where statistical data were not reported, effect was denoted as not evaluated.

Primary authors were contacted for clarification on data, where necessary. Two reviewers independently extracted data; discrepancies were resolved through consensus or referral to a third reviewer.

Methods of synthesis

Meta-analysis was not reported due to significant heterogeneity; data were presented as a narrative synthesis. Data were reported separately for type of treatment categorisation.

Sensitivity analysis was performed to assess the impact of studies with mismatch between the unit of allocation (for example clinician) and the unit of analysis (such as individual patients without adjustment for clustering).

Results of the review

Thirty-six RCTs (37 publications) were included in the review; 3,417 healthcare professionals and 202,491 patients. The mean quality score was 6.4 (range 2 to 10); 21 RCTs reported adequate allocation concealment, 10 RCTs went some way to employ cluster randomisation.

Twenty-two of 35 RCTs reported improvements in process of care outcomes; three of 20 RCTs reported improvements in patient outcomes.

Management assistants – alerts and reminders (11 RCTs): Nine of the 11 RCTs that assessed process of care outcomes reported improvements; none of the four RCTs that assessed patient outcomes reported improvements.

Management assistants – guidelines and algorithms (nine RCTs): Three of eight RCTs that assessed process of care outcomes showed improvements while none of the four RCTs that assessed patient outcomes reported improvements.

Diagnostic assistants (three RCTs): Two of the three RCTs showed improvements in process of care, neither of the two RCTs that assessed patient outcomes showed an improvement.

Medication dosing assistants (14 RCTs): Nine of the 14 RCTs that assessed process of care outcomes demonstrated improvements. Three of 10 RCTs that assessed patient outcomes reported positive effects while one reported negative effects.

Sensitivity analysis did not identify any significant impact from studies with mismatch on process of care outcomes or patient outcomes.

Cost information

Four RCTs reported costs relating to costs of installing computer systems and associated equipment and cost saving benefits (as reported in the review).

Authors' conclusions

Most computerised clinical decision support systems showed improvements in process of care, but patient outcomes were rarely assessed and were much less likely to show benefit. Computerised clinical decision support systems for acute medical care have not matured sufficiently to enable clinical decision makers to embrace the technology for clinical application.

CRD commentary

The review question and supporting inclusion criteria were broadly stated. Several relevant sources were searched for relevant data without language restrictions. Although publication bias was not formally assessed, the authors suggested that bias was likely to exist. Study quality was assessed using appropriate criteria and results were fully reported. It was unclear whether quality assessment was performed in duplicate, which meant that reviewer error and bias could not be ruled out. Methods employed in data extraction meant that in some instances studies could be evaluated as statistically significantly positive even when no pre-specified outcomes were reported. The synthesis used a vote-counting method to evaluate the evidence base which meant that study size and quality were not taken into account in considering the impact of a study on the overall result. The authors acknowledged multiple limitations of the synthesis presented and the included studies which contributed to it.

The authors' conclusions may have reflected the evidence but the wide variability in studies and limitations that surrounded the determination of study significance as well as the use of vote counting methods meant that the conclusions could not be regarded as reliable.

Implications of the review for practice and research

Practice: The authors stated that computerised clinical decision support systems for acute medical care have not matured sufficiently to enable clinical decision makers to embrace the technology for clinical application.

Research: The authors stated that high quality studies were needed to rigorously evaluate the effect of computerised clinical decision support systems on patient outcomes.

Funding

Canadian Institutes of Health Research Synthesis.

Bibliographic details

Sahota N, Lloyd R, Ramakrishna A, Mackay JA, Prorok JC, Weise-Kelly L, Navarro T, Wilczynski NL, Haynes RB, CCDSS Systematic Review Team. Computerized clinical decision support systems for acute care management: A decision-maker-researcher partnership systematic review of effects on process of care and patient outcomes. Implementation Science 2011; 6(91) [PMC free article: PMC3169487] [PubMed: 21824385]

Other publications of related interest

Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Hanes RB. Effects of computerised clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005; 293(10):1223-1238.

Haynes RB, Wilcznski N, the Computerised Clinical Decision Support System (CCDSS) Systematic Review Team. Effects of computerised clinical decision support systems on practitioner performance and patient outcomes: Methods of a decision-maker-researcher partnership systematic review. Implementation Science 2010; 5:12.

Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, Nieuwlaat R, Souza NM, Beyene J, Van Spall HG, Garg AX, Haynes RB. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ 2013; 346: f657.

Indexing Status

Subject indexing assigned by NLM

MeSH

Acute Disease; Algorithms; Biomedical Research; Cooperative Behavior; Decision Making; Decision Support Systems, Clinical; Humans; Monitoring, Physiologic /methods; Patient Care; Treatment Outcome

AccessionNumber

12011006655

Database entry date

11/06/2012

Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.

CRD has determined that this article meets the DARE scientific quality criteria for a systematic review.

Copyright © 2014 University of York.

PMID: 21824385

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...