• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of implemsciBioMed CentralBiomed Central Web Sitesearchsubmit a manuscriptregisterthis articleImplementation Science : ISJournal Front Page
Implement Sci. 2011; 6: 92.
Published online Aug 3, 2011. doi:  10.1186/1748-5908-6-92
PMCID: PMC3170626

Computerized clinical decision support systems for chronic disease management: A decision-maker-researcher partnership systematic review

Abstract

Background

The use of computerized clinical decision support systems (CCDSSs) may improve chronic disease management, which requires recurrent visits to multiple health professionals, ongoing disease and treatment monitoring, and patient behavior modification. The objective of this review was to determine if CCDSSs improve the processes of chronic care (such as diagnosis, treatment, and monitoring of disease) and associated patient outcomes (such as effects on biomarkers and clinical exacerbations).

Methods

We conducted a decision-maker-researcher partnership systematic review. We searched MEDLINE, EMBASE, Ovid's EBM Reviews database, Inspec, and reference lists for potentially eligible articles published up to January 2010. We included randomized controlled trials that compared the use of CCDSSs to usual practice or non-CCDSS controls. Trials were eligible if at least one component of the CCDSS was designed to support chronic disease management. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of relevant outcomes.

Results

Of 55 included trials, 87% (n = 48) measured system impact on the process of care and 52% (n = 25) of those demonstrated statistically significant improvements. Sixty-five percent (36/55) of trials measured impact on, typically, non-major (surrogate) patient outcomes, and 31% (n = 11) of those demonstrated benefits. Factors of interest to decision makers, such as cost, user satisfaction, system interface and feature sets, unique design and deployment characteristics, and effects on user workflow were rarely investigated or reported.

Conclusions

A small majority (just over half) of CCDSSs improved care processes in chronic disease management and some improved patient health. Policy makers, healthcare administrators, and practitioners should be aware that the evidence of CCDSS effectiveness is limited, especially with respect to the small number and size of studies measuring patient outcomes.

Background

Chronic conditions present patients, practitioners, and healthcare systems with some unique demands, including recurrent visits, adherence to complex care plans, long-term disease and treatment monitoring, behavior modification, and patient self-management. For the many patients with multiple co-morbidities [1], overlapping or diverging care plans may further complicate these processes.

Computerized clinical decision support systems (CCDSSs) may help practitioners meet the requirements of chronic care. These systems analyze a patient's characteristics to provide tailored recommendations for diagnosis, treatment, patient education, adequate follow-up, and timely monitoring of disease indicators. For example, Holbrook et al. [2,3] gave providers and diabetic patients access to a web-based system that offered care advice, allowed monitoring of diabetes risk factors, and tracked key care targets. As with any health intervention, however, rigorous testing is warranted to determine whether CCDSSs improve chronic care processes and patient outcomes.

In our previous review of the effects of CCDSSs [4], we analyzed 100 randomized and non-randomized studies published until September 2004, 40 of which assessed the effects of CCDSSs on disease management. Of these 40 studies, 37 measured processes of care of which 62% (23) showed an improvement, and 27 measured patient outcomes of which 19% (5) showed an improvement. The quality of the studies varied widely, but improved over time.

Many new randomized controlled trials (RCTs) have been published in this field since our previous work, potentially documenting important advances. Recognizing that the management of chronic disease has unique characteristics, we wished to review the impact of CCDSSs on the quality and effectiveness of chronic care. We had the opportunity to include the perspectives of senior hospital managers and front-line healthcare practitioners to ensure that relevant data were extracted and summarized--a level of stakeholder engagement that has not been included in other reviews [5-8].

Methods

We previously published the details of our review protocol, openly accessible at http://www.implementationscience.com/content/5/1/12[9]. These methods are briefly summarized here, along with details specific to this review of CCDSSs for chronic disease management.

Research question

Do CCDSSs improve chronic disease management processes or patient outcomes?

Partnering with decision makers

We conducted this review in partnership with individuals responsible for implementing CCDSSs in our region [9]. Decision makers, both managers and clinicians, met with the review team periodically to discuss direction and specific details for the data extraction, analysis, presentation and interpretation of results.

Search strategy

Full details of our search strategy are in our review protocol [9]. In summary, we searched MEDLINE, EMBASE, Ovid's Evidence-Based Medicine Reviews, and Inspec until 6 January 2010, and reviewed the reference lists of included RCTs and relevant systematic reviews. We screened articles for eligibility in two stages: a duplicate, independent review of titles and abstracts followed by a duplicate, independent, full-text review of potentially eligible articles, with a third reviewer resolving disagreements.

Study selection

We selected RCTs of a CCDSS used by a health care provider for management of chronic conditions, published up to 6 January 2010 in any language that measured CCDSS impact on processes of care or patient outcomes. We included RCTs in any language that compared patient care with a CCDSS to routine care without a CCDSS and evaluated clinical performance (i.e., a measure of process of care) or a patient outcome. Additionally, to be included in the review, the CCDSS had to provide patient-specific advice that was reviewed by a healthcare practitioner before any clinical action. Studies were excluded if the system was used solely by students, only provided summaries of patient information, provided feedback on groups of patients without individual assessment, only provided computer-aided instruction, or was used for image analysis. Trials included in our previous review [4] were included if they were eligible. Trials of CCDSSs for managing narrow therapeutic index medications used in some chronic conditions (such as warfarin in atrial fibrillation [10]) were not included in this review, but are discussed in our review for therapeutic drug monitoring and dosing.

Data extraction

To meet the needs of our management and clinical partners, we extracted study characteristics (e.g., study design, size, setting, authorship, funding, and year of publication) and system characteristics (e.g., integration with other systems, user interface elements, methods of data entry and delivery of recommendations, target users, and implementation details such as pilot testing and user training). Disagreements were resolved by a third reviewer or by consensus. We contacted primary authors to provide missing data and to assess the accuracy of the extracted data; 78% (43/55) provided input. For the remaining trials, a trained reviewer assessed the extraction form against the full-text to confirm accuracy.

Assessment of study quality

Using a 10-point scale, pairs of reviewers independently evaluated the selected trials on five dimensions of quality, including concealment of allocation, appropriate unit of allocation, appropriate adjustment for baseline differences, appropriate blinding of assessment, and adequate follow-up [9]. We used a 2-tailed Mann-Whitney U test to compare methodologic scores between trials published before the year 2000 and those published later to determine if trial quality has improved with time.

Assessment of CCDSS intervention effects

We assessed the effectiveness of CCDSSs in each trial for improving process of care and patient outcomes. We defined process outcomes as changes in care activities such as diagnosis, treatment, and monitoring of disease. Examples of patient outcomes included changes in blood pressure, clinical events and health-related quality of life. We judged a CCDSS effective if it produced a statistically significant (p < 0.05) improvement in a primary chronic disease outcome or in ≥50% of multiple relevant pre-specified outcomes. We considered primary any outcome that trial reports described as 'primary' or 'main.' If authors did not designate a primary outcome, we considered the outcome used to calculate the trial's sample size to be primary, if reported. When there were no pre-specified outcomes, the system was considered effective if it produced an improvement in ≥50% of all reported chronic disease outcomes. Our assessment criteria are more specific than those used in our 2005 review [4]; therefore, the assignment of effect was adjusted for some trials included in the review.

Data synthesis and analysis

We summarized data using proportions, medians, and ranges. Denominators vary in some proportions because not all trials reported relevant information. All analyses were carried out using SPSS, version 15.0. We did not attempt a meta-analysis because of study-level differences in participants, clinical settings, disease conditions, interventions, and outcomes measured.

We conducted a sensitivity analysis to assess the possibility of biased results in studies with a mismatch between the unit of allocation (e.g., clinicians) and the unit of analysis (e.g., individual patients without adjustment for clustering). We compared success rates between studies with matched and mismatched analyses using chi-square for comparisons. No differences in reported success were found for either process of care outcomes (Pearson X2 = 1.41, p = 0.24) or patient outcomes (Pearson X2 = 1.45, p = 0.23). Accordingly, results have been reported without distinction for mismatch.

Results

Figure Figure11 shows a flow diagram of included and excluded trials. We identified 166 trials of CCDSSs and Cohen's κ for reviewer agreement on trial eligibility was 0.93 (95% confidence interval [CI], 0.91 to 0.94). In this review, we included 71 publications describing 55 trials (33% of total) about management of chronic diseases [2,3,11-79]. Thirty-eight included studies contributed outcomes to both this review and other CCDSS interventions in the series; three studies [30,53,62] to four reviews, 12 studies [21,25,28,31-33,42-44,51,52,54,55,57-61,74] to three reviews, and 23 studies [2,3,11,12,18,19,23,27,35-38,40,41,45,46,48-50,56,67,71-73,77,79] to two reviews; but we focused here on outcomes relevant to the management of chronic disease.

Figure 1
Flow diagram of included and excluded studies for the update 1 January 2004 to 6 January 2010 with specifics for chronic disease management*. * Details provided in: Haynes RB et al. [9] Two updating searches were performed, for 2004 to 2009 and to 6 January ...

Summary of trial quality is reported in Additional file 1, Table S1; system characteristics in Additional file 2, Table S2; study characteristics in Additional file 3, Table S3; outcome data in Table Table11 and Additional file 4, Table S4; and other CCDSS-related outcomes in Additional file 5, Table S5.

Table 1
Results for CCDSS trials of chronic disease management

Study quality

Additional file 1, Table S1 presents details of our methodological quality assessment. Of the 55 trials, 53% reported adequate concealment of allocation [2,3,13,18,20,27,29,31-34,37,39,40,47-58,60-67,72-75]; 78% showed no differences in baseline characteristics between study groups or adjusted accordingly [2,3,11-13,18-21,23-25,28,29,34,36,38-58,60-67,70-72,74-76,78,79]; 53% allocated entire wards or practices to each study group [11,12,14-18,25,28-35,37,39,46-49,51,54-59,62-64,67,70,73,76-79]; all except one used objective outcomes or blinding of outcome assessments [23]; and 60% achieved a ≥90% follow-up rate for their unit of analysis [11-13,18-24,27,30,35,36,39,40,46,47,50-55,59-62,65-70,73,74,76,77]. The overall quality of trials was good (median methods score, 8; ranging from 2 to 10) and improved with time (median methods score before versus after year 2000, 7 versus 8, 2-tailed Mann-Whitney U = 137; p = 0.005), possibly because early trials often failed to conceal allocation or to achieve adequate follow-up.

CCDSS and study characteristics

Additional file 2, Table S2 describes CCDSS design and implementation characteristics. Denominators vary because not all trials reported on all features considered. Fifty-nine percent (32/54) of CCDSSs were integrated with electronic medical records [2,3,14-18,21-23,25,26,28,29,31-40,42-44,46,48,49,51,53-55,60-64,67,73,74,76,79], and 17% (8/47) were also integrated with computerized physician order entry systems [22,29,36,38,46,48,49,53,60,61]. Fifty-three percent (25/47) automatically obtained data needed to give recommendations from electronic medical records [2,3,18,21-23,25,28,34-36,38,40,46,48,49,51,53-55,60-64,67,73,74,76,79]; 36% (17/47) relied on practitioners to enter the data [2,3,14-17,23,30,39,41,45,48,49,52-58,67-69,72,75]; and 26% (12/47) used research staff for this purpose [18,24,36,41,47,50,59,65,66,72,75,77,79]. Advice was provided at the time of care in 85% of trials (46/54) [2,3,13-18,20-28,30-36,38-40,42-49,51-59,62-70,73-79] most often on a desktop or laptop computer (51%; 26/51) [2,3,14-17,21,22,28,30,34-39,46-49,51,53-56,62-64,67,70,72-74] or by existing non-prescribing staff (22%; 11/51) [18,23-26,30,40,42-44,71,76,79]. Fifty-three percent (29/54) provided advice to other healthcare practitioners in addition to physicians [2,3,11,12,14-18,22,23,25,26,28-36,38,40,45-47,53,57-59,63,64,67,71,73,76,77,79] and 15% (8/55) directly advised patients in addition to practitioners [2,3,11-13,18,19,29,41,74]. Sixty-four percent (25/39) of systems were pilot tested [11-19,22,23,26,28,31-34,36,37,39,46,47,50,51,56,59-61,63,64,67,72,77] and healthcare professionals were trained to use them in 72% (34/47) [2,3,11-17,19-22,25,29-33,35,36,38,39,46-61,63,64,67-69,73,77,78]. Reports rarely described the CCDSS user interface characteristics.

Seventy-three percent of trials (40/55) declared that at least one author was involved in the development of the system [2,3,11-13,18,19,22-26,28,30,34,36,38-51,53-61,63,64,67-70,72,73,76,77,79] and three trials indicated that all authors were independent of development [14-17,31-33,78].

Additional file 3, Table S3 provides further details of the CCDSS intervention, care setting, study funding source, and year of publication. Trials included a total of 7,335 practitioners (median, 72; ranging from 5 to 1,378 [when reported]) caring for 381,562 patients (median, 719; ranging from 27 to 156,772 [when reported]) in 974 clinics (median, 13; ranging from 1 to 112 [when reported]) across 705 distinct sites (median, 4; ranging from 1 to 112 [when reported]). Eight trials did not report their source of funding [21,26,36,40,71-73,75]. Of the remaining 47, 74% (n = 35) were publicly funded, 17% (n = 8) were conducted with only private funds, [14-17,19,27,48,49,52,60-62,70], and 9% (n = 4) were conducted with a combination of private and public funding [20,29,54,55,75]. The earliest trial was published in 1977 [45], but over one-half (62%) were published after our previous search in September 2004 [2,3,11-20,27-29,34-37,46-53,57-66,68-75].

CCDSS effects

Table Table11 summarizes the effects of all systems for improving process of care and patient outcomes and Additional file 4, Table S4 provides further detail regarding systems and individual outcomes selected for evaluation.

Eighty-seven percent (48/55) of trials measured effects on chronic disease management processes [2,3,11,12,14-19,21-33,35-44,46-69,71,73-76,78,79], and 52% (25/48) demonstrated improvement [2,3,11,12,18,19,21,23,27-30,35,36,40,42-44,50,57-61,63,64,68,69,71,73,74,76,78,79]. Sixty-five percent (36/55) measured impact on patient outcomes [2,3,11-20,22,29,31-39,41-45,50,52-56,59-62,65-67,70,72,74,75,77-79] and 31% (11/36) of these demonstrated benefit on measures such as health-related quality of life, rates of hospitalization, unscheduled care visits, and a host of disease-specific clinical outcomes [2,3,13,18-20,42-44,52,56,59,72,78].

Diabetes

Thirteen trials described systems primarily supporting diabetes care (median quality score, 7; ranging from 2 to 10) [2,3,11-26]. Fifty-five percent (6/11) reported improvements in processes of care including treatment and monitoring [2,3,11,12,18,19,21,23], while 62.5% (5/8) reported improvements in corresponding patient outcomes including blood pressure, HbA1c, and low-density lipoprotein (LDL) cholesterol [2,3,13,18-20]. The seven trials published since 2005 appeared to show success more consistently: four of five improved the process of care [2,3,11-13,18-20], and five of seven improved patient outcomes [2,3,13,18-20].

Systems in five diabetes trials targeted patients in addition to practitioners [2,3,11-13,18,19]. Of these, all four trials that measured process effects demonstrated benefit [2,3,11,12,18,19], and four reported improvement in patient outcomes [2,3,13,18,19].

Several recent trials were conducted in primary community clinics whereas most previous trials were conducted in hospitals. For example, in two trials conducted across multiple practices, CCDSSs provided patient-specific reminders during visits and notified at-risk patients of their care targets and upcoming appointments [2,3,11,18]. Both trials demonstrated improvements in composite process measures comprising timely completion of foot and eye exams, and monitoring of blood pressure, HbA1c, lipoproteins, and renal function. Both trials also showed improvements in corresponding composite patient outcomes.

Diabetes and other conditions

CCDSSs in five trials (median score, 7; ranging from 6 to 8) provided recommendations for a host of conditions in conjunction with diabetes, including dyslipidemia, hypertension, obesity, and heart failure [27-33]. Their effects on diabetes outcomes could not be isolated. All five measured process of care, and 80% (4/5) found improvements [27-30]. Only one measured corresponding patient outcomes, but showed no benefit [31-33].

Hypertension

The 10 trials focusing primarily on hypertension management (median score, 7; ranging from 4 to 10) were older, with 70% (7/10) published before 2005 [34-45].

Eight of 10 trials assessed impact on process of care using measures such as adherence to recommendations for blood pressure control [35-44], patient satisfaction, and number of scheduled care visits, and four demonstrated improvements [35,36,40,42-44].

In contrast to diabetes systems, however, hypertension systems showed little or no patient benefit. Of the nine trials that reported patient outcomes, such as blood pressure and health-related quality of life [34-39,41-45], only one found benefit [42-44]. This multi-component system improved patients' perceived health status by giving suggestions for the management of hypertension, obesity, and renal disease. The trial, however, was of poor quality (methods score 4), and the nature of the intervention prevented isolating effects related to hypertension.

Dyslipidemia

Four trials evaluated systems that focused primarily on dyslipidemia [57-62]. All were conducted in primary care settings and published after 2005 (median quality score, 8.5; ranging from 7 to 10).

Three trials measured effects on process of care and demonstrated improvements in lipid monitoring and treatment [57-61], but only one of three trials measuring patient outcomes found a benefit [59]. This CCDSS generated patient-specific reminders that were mailed to primary care physicians and nurses; highlighted the patient's risk factors, lipoprotein values, and current medications; and recommended initiation or adjustment of lipid-lowering treatment when appropriate. The trial detected improvements in blood lipid monitoring and treatment management, as well as relative reductions in patients' LDL cholesterol.

Asthma and chronic obstructive pulmonary disease (COPD)

The nine trials of systems supporting asthma care were of excellent quality (median score, 9; ranging from 8 to 10) and relatively new (7/9 published after September 2004), but the systems were generally ineffective [46-56]. All trials measured effects on process of care (including rates of spirometry, thorax radiography, IgE levels, and allergy testing; medication prescriptions and influenza vaccinations; and use of rescue medications) but only one demonstrated benefit [50].

Two of five trials measuring patient outcomes found an impact [52,56]. One system delivered asthma recommendations in primary care, made prognostic predictions by matching patients to similar known cases, and allowed users to print self-management plans for their patients [56]. The trial demonstrated a reduction in acute asthma exacerbations and patient-initiated primary care visits. Another system delivered guideline recommendations to general practitioners and pneumologists, and proved to be more cost effective at improving quality of life than usual asthma care [52].

Three asthma systems also gave advice for management of COPD [48,49,51,53]. All of these measured process of care but detected no effects. One trial also measured patient outcomes but did not show benefit [53].

Cardiac care

Systems in four methodologically strong trials (median score, 9.5; ranging from 8 to 10) focused on heart failure [65,66], cardiac rehabilitation [63,64], ischemic heart disease [67], and angina [54,55]. All measured process of care using adherence to guideline recommendations but only one found benefit [63,64]. The CCDSS for cardiac rehabilitation used electronic medical records and needs assessment data to generate recommendations for exercise training, education, lifestyle change, and stress management [63,64]. The trial demonstrated improved guideline adherence, but patient outcomes were not studied. The other three trials measured effects on quality of life as a patient outcome, but none found benefit [54,55,65-67].

Other care

We did not group the remaining 12 trials due to their diverse primary indications. They focused on urinary incontinence [78], cancer [75], osteoporosis [74], renal disease [70], functional deficits [77], obesity [68,69], dementia [73], rheumatoid arthritis [72], advance directives [76], and various non-specific indications [71,79]. Most trials found improvements in care process but only two demonstrated benefit to patients: one reduced urinary incontinence in nursing home patients [78], and the other improved likelihood of remission in patients with early rheumatoid arthritis through CCDSS-guided management of methotrexate [72].

Costs and practical process related outcomes

Four trials used cost-effectiveness as an outcome (see Additional file 4, Table S4) [14-17,52,65,66,75]. Only one trial demonstrated improvement to patient outcomes overall, and the CCDSS was also more cost-effective than usual asthma care [52].

Additional file 5, Table S5 summarizes cost-related findings of the 12 trials that statistically compared costs of care between the study groups: six reported no difference with CCDSS compared to usual care [14-17,29,38,52,67,75], four reported savings with the CCDSS [11,12,50,62,71], and two reported that the CCDSS increased some costs [65-67].

In addition to process of care and patient outcomes, we looked for effects on user satisfaction and workflow (see Additional file 5, Table S5). Only seven trials reported a formal effort of assessing user satisfaction: 3 found users satisfied overall [19,28,63,64], one found them unsatisfied [54,55], and the remaining three showed mixed results [2,3,31-33,56]. The authors of five other studies commented that users were satisfied in informal evaluations [13,18,36,59,72].

Two trials made formal attempts to measure systems' impact on user workflow and reported mixed results [23,63,64].

Discussion

This review was done in partnership with key decision makers to summarize the effectiveness of clinical decision support technology for the management of chronic conditions. We considered studies 'positive' if they showed a statistically significant improvement in at least 50% of relevant outcomes. CCDSSs often improved the process of patient care. When assessed, effects on any patient outcomes were rarely found, but may have been underestimated: 56% of trials reporting these outcomes declared them primary [11-20,22,29,34,35,38,50,52,56,59-62,67,70,72], and the remaining trials may not have been large enough or long enough to detect such outcomes. No study showed convincing evidence of benefit for major patient outcomes.

Nevertheless, results from recent diabetes management trials are encouraging. Several of these systems were deployed in general community practice and those that engaged both patients and providers were consistently effective. These systems may become increasingly popular with the advent of patient-controlled electronic medical records. Systems addressing several conditions, including but not limited to diabetes, generally improved care but only one measured patient outcomes [31-33] (no effect). In dyslipidemia, systems improved lipid monitoring and treatment, but only one reduced blood lipids [59]. The few dyslipidemia trials were recent and may represent a promising area for future research.

Conversely, most trials in hypertension measured patient outcomes and almost never found benefits, and only some showed improvements in the process of care. Asthma and COPD systems mostly failed to show effectiveness, despite being tested in recent, high-quality trials. The small collection of trials in heart failure, ischemic heart disease, cardiac rehabilitation, and angina also rarely show effects, with improvement only in rehabilitation processes. The remaining systems, too diverse to group, often improved care processes but were seldom found to benefit patients.

While systems in diabetes appear to achieve success with respect to patient outcomes more often than systems in asthma and hypertension, we did not pre-specify this comparison and, given the play of chance and many possible confounders, we cannot confidently assert that the pattern is real. It is plausible that the effectiveness of CCDSS recommendations at improving patient outcomes for some indications is limited by the absence of high-quality clinical evidence in that area. Even the most scientifically sound recommendations, however, will fail to improve health outcomes if patients do not adhere to prescribed treatments--a very common problem [80]. Unfortunately, our suggestions regarding the discrepancy remain purely speculative because studies did not explore reasons for failure, and we do not have enough trials to test these hypotheses reliably.

The growing use of CCDSSs and their potential for benefit and harm highlight the importance of evaluating these systems in well-conducted randomized clinical trials. The increase in number and quality of trials is encouraging, but results remain mixed, and few trials investigated the mechanisms behind their findings. Careful description of study and system design in trial reports, as well as assessments of effectiveness and acceptability of system features, would support progress in this area.

CCDSSs may represent a cost-effective way of improving chronic disease outcomes. However, the economic effects of systems are not readily assessed based on available data. The costs of design, local implementation, ongoing maintenance, and user support can be high, and may be further elevated by the unique nature of chronic care. This warrants cost-effectiveness analyses, but only four trials [14-17,52,65,66,75] reported such data and little cost data of any kind are available across studies. If cost savings exist, however, current results suggest that they are modest.

The benefits we can expect from the use of computerized decision support are not clear. Policy makers promoting the use of CCDSS, as well as healthcare administrators and practitioners considering local implementation, should be aware that the evidence of CCDSS effectiveness is limited, especially with respect to the small number and size of studies of patient outcomes. Further, evidence of benefit comes mainly from a few 'trail blazer' institutions with much in-house informatics expertise, evaluating home-grown systems developed over many years. As a result, trials in this review may not represent the effects in less technically endowed settings or from commercially available systems, the capabilities of which have been shown to vary greatly [81].

Our review has some potential limitations. Great heterogeneity in CCDSS design, purpose, and targets for evaluation prevented us from conducting a meta-analysis. Instead, we used a binary measure of effect, where we considered studies 'positive' if they showed a statistically significant improvement in at least 50% of relevant outcomes. Thus, some of the studies we considered to show no effect found improvement on a minority of secondary or non-prespecified outcomes. These findings could be real but could also be due to post hoc unplanned analyses and multiple testing. Readers should refer to the Methods section for a more detailed account of our effect assessment.

We were unable to assess the risk of publication bias in this literature. Given that most systems were studied by their own developers, we suspect that publication bias is likely, and even our findings of modest effects may overestimate the true likelihood of seeing benefit from CCDSSs.

Our method of summarizing the evidence by vote counting inflates the risk of Type 2 error [82] and should generally be approached with caution. However, our results remain essentially unchanged from our 2005 review [4] and are comparable to another major review conducted by Kawamoto and colleagues [83], and a recent 'umbrella ' review of high-quality systematic reviews of CCDSSs in hospital settings [84]. Another recent review of reminder systems [5] (a subset of CCDSS) summarized evidence by effect size meta-analysis and qualified the impact of these interventions as falling below the thresholds of clinical importance. Given the similar conclusions of these other systematic reviews and the risk of publication bias in the CCDSS literature, we have little reason to believe that our methods underestimate the benefit from these systems.

Finally, we observed improvements in the quality of trials over time but this trend may have resulted from better reporting in more recent studies.

Conclusions

CCDSSs can improve chronic disease management processes and, in some cases, patient outcomes. Recent trials in diabetes care show the most promising results. The mechanisms behind systems' success or failure remain understudied. Future trials with clear descriptions of system design, local context, implementation strategy, costs, adverse outcomes, user satisfaction, and impact on user workflow will better inform CCDSS development and decisions about local implementation.

Competing interests

RBH, PSR, SM, HCG, AXG, RJS, JAM, NMS, LWK, and TN received support through the Canadian Institutes of Health Research Synthesis Grant: Knowledge Translation KRS 91791 for the submitted work. PSR was also supported by an Ontario Graduate Scholarship, a Canadian Institutes of Health Research Strategic Training Fellowship, and a Canadian Institutes of Health Research 'Banting and Best' Master's Scholarship. Additionally, PSR is a co-applicant for a patent concerning computerized decision support for anticoagulation, which was not discussed in this review, and has recently received awards from organizations that may benefit from the notion that information technology improves health care, including COACH (Canadian Organization for Advancement of Computers in Healthcare), the National Institutes of Health Informatics, and Agfa HealthCare Corp. RJS is the owner of Fig.P Software Incorporated, which develops and sells a chronic disease management system that is not a subject of this review. HCG has/had financial relationships with the following organisations in the previous three years: Sanofi Aventis, GlaxoSmithKline, Eli Lilly, Novo Nordisk, Astra Zeneca, BMS, Roche, Bayer, Janssen Ortho, Solvay, BI, Servier. RBH is acquainted with several CCDSS developers and researchers, including authors of papers included in this review.

Authors' contributions

RBH was responsible for study conception and design; acquisition, analysis, and interpretation of data; drafting and critical revision of the manuscript; obtaining funding; study supervision. He is the guarantor. PSR acquired, analyzed, and interpreted data; drafted and critically revised the manuscript; and conducted statistical analysis. SM acquired data; drafted and critically revised the manuscript. HCG analyzed and interpreted data; and critically revised the manuscript. AXG acquired, analyzed, and interpreted data; and critically revised the manuscript. RJS analyzed and interpreted the data. JAM acquired, analyzed, and interpreted data; and critically revised the manuscript. LWK and TN acquired data and drafted the manuscript. NLW acquired, analyzed, and interpreted data; provided administrative, technical, or material support; and provided study supervision. All authors read and approved the final manuscript.

Supplementary Material

Additional file 1:

Table S1. Study methods scores for trials of chronic disease management. Methods scores for the included studies.

Additional file 2:

Table S2. CCDSS characteristics for trials of chronic disease management. CCDSS characteristics of the included studies.

Additional file 3:

Table S3. Study characteristics for trials of chronic disease management. Study characteristics of the included studies.

Additional file 4:

Table S4. Results for CCDSS trials of chronic disease management. Details results of the included studies.

Additional file 5:

Table S5. Costs and CCDSS process-related outcomes for trials of chronic disease management. Cost and CCDSS process-related outcomes for the included studies.

Acknowledgements

The research was funded by a Canadian Institutes of Health Research Synthesis Grant: Knowledge Translation KRS 91791. The members of the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team included the Principal Investigator, Co-Investigators, Co-Applicants/Senior Management Decision-makers, Co-Applicants/Clinical Service Decision-Makers, and Research Staff. The following were involved in collection and/or organization of data: Jeanette Prorok, MSc, McMaster University; Nathan Souza, MD, MMEd, McMaster University; Brian Hemens, BScPhm, MSc, McMaster University; Robby Nieuwlaat, PhD, McMaster University; Shikha Misra, BHSc, McMaster University; Jasmine Dhaliwal, BHSc, McMaster University; Navdeep Sahota, BHSc, University of Saskatchewan; Anita Ramakrishna, BHSc, McMaster University; Pavel Roshanov, BSc, McMaster University; Tahany Awad, MD, McMaster University. Nicholas Hobson, DiplT, Chris Cotoi, BEng, EMBA, and Rick Parrish, DiplT, at McMaster University provided programming and information technology support.

References

  • Fortin M, Bravo G, Hudon C, Vanasse A, Lapointe L. Prevalence of multimorbidity among adults seen in family practice. Ann Fam Med. 2005;3(3):223–228. doi: 10.1370/afm.272. [PMC free article] [PubMed] [Cross Ref]
  • Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H. Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ. 2009;181(1-2):37–44. doi: 10.1503/cmaj.081272. [PMC free article] [PubMed] [Cross Ref]
  • Holbrook A, Keshavjee K, Lee H, Bernstein B, Chan D, Thabane L, Gerstein H, Troyan S, COMPETE II Investigators. Individualized electronic decision support and reminders can improve diabetes care in the community. AMIA Annu Symp Proc. 2005. p. 982. [PMC free article] [PubMed]
  • Garg AX, Adhikari NK, McDonald H, Rosas-Arellano M, Devereaux PJ, Beyene J, Sam J, Haynes RB. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA. 2005;293(10):1223–1238. doi: 10.1001/jama.293.10.1223. [PubMed] [Cross Ref]
  • Shojania KG, Jennings A, Mayhew A, Ramsay C, Eccles M, Grimshaw J. Effect of point-of-care computer reminders on physician behaviour: a systematic review. CMAJ. 2010;182(5):E216–E225. doi: 10.1503/cmaj.090578. [PMC free article] [PubMed] [Cross Ref]
  • Pearson SA, Moxey A, Robertson J, Hains I, Williamson M, Reeve J, Newby D. Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007) BMC Health Serv Res. 2009;9:154. doi: 10.1186/1472-6963-9-154. [PMC free article] [PubMed] [Cross Ref]
  • Durieux P, Trinquart L, Colombet I, Nies J, Walton R, Rajeswaran A, Rege Walther M, Harvey E, Burnand B. Computerized advice on drug dosage to improve prescribing practice. Cochrane Database Syst Rev. 2008. p. CD002894. [PubMed]
  • Berner ES. Clinical decision support systems: State of the art. AHRQ Publication No 09-0069-EF. Rockville, Maryland: Agency for Healthcare Research and Quality; 2009.
  • Haynes RB, Wilczynski NL. the Computerized Clinical Decision Support System (CCDSS) Systematic Review Team. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: methods of a decision-maker-research partnership systematic review. Implement Sci. 2010;5:12. [PMC free article] [PubMed]
  • Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, Kent DL, White RH. A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin. National Consortium of Anticoagulation Clinics. J Gen Intern Med. 1994;9(3):131–139. doi: 10.1007/BF02600026. [PubMed] [Cross Ref]
  • MacLean CD, Gagnon M, Callas P, Littenberg B. The Vermont Diabetes Information System: a cluster randomized trial of a population based decision support system. J Gen Intern Med. 2009;24(12):1303–1310. doi: 10.1007/s11606-009-1147-x. [PMC free article] [PubMed] [Cross Ref]
  • MacLean CD, Littenberg B, Gagnon M. Diabetes decision support: initial experience with the Vermont diabetes information system. Am J Public Health. 2006;96(4):593–595. doi: 10.2105/AJPH.2005.065391. [PMC free article] [PubMed] [Cross Ref]
  • Christian JG, Bessesen DH, Byers TE, Christian KK, Goldstein MG, Bock BC. Clinic-based support to help overweight patients with type 2 diabetes increase physical activity and lose weight. Arch Intern Med. 2008;168(2):141–146. doi: 10.1001/archinternmed.2007.13. [PubMed] [Cross Ref]
  • Cleveringa FG, Gorter KJ, van den Donk M, Rutten GE. Combined task delegation, computerized decision support, and feedback improve cardiovascular risk for type 2 diabetic patients: a cluster randomized trial in primary care. Diabetes Care. 2008;31(12):2273–2275. doi: 10.2337/dc08-0312. [PMC free article] [PubMed] [Cross Ref]
  • Cleveringa FGW, Gorter KJ, van den Donk M, Pijman PLW, Rutten GEHM. Task delegation and computerized decision support reduce coronary heart disease risk factors in type 2 diabetes patients in primary care. Diabetes Technol Ther. 2007;9(5):473–481. doi: 10.1089/dia.2007.0210. [PubMed] [Cross Ref]
  • Cleveringa FG, Welsing PM, van den Donk M, Gorter KJ, Niessen LW, Rutten GE, Redekop WK. Cost-effectiveness of the diabetes care protocol, a multifaceted computerized decision support diabetes management intervention that reduces cardiovascular risk. Diabetes Care. 2010;33(2):258–263. doi: 10.2337/dc09-1232. [PMC free article] [PubMed] [Cross Ref]
  • Cleveringa FGW, Minkman MH, Gorter KJ, van den Donk M, Rutten G. Diabetes Care Protocol: effects on patient-important outcomes. A cluster randomised non-inferiority trial in primary care. Diabetes Med. 2010;27(4):442–450. doi: 10.1111/j.1464-5491.2010.02968.x. [PubMed] [Cross Ref]
  • Peterson KA, Radosevich DM, O'Connor PJ, Nyman JA, Prineas RJ, Smith SA, Arneson TJ, Corbett VA, Weinhandl JC, Lange CJ, Hannan PJ. Improving diabetes care in practice: findings from the TRANSLATE trial. Diabetes Care. 2008;31(12):2238–2243. doi: 10.2337/dc08-2034. [PMC free article] [PubMed] [Cross Ref]
  • Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, Gruber-Baldini A. WellDoc™ mobile diabetes management randomized controlled trial: change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther. 2008;10(3):160–168. doi: 10.1089/dia.2008.0283. [PubMed] [Cross Ref]
  • Augstein P, Vogt L, Kohnert KD, Freyse EJ, Heinke P, Salzsieder E. Outpatient assessment of Karlsburg Diabetes Management System-based decision support. Diabetes Care. 2007;30(7):1704–1708. doi: 10.2337/dc06-2167. [PubMed] [Cross Ref]
  • Filippi A, Sabatini A, Badioli L, Samani F, Mazzaglia G, Catapano A, Cricelli C. Effects of an automated electronic reminder in changing the antiplatelet drug-prescribing behavior among Italian general practitioners in diabetic patients: an intervention trial. Diabetes Care. 2003;26(5):1497–1500. doi: 10.2337/diacare.26.5.1497. [PubMed] [Cross Ref]
  • Meigs JB, Cagliero E, Dubey A, Murphy-Sheehy P, Gildesgame C, Chueh H, Barry MJ, Singer DE, Nathan DM. A controlled trial of web-based diabetes disease management: the MGH diabetes primary care improvement project. Diabetes Care. 2003;26(3):750–757. doi: 10.2337/diacare.26.3.750. [PubMed] [Cross Ref]
  • Lobach DF, Hammond W. Computerized decision support based on a clinical practice guideline improves compliance with care standards. Am J Med. 1997;102(1):89–98. doi: 10.1016/S0002-9343(96)00382-8. [PubMed] [Cross Ref]
  • Nilasena DS, Lincoln MJ. A computer-generated reminder system improves physician compliance with diabetes preventive care guidelines. Proc Annu Symp Comput Appl Med Care. 1995. pp. 640–645. [PMC free article] [PubMed]
  • Mazzuca SA, Vinicor F, Einterz RM, Tierney WM, Norton JA, Kalasinski LA. Effects of the clinical environment on physicians' response to postgraduate medical education. Am Educ Res J. 1990;27(3):473–488.
  • Thomas JC, Moore A, Qualls PE. The effect on cost of medical care for patients treated with an automated clinical audit system. J Med Syst. 1983;7(3):307–313. doi: 10.1007/BF00993294. [PubMed] [Cross Ref]
  • Derose SF, Dudl JR, Benson VM, Contreras R, Nakahiro RK, Ziel FH. Point of service reminders for prescribing cardiovascular medications. Am J Manag Care. 2005;11(5):298–304. [PubMed]
  • Sequist TD, Gandhi TK, Karson AS, Fiskio JM, Bugbee D, Sperling M, Cook EF, Orav EJ, Fairchild DG, Bates DW. A randomized trial of electronic clinical reminders to improve quality of care for diabetes and coronary artery disease. J Am Med Inform Assoc. 2005;12(4):431–437. doi: 10.1197/jamia.M1788. [PMC free article] [PubMed] [Cross Ref]
  • Martin DC, Berger ML, Anstatt DT, Wofford J, Warfel D, Turpin RS, Cannuscio CC, Teutsch SM, Mansheim BJ. A randomized controlled open trial of population-based disease and case management in a Medicare Plus Choice health maintenance organization. Prev Chronic Dis. 2004;1(4):A05. [PMC free article] [PubMed]
  • Demakis JG, Beauchamp C, Cull WL, Denwood R, Eisen SA, Lofgren R, Nichol K, Woolliscroft J, Henderson WG. Improving residents' compliance with standards of ambulatory care: results from the VA Cooperative Study on Computerized Reminders. JAMA. 2000;284(11):1411–1416. doi: 10.1001/jama.284.11.1411. [PubMed] [Cross Ref]
  • Hetlevik I, Holmen J, Krüger O. Implementing clinical guidelines in the treatment of hypertension in general practice. Evaluation of patient outcome related to implementation of a computer-based clinical decision support system. Scand J Prim Health Care. 1999;17(1):35–40. doi: 10.1080/028134399750002872. [PubMed] [Cross Ref]
  • Hetlevik I, Holmen J, Kruger O, Kristensen P, Iversen H. Implementing clinical guidelines in the treatment of hypertension in general practice. Blood Press. 1998;7(5-6):270–276. doi: 10.1080/080370598437114. [PubMed] [Cross Ref]
  • Hetlevik I, Holmen J, Krüger O, Kristensen P, Iversen H, Furuseth K. Implementing clinical guidelines in the treatment of diabetes mellitus in general practice. Evaluation of effort, process, and patient outcome related to implementation of a computer-based decision support system. Int J Technol Assess. 2000;16(1):210–227. doi: 10.1017/S0266462300161185. [PubMed] [Cross Ref]
  • Bosworth HB, Olsen MK, Dudley T, Orr M, Goldstein MK, Datta SK, McCant F, Gentry P, Simel DL, Oddone EZ. Patient education and provider decision support to control blood pressure in primary care: a cluster randomized trial. Am Heart J. 2009;157(3):450–456. doi: 10.1016/j.ahj.2008.11.003. [PubMed] [Cross Ref]
  • Hicks LS, Sequist TD, Ayanian JZ, Shaykevich S, Fairchild DG, Orav EJ, Bates DW. Impact of computerized decision support on blood pressure management and control: a randomized controlled trial. J Gen Intern Med. 2008;23(4):429–441. doi: 10.1007/s11606-007-0403-1. [PMC free article] [PubMed] [Cross Ref]
  • Borbolla D, Giunta D, Figar S, Soriano M, Dawidowski A, de Quiros FG. Effectiveness of a chronic disease surveillance systems for blood pressure monitoring. Stud Health Technol Inform. 2007;129(Pt 1):223–227. [PubMed]
  • Mitchell E, Sullivan F, Watt G, Grimshaw JM, Donnan PT. Using electronic patient records to inform strategic decision making in primary care. Stud Health Technol Inform. 2004;107(Pt2):1157–1161. [PubMed]
  • Murray MD, Harris LE, Overhage JM, Zhou XH, Eckert GJ, Smith FE, Buchanan NN, Wolinsky FD, McDonald CJ, Tierney WM. Failure of computerized treatment suggestions to improve health outcomes of outpatients with uncomplicated hypertension: results of a randomized controlled trial. Pharmacotherapy. 2004;24(3):324–337. doi: 10.1592/phco.24.4.324.33173. [PubMed] [Cross Ref]
  • Montgomery AA, Fahey T, Peters TJ, MacIntosh C, Sharp DJ. Evaluation of computer based clinical decision support system and risk chart for management of hypertension in primary care: randomised controlled trial. BMJ. 2000;320(7236):686–690. doi: 10.1136/bmj.320.7236.686. [PMC free article] [PubMed] [Cross Ref]
  • Rossi RA, Every NR. A computerized intervention to decrease the use of calcium channel blockers in hypertension. J Gen Intern Med. 1997;12(11):672–678. doi: 10.1046/j.1525-1497.1997.07140.x. [PMC free article] [PubMed] [Cross Ref]
  • McAlister NH, Covvey HD, Tong C, Lee A, Wigle ED. Randomised controlled trial of computer assisted management of hypertension in primary care. Br Med J (Clin Res Ed) 1986;293(6548):670–674. doi: 10.1136/bmj.293.6548.670. [PMC free article] [PubMed] [Cross Ref]
  • Rogers JL, Haring OM, Goetz JP. Changes in patient attitudes following the implementation of a medical information system. QRB Qual Rev Bull. 1984;10(3):65–74. [PubMed]
  • Rogers JL, Haring OM, Wortman PM, Watson RA, Goetz JP. Medical information systems: assessing impact in the areas of hypertension, obesity and renal disease. Med Care. 1982;20(1):63–74. doi: 10.1097/00005650-198201000-00005. [PubMed] [Cross Ref]
  • Rogers JL, Haring OM. The impact of a computerized medical record summary system on incidence and length of hospitalization. Med Care. 1979;17(6):618–630. doi: 10.1097/00005650-197906000-00006. [PubMed] [Cross Ref]
  • Coe FL, Norton E, Oparil S, Tatar A, Pullman TN. Treatment of hypertension by computer and physician-a prospective controlled study. J Chronic Dis. 1977;30(2):81–92. doi: 10.1016/0021-9681(77)90077-7. [PubMed] [Cross Ref]
  • Fiks AG, Hunter KF, Localio AR, Grundmeier RW, Bryant-Stephens T, Luberti AA, Bell LM, Alessandrini EA. Impact of electronic health record-based alerts on influenza vaccination for children with asthma. Pediatrics. 2009;124(1):159–169. doi: 10.1542/peds.2008-2823. [PubMed] [Cross Ref]
  • Poels PJ, Schermer TR, Thoonen BP, Jacobs JE, Akkermans RP, de Vries Robbe PF, Quanjer PH, Bottema BJ, van Weel C. Spirometry expert support in family practice: a cluster-randomised trial. Prim Care Respir J. 2009;18(3):189–197. doi: 10.4104/pcrj.2009.00047. [PubMed] [Cross Ref]
  • Martens JD, van der Weijden T, Severens JL, de Clercq PA, de Bruijn DP, Kester AD, Winkens RA. The effect of computer reminders on GPs' prescribing behaviour: a cluster-randomised trial. Int J Med Inform. 2007;76(Suppl 3):S403–S416. [PubMed]
  • Martens JD, van der Aa A, Panis B, van der Weijden T, Winkens RA, Severens JL. Design and evaluation of a computer reminder system to improve prescribing behaviour of GPs. Stud Health Technol Inform. 2006;124:617–623. [PubMed]
  • Kattan M, Crain EF, Steinbach S, Visness CM, Walter M, Stout JW, Evans Iii R, Smartt E, Gruchalla RS, Morgan WJ. A randomized clinical trial of clinician feedback to improve quality of care for inner-city children with asthma. Pediatrics. 2006;117(6):e1095–e1103. doi: 10.1542/peds.2005-2160. [PubMed] [Cross Ref]
  • Kuilboer MM, van Wijk MA, Mosseveld M, van der Does E, de Jongste JC, Overbeek SE, Ponsioen B, van der Lei J. Computed critiquing integrated into daily clinical practice affects physicians' behavior--a randomized clinical trial with AsthmaCritic. Methods Inf Med. 2006;45(4):447–454. [PubMed]
  • Plaza V, Cobos A, Ignacio-Garcia JM, Molina J, Bergonon S, Garcia-Alonso F, Espinosa C, Grupo Investigador A. [Cost-effectiveness of an intervention based on the Global INitiative for Asthma (GINA) recommendations using a computerized clinical decision support system: a physicians randomized trial] Med Clin (Barc) 2005;124(6):201–206. doi: 10.1157/13071758. [PubMed] [Cross Ref]
  • Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD. Can computer-generated evidence-based care suggestions enhance evidence-based management of asthma and chronic obstructive pulmonary disease? A randomized, controlled trial. Health Serv Res. 2005;40(2):477–497. doi: 10.1111/j.1475-6773.2005.0t369.x. [PMC free article] [PubMed] [Cross Ref]
  • Eccles M, McColl E, Steen N, Rousseau N, Grimshaw J, Parkin D, Purves I. Effect of computerised evidence based guidelines on management of asthma and angina in adults in primary care: cluster randomised controlled trial. BMJ. 2002;325(7370):941. doi: 10.1136/bmj.325.7370.941. [PMC free article] [PubMed] [Cross Ref]
  • Rousseau N, McColl E, Newton J, Grimshaw J, Eccles M. Practice based, longitudinal, qualitative interview study of computerised evidence based guidelines in primary care. BMJ. 2003;326(7384):314. doi: 10.1136/bmj.326.7384.314. [PMC free article] [PubMed] [Cross Ref]
  • McCowan C, Neville RG, Ricketts IW, Warner FC, Hoskins G, Thomas GE. Lessons from a randomized controlled trial designed to evaluate computer decision support software to improve the management of asthma. Med Inform Internet. 2001;26(3):191–201. doi: 10.1080/14639230110067890. [PubMed] [Cross Ref]
  • Bertoni AG, Bonds DE, Chen H, Hogan P, Crago L, Rosenberger E, Barham AH, Clinch CR, Goff DC Jr. Impact of a multifaceted intervention on cholesterol management in primary care practices: guideline adherence for heart health randomized trial. Arch Intern Med. 2009;169(7):678–686. doi: 10.1001/archinternmed.2009.44. [PMC free article] [PubMed] [Cross Ref]
  • Rosenberger EL, Goff DC Jr, Blackwell CS, Williams DT, Crago OL, Ellis SD, Bertoni AG, Bonds DE. Implementing a palm pilot intervention for primary care providers: lessons learned. Contemp Clin Trials. 2009;30(4):321–325. doi: 10.1016/j.cct.2009.03.009. [PMC free article] [PubMed] [Cross Ref]
  • Gilutz H, Novack L, Shvartzman P, Zelingher J, Bonneh DY, Henkin Y, Maislos M, Peleg R, Liss Z, Rabinowitz G, Vardy D, Zahger D, Ilia R, Leibermann N, Porath A. Computerized community cholesterol control (4C): meeting the challenge of secondary prevention. Israel Med Assoc J. 2009;11(1):23–29. [PubMed]
  • Lester WT, Grant RW, Barnett GO, Chueh HC. Randomized controlled trial of an informatics-based intervention to increase statin prescription for secondary prevention of coronary disease. J Gen Intern Med. 2006;21(1):22–29. doi: 10.1111/j.1525-1497.2005.00268.x. [PMC free article] [PubMed] [Cross Ref]
  • Lester WT, Grant R, Barnett GO, Chueh H. Facilitated lipid management using interactive e-mail: preliminary results of a randomized controlled trial. Stud Health Technol Inform. 2004;107(Pt 1):232–236. [PubMed]
  • Cobos A, Vilaseca J, Asenjo C, Pedro-Botet J, Sanchez E, Val A, Torremade E, Espinosa C, Bergonon S. Cost effectiveness of a clinical decision support system based on the recommendations of the European Society of Cardiology and other societies for the management of hypercholesterolemia: Report of a cluster-randomized trial. Dis Manag Health Out. 2005;13(6):421–432. doi: 10.2165/00115677-200513060-00007. [Cross Ref]
  • Goud R, de Keizer NF, ter Riet G, Wyatt JC, Hasman A, Hellemans IM, Peek N. Effect of guideline based computerised decision support on decision making of multidisciplinary teams: cluster randomised trial in cardiac rehabilitation. BMJ. 2009;338:b1440. doi: 10.1136/bmj.b1440. [PMC free article] [PubMed] [Cross Ref]
  • Goud R, Jaspers MW, Hasman A, Peek N. Subjective usability of the CARDSS guideline-based decision support system. Stud Health Technol Inform. 2008;138:193–198. [PubMed]
  • Feldman PH, Murtaugh CM, Pezzin LE, McDonald MV, Peng TR. Just-in-time evidence-based e-mail 'reminders' in home health care: impact on patient outcomes. Health Serv Res. 2005;40(3):865–885. doi: 10.1111/j.1475-6773.2005.00389.x. [PMC free article] [PubMed] [Cross Ref]
  • Murtaugh CM, Pezzin LE, McDonald MV, Feldman PH, Peng TR. Just-in-time evidence-based e-mail 'reminders' in home health care: impact on nurse practices. Health Serv Res. 2005;40(3):849–864. doi: 10.1111/j.1475-6773.2005.00388.x. [PMC free article] [PubMed] [Cross Ref]
  • Tierney WM, Overhage JM, Murray MD, Harris LE, Zhou XH, Eckert GJ, Smith FE, Nienaber N, McDonald CJ, Wolinsky FD. Effects of computerized guidelines for managing heart disease in primary care. J Gen Intern Med. 2003;18(12):967–976. doi: 10.1111/j.1525-1497.2003.30635.x. [PMC free article] [PubMed] [Cross Ref]
  • Lee NJ, Chen ES, Currie LM, Donovan M, Hall EK, Jia H, John RM, Bakken S. The effect of a mobile clinical decision support system on the diagnosis of obesity and overweight in acute and primary care encounters. ANS Adv Nurs Sci. 2009;32(3):211–221. [PubMed]
  • Bakken S, Cook SS, Curtis L, Desjardins K, Hyun S, Jenkins M, John R, Klein WT, Paguntalan J, Roberts WD, Soupios M. Promoting patient safety through informatics-based nursing education. Int J Med Inform. 2004;73(7-8):581–589. doi: 10.1016/j.ijmedinf.2004.04.008. [PubMed] [Cross Ref]
  • Locatelli F, Covic A, Macdougall IC, Scherhag A, Wiecek A. ORAMA Study Group. Effect of computer-assisted European Best Practice Guideline implementation on adherence and target attainment: ORAMA results. J Nephrol. 2009;22(5):662–674. [PubMed]
  • Javitt JC, Rebitzer JB, Reisman L. Information technology and medical missteps: evidence from a randomized trial. J Health Econ. 2008;27(3):585–602. doi: 10.1016/j.jhealeco.2007.10.008. [PubMed] [Cross Ref]
  • Verstappen SMM, Jacobs JWG, Van der Veen MJ, Heurkens AHM, Schenk Y, Ter Borg EJ, Blaauw AAM, Bijlsma JWJ. Intensive treatment with methotrexate in early rheumatoid arthritis: aiming for remission. Computer Assisted Management in Early Rheumatoid Arthritis (CAMERA, an open-label strategy trial) Ann Rheum Dis. 2007;66(11):1443–1449. doi: 10.1136/ard.2007.071092. [PMC free article] [PubMed] [Cross Ref]
  • Downs M, Turner S, Bryans M, Wilcock J, Keady J, Levin E, O'Carroll R, Howie K, Iliffe S. Effectiveness of educational interventions in improving detection and management of dementia in primary care: cluster randomised controlled study. BMJ. 2006;332(7543):692–696. doi: 10.1136/bmj.332.7543.692. [PMC free article] [PubMed] [Cross Ref]
  • Feldstein A, Elmer PJ, Smith DH, Herson M, Orwoll E, Chen C, Aickin M, Swain MC. Electronic medical record reminder improves osteoporosis management after a fracture: a randomized, controlled trial. J Am Geriatr Soc. 2006;54(3):450–457. doi: 10.1111/j.1532-5415.2005.00618.x. [PubMed] [Cross Ref]
  • McDonald MV, Pezzin LE, Feldman PH, Murtaugh CM, Peng TR. Can just-in-time, evidence-based 'reminders' improve pain management among home health care nurses and their patients? J Pain Symptom Manage. 2005;29(5):474–488. doi: 10.1016/j.jpainsymman.2004.08.018. [PubMed] [Cross Ref]
  • Dexter PR, Wolinsky FD, Gramelspacher GP, Zhou XH, Eckert GJ, Waisburd M, Tierney WM. Effectiveness of computer-generated reminders for increasing discussions about advance directives and completion of advance directive forms. A randomized, controlled trial. Ann Intern Med. 1998;128(2):102–110. [PubMed]
  • Rubenstein LV, McCoy JM, Cope DW, Barrett PA, Hirsch SH, Messer KS, Young RT. Improving patient quality of life with feedback to physicians about functional status. J Gen Intern Med. 1995;10(11):607–614. doi: 10.1007/BF02602744. [PubMed] [Cross Ref]
  • Petrucci K, Petrucci P, Canfield K, McCormick KA, Kjerulff K, Parks P. Evaluation of UNIS: Urological Nursing Information Systems. Proc Annu Symp Comput Appl Med Care. 1991. pp. 43–47. [PMC free article] [PubMed]
  • McDonald CJ, Hui SL, Smith DM, Tierney WM, Cohen SJ, Weinberger M, McCabe GP. Reminders to physicians from an introspective computer medical record. A two-year randomized trial. Ann Intern Med. 1984;100(1):130–138. [PubMed]
  • Sackett DL, Snow JC. In: Adherence in Health Care. Haynes RB, Taylor DW, Sackett DL, editor. Baltimore: Johns Hopkins University Press; 1979. The magnitude of adherence and non-adherence.
  • Wright A, Sittig DF, Ash JS, Sharma S, Pang JE, Middleton B. Clinical decision support capabilities of commercially-available clinical information systems. J Am Med Inform Assoc. 2009;16(5):637–644. doi: 10.1197/jamia.M3111. [PMC free article] [PubMed] [Cross Ref]
  • Hedges LV, Olkin I. Statistical methods for meta-analysis. Orlando: Academic Press; 1985.
  • Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005;330(7494):765. doi: 10.1136/bmj.38398.500764.8F. [PMC free article] [PubMed] [Cross Ref]
  • Jaspers MW, Smeulers M, Vermeulen H, Peute LW. Effects of clinical decision-support systems on practitioner performance and patient outcomes: a synthesis of high-quality systematic review findings. J Am Med Inform Assoc. 2011;18(3):327–334. doi: 10.1136/amiajnl-2011-000094. [PMC free article] [PubMed] [Cross Ref]

Articles from Implementation Science : IS are provided here courtesy of BioMed Central
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...