Home > DARE Reviews > Effectiveness and efficiency of...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. York (UK): Centre for Reviews and Dissemination (UK); 1995-.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet].

Effectiveness and efficiency of guideline dissemination and implementation strategies

JM Grimshaw, RE Thomas, G MacLennan, C Fraser, CR Ramsay, L Vale, P Whitty, MP Eccles, L Matowe, L Shirran, M Wensing, R Dijkstra, and C Donaldson.

Review published: 2004.

Link to full article: [Journal publisher]

CRD summary

This well-conducted review found that there was a limited evidence base to support decisions about which guideline dissemination and implementation strategies were likely to be effective in changing doctors’ behaviour and/or improving outcomes for patients. The authors’ conclusions reflect the evidence presented and are likely to be reliable.

Authors' objectives

To assess the effectiveness of guideline dissemination and implementation strategies in promoting improved professional practice.

Searching

The authors searched MEDLINE, HealthSTAR, Cochrane Central Register of Controlled Trials, EMBASE, SIGLE and EPOC (Cochrane Effective Practice and Organisation of Care group) specialised register up to 1998. The authors checked reference lists of 51 previous systematic reviews. Search terms were reported in an appendix. There were no language restrictions.

Study selection

Randomised controlled trials (RCTs), non-randomised controlled clinical trials (CCTs), controlled before-and-after studies and interrupted time series studies of interventions to introduce guidelines into clinical practice were eligible for the review. RCTs and CCTs could involve allocation to groups at the individual or cluster level. Guidelines were defined as systematically developed statements to assist decisions about health care. Studies had to involve medically qualified healthcare professionals and report objective measures of provider behaviour (process of care) and/or patient outcome.

Included studies used a variety of single and multifaceted interventions to support guidelines, including educational materials, educational meetings, reminders and audit and feedback. Most studies were conducted in USA (71%) or UK (11%) and in primary care (39%), in-patient (19%) or generalist out-patient (19%) settings. Most studies reported outcomes related to process of care; patient outcomes were reported for some interventions. Few studies evaluated patient-directed interventions.

Two reviewers independently selected studies for the review. Disagreements were resolved by consensus in discussion with a third reviewer.

Assessment of study quality

Validity was assessed independently by two reviewers who used criteria developed by EPOC (listed in the report), which varied according to study design.

Data extraction

Two reviewers independently extracted data. Effect sizes were standardised so that a positive difference between post-intervention percentages or means represented a good outcome. If studies only reported or analysed within-group pre- versus post-intervention comparisons, an attempt was made to reanalyse the between-group post-intervention comparison. If cluster RCTs had potential unit of analysis errors (data analysed without taking account of clustering), the reviewers attempted to reanalyse the trial using cluster level data. Interrupted time series studies were reanalysed where possible using time series regression methods.

Methods of synthesis

Studies were synthesised qualitatively using an explicit analytical framework. Separate analyses were undertaken for comparisons of single interventions against no intervention and intervention controls and of multifaceted interventions against no intervention and intervention controls. For study designs other than interrupted time series, number of comparisons that showed a positive effect, median effect size across all comparisons and across comparisons without unit of analysis errors, and number of comparisons that showed statistically significant effects were reported separately for each design. For multifaceted interventions, separate analyses were performed for combinations that included educational outreach and combinations with more than four comparisons. The authors assessed whether the effect of multifaceted interventions increased with the number of components.

Results of the review

A total of 235 studies (309 comparisons) was included in the review. The overall quality of the studies was considered poor. Most comparisons (73%) involved multifaceted interventions.

Most studies that reported dichotomous process of care outcomes reported improvements, but there were considerable differences within interventions of the same type as well as between different interventions. All studies that compared audit and feedback with a no-intervention control reported improvements in care, but the effects were generally small. Reminders were evaluated for a wide range of behaviours across different settings; improvements in dichotomous process of care measures were observed in 28 out of 33 comparisons, with a median effect size (post-intervention absolute difference between groups) in cluster RCTs of 14.1%. There were 23 comparisons involving multifaceted interventions that included educational outreach. Most of these reported modest to moderate effects (absolute difference between groups after intervention ranged from -5.6 to +17.4%). There was no relationship between the effects of multifaceted interventions and the number of components.

Results of many other analyses were reported; these included some evaluations of effects of guideline dissemination strategies on outcomes of care.

Cost information

Sixty-three studies included in the review reported either cost analyses or economic evaluations that attempted to assess costs or cost-effectiveness of different guideline implementation strategies. Reporting was generally poor and only two studies reported on resource use and costs of guideline development, dissemination and implementation (details in the report).

Authors' conclusions

There was a limited evidence base to support decisions about which guideline dissemination and implementation strategies were likely to be effective and efficient under different circumstances.

CRD commentary

The review had clear inclusion criteria and involved a thorough search without language restrictions. It appeared that no attempts were made to locate unpublished studies, so the review may have been at risk of publication bias. Study selection, validity assessment and data extraction were performed in duplicate, which minimised risk of errors or bias affecting the review process. The validity assessment used appropriate criteria for different study designs and the results were used in the synthesis. Full details of included studies were presented in the report and online appendices. The synthesis used appropriate methods, which included consideration of unit of analysis errors.

This was a well-conducted review. The authors' conclusions reflect the limitations of the evidence base and heterogeneity of the included studies and are likely to be reliable.

Implications of the review for practice and research

Practice: The authors stated that decision-makers should choose interventions based on feasibility, costs and benefits (including the implications of changes in clinical practice following implementation of guidelines). They also stated that wherever possible, interventions should include paper-based or computerised reminders.

Research: The authors stated that further rigorous evaluations of different dissemination and implementation strategies were required; they specifically recommended evaluations of relatively cheap interventions, such as printed educational materials, that have the potential to be efficient.

Funding

Health Technology Assessment Programme Methodology Panel (project number 94/08/29)

Bibliographic details

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay CR, Vale L, Whitty P, Eccles MP, Matowe L, Shirran L, Wensing M, Dijkstra R, Donaldson C. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technology Assessment 2004; 8(6): 1-84. [PubMed: 14960256]

Indexing Status

Subject indexing assigned by NLM

MeSH

Cost-Benefit Analysis; Great Britain; Information Dissemination; Medical Informatics; Practice Guidelines as Topic; State Medicine

AccessionNumber

12004008153

Database entry date

18/08/2010

Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.

CRD has determined that this article meets the DARE scientific quality criteria for a systematic review.

Copyright © 2014 University of York.

PMID: 14960256