NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Cover of Empirical Evaluation of the Association Between Methodological Shortcomings and Estimates of Adverse Events

Empirical Evaluation of the Association Between Methodological Shortcomings and Estimates of Adverse Events

Technical Reviews, No. 13

Investigators: , MD, , PhD, , MPH, , MD, MPH, and , MD, MPH.

Oregon Evidence-based Practice Center, Portland, OR
Rockville (MD): Agency for Healthcare Research and Quality (US); .
Report No.: 07-0003

Structured Abstract

Objective:

Accurate harms data are necessary to appropriately assess the balance between benefits and harms of interventions. Numerous deficiencies in the quality and reporting of harms associated with clinical interventions have been reported. Little is known, however, about whether such perceived methodological shortcomings are associated with lower estimates of harms.

Study Design and Setting:

Studies reporting harms associated with (1) carotid endarterectomy (CEA) for symptomatic stenosis, (2) rofecoxib for arthritis, and (3) CEA for asymptomatic stenosis were identified using published systematic reviews. A standardized abstraction form, including eight pre-defined criteria for assessing the quality of harms reporting, was used to extract data. Univariate and multivariate analyses were performed to empirically evaluate the association between quality criteria and estimates of harms. A quality-rating instrument for studies reporting harms was developed based on the results of the analyses of studies of CEA for symptomatic stenosis. The quality-rating instrument was tested on the other data sets.

Results:

In 111 studies of CEA for symptomatic stenosis, meeting five of the eight quality criteria was associated with significantly higher rates of stroke or death. A quality-rating instrument with four of the five criteria predicted adverse events (5.7 percent in studies rated “adequate”, compared to 3.9 percent in studies rated “inadequate” [p=0.0003]). In multivariate analyses, the quality rating assignment remained significant when controlling for other clinical and study-related variables. Different quality criteria, however, predicted lower estimates of risk for myocardial infarction in 16 trials of rofecoxib, and none of the quality criteria predicted lower estimates of stroke of death in 18 studies of CEA for asymptomatic stenosis. Evaluation of the latter two data sets was limited by small numbers of trials and low rates of evaluated adverse events.

Conclusions:

The presence of methodological shortcomings can predict lower estimates of serious harms. Clinicians and researchers should carefully consider the potential effects of individual methodological shortcomings when evaluating estimates of harms associated with clinical interventions. However, we were unable to develop a generic summary quality-rating instrument for studies of harms because specific methodological shortcomings were not consistently predictive across data sets.

540 Gaither Road, Rockville, MD 20850. www​.ahrq.gov

Prepared for: Agency for Healthcare Research and Quality, U.S. Department of Health and Human Services.1 Contract No. 290-02-0024. Prepared by: Oregon Evidence-based Practice Center, Portland, OR.

Suggested citation:

Chou R, Fu R, Carson S, Saha S, Helfand M. Empirical Evaluation of the Association Between Methodological Shortcomings and Estimates of Adverse Events. Technical Review No. 13. (Prepared by the Oregon Evidence-based Practice Center under Contract No. 290-02-0024.) AHRQ Publication No. 07-0003. Rockville, MD: Agency for Healthcare Research and Quality. June 2006.

This report is based on research conducted by the Oregon Evidence-based Practice Center (EPC) under contract to the Agency for Healthcare Research and Quality (AHRQ), Rockville, MD (Contract No. 290-02-0024). The findings and conclusions in this document are those of the authors, who are responsible for its content, and do not necessarily represent the views of AHRQ. No statement in this report should be construed as an official position of AHRQ or of the U.S. Department of Health and Human Services.

The information in this report is intended to help clinicians, employers, policymakers, and others make informed decisions about the provision of health care services. This report is intended as a reference and not as a substitute for clinical judgment.

This report may be used, in whole or in part, as the basis for the development of clinical practice guidelines and other quality enhancement tools, or as a basis for reimbursement and coverage policies. AHRQ or U.S. Department of Health and Human Services endorsement of such derivative products may not be stated or implied.

Dr. Saha was supported by a Research Career Development award from the Health Services Research and Development Service of the Department of Veterans Affairs and by a Generalist Physician Faculty Scholar award from the Robert Wood Johnson Foundation.

1

540 Gaither Road, Rockville, MD 20850. www​.ahrq.gov

Bookshelf ID: NBK44049PMID: 20734509
PubReader format: click here to try

Views

  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (381K)

Related information

Related citations in PubMed

See reviews...See all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...