• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of bmjBMJ helping doctors make better decisionsSearch bmj.comLatest content
BMJ. Sep 8, 2001; 323(7312): 546.
PMCID: PMC48159

Assessment of therapeutic safety in systematic reviews: literature review

Edzard Ernst, director and Max H Pittler, research fellow

Evidence based medicine aims to objectively and continually evaluate all medical interventions with a view to incorporating the knowledge gained into routine healthcare practice. The ultimate goal is to enhance patient care. The most powerful tools used in evidence based medicine are systematic reviews and meta-analyses.1 Generally, these studies focus on the efficacy or effectiveness of therapeutic interventions; indisputably, however, information on safety is equally important for making informed, evidence based decisions on the value of a given treatment. We evaluated two major medical databases to assess the extent to which therapeutic safety is addressed in systematic reviews and meta-analyses.

Methods and results

We performed systematic literature searches to identify all systematic reviews and meta-analyses on the efficacy or effectiveness and the safety of any type of therapeutic intervention. Data sources were Medline (1966-December 2000) and the Cochrane Library (2000, issue 4). The title, abstract, and keywords of papers were searched using the terms efficacy, effectiveness, systematic review, meta-analysis, safety, adverse effect, adverse event, and adverse reaction and their derivatives. No language restrictions were imposed. The title and abstract of all studies were read and data were according to the criteria defined. Studies were included only if they stated that they were systematic reviews or meta-analyses or if they indicated that the available data were systematically searched for, appraised, and summarised. Studies that included no abstract or that provided no data on humans were excluded. Data were assessed according to the year of the study, and the studies were categorised into three categories: systematic reviews on efficacy or effectiveness, systematic reviews that included safety aspects as a secondary outcome measure (safety category A), and systematic reviews with a primary focus on the safety of the therapy (safety category B). Study selection, data extraction, and evaluation were performed independently by the two investigators, and discrepancies were resolved through discussion.

The results (table) showed a continuous increase in the number of systematic reviews and meta-analyses published between 1966 and December 2000. The number of studies that included safety aspects as a secondary outcome measure (category A) amounted to just over one quarter of the total number of studies on efficacy or effectiveness. The number of systematic reviews that assessed safety aspects as the primary focus of the investigation (category B) amounted to only 3-5% (table).

Comment

Systematic reviews and meta-analyses provide little information on the safety aspects of therapeutic interventions. Most systematic reviews include data on safety only in as much as the primary studies, usually randomised clinical trials, report on adverse events. This seems inadequate. Randomised clinical trials typically assess only a small number of patients and thus the chances of detecting rare adverse events are small. Randomised clinical trials are usually of short duration and thus cannot identify delayed adverse events.2 The assessment of safety has to go far beyond randomised clinical trials and should use various methods, including post-marketing surveillance studies, spontaneous reporting schemes, and epidemiological investigations.3 Systematic reviews on the safety of therapeutic interventions should take these considerations into account and should combine data from various types of studies.4 Safety aspects have so far not been adequately investigated by systematic reviews. One challenge of evidence based medicine therefore is to address safety issues more systematically than has been done before.

Table
Systematic reviews and meta-analyses of therapeutic interventions in Medline and the Cochrane Library. Values are numbers (percentages) of trials

Footnotes

Funding: None.

Competing interests: None declared.

References

1. Mulrow C. Rationale of systematic reviews. BMJ. 1994;309:597–599. [PMC free article] [PubMed]
2. Rawlins MD. Pharmacovigilance: paradise lost, regained or postponed? J R Coll Physicians Lond. 1995;29:41–49. [PubMed]
3. Ernst E, Barnes J. Methodological approaches to investigating the safety of complementary medicine. Complement Ther Med. 1998;6:115–121.
4. Stevinson C, Ernst E. Safety of hypericum in patients with depression. CNS Drugs. 1999;11:125–132.

Articles from BMJ : British Medical Journal are provided here courtesy of BMJ Group
PubReader format: click here to try

Formats: