NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

West SL, Garbutt JC, Carey TS, et al. Pharmacotherapy for Alcohol Dependence. Rockville (MD): Agency for Health Care Policy and Research (US); 1999 Jan. (Evidence Reports/Technology Assessments, No. 3.)

  • This publication is provided for historical reference only and the information may be out of date.

This publication is provided for historical reference only and the information may be out of date.

Cover of Pharmacotherapy for Alcohol Dependence

Pharmacotherapy for Alcohol Dependence.

Show details


This chapter documents the procedures that the Research Triangle Institute-University of North Carolina, Chapel Hill (RTI-UNC) Evidence-based Practice Center (EPC) used to develop a comprehensive evidence report that describes and contrasts the pharmacotherapies for alcohol dependence in adults. To set the framework for the review, we present first the key questions and their underlying causal pathways. This is followed by a detailed description of the literature search, e.g., documenting the inclusion and exclusion criteria for literature acquisition, selecting relevant Medical Subject Headings (MeSH terms), listing databases searched, retrieving the gray literature, and specifying the eligibility criteria for study inclusion. Once it was determined that studies met the inclusion/exclusion criteria and were eligible for inclusion, the data were abstracted onto data extraction forms; development of these forms is also described in this chapter.

This chapter also discusses quality issues, i.e., the RTI-UNC EPC's quality control procedures with regard to determining a study's eligibility for inclusion, carrying out data abstraction, and developing the quality rating for individual studies. An evidence report requires an extensive search for all types of literature, and some published works are of higher methodologic quality than others. The RTI-UNC EPC developed a quality rating form that was specific to studies of pharmacologic treatment for alcohol dependence. This section describes the development of the rating system and its use in the analysis.

The RTI-UNC EPC's procedures for determining whether supplemental analyses could be performed are also discussed in this section.

Key Questions and Causal Pathways

The five key questions for this report address the efficacy of medications used for the treatment of alcoholism. The RTI-UNC EPC developed the key questions with input from its Technical Expert Advisory Group (TEAG) on additional factors to consider in the evaluation of each of the therapies.

Final Key Questions

Our evidence synthesis addresses five questions relevant to the pharmacotherapy for treatment of alcohol-dependent patients. The first three questions relate to three agents used to reduce craving, reduce drinking days and relapse rates, and enhance abstinence. Two of these therapies have Food and Drug Administration (FDA) approval for use in alcohol dependence; the third is still under review. These agents have been in use for different periods of time and, as will be seen, the amount of evidence available for them differed considerably. The last two questions involve pharmacotherapies that, although used principally to treat other conditions, have been found useful by some clinicians for treating alcohol dependence; and/or there is some evidence of their efficacy in reducing alcohol intake.


What is the efficacy of disulfiram relative to placebo in treating alcohol dependence?


What is the efficacy of opiate antagonists (naltrexone and nalmefene) relative to placebo in treating alcohol dependence?


What is the efficacy of acamprosate relative to placebo in treating alcohol dependence?


What is the efficacy of serotonergic agents relative to placebo in the treatment of alcohol dependence?


What is the efficacy of lithium relative to placebo in the treatment of alcohol dependence?

In the context of alcohol dependence pharmacotherapies, efficacy refers to the medication's ability to reduce (or eliminate) alcohol intake and the desire to seek alcohol under the ideal setting and circumstances of a clinical trial. The efficacy of each of the drugs specified in our key questions was determined by assessing the following factors: reduction in the average number of standard drinks of alcohol; number or percentage of drinking/nondrinking days; number of episodes of heavy drinking (>5 drinks per day for men, >4 drinks per day for women); time to return to drinking; overall resumption of drinking over the course of the study (the definition of which may differ among studies); reduction in relapse rates where the definition of relapse uses one or more of the aforementioned outcomes; and craving or urge for alcohol.

Examination of treatment efficacy must be balanced with assessment of treatment harm. Evidence on treatment side effects - i.e., harm - is sometimes found within randomized controlled trials (RCTs), but it may also be identified through prospective cohort studies or secondary data sources. We used all types of controlled studies to evaluate the side effects of these medications. However, data on harms were not systematically sought outside the controlled clinical trials identified by our literature review. Therefore, by definition, harms as reported in this evidence report are limited in scope. (As with the efficacy studies, the information on harms will be presented in the form of summary tables at the conclusion of this report.)


Disulfiram is classified as an alcohol-sensitizing agent or deterrent drug. This class of drugs works by deterring the problem drinker from drinking by producing a very unpleasant physiological reaction when the person consumes alcohol. Additional information on disulfiram's mechanism of action for the treatment of alcohol dependence is provided in Chapter 1. The hypothesized causal pathway for the use of disulfiram in the treatment of alcohol dependence is provided in Figure 1.

Figure 1. Hypothesized causal pathway for the use of disulfiram for alcohol dependence.


Figure 1. Hypothesized causal pathway for the use of disulfiram for alcohol dependence.

Opiate Antagonists (Naltrexone and Nalmefene)

Naltrexone, an opioid antagonist, was originally developed for the treatment of heroin addiction. The FDA first approved its use for treating alcohol-dependent patients in the United States in 1994. Naltrexone binds to opioid receptors in the central nervous system (CNS) and competitively inhibits the actions of opioid drugs (both pure agonists and agonists/antagonists) and endogenous opioids (see Chapter 1 for a more thorough discussion of the opiate antagonists' mechanism of action for the treatment of alcohol dependence). In the work that follows, we occasionally refer also to nalmefene, which is another opiate antagonist. Thus, opiate antagonists should be understood to include naltrexone and nalmefene. The hypothesized causal pathway for the use of opiate antagonists for the treatment of alcohol dependence is provided in Figure 2.

Figure 2. Hypothesized causal pathway for the use of opiate antagonists (naltrexone and nalmefene) for alcohol dependence.


Figure 2. Hypothesized causal pathway for the use of opiate antagonists (naltrexone and nalmefene) for alcohol dependence.


Acamprosate, also known as calcium acetyl homotaurinate, has been approved for treating alcoholism in Europe for several years and is now in wide use there; it has demonstrated promising results in regard to abstinence and dropout rates (see Chapter 1 for a more thorough discussion of acamprosate's mechanism of action for the treatment of alcohol dependence). The hypothesized causal pathway for the use of acamprosate in alcohol dependence is provided in Figure 3.

Figure 3. Hypothesized causal pathway for the use of acamprosate for alcohol dependence.


Figure 3. Hypothesized causal pathway for the use of acamprosate for alcohol dependence.

Serotonergic Agents

An extensive literature exists on abnormalities of serotonergic function in alcohol-dependent humans and alcohol-preferring rats. Serotonergic agents represent a group of medications that modulate serotonin neurotransmission. For this review, some are individual drugs, and others belong to the subclass known as selective serotonin (5-HT) reuptake inhibitors (SSRIs). Chapter 1 provides a more thorough discussion of the serotonergic agent's mechanism of action for the treatment of alcohol dependence. The hypothesized causal pathway for the use of serotonergic agents in treating alcohol dependence is provided in Figure 4.

Figure 4. Hypothesized causal pathway for the use of serotonergic agents for alcohol dependence.


Figure 4. Hypothesized causal pathway for the use of serotonergic agents for alcohol dependence.

Serotonergic agents that have appeared in the literature for the treatment of alcohol dependence include buspirone, ondansetron, and several of the SSRIs (fluoxetine, fluvoxamine, paroxetine, sertraline, citalopram, and viqualine). Other serotonergic agents, such as ritanserin, initially had been reported to reduce alcohol intake. Communications with the representatives of the pharmaceutical manufacturer (August 1997) indicate that ritanserin is no longer being evaluated for use in alcohol dependence, presumably because of lack of efficacy. Thus, ritanserin is not included in the summary tables.


Because the literature contains several evaluations of lithium as a potential pharmacotherapy for alcohol dependence, the American Society of Addiction Medicine (ASAM) suggested that lithium be included in our review. Lithium has been a mainstay of therapy in the treatment of bipolar affective disorder, although the literature in the area of alcohol dependence is somewhat limited. The mechanism of action has not been established, but lithium may be helpful in treating alcoholism owing to its antidepressant or mood-stabilizing effects. The exact mechanism of action of lithium is unknown though lithium enhances serotonergic activity and affects a variety of second messenger systems (see Chapter 1 for a more thorough discussion of lithium's mechanism of action for the treatment of alcohol dependence). The hypothesized causal pathway for use of lithium in treating alcohol dependence is provided in Figure 5.

Figure 5. Hypothesized causal pathway for lithium use in alcohol dependence.


Figure 5. Hypothesized causal pathway for lithium use in alcohol dependence.

Comparisons Across Drugs

Apart from understanding the efficacy (and harms) of these agents taken one at a time, an important related issue is whether one of these agents appears to be more efficacious than one or more of the others. Few data are available from controlled trials that directly compare these agents. However, one very small study (Gerra, Caccavari, Delsignore, et al., 1992) did address the issue of comparisons across drugs (acamprosate vs. fluoxetine vs. placebo).

Literature Search

This portion of Chapter 2 describes the literature search process and specifies the inclusion/exclusion criteria and MeSH terms for each of the literature database searches conducted; it also describes the gray literature search strategy. The procedures for identifying the studies to include in the evidence report based on a review of titles and abstracts are documented in this section as well.

Inclusion and Exclusion Criteria for the Literature Search

The criteria listed in Table 3 provide the basis for the systematic search strategy. They map to the key questions with regard to the populations being studied, the treatment setting for patients with alcohol dependence, the countries where these studies are typically done, and the publication languages.

Table 3. Pharmacotherapy for alcohol dependence: Inclusion/exclusion criteria.


Table 3. Pharmacotherapy for alcohol dependence: Inclusion/exclusion criteria.

Table 4 lists the MeSH terms that were used and the final search results. The initial search was conducted in October 1997, with subsequent partial searches to ensure identification of all relevant articles. As indicated in Table 4, each search used the "explode" function, which includes all the individual brand and generic drug names with the need to list all the names separately. With regard to the condition of interest, because "alcohol dependence" does not have its own MeSH entry, the terms alcoholism and alcohol drinking were used.

Table 4. Systematic search strategy and results.


Table 4. Systematic search strategy and results.

In this search, "study characteristics" included: analytic studies, case-control studies, retrospective studies, cohort studies, longitudinal studies, follow-up studies, prospective studies, cross-sectional studies, clinical protocols, clinical trials (phases I-IV), controlled clinical trials, RCTs, intervention studies, and sampling studies. "Study design" included: cross-over studies, double-blind method, matched pair analysis, meta-analysis, random allocation, reproducibility of results, and sample size.

Main Databases Searched

Searches initially were conducted of the U.S. National Library of Medicine (NLM) databases MEDLINE and HealthSTAR.


The MEDLINE database is widely recognized as the premier source for bibliographic coverage of biomedical literature. MEDLINE encompasses information from Index Medicus, Index to Dental Literature, and International Nursing, as well as other sources of coverage in the areas of communication disorders, population biology, and reproductive biology. More than 8.5 million records from more than 3,600 journals are indexed. Our MEDLINE search was done back to 1966.


HealthSTAR, produced cooperatively by the NLM and the American Hospital Association, contains citations to the published literature on health services, technology, administration, and research; it replaces the former Health Planning and Administration database (HLTH). HealthSTAR focuses on both the clinical and nonclinical aspects of health care delivery, including the following topics: evaluation of patient outcomes; effectiveness of procedures, programs, products, services, and processes; administration and planning of health facilities, services, and manpower; health insurance; health policy; health services research; health economics and financial management; laws and regulation; personnel administration; quality assurance; licensing; and accreditation. The database contains citations and abstracts (when available) to journal articles, monographs, technical reports, meeting abstracts and papers, book chapters, government documents, and newspaper articles from 1975 to the present. Citations are indexed with the NLM MeSH terms to ensure compatibility with other NLM databases. Information in HealthSTAR is derived from MEDLINE, CATLINE, the Hospital Literature Index, and selected journals. Additional records specially indexed for this database do not appear in any other NLM database.

Other Databases Searched

The same search strategy was used in several other databases as well. Because the MeSH controlled vocabulary is not standard across other databases, analogous terminology was used to retrieve the widest range of articles from these sources.

International Pharmaceutical Abstracts

Produced by the American Society of Health-System Pharmacists, International Pharmaceutical Abstracts (IPAB) provides worldwide coverage of pharmaceutical science and health-related literature. Its comprehensive bibliographic citations are valuable to researchers, librarians, and medical professionals. Coverage includes drug therapy, toxicity, and pharmacy practice, as well as legislation, regulation, technology, utilization, biopharmaceutics, information processing, education, economics, and ethics as related to pharmaceutical science and practice. The database currently contains more than 246,000 records.

EMBASE Drugs and Pharmacology

The EMBASE Drugs and Pharmacology database, produced by Elsevier Science, contains the most important citations and abstracts to the worldwide drug literature. It is a subset of EMBASE, the Excerpta Medica database, which indexes more than 3,500 international journals in the following fields: drug research, pharmacology, pharmaceutics, toxicology, clinical and experimental human medicine, health policy and management, public health, occupational health, environmental health, drug dependence and abuse, psychiatry, forensic medicine, and biomedical engineering and instrumentation. The database contains more than 1.4 million records, allowing searchers to focus on the pharmaceutical industry. Coverage includes effects, use, and administration of all current, potential, and experimental drugs, side effects, manufacturers, and trade names of approved and prospective drugs.

Alcohol and Alcohol Problems Science Database

The Alcohol and Alcohol Problems Science Database (ETOH) contains more than 92,000 bibliographic records with abstracts to alcohol-related scientific documents from U.S. and international sources. It is produced by the U.S. National Institutes of Health and the U.S. National Institute on Alcohol Abuse and Alcoholism (NIAAA). The database covers all aspects of alcoholism research, including psychology, psychiatry, physiology, biochemistry, epidemiology, sociology, animal studies, treatment and prevention, employee assistance programs, drinking and driving, and public policy. Document types include journal articles, books, monographs, selected book chapters, reports, conference papers and proceedings, unpublished papers, and abstracts of dissertations.


Produced by the American Psychological Association, PsycINFO covers the professional and academic literature in psychology and related disciplines, including medicine, psychiatry, nursing, sociology, education, pharmacology, physiology, linguistics, and other areas. PsycINFO's worldwide coverage includes references and abstracts for more than 1,300 journals in more than 20 languages, as well as book chapters and books in the English language. The database includes information from empirical studies, case studies, surveys, bibliographies, literature reviews, discussion articles, conference reports, and dissertations. PsycINFO currently contains more than 1 million references, with more than 57,000 references added annually. PsycINFO is the world's most comprehensive source for bibliographic coverage of psychology and behavioral sciences literature.

The Cochrane Collaboration

The Cochrane Collaboration prepares, maintains, and disseminates systematic, up-to-date reviews of all relevant RCTs of health care. Each reviewer belongs to a collaborative review group, and each collaborative review group is coordinated by an editorial team that assembles an edited report for dissemination.

PSDI: The Practice and Service Development Initiative

The United Kingdom National Health Service Centre for Reviews and Dissemination (CRD) promotes the application of research-based knowledge in health care relevant to all health care professions. The Practice and Service Development Initiative is a project located within CRD that focuses on disseminating relevant research to nurses, midwives, health visitors (paraprofessional health workers in the United Kingdom), and the professions allied to medicine (PAMs). CRD is funded by the National Health Service and the Health Departments of Scotland, Wales, and Northern Ireland; a contribution to the Centre is also made by the University of York and the Health Education Authority. The views expressed in these databases are those of the authors and not necessarily those of the NHS Executive or the three Health Departments.

Compilation into ProCite®

The results from all searches were compiled into a ProCite® bibliographic database, removing all duplicate records. The bibliographic software was used to tag eligible articles and, for articles determined to be ineligible, the reason for their exclusion from the evidence report.

Our search strategy identified many articles for each search term taken one at a time (in most cases, in the thousands). When the search terms were taken in appropriate combinations, the strategy identified 487 published papers. The search results for the major therapy groupings are provided in Table 5; our five main classes of drugs are shown in bold. This number includes papers that were identified by reviewing the table of contents from symposia proceedings and retrieval of additional papers identified from review articles. This number also includes preliminary publications of trials that subsequently have been published in a more complete form. In those instances the most complete published study was abstracted and used in the evidence tables although the earlier publications were also noted. Additionally, subgroup or followup analyses of published controlled trials were abstracted but then included as a subgroup or followup analysis to the main trial - this was done to prevent subjects from being included in more than one publication.

Table 5. Search results by type or name of pharmaceutical agent.


Table 5. Search results by type or name of pharmaceutical agent.

The final database contained 375 publications. This number is lower than would be produced by summing over the number of articles retrieved (n=487) for each drug type, because some articles addressed more than one drug.

Title and Abstract Review

Two abstractors independently evaluated the titles and abstracts for the 375 articles for inclusion in the evidence report. When abstracts were not available from the literature databases, they were obtained from the original article and then subjected to review. The scientific director for this project adjudicated any disagreements between the two abstractors at this stage.

Exclusion of Articles

Of the 375 articles in the final database, many were excluded. Table 6 provides the main reasons for excluding articles along with the number of articles excluded for each criterion. By and large, articles were excluded (i.e., not abstracted) for the following reasons: they were opinion/editorials, reviews, or biological studies (e.g., of tissue damage or biological markers); they were animal studies; or they did not address alcohol outcomes or evaluate alcohol dependence drugs. After evaluation of the 375 articles in the database, 53 articles remained for review and inclusion in the efficacy evidence tables, and 34 articles were reviewed for harms data.

Table 6. Principal reasons for excluding articles after title/abstract review and numbers of articles excluded.


Table 6. Principal reasons for excluding articles after title/abstract review and numbers of articles excluded.

Gray Literature

Useful data for an evidence report often can be found in symposium proceedings, government monographs, industry reports, articles submitted for publication, and other nontraditional sources. Collectively, this is considered the "gray literature." For the purposes of this report, gray literature was defined as information that has not appeared in peer-reviewed journals but is available as follows:

  • Government documents and monographs (aside from FDA documents).
  • Industry reports and studies.
  • Unpublished studies and works in progress.
  • Conference proceedings and symposia.
  • FDA Medical Officers Review (MOR).

Individuals were contacted in industry, government, and professional societies (including ASAM) to elicit information or reports that are not retrievable through literature database searches (i.e., specifically through MEDLINE or EMBASE). This was done for the following medications: disulfiram; the opiate antagonists naltrexone and nalmefene; acamprosate; and the serotonergic agents.

Federal Agency Sources

Librarians at NIAAA and at the National Clearinghouse for Alcohol and Drug Information (NCADI) were contacted with a request to search for any documents on alcohol dependence and one or more of our drugs of interest, excluding the peer-reviewed literature. The NIAAA library produced 18 documents; the NCADI database produced 10 documents. The document abstracts were reviewed from both of these library searches to determine whether the materials would meet our eligibility criteria; 12 publications were added to our bibliography.

Freedom of Information Act (FOIA) requests were submitted to the FDA to obtain information on two topics as they related to treatments for alcohol dependence. The first was for Adverse Drug Reaction (ADR) reports for the key therapies. The second was for information submitted and/or reviewed by the FDA regarding: (a) investigational new drug (IND) and new drug approval (NDA) or extended (change of label) NDA applications for each of the drugs of interest; (b) drug abuse, psychopharmacologic drugs, or any other advisory committee minutes for meetings specifically for the pharmacotherapies for alcohol dependence; and (c) summaries and evidence reports prepared by FDA staff or pharmaceutical companies for the drugs of interest and their use in alcohol dependence.

ADR information provided by the FDA was briefly reviewed by both the task leader and scientific director. They determined that the information would not be appropriate for inclusion in the evidence report. The ADR reports provide only instances of potential adverse events that may be related to therapy, but they lack appropriate denominator data; thus, it would not be possible to draw any firm or defensible conclusions. Because of time constraints and the extensiveness of the literature requested, the FOIA request was modified to ask for the FDA MOR of efficacy data relevant to the drugs of interest - specifically, disulfiram; the opiate antagonist naltrexone; and the serotonergic agents. Information for the oral form of nalmefene and for acamprosate, which are currently only INDs, was not available. The FOIA information was reviewed for drug efficacy in alcohol-dependent individuals.

A review of all MORs - giving particular care to references to alcohol dependence or the use of these agents in treating alcohol dependence - indicated that, with two exceptions, efficacy of these agents in the treatment of alcohol dependence is not mentioned. For the naltrexone relabeling application for alcohol dependence, the MOR referenced efficacy data submitted on two completed studies, a clinical trial by Volpicelli, Alterman, Hayashida, et al. (1992) and a study by O'Malley, Jaffe, Chang, et al. (1992). In addition, the chairman of the FDA's Panel on Psychiatric Drugs accepted a bibliography of 11 published works dated from 1949 to 1966 as evidence of disulfiram efficacy.

Industry Sources

All research directors from the pharmaceutical companies that produce the key therapies were contacted to identify additional relevant studies. Several published articles were provided, but all were already in our bibliography. One book on disulfiram was recommended; it was obtained and reviewed.

Other Public- or Private-Sector Sources

The TEAG identified several conference and symposia proceedings that they believed would contain original research on the pharmacotherapy for alcohol dependence. These publications or documents were reviewed for inclusion eligibility; 12 publications were added to the bibliography.

Data Abstraction Process

Data Extraction Form

The task leader and scientific director worked with the core project staff to develop a data extraction form to use for entry of relevant information from the eligible publications that addressed efficacy. The data extraction form was developed in an iterative fashion with extensive communication between the methodologists and alcohol researchers. The process was begun by outlining the relevant study designs and outcomes and identifying the variables most important to address for studies involving the efficacy of medications for alcohol dependence. These included:

  • Study designs.
  • Practice setting.
  • Description of patient populations, e.g., severity of dependence, instrument for diagnosing alcohol dependence, inclusion/exclusion parameters, and prior alcohol treatments.
  • Interventions, e.g., dose titration and maximum dosing, compliance with medication, length of study.
  • Co-interventions.
  • Data collection techniques for assessing outcome.
  • Statistical analysis.
  • Outcome measures, e.g., drinks per day or per week, relapse, craving, compliance with treatment.
  • Side effects of treatment.
  • Limitations noted by authors.
  • Limitations noted by reviewers.
  • Conclusions noted in article and whether the reviewer concurred with the stated conclusions.

These variables were incorporated into the data extraction form, whose basic structure was taken from one developed for spinal cord clinical guidelines work. The draft form underwent an extensive review process that included pretesting on several randomly selected alcohol articles. After each pretest, the form was revised to increase its utility and efficiency. After extensive pretesting, the form was also sent to the TEAG for comment. Several TEAG members suggested inclusion of pretreatment variables such as detoxification, length of abstinence, and pre-randomization psychosocial therapies; the extraction form was revised accordingly. The final data extraction form can be found in Appendix C.

The RTI-UNC EPC used two types of abstractors for the data extraction process, content reviewers and methods abstractors. The content abstractors are basic science researchers who specialize in understanding alcohol effects on the brain and other physiologic aspects of this disease. The methods abstractors are more generally trained in qualitative and quantitative methods such as epidemiology, economics, and statistics. In addition, reviewers included an epidemiologist with special expertise in pharmacotherapy (task leader), a psychiatrist with special expertise in treating patients with alcoholism (scientific director), and an internist with broad clinical responsibilities and a substantial research background (senior advisor to the project).

To collect high-quality data, an extraction guide was developed for use during training and as an ongoing reference for each of the items in the data extraction form. In addition to providing guidance on specific items, the guide also delineated general rules for this process that were intended, insofar as possible, to guarantee consistency across abstractors. For example, abstractors were instructed that the outcomes data for the results section of the data extraction form could come only from text or tables, not from graphic data presentations. Furthermore, when inconsistencies occurred between results stated in the text of an article and the results presented in tabular form, the abstractors were instructed to use the data from the text. Abstractors also were instructed to extract the alcohol outcomes data from the intent-to-treat analyses whenever possible. Subgroup analyses (e.g., compliant patients) were extracted onto a separate section of the abstraction form.

Each of the abstractors had used drafts of the data extraction form (e.g., during pretesting). Nonetheless, a formal training session was held to discuss the extraction guide and various changes to the data extraction form, provide final guidance on the abstracting procedures, and answer questions with regard to the final instruments.

At the completion of abstractor training, the data abstraction process began. The task leader, scientific director, and both the RTI and UNC research coordinators monitored progress. To this end, the task leader and scientific director conducted extensive reviews of the first three data extraction forms completed by each abstractor and provided both individual and group feedback for particularly problematic items. This was done to enhance consistency between abstractors.

Two especially difficult problems in the abstraction process were (1) accounting for comorbidities and (2) capturing information on followup of initial interventions. Addressing these issues ultimately required the development of two separate supplemental forms. These additional forms are also provided in Appendix C.

Side-Effect Profile Form

The harms literature was abstracted onto a side effect profile form (Appendix C) that elicited data on the following:

  • Type of side effects.
  • Total sample size of the study.
  • Frequency of the side effect in the key therapy and placebo groups.
  • Whether there was a statistically significant difference between the key therapy and placebo groups.
  • Methodology for collecting side effect data.

This information was sought for all of the eligible efficacy articles reviewed (n=53) as well as from articles on side effects and all others that included a control group for comparison purposes. Because the main emphasis of this report was efficacy and not harms, these data were not collected by dual abstraction. A registered pharmacist (who is also a Ph.D. candidate) collected the side effect and withdrawal data for the evidence tables on harms. However, it should again be emphasized that an extensive search for harms data was not completed as part of this evidence report; the listing of harms reflects those that have been detected in clinical trials and not in general clinical practice.

Quality Control and Adjudication

Quality control to determine eligibility for abstraction has been described (see "Title and Abstract Review"). In short, all titles and abstracts were subject to dual review, and the scientific director adjudicated discrepancies. Each article was abstracted by two independent abstractors - one an expert in alcohol research (content reviewer) and the other a methodologist. Abstractors were blinded to the authors' names and the institution that produced the work. Owing to the time-consuming nature of deleting all reference to the journal title and the distinct formatting of certain journals (such as the Archives of General Psychiatry), the abstractors were not blinded to the journal title.

In concordance with an article by Morley, Finney, Monahan, et al. (1996), the RTI-UNC EPC noted that the quality of the literature and the heterogeneity of outcomes caused considerable difficulties in the data abstraction process as discussed in Chapter 4. As can be seen by reviewing the data extraction form (Appendix C), a great deal of information was abstracted from each article. In reviewing the completed abstracts, it was noted that agreement was good for the core variables (sample size, study design, outcomes, etc.), but agreement was lower for the more peripheral variables. With such a large number of variables abstracted, many disagreements were found that required adjudication. Because the adjudication process is time-consuming, the RTI-UNC EPC staff decided to have an additional reviewer, the task leader or scientific director, complete a third form for all articles. In reporting on this whole process and our final results, the primary study design and results are based on this third review, which took into account the reporting of the content and methods abstractors, whereas the quality rating, study limitations, and conclusions are those provided by the method and content reviewers.


An important aspect of the dual abstraction process is to determine the reliability of data collection between the two reviewers. To this end, the task leader, scientific director, and the clinical methodologist selected the following eight key variables for reliability assessment:

  • Study design (categorical variable).
  • Diagnostic criteria for alcohol dependence (categorical variable).
  • Total number of controls entered into the trial (continuous variable).
  • Maximum dose per day for intervention group 1 (continuous variable).
  • Whether compliance with treatment was measured (continuous variable).
  • Average number of standard drinks for intervention group 1 (continuous variable).
  • Average number of episodes of heavy drinking for intervention group 1 (continuous variable).
  • Average time to first drink for intervention group 1 (continuous variable).

From the reviewed articles (n=53), nine articles were randomly sampled to assess the reliability of abstraction between the content and method abstractors. The kappa statistic, which corrects for chance agreement, was used. Kappa was not evaluated for four of the eight variables because there was near perfect agreement between the reviewers but no variation among the nine articles for these variables (study design, average number of standard drinks for intervention group 1, average number of episodes of heavy drinking for intervention group 1, and average time to first drink for intervention group 1). For example, for study design, all the abstractors said the articles were RCTs with only one disagreement. Thus, the observed agreement was 87.5 percent, with a chance agreement of 87.5 percent as well. Since kappa takes the chance agreement into account, its computation in these circumstances produces a low kappa statistic that does not actually reflect the excellent agreement observed. For this reason, the kappas for the variables with no variation among the nine articles have been omitted.

For the variables with variation among the articles, substantial agreement was noted, as follows:

  • Criteria for alcohol dependence (kappa 0.84).
  • Total controls initially entered in the trial (kappa 0.74).
  • Maximum dose per day for intervention group 1 (kappa 0.66).
  • Compliance with treatment not measured (kappa 0.77).

A kappa value greater than 0.60 is considered very good; a kappa greater than 0.80 is considered almost perfect.

Quality Rating Development and Analysis

Quality of the evidence can be judged on two levels: at the level of the individual article and in summary over the spectrum of articles addressing each of the key therapies. This section of the report describes our approaches to the development of quality ratings on both levels.

Quality of Individual Articles

The RTI-UNC approach to assessing the quality of the individual articles (see Figure 6) takes into account the internal and external validity of each study. It encompasses, among other issues, study design, measurement issues, statistical analyses, and the appropriateness of the conclusions drawn.

Figure 6. Quality rating scale for alcohol dependence pharmacotherapy review.


Figure 6. Quality rating scale for alcohol dependence pharmacotherapy review.

To assess the internal validity of a study - i.e., the likelihood that the design and conduct of the study minimize systematic error (bias) - the following factors were evaluated from each study:

  • Sample size and statistical power.
  • Selection bias and inclusion criteria.
  • Selection of comparison groups.
  • Randomization and comparability of the groups.
  • Definition of the intervention and exposures with an assessment of patient compliance with therapy.
  • Definition of outcome measures, attrition rates, confounding variables, data collection methods, and observation bias.
  • Methods of statistical analysis.

Statistical considerations included assessing the adequacy of the sample size for drawing conclusions and identifying whether the authors did a power calculation for small studies. When the authors of a given article presented numerous descriptive analyses in a single publication, our rating addressed whether they had taken these multiple comparisons into account in their analyses. For studies that used regression techniques, the rating assessed whether the number of variables for the model exceeded the limitations imposed by the size of the population studied. The rating scale also addressed the clinical relevancy of the statistical findings.

With regard to diagnostic and outcome measures, the quality rating instrument assessed whether the study design incorporated validated measurement instruments or relied on unstructured and/or unvalidated instruments. In addition, the abstractors evaluated whether the diagnostic criteria and outcomes were clearly specified and appropriately measured.

External validity - i.e., whether the findings of the study can be generalized to populations other than those in the study - was also considered for the quality rating. Especially important for this evidence report was whether the results from studies that took place outside of North America are relevant to U.S. population groups.

The quality rating instrument was based on a 40-point scale with higher scores denoting better studies. Each abstractor completed the quality rating for each article reviewed. The individual scores provided in the evidence table for each article were derived by calculating the following ratio: the points from the responses circled by the abstractor (numerator) divided by the total number of points that could have been awarded, typically 40 points (denominator). When the items were determined to be not applicable, "NA," then we removed total number of points for that item from both the numerator and the denominator. Quality rating scores (Figure 6) for each article were not averaged across the two abstractor types, i.e., content and methods abstractors, because of the large discrepancies noted. Generally, content abstractors tended to give higher (better) scores than did the methods experts. Because averaging the two abstractors' scores would mask these patterns, both scores are presented for comparison.

Grading of the Evidence

According to a report developed for AHCPR by the RTI-UNC EPC Co-Director (Lohr, 1998), several systems have been proposed in the past 10 to 15 years for "grading the quality or strength of the evidence" on a given clinical topic or causal pathway. However, little or no consensus exists about which specific system is best, and thus there was little guidance on what approach to use in the case of the efficacy of a specific pharmacotherapy for alcohol dependence. Like the Jovell and Navarro-Rubio (1995) categorization, the RTI-UNC EPC developed a four-level grading scheme for judging the overall efficacy of each therapy. Our grading scheme was based on several issues: the magnitude of the outcomes reported for each of the pharmacotherapies (abstinence, days to first drink, etc.), the quality rating scores for individual studies, sample sizes of the studies evaluated, and the consistency of the evidence over all the studies. Scores were not assigned for each of these variables; instead, our quality is based on an adjudicated rating that was initially provided by the task leader, scientific director, and clinical methodologist as follows:

  • Good (A): There are sufficient data for evaluating efficacy. The sample size is adequate. The data are consistent and indicate that the key drug is clearly superior to placebo for treating alcohol dependence.
  • Fair (B): There are sufficient data for evaluating efficacy. The sample size is adequate. The data indicate that there are inconsistencies in the findings for the alcohol outcomes between the key therapy and placebo such that the efficacy of the key therapy for treating alcohol dependence is not clearly established.
  • Poor (C): There are sufficient data for evaluating efficacy. The sample size is adequate. The data show that the key therapy is no more efficacious for treating alcohol dependence than placebo.
  • Incomplete evidence (I): There are insufficient data for assessing the efficacy of the key therapy for treating alcohol dependence based on limited sample size or poor methodology.

Although a harms grade was developed for each therapy, it was not based on a thorough search of the literature for adverse effects associated with the therapies under review. As was described in the section on "Data Abstraction Process," the harms data were taken from all controlled studies, whether or not they were eligible for inclusion for the efficacy assessments. The clinicians (i.e., the scientific director and the clinical methodologist) were responsible for grading the harms literature. Their assessment was based on the following issues: the seriousness of the side effects for each key drug, whether the side effects noted were known adverse effects of the drugs, whether there was a statistically significant difference between the active (key therapy) and placebo groups, and how the information on side effects was collected. The following categories were used to grade the harms literature.

A "low" probability of risk was given when:

  • The side effects were not life-threatening.
  • For side effects due to the key therapy where the frequency statistically was significantly different from that of placebo, most are known side effects of the drug.

A "high" probability of risk was given when:

  • Any of the side effects were life-threatening.
  • A side effect not commonly seen was reported; it was serious, and its frequency of occurrence was greater in the key therapy group compared to the placebo group.

The RTI-UNC EPC determined that efficacy should be judged separately from side effects. Thus, the grading of the evidence provides a letter score for efficacy (A, B, C, I) and a dichotomous score for safety (low or high) for each of the key therapies. In this way, the clinicians can choose the best therapy based on their patients' comorbidities and past experiences of potential adverse drug reactions.

Development of the Evidence Tables

A prototype, first draft of our evidence tables was created for the opiate antagonists, naltrexone and nalmefene, using the format of evidence tables typically found in review papers. Thus, the original tables included: citation, study design, inclusion/exclusion criteria, length of active treatment, sample size, data collection methods, statistical analyses, results, conclusions, and other important issues. This prototype version was sent to both AHCPR and the TEAG for review and comment.

The TEAG suggested revisions that would split the prototype table into two separate tables for evaluating the efficacy of each key therapy - one for study design and the other for outcomes. The content of these two tables is listed in Table 7. To provide the available data on the potential risks of each key therapy, evidence tables on harms also were developed.

Table 7. Content of evidence tables on study design and outcomes.


Table 7. Content of evidence tables on study design and outcomes.

The information in the tables on study design is self-explanatory, although numerous abbreviations were used to increase the amount of data entered into the tables for each study; a key to the abbreviation is provided. For the outcomes tables, with the exception of the "conclusion" column, all available data are provided for each of the studies reviewed without regard to the statistical significance of the findings. For the conclusions, however, only those findings that are statistically significant at p < 0.05 are reported. The limitations specified in these tables include both those reported by the study's authors and those noted by our content and methods abstractors.

Using both study design and outcomes tables to present the evidence on our key therapies has advantages and drawbacks. The most important benefit is that by having many categories for each table, the consistency of the data presented can be enhanced, thereby making the reader's task in reviewing the studies much more efficient. In addition, users of the evidence tables will find it easier to see which outcomes were evaluated for each drug and to get a general sense of the weight of evidence provided by each study.

The main drawback of separating the evidence into two tables is that important design characteristics may be missed when reviewing only the outcomes tables. Only by reviewing the study design tables in conjunction with the outcomes tables can readers evaluate the internal and external validity of a study and know what confidence to put in the reported results. When reviewing the outcomes of a study, it is important to know whether the patients who completed the study were similar to those who did not and, more importantly, to the population that could have been selected for study. For this evaluation, users will need to consult both tables.

Peer Reviewer Process

Among the more important activities in producing a credible evidence report is the conduct of an unbiased and broadly based review of the draft report. Such a review, here termed "peer review," should provide a wide array of scientists, methodologists, users, and laypersons adequate opportunity to comment on the report and to identify problems in fact, interpretation, or presentation. The selection process for peer review and the names of all the peer reviewers are listed in Appendix B.

Supplemental Analysis

The presumption at the start of this project was that at least some of the literature that would be reviewed for the evidence report would lend itself to some form of meta-analysis. It was also assumed, should this be correct, that it might be possible to propose conducting some type of cost-effectiveness analysis.

A meta-analysis of one or more common measures over the five key therapies would be extremely helpful to clinicians. Such an analysis would show the relative efficacy and strength of the evidence (degree of uncertainty surrounding the efficacy estimate) of each key therapy in an easy-to-read, summary graph. However, few of the studies that were reviewed, even within one of the key therapy groups, had common interventions (e.g., dosages and counseling varied greatly), alcohol outcomes measures, or durations of followup. Further, no two studies had all three of these dimensions in common. Therefore, to conduct a meta-analysis would have required extreme assumptions and adjustments. We concluded that this was not in the best interests of AHCPR or the ultimate users of the evidence report. In the end, it was concluded that a meta-analysis could not be justified at this time.

PubReader format: click here to try


  • PubReader
  • Print View
  • Cite this Page

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...