Home > DARE Reviews > The application of quantitative methods...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. York (UK): Centre for Reviews and Dissemination (UK); 1995-.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet].

The application of quantitative methods for identifying and exploring the presence of bias in systematic reviews: PDE-5 inhibitors for erectile dysfunction

GE Bekkering, AM Abou-Setta, and J Kleijnen.

Review published: 2008.

CRD summary

This review of the efficacy of sildenafil, tadalafil and vardenafil for the treatment of erectile dysfunction concluded that all three drugs are highly effective; no differences were identified between them. The authors thoroughly investigated possible sources of bias in the review; their conclusions were in line with the evidence presented and likely to be reliable.

Authors' objectives

To evaluate the effects of bias and study quality on pooled outcomes in a meta-analysis of phosphodiesterase-5 (PDE-5) inhibitors for erectile dysfunction (ED).


The authors searched 15 electronic databases (listed) to July 2006. Search terms were reported and a sensitive filter was used to identify randomised controlled trials (RCTs). Seven trial registries and the websites of regulatory agencies were searched. Manufacturers were contacted for additional studies. Reference lists of included studies and recent systematic reviews were screened. Searches were not limited by language or publication status.

Study selection

RCTs that evaluated the efficacy or safety of PDE-5 inhibitors (sildenafil, tadalafil or vardenafil) compared with placebo or another PDE-5 inhibitor in men with erectile dysfunction were eligible for the review. Crossover trials (first part only) were included if they met the other criteria. The main outcomes used for the review were those most commonly reported in the included trials, namely the final International Index of Erectile Function (IIEF) EF domain score and the proportion of participants reporting improved EF (general efficacy question, GEQ). The included trials evaluated fixed and flexible doses of PDE-5 inhibitors taken on demand or daily. The majority of trials were in broad populations, but some were confined to men with specific conditions such as diabetes or spinal cord injury. One reviewer selected studies for the review.

Assessment of study quality

Validity was assessed based on method of randomisation, allocation concealment, blinding of patients, care givers and assessors, formal test of blinding, comparability of groups at baseline and use of intention-to-treat analysis. The assessment was done by one reviewer with checking by a second reviewer.

Data extraction

Data were extracted to calculate the difference in outcome between groups – odds ratio (OR) for dichotomous outcomes and mean difference for continuous outcomes – for each included study. Data extractions were done by one reviewer using a standard form and checked by a second reviewer. Authors were contacted to obtain missing data.

Methods of synthesis

Studies were pooled by meta-analysis using random-effects models with weighting by inverse variance. Adjusted indirect comparisons between different PDE-5 inhibitors were calculated using the method of Bucher et al. Heterogeneity was assessed using the Χ2 test and I2 statistic, with I2 more than 50 per cent considered to represent substantial heterogeneity. For comparisons with at least 10 studies, funnel plots were used to examine small study effects; funnel plot asymmetry was tested using Begg's and Egger's tests. Where there was evidence of small study bias, the trim and fill technique was used to estimate the number of missing studies and estimate an adjusted treatment effect. Meta-regression analyses were used to examine the effect of selective exclusion and methodological quality on treatment effect.

Results of the review

A total of 123 RCTs were included in the review, of which 88 (approximately 25,000 participants) were included in the meta-analysis. Fifty-two RCTs evaluated sildenafil, 16 vardenafil, 19 tadalafil and one evaluated a combination of drugs. Methodological quality was poorly reported. All three drugs were highly significantly more effective than placebo with pooled weighted mean differences of 7.27 to 7.55 for IIEF score and pooled ORs of 10.26 to 10.56 for the GEQ. Statistical heterogeneity was significant for some drugs for one or both outcomes. One direct comparison between sildenafil and vardenafil, and indirect comparisons among all three drugs, found no significant differences between them. There was evidence of significant small study effects for tadalafil on IIEF score and for tadalafil and vardenafil on GEQ. But, adjusted estimates of effect using the trim and fill method did not differ substantially from the unadjusted ones. Estimates of effect decreased with time for all three drugs.

Authors' conclusions

All three drugs were highly effective for the treatment of ED with no differences between them identified.

CRD commentary

This review had clear inclusion criteria for participants, interventions and study designs. Inclusion criteria for outcomes were broad, but this appeared unlikely to have affected study selection or the findings of the review. The authors searched a wide range of sources without restrictions by language or publication status. Validity was assessed using appropriate criteria. Study selection was performed by one reviewer, increasing the risk of errors or bias at this stage, but checking was used to reduce these risks during validity assessment and data extraction. Relevant details of included studies were presented in tables. Studies were pooled by meta-analysis and possible sources of bias in the analyses were investigated extensively. The authors' conclusions were in line with the evidence presented and with previous reviews in the field, and were likely to be reliable.

The authors reported that they have been involved in a review of PDE-5 inhibitors funded by Pfizer Ltd.

Implications of the review for practice and research

Practice: The authors did not state any implications for practice.

Research: The authors stated that comprehensive assessment of bias should be performed routinely in systematic reviews. They also stated that future trials of PDE inhibitors should test adequacy of blinding of participants and should include all randomised patients in analyses.


Not stated.

Bibliographic details

Bekkering G E, Abou-Setta A M, Kleijnen J. The application of quantitative methods for identifying and exploring the presence of bias in systematic reviews: PDE-5 inhibitors for erectile dysfunction. International Journal of Impotence Research 2008; 20(3): 264-277. [PubMed: 18059501]

Indexing Status

Subject indexing assigned by NLM


Bias (Epidemiology); Carbolines /therapeutic use; Controlled Clinical Trials as Topic; Erectile Dysfunction /drug therapy; Humans; Imidazoles /therapeutic use; Male; Meta-Analysis as Topic; Phosphodiesterase Inhibitors /therapeutic use; Piperazines /therapeutic use; Purines /therapeutic use; Review Literature as Topic; Sulfones /therapeutic use; Treatment Outcome; Triazines /therapeutic use



Database entry date


Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.

CRD has determined that this article meets the DARE scientific quality criteria for a systematic review.

Copyright © 2014 University of York.

PMID: 18059501


PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...