Home > DARE Reviews > Effect of study design and quality on...

PubMed Health. A service of the National Library of Medicine, National Institutes of Health.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet]. York (UK): Centre for Reviews and Dissemination (UK); 1995-.

Database of Abstracts of Reviews of Effects (DARE): Quality-assessed Reviews [Internet].

Effect of study design and quality on unsatisfactory rates, cytology classifications, and accuracy in liquid-based versus conventional cervical cytology: a systematic review

E Davey, A Barratt, L Irwig, SF Chan, P Macaskill, P Mannes, and AM Saville.

Review published: 2006.

CRD summary

This review concluded that there was no evidence that liquid-based cytology reduced the proportion of unsatisfactory slides, or detected more high-grade lesions in high-quality studies, than conventional cytology. Liquid-based cytology did not perform better than conventional cytology; large randomised controlled trials are required. Despite some methodological limitations, this was generally a well-conducted review and the authors' conclusions are likely to be reliable.

Authors' objectives

To examine the relative performance of liquid-based cytology and conventional cytology in cervical screening, and to assess the effects of study design and quality on the results.

Searching

MEDLINE (1966 to January 2004) and EMBASE (1973 to March 2004) were searched for studies published in peer-reviewed journals; the search terms were stated. In addition, the authors searched the reference lists of published reviews and primary studies.

Study selection

Study designs of evaluations included in the review

Primary studies were eligible for inclusion; no further inclusion criteria for the study design were specified.

Specific interventions included in the review

Studies comparing liquid-based cytology with conventional cytology, with both techniques done by manual reading (not an automated screening system), were eligible for inclusion. Most of the included studies used ThinPrep; others used AutocytePrep, CytoRich, Cytoscreen and Papspin liquid-based cytology.

Reference standard test against which the new test was compared

No inclusion criteria regarding a reference standard were reported. The reference standards used in the included studies, where these existed, were cytology alone, colposcopy with or without histology, or a combination of cytology and histology.

Participants included in the review

No inclusion criteria for the participants were reported. Where reported, the included studies were set in referral clinics, screening settings, or mixed screening and clinic settings.

Outcomes assessed in the review

The outcomes included in the review were the proportion of unsatisfactory slides; the proportions of slides classified as normal, atypical squamous cells of undetermined significance (ASCUS), low-grade squamous intraepithelial lesion (LSIL), or high-grade squamous intraepithelial lesion (HSIL); and the accuracy for detecting reference standard HSIL.

How were decisions on the relevance of primary studies made?

Two reviewers assessed studies for inclusion and any disagreements were resolved by consensus.

Assessment of study quality

The quality of the included studies was assessed according to study design (independent sample or paired-sample), study setting (screening, referral clinic, mixed, unspecified), randomisation, whether a reference standard had been used, type of reference standard, method of treatment allocation, masked reading of slides, and masked (to test) application of the reference standard. Applicability and reproducibility of the study were also assessed. Studies were then classed as ideal, high quality, medium quality, or low quality.

Two reviewers, who were blind to the study results, independently appraised the quality of the studies. Any disagreements were resolved by consensus and, where necessary, a third reviewer was consulted. When required, authors were contacted to verify details of masking.

Data extraction

Two reviewers extracted the data for the review. Any disagreements were resolved by consensus and, where necessary, a third reviewer was consulted. Where required, authors were contacted for confirmation of details of tabulated data. For each study, the percentage of slides classified as unsatisfactory by conventional cytology was subtracted from the percentage of slides classified as unsatisfactory by liquid-based cytology. Also, the percentages of slides classified as normal, ASCUS, LSIL, and HSIL by conventional cytology and by liquid-based cytology were obtained, and the differences in percentages calculated for each classification.

Methods of synthesis

How were the studies combined?

Studies of medium and high quality that presented sufficient data to calculate the sensitivity and specificity of liquid-based cytology and conventional cytology to detect high-grade reference standard disease were plotted in receiver operating characteristic (ROC) space using test thresholds of ASCUS and LSIL. Differences in the percentage of slides classified as unsatisfactory by conventional cytology and by liquid-based cytology were combined using the random-effects method of DerSimonian and Laird, and 95% confidence intervals (CIs) were calculated.

Studies of medium and high quality that allowed the calculation of sensitivity and specificity were combined in a narrative.

How were differences between studies investigated?

Differences in the percentage of slides classified as unsatisfactory by conventional cytology and by liquid-based cytology were displayed in a forest plot, and the chi-squared statistic was used to assess statistical heterogeneity between studies.

The studies were stratified by study quality, setting, design (paired or independent) and liquid-based cytology proprietary name (both individually and grouped as either ThinPrep or AutocytePrep and CytoRich combined), and were investigated using random-effects meta-regression. Non-parametric methods (Mann-Whitney or Kruskal-Wallis tests) were also used.

Results of the review

Fifty-six studies (58 datasets) were included in the review (nreported as over 1.25 million slides). One study was a randomised controlled trial (RCT); the design of the other included studies was not reported.

No studies were rated as ideal; 5 were rated as high quality, 32 as medium quality and 19 as low quality.

The summary estimate of the differences in percentages of unsatisfactory slides (liquid-based cytology minus conventional cytology) was not statistically significant (-0.14%, 95% CI: -0.33, 0.06). Statistically significant heterogeneity was detected (p<0.0001). The median of the differences across all studies was -0.17% (interquartile range: -0.98 to 0.37). There were no significant differences in results according to study quality, design or setting. The meta-regression results were consistent with these findings; although evidence of an association between setting and results was found, the differences were not clinically significant. There was a significant difference according to liquid-based cytology proprietary name when examined individually (p=0.01) and when grouped according to manufacturer (p=0.003). However, the differences were small (ThinPrep 0.12%, AutocytePrep or CytoRich -1.08%).

Classification of HSIL varied with study quality (p=0.04), with conventional cytology classifying more slides as HSIL than liquid-based cytology in high-quality studies only. Liquid-based cytology classified more slides as LSIL than conventional cytology in all quality categories. Overall, conventional cytology classified more slides as ASCUS than liquid-based cytology, but the results varied with study quality: in studies of medium and high quality, liquid-based cytology classified more slides as ASCUS than did conventional cytology (p=0.05); in low-quality studies, conventional cytology classified more slides as ASCUS than liquid-based cytology. Differences between liquid-based and conventional cytology classifications of normal, ASCUS and LSIL slides did not vary significantly between paired and independent studies. Liquid-based cytology classified more slides as HSIL than conventional cytology in both study designs, but the difference was greater in independent than in paired studies (p=0.03). There was no difference in results according to setting. The results of the meta-regression were consistent with these findings for quality, study design and setting. No significant difference was found in any cytological classification with different liquid-based cytology proprietary names (individually or grouped).

Based on ROC plots using seven data sets, there was no evidence that liquid-based cytology improved accuracy compared with conventional cytology in the detection of high-grade disease.

Authors' conclusions

There was no evidence that liquid-based cytology reduced the proportion of unsatisfactory slides, or detected more high-grade lesions in high-quality studies, than conventional cytology. The review did not suggest that liquid-based cytology performed better than conventional cytology; large RCTs are required.

CRD commentary

The authors set out a clear objective at the beginning of the review, but inclusion criteria were only defined for the interventions and outcomes. Appropriate sources were searched, but no attempts to identify unpublished studies appear to have been made and publication bias was not assessed. Methods were used to help reduce the risk of error and bias in the study selection, quality assessment and data extraction processes. Validity was assessed using appropriate criteria.

Details of the individual studies were not presented, which made it difficult to assess the validity of statistically combining the studies and the generalisability of the results. However, statistical heterogeneity was assessed and the potential effects of several variables on the results were investigated. Despite some limitations, this was generally a well-conducted review and the authors' conclusions are likely to be reliable.

Implications of the review for practice and research

Practice: The authors stated that although liquid-based cytology was not more accurate than conventional cytology, equivalent performance might be sufficient if liquid-based cytology has other advantages, such as the opportunity for concurrent human papillomavirus DNA testing, a reduction in reading times, or if it is more economical than conventional cytology.

Research: The authors stated that there is a need for large RCTs that incorporate colposcopy and biopsy of women who have positive results on either conventional or liquid-based cytology, and histology should be read blind to cytology results as a reference standard. Ideally, colposcopy and biopsy should also be blind to cytology results. These large-scale trials should be integrated into routine services.

Funding

National Health and Medical Research Council, Australia, grant number 211205.

Bibliographic details

Davey E, Barratt A, Irwig L, Chan S F, Macaskill P, Mannes P, Saville A M. Effect of study design and quality on unsatisfactory rates, cytology classifications, and accuracy in liquid-based versus conventional cervical cytology: a systematic review. Lancet 2006; 367: 122-132. [PubMed: 16413876]

Indexing Status

Subject indexing assigned by NLM

MeSH

Cytological Techniques /methods; Female; Humans; Quality Assurance, Health Care /methods /standards; Sensitivity and Specificity; Uterine Cervical Neoplasms /classification /pathology

AccessionNumber

12006000440

Database entry date

30/04/2007

Record Status

This is a critical abstract of a systematic review that meets the criteria for inclusion on DARE. Each critical abstract contains a brief summary of the review methods, results and conclusions followed by a detailed critical assessment on the reliability of the review and the conclusions drawn.

CRD has determined that this article meets the DARE scientific quality criteria for a systematic review.

Copyright © 2014 University of York.

PMID: 16413876

PubMed Health Blog...

read all...

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...