• We are sorry, but NCBI web applications do not support your browser and may not function properly. More information
Logo of canfamphysLink to Publisher's site
Can Fam Physician. Jul 2007; 53(7): 1192–1197.
PMCID: PMC1949303

Language: English | French

Validation of self-reported history of colorectal cancer screening

Shariq Khoja, MD PhD
Assistant Professor in the Department of Community Health Sciences at the Aga Khan University in Karachi, Pakistan
S. Elizabeth McGregor, PhD
Epidemiologist in the Division of Population Health and Information at the Alberta Cancer Board in Calgary, Alta

Abstract

OBJECTIVE

This study aims to determine the validity of self-reported history of colorectal cancer (CRC) testing consisting of fecal occult blood tests (FOBTs)in the past 2 years or endoscopy (flexible sigmoidoscopy or colonoscopy) in the past 5 years by comparing it with reports provided by physicians.

DESIGN

A cross-sectional design was used for this study.

SETTING

Study participants were selected directly from the city’s population. Self-reported history of CRC testing was validated using records obtained from their physicians’ offices.

PARTICIPANTS

Participants were adults of 50 to 74 years, living within the boundaries of Calgary Health Region in Alberta.

INTERVENTIONS

Participants were recruited by a random-digit dial telephone survey of adults aged 50 to 74 years (n = 598). Following a phased process, a subset of these people (n = 200) agreed to provide names of their physicians to be contacted for their histories of CRC testing. Physicians’ reports were used to measure validity of self-reported history.

MAIN OUTCOME MEASURES

Agreement between self-reported history and physician’s records was measured using κ statistics and concordance. Validity of self-report was measured by calculating sensitivity, specificity, positive predictive values, and negative predictive values. Reasons for testing reported by the participants were compared with those reported by their physicians.

RESULTS

Complete information was received for 146 participants, revealing a 34.2% testing rate for CRC. Intermediate level of agreement for testing history (κ = 0.66 and concordance = 84.9%) was found between the 2 types of reporting for CRC testing. Self-reported history showed sensitivity of 76.0% (95% CI = 61.8%–86.9%) and specificity of 89.6% (95% CI = 81.7%–94.9%). High specificity was also observed for self-reporting of the individual tests, but low sensitivity was seen for the reporting of FOBT in the last 2 years. Most participants who correctly recalled the testing history also accurately identified the reason for testing (concordance = 80.0% for FOBT and 69.6% for endoscopy).

CONCLUSION

Self-reported history of CRC testing and physicians’ reports showed dependable agreement. Physicians need to probe their patients further for the history of FOBT. These results can be useful in clinical practice to determine the CRC screening status of the patients.

RÉSUMÉ

OBJECTIF

Déterminer la validité de ce que rapportent les patients concernant les tests de dépistage du cancer colorectal (CCR) qu’ils ont subis, soit par recherche du sang occulte dans les selles (RSOS) au cours des 2 dernières années ou par endoscopie (sigmoïdoscopie ou endoscopie flexible) au cours de 5 dernières années, en la comparant à ce qui est consigné dans les dossiers des médecins.

TYPE D’ÉTUDE

Étude transversale.

CONTEXTE

Les participants ont été choisis directement à partir de la population de la ville. L’histoire des examens de dépistage du CCR rapporté par les patients a été validée par confrontation avec les dossiers de leurs médecins.

PARTICIPANTS

Adultes âgés de 50 à 74 ans vivant dans les limites de la région sanitaire de Calgary, Alberta.

INTERVENTIONS

Des sujets de 50 à 74 ans ont été recrutés par 598 appels téléphoniques (numéros aléatoires) à l’aide d’appareils automatiques. Après certains arrangements, un sous-groupe de ces sujets (n = 200) ont consenti à donner le nom de leurs médecins en vue d’obtenir les tests de dépistage du CCR déjà effectués. Les rapports des médecins ont servi à vérifier l’exactitude de ce que rapportaient les patients.

PRINCIPAUX PARAMÈTRES MESURÉS

La concordance entre les informations rapportées par les patients et les dossiers des médecins a été mesurée en comparant lesdeux versions et en utilisant le test statistique κ. La validité des données rapportées par les patients a été mesurée en calculant la sensibilité, la spécificité ainsi que les valeurs prédictives positives et négatives. On a aussi comparé ce que les patients et les médecins rapportent comme raisons pour les tests.

RÉSULTATS

Les informations complètes de 146 participants ont révélé un taux de dépistage du CCR de 34,2%. Un niveau intermédiaire de concordance a été observé entre les deux sources d’information pour l’histoire du dépistage (κ = 0,66; concordance de 84,9%). L’histoire rapportée par les patients montrait une sensibilité de 76% (IC à 95% = 61,8–86,9%) et une spécificité de 89,6% (IC à 95% = 81,7–94,9%). On a aussi observé une spécificité élevée pour les différents tests rapportés par les patients, mais un faible niveau de sensibilité pour la RSOS au cours des 2 dernières années. La plupart des participants qui se souvenaient bien de l’histoire du dépistage avaient aussi identifié correctement les raisons pour les tests (concordance de 80,0% pour la RSOS et de 69,6% pour l’endoscopie).

CONCLUSION

On a constaté une bonne concordance entre ce que les patients rapportent du dépistage du CCR et les dossiers des médecins. Les médecins doivent toutefois questionner davantage leurs patients sur la RSOS déjà effectuée. Ces résultats peuvent être utiles en clinique pour déterminer l’état du dépistage du CCR chez les patients.

Self-report is the most common method of determining adherence to screening guidelines for various cancers. This method is widely used in clinical practice and research to determine whether patients have been screened for various cancers.1,2 Validity of self-report varies with type, frequency, and duration of screening tests.1,3 Such validity has been tested extensively for breast, cervical, and prostate cancers,1,36 which showed high sensitivity and agreement. Few studies have tested the validity of self-report in assessing adherence to colorectal cancer (CRC) screening guidelines,1,2,7 but all such studies have found patients to over-report the rates of screening tests. Only 1 study has reported on the validity of self-reported CRC screening in Canada’s publicly funded health care system, where the sample was drawn from the family members of CRC patients already enrolled with a provincial familial colon cancer registry.8 The current study assesses validity of self-reported screening and the reasons for testing in a general population sample.

Self-reported history provides useful information on compliance with screening guidelines and evaluating extent of screening programs.2 Studies on the validity of self-reported history not only provide useful information to physicians and planners of health care in making appropriate decisions, but also change the way physicians communicate the need, process, and results of screening tests to their patients.1

The Canadian Task Force on Preventive Health Care recommends annual or biennial screening with fecal occult blood tests (FOBTs) or flexible sigmoidoscopy for adults aged 50 and older.9 Other guidelines (such as those of the Canadian Association of Gastroenterology) recommend that all asymptomatic men and women over 50 years of age with no family history of CRC should undergo one of the following: FOBT every 2 years, flexible sigmoidoscopy alone or combined with FOBT every 5 years, double-contrast barium enema every 5 years or colonoscopy every 10 years.10

The objective of this study was to validate the self-reported screening history of CRC for adults aged 50 to 74 years, living in the Calgary Health Region in the province of Alberta.

METHODS

This validation study was embedded in a broader pilot test of a random-digit dial (RDD) telephone survey and an in-depth interview about CRC screening in Calgary Health Region. This health region has a total geographic area of 39 260 km2 and a total population of 1 122 521 living predominantly in urban areas (approximately 84.5%).11 A cross-sectional design was used to compare the self-reported CRC screening history with the records provided by the physicians. Data were collected in 3 steps. The first step consisted of RDD to recruit participants and obtain self-reports of CRC screening over the telephone. Participants were in the age group of 50 to 74 years, living within the boundaries of the Calgary Health Region. At the end of the RDD survey, participants were invited to take part in an in-depth interview conducted in person by trained interviewers. At the end of the in-depth interview, interviewers requested permission to contact participants’ physicians to obtain their histories of CRC testing. Consenting participants were asked to provide names and addresses of physicians they had visited within the past 6 years. Each identified physician was sent an explanatory letter, a copy of the signed consent form from the patient, and a form on which the physician could indicate the tests performed, reasons for testing, and the dates of the tests over the past 5 years. All the information was merged into a SAS12 data set for data analysis.

Agreement between self-reported history and physician’s records was measured using κ statistics and by checking concordance between the 2 reports.1,3 Validity of self-report was measured by comparing self-reported history with records provided by the physicians, and by calculating sensitivity, specificity, positive predictive values, and negative predictive values.1,3 Finally, the reasons for testing reported by the participants were compared with those reported by their physicians.

The information provided by the participants during the RDD survey about any kind of testing (ie, FOBT in the last 2 years or endoscopy—sigmoidoscopy or colonoscopy—in the last 5 years) was compared with the information provided by the physicians for these tests in the same period. Comparisons were also made separately for FOBT within the last 2 years and endoscopy within the last 5 years. Prevalence rates of CRC testing for the entire population of 598 were recalculated based on the sensitivity and specificity results calculated in the validation study. This study was approved by the Conjoint Health Research Ethics Board at the University of Calgary.

RESULTS

The RDD telephone survey was conducted between June and August 2003, with 598 people aged between 50 and 74 years (response rate = 48.0%). Among the people surveyed, 52.7% (n = 315) agreed to receive information regarding the in-depth interview. A total of 221 in-person interviews were completed, giving a response rate of 70.2%. Among the people interviewed, 90.5% (n = 200) agreed to participate in the validation study. In total, 333 physicians were contacted (median = 1 physician per participant; range 1 to 6), of whom 246 returned completed forms (response rate = 73.9%). This resulted in 146 patient records with responses from all physicians identified by the participants available for the validation study. The final sample was demographically very similar to the original sample of 598 participants, but the people in the final sample were slightly more likely to have a usual source of care and to visit their physicians for regular physical examinations (Table 1).

Table 1
Comparison of demographic characteristics between the overall and the final validation samples

Comparing self-reports with physicians’ records for any type of testing

According to physicians’ records, the overall prevalence of CRC testing (FOBT in the last 2 years or endoscopy in the last 5 years), was 34.2% (n = 50). Agreement on the reporting of CRC testing was found to be intermediate (κ = 0.66, 95% CI = 0.53–0.79, concordance = 84.9%). The overall sensitivity of participants’ responses, in comparison with physician records, was 76.0%, whereas the overall specificity was 89.6%. One reason for lower sensitivity could be “telescoping” (reporting a more recent date than the actual date for CRC screening tests). Telescoping was observed in 25% (3 in 12) of people who were classified as false positives for FOBT or endoscopy. Telescoping was also observed among 26% (10 in 38) of people who were classified as true positives because, despite reporting a more recent date for the test, both the self-reported date and the actual date fell within the set period of 2 years for FOBT and 5 years for endoscopy. Sensitivity and specificity for any CRC testing are shown in Table 2.

Table 2
Accuracy of self-reported colorectal cancer screening history compared with medical records

Comparing self-reports with physicians’ records for individual tests

According to physicians’ reports, the prevalence of FOBT in the past 2 years was 18.5% (n = 27), whereas the prevalence of endoscopy in the past 5 years was 20.6% (n = 30). Agreement between self-reports and physicians’ reports was tested for both FOBT in the last 2 years and endoscopy in the last 5 years (Table 2). Agreement on the reporting of FOBT was found to be low (κ = 0.47, 95% CI = 0.28–0.65, concordance = 84.2%). The reason for high concordance, despite low κ score, could be that most of the participants did not report any FOBT, which was confirmed by their physicians. The overall sensitivity of patients’ responses, in comparison with physician records, was 55.6%, whereas the overall specificity was 90.8%. Forty-four percent (12 in 27) of people reporting FOBTs were classified as false positives, resulting in low sensitivity for FOBT. False positives were more likely to be women (61.5% vs 28.6%, P = .08), and those with a high school education or less (53.8% vs 31.2%, P = .09). Reasons for the false positives were confusion with other CRC screening tests (6 in 12), unknown (5 in 12), or telescoping (1 in 12). Agreement on the reporting of endoscopy was found to be intermediate (κ = 0.74, 95% CI = 0.60–0.88, concordance=91.8%). The overall sensitivity of patients’ responses was 76.7%, whereas the overall specificity was 95.7%. Sensitivity and specificity for specific tests are shown in Table 2.

Measuring agreement on reasons for testing

Although limited by the low prevalence of CRC screening tests, we analyzed the agreement between reasons for testing reported by the participants and their physicians. These comparisons were made only for the participants who reported CRC screening that was validated by the physicians (true positives). Among the participants who correctly reported FOBT testing in the past 2 years, 80.0% (12 in 15) reported being tested for screening or because of positive family history. The self-reported reasons in this group also matched the reasons provided by the physicians 80.0% (12 in 15) of the time. Among those who correctly reported endoscopy testing in the past 5 years, 60.9% (14 in 23) of participants reported being tested for screening or because of positive family history. The self-reported reasons in this group matched the reasons provided by the physicians 73.9% (17 in 23) of the time. The reasons provided by the participants for any type of screening matched with the reasons provided by their physicians 76.3% (29 in 38) of the time.

DISCUSSION

Our comparison of patients’ self-reported history of CRC screening and physicians’ reports showed dependable agreement in the types of tests and concordance in the reasons for testing. However, assessment of validity showed low sensitivity and positive predictive values, which might be due to telescoping. This disparity can make it difficult for physicians and health care planners to decide future strategies at both the individual and the population levels. Our data could not find any specific demographic or other characteristics to distinguish people with high or low reliability for self-reporting of CRC testing, except that women and those with less education were more likely to provide false-positive information on having FOBT in the last 2 years. High concordance rates for the reason for conducting FOBT and colonoscopy also made self-reporting more reliable.

The major strength of this study is that the sample was drawn from the general population rather than from physicians’ offices or hospitals. A weakness of the study could be the low overall prevalence of CRC screening, which substantially reduced the number of people having particular tests and thus affected the power of the results. Another limitation could be the sex distribution of our sample, which differed from the population figures.11

The effectiveness of physicians’ records as the standard for evaluating self-reported information in this study could also be questioned. Physician records have been used as standard for validating self-reported histories in several studies in Canada13,14 and elsewhere.15 One study reported the accuracy of self-reported use of antidepressants among cancer patients by using physician records as standards.13 One more study from the Netherlands used physician records as a comparison to measure quality of cancer registry data.15 Another study, which compared patient and physician records for Papanicolaou smear histories, recommended that, if the physician response rate is high, physician records should be used as a standard to evaluate screening programs.14

These results strengthen our knowledge about the reliability of self-reporting of CRC screening. The results also indicate that the reliability of self-reporting for more invasive tests, such as sigmoidoscopy and colonoscopy, is higher than for FOBT, which is easier to confuse with other types of stool tests.3 The only study showing high sensitivity for FOBT (92.4%) was conducted by Mandelson, who actually did a 5-year follow-up study among women aged 50 to 79 years for FOBT only.4 This indicates that, by having a shorter follow-up time and focusing on more than 1 CRC screening test, patients might confuse both the timing and the types of tests.1,36 Thus, it is important that physicians explain the test clearly while asking about the history of FOBT, and probe further about the exact timings of tests. Believing patient information about FOBT, without probing it further, might result in longer than recommended intervals for CRC screening.

The results also highlight the need to maintain up-to-date records of CRC screening, and to make that information accessible to other practitioners through a medium such as electronic health records. Availability of such information might help family physicians make appropriate decisions about educating patients and recommending CRC screening tests to their patients, in accordance with the available guidelines.

Conclusion

This study shows that self-reported history for CRC screening is reliable for testing status and reasons for testing. Physicians need to explain the test and to probe further for information about FOBT. Results of this study can help physicians improve their patients’ adherence to CRC screening guidelines.

Acknowledgment

Alberta Cancer Foundation, Medical Services Incorporated Foundation, and Alberta Heritage Foundation for Medical Research provided funding for this study. Dr Hilsden is an Alberta Heritage Foundation for Medical Research Population Health Investigator.

Notes

EDITOR’S KEY POINTS

  • Few studies have tested the validity of self-report in assessing adherence to screening guidelines for colorectal cancer (CRC), though all such studies have found that patients over-report the rates of screening tests.
  • Results of the study indicate that the reliability of self-reporting for more invasive tests, such as sigmoidoscopy and colonoscopy, is higher than for fecal occult blood tests, which is easier to confuse with other types of stool tests. This highlights the need to maintain up-to-date records of CRC screening and to make that information accessible to other practitioners through a medium such as electronic health records.

POINTS DE REPÈRE DU RÉDACTEUR

  • Peu d’études ont vérifié la validité de ce que les patients rapportent lorsqu’on évalue la fidélité aux directives pour le dépistage du cancer colo-rectal (CCR), et pourtant, toutes ces études trouvaient que les patients rapportent un taux excessif de tests de dépistage.
  • Les résultats de cette étude indiquent que la fiabilité des rapports des patients pour les tests plus effractifs, tels la sigmoïdoscopie et la colonoscopie, est plus élevée que pour la recherche du sang occulte dans les selles, ce dernier test pouvant plus facilement être confondu avec d’autre types d’examens des selles. Cela souligne la nécessité de maintenir à jour les données de dépistage du CCR et de rendre cette information accessible aux autres intervenants via un médium tel que des dossiers de santé électroniques.

Footnotes

This article has been peer reviewed.

Contributors

Drs Khoja, McGregor, and Hilsden contributed to concept and design of the study; data gathering, analysis, and interpretation; and preparing the manuscript for submission.

Competing interests

None declared

References

1. Hiatt RA, Perez-Stable EJ, Quesenberry C, Jr, Sabogal F, Otero-Sabogal R, McPhee SJ. Agreement between self-reported early cancer detection practices and medical audits among Hispanic and non-Hispanic white health plan members in northern California. Prev Med. 1995;24:278–85. [PubMed]
2. Baier M, Calonge N, Cutter G, McClatchey M, Schoentgen S, Hines S, et al. Validity of self-reported colorectal cancer screening behavior. Cancer Epidemiol Biomarkers Prev. 2000;9:229–32. [PubMed]
3. Gordon NP, Hiatt RA, Lampert DI. Concordance of self-reported data and medical record audit for six cancer screening procedures. J Natl Cancer Inst. 1993;85(7):566–70. [PubMed]
4. Mandelson MT, LaCroix AZ, Anderson LA, Nadel MR, Lee NC. Comparison of self-reported fecal occult blood testing with automated laboratory records among older women in a health maintenance organization. Am J Epidemiol. 1999;150:617–21. [PubMed]
5. Bowman JA, Sanson-Fisher R, Redman S. The accuracy of self-reported Pap smear utilisation. Soc Sci Med. 1997;44(7):969–76. [PubMed]
6. McPhee SJ, Nguyen TT, Shema SJ, Nguyen B, Somkin C, Vo P, et al. Validation of recall of breast and cervical cancer screening by women in an ethnically diverse population. Prev Med. 2002;35:463–73. [PubMed]
7. Lipkus IM, Rimer BK, Lyna PR, Pradhan AA, Conaway M, Woods-Powell CT. Colorectal screening patterns and perceptions of risk among African-American users of a community health center. J Community Health. 1996;21(6):409–27. [PubMed]
8. Madlensky L, McLaughlin J, Goel V. A comparison of self-reported colorectal cancer screening with medical records. Cancer Epidemiol Biomarkers Prev. 2003;12:656–9. [PubMed]
9. Canadian Task Force on Preventive Health Care. Colorectal cancer screening. Recommendation statement from the Canadian Task Force on Preventive Health Care. CMAJ. 2001;165(2):206–8. [PMC free article] [PubMed]
10. Leddin D, Hunt R, Champion M, Cockeram A, Flook N, Gould M, et al. Guidelines on colon cancer screening. Can J Gastroenterol. 2004;18(2):93–9. [PubMed]
11. Calgary Health Region [website] Calgary, Alta: Calgary Regional Health Authority; 2004. [Accessed 2005 April 3]. Available from: www.calgaryhealthregion.ca.
12. SAS Institute Inc. SAS/STAT user’s guide. 4. Cary, North Carolina: SAS Institute Inc; 2001.
13. Cotterchio M, Kreiger N, Darlington G, Steingart A. Comparison of self-reported and physician-reported antidepressant medication use. Ann Epidemiol. 1999;9(5):283–9. [PubMed]
14. Schouten LJ, Jager JJ, van den Bandt PA. Quality of cancer registry data: a comparison of data provided by clinicians with those of registration personnel. Br J Cancer. 1993;68(5):974–7. [PMC free article] [PubMed]
15. Walter SD, Clarke EA, Hatcher J, Stitt LW. A comparison of physician and patient reports of Pap smear histories. J Clin Epidemiol. 1988;41(4):401–10. [PubMed]

Articles from Canadian Family Physician are provided here courtesy of College of Family Physicians of Canada
PubReader format: click here to try

Formats:

Related citations in PubMed

See reviews...See all...

Cited by other articles in PMC

See all...

Links

  • MedGen
    MedGen
    Related information in MedGen
  • PubMed
    PubMed
    PubMed citations for these articles

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...