Evidence Table 4, KQ5: Effective approaches for monitoring use and quality of colorectal cancer screening - Enhancing the Use and Quality of Colorectal Cancer Screening

Evidence Table 4KQ5: Effective approaches for monitoring use and quality of colorectal cancer screening

STUDY: The use of screening colonoscopy for patients cared for by the Department of Veterans Affairs	Authors, ref ID: El-Serag, Peterson, Hampel, Richardson, Cooper ¹⁰⁰ Year of publication: 2006 Dates of data collection: 1998 – 2003 Trial name: NA

OBJECTIVE OR AIM:	To investigate whether colonoscopy use increased disproportionately in the VA system and changes in rates of FS, DCBE, and FOBT use between October 1, 1998, and September 30, 2003.

DESIGN:	Setting: United States, VA medical centers Study design: cross-sectional Duration (mean followup): NA Overall study size (N enrolled/N analyzed):

Sample size:	NA
Describe intervention: NA

RECRUITMENT: (population-based, clinic-based, volunteer, other)	Population based- from VA databases

INCLUSION CRITERIA:	Procedures for VA users aged 49 to 75 years.

EXCLUSION CRITERIA:	NR

POPULATION CHARACTERISTICS:	Feature			Screening c-scope (n=178,853)		FOBT (n=1,635,364)				DCBE (n=78,830)				FS (n=217,327)
Mean age & range (years):
Colonoscopy: 62.3 (7.6) years	Race
FOBT: 63.9 (7.8)	White			105,746 (59.1)		815,582 (49.9)				45,894 (58.2)				118,789 (54.7)
DCBE: 62.6 (7.8)	Black			17,934 (10.0)		133,822 (8.2)				10,458 (13.3)				20,358 (9.4)
FS: 61/8 (7.7)	Other			55,173 (30.8)		685,960 (41.9)				22,478 (28.5)				78,180 (36.0)
Sex (% female):	Sex
	Male			174,356 (97.5(		1,583,775 (96.8)				76,427 (97.0)				211,824 (97.5)
	Female			4,497 (2.5)		51,589 (3.2)				2,403 (3.0)				5,503 (2.5)
Race:
Other:	The mean (SD) age of those undergoing screening colonoscopy was 62.3 (7.6) years; FOBT, 63.9 (7.8) years; DCBE, 62.6 (7.8) years; and FS, 61.8 (7.7) years.

Attrition/Drop-out (not available for endpoint measurement):	NR
Adherence:
Contamination:
Response Rates (e.g. for surveys):

STATISTICAL ANALYSES:	The frequencies and proportions of the 4 types of CRC screening procedures, and of unique individuals undergoing these procedures, were calculated for each fiscal year. The temporal changes and potential determinants (age, sex, and race) of screening colonoscopy (vs other CRC screening tests) were examined in unadjusted and adjusted logistic regression analyses. Similar calculations were performed for screening colonoscopy (vs other colonoscopy). Statistical comparisons of these proportions were not performed because of overlapping groups. The authors used the predictive values to perform a sensitivity analysis of the calculated proportions for screening colonoscopy.

ASSESSMENT OF EXPOSURES AND POTENTIAL CONFOUNDERS:	National inpatient and outpatient VA databases used to assess potential confounders/covariates

OUTCOME ASSESSMENT:	National inpatient and outpatient VA databases were searched for codes indicative of colonoscopy, FS, FOBT, and DCBE recorded during fiscal years 1998 to 2003. The authors also used the VA Patient Treatment File, which contains hospital discharge records and up to 10 diagnostic codes, 5 operating room procedures, and 32 nonoperating room procedures coded according to the International Classification of Diseases, Ninth Revision, Clinical Modification. The indications for CRC screening tests were classified as screening, diagnosis, or surveillance based on the predefined Algorithm. All FOBT procedures were designated as screening procedures. Flexible sigmoidoscopy and DCBE were considered diagnostic in the presence of specific conditions recorded within the year before the date of the procedure. They were considered to be surveillance procedures in the presence of a second set of prespecified conditions (coded as 17–28). The remaining procedures were considered to be screening procedures Colonoscopy was considered for CRC screening in the absence of conditions associated with diagnostic or surveillance indications and if no colonoscopy had been performed within the past 4 years. Because of concerns of the accuracy of diagnosis and procedure codes for specifying procedural indications, the authors also conducted a medical record review study in a subset of colonoscopic procedures nested within the main study cohort to validate and refine the algorithm that was used. A review of procedure, pathology, and progress notes was performed by 2 boardcertified gastroenterologist investigators who were blinded to the designated status based on the VA administrative data sets. They categorized indications for procedures as screening, surveillance, or diagnostic. A total of 303 medical records of unique patients with colonoscopy performed at the Michael E. DeBakey VA Medical Center, between October 6, 1999, and September 30, 2003, were identified at random using a computer generated algorithm, and reviewed from the national databases (ie, a subset from the main study cohort). Agreement between the 2 reviewers was achieved in 92.0% of cases, and differences were resolved by discussion. The predictive values of the database algorithm for identifying screening colonoscopy (compared with the medical record as a gold standard) were calculated.

RESULTS:

KQ2 - What factors influence the use of colorectal cancer screening?	NA

KQ3 - Which strategies are effective in increasing the appropriate use of colorectal cancer screening and followup?	NA

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	The authors calculated that the algorithm has approximately 70.1% sensitivity and 71.6% specificity to define screening colonoscopy. The findings of medical record review were then applied in a sensitivity analysis to recalculate the estimated annual frequency of screening colonoscopy. Apart from reducing the total number of screening colonoscopies by up to 25%, changing the definition of screening colonoscopy had little effect on the observed trends. Concordance or kappa not calculated

QUALITY RATING:	Fair

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?														NA

Were the drop-out or response rates acceptable (≤ 20%)? [If between 20% and 60%, check other and explain.]						X

Were the differential drop-out or response rates acceptable (≤ 15%)?						X

Were intervention/exposure measures valid, reliable, and equally applied?						X

Were the outcome assessors blinded to the intervention or exposure status of subjects?														NA

Were outcome measures valid, reliable, and equally applied?						x								Algorithm used to differentiate screening from diagnostic procedures, verified through dual review (see statistical methods section of abstraction) Concordance/kappa not calculated

Does the analysis control for baseline differences?						x								Logistic regression models used

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?						x

Were the statistical methods used to assess the abstracted outcomes appropriate?						x

Quality Rating (Good, Fair, or Poor): Fair

STUDY:	Authors, ref ID: Fiscella¹⁰¹ Year of publication: 2006 Dates of data collection: 1998–2002 Trial name: NR

OBJECTIVE OR AIM:	To determine whether estimates of racial and racial disparities in receipt of six different types of largely preventive procedures differ between self-report and Medicare claims data

DESIGN:	Setting: Medicare claims data Study design: observational Duration (mean followup): 2 to 4 years Overall study size (N enrolled/N analyzed): 88509 # observations (n = 1474 for colorectal testing)

Sample size:	1474
Describe intervention:	Includes fecal occult blood testing, sigmoidoscopy, or colonoscopy

RECRUITMENT: (population-based, clinic-based, volunteer, other)	Population

INCLUSION CRITERIA:	Medicare Beneficiaries 65 and older who participated in the Medicare Current Beneficiary Survey, 1999–2002.

EXCLUSION CRITERIA:	Participated in facility interviews (i.e., resided in long-term care facilities), were less than 65 years of age (i.e., were Medicare recipients due to having a qualifying disability), reported race/ethnicity other than Hispanic, non- Hispanic African American, or non-Hispanic White, i.e. majority, were enrolled in a Medicare HMO, or were not eligible for Medicare B (or Medicare A and B) coverage dropped due to incomplete claims.

POPULATION CHARACTERISTICS:	Overall
Mean age & range (years):	65–69 16.7% 70–74 35.1% 75–79 26.7% 80–84 14.3% 85+ 7.2%
Sex (% female):	56% female
Race:	Hispanic, African American 8.8% Majority 91.2%
Other:

Attrition/Drop-out (not available for endpoint measurement):	NA
Adherence:
Contamination:
Response Rates (e.g. for surveys):

STATISTICAL ANALYSES:	Describe: used SAS Survey and logistic regression to assess the relationship, adjusted, between minority status and self report

ASSESSMENT OF EXPOSURES AND POTENTIAL CONFOUNDERS:	Predictors of self report

OUTCOME ASSESSMENT:	Outcome Measures: • Prevalence of receipt of FOBT, FS, or colonoscopy as measured by: • Self-report in the MCBS of having any of the tests in the last year (MCBS) (indication was not specified) • Medicare claims, including both screening and diagnostic codes (administrative data)

RESULTS:

KQ2 - What factors influence the use of colorectal cancer screening?	NA

KQ3 - Which strategies are effective in increasing the appropriate use of colorectal cancer screening and followup?	NA

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	Outcomes: Unadjusted prevalence of CRC screening White: Survey 38.0 Administrative 30.1 Minority: Survey 34.8 Administrative 20.4 Concordance between self-report and administrative data (measured by kappa score) for CRC screening White 0.37 Minority 0.19 The authors also calculated an odds ratios for reporting a procedure in the absence of a claim, or vice versa. Minorities were more likely to report receipt of CRC screening in the absence of a claim (OR=1.92, 95% CI, 1.32‐ 2.79), with little change after adjustment for age, gender, income, educational level, health status, proxy response, and supplemental insurance. Having a claim for CRC testing in the absence of self-report did not differ by race or ethnicity.

QUALITY RATING:	Good

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?														NA

Were the drop-out or response rates acceptable (≤ 20%)? [If between 20% and 60%, check other and explain.]														Response rate for MCBS not reported. Possible that non-respondents differ from respondents in terms of associations between self report and administrative claims.

Were the differential drop-out or response rates acceptable (≤ 15%)?

Were intervention/exposure measures valid, reliable, and equally applied?														NA

Were the outcome assessors blinded to the intervention or exposure status of subjects?														NA

Were outcome measures valid, reliable, and equally applied?						x

Does the analysis control for baseline differences?						x

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?						x

Were the statistical methods used to assess the abstracted outcomes appropriate?						x

Quality Rating (Good, Fair, or Poor): Good

STUDY:	Authors, ref ID: Hall et al.¹⁰² Year of publication: 2004 Dates of data collection: December 1999 – June 2001 Trial name: NA

OBJECTIVE OR AIM:	To examine the accuracy of self-reports of prostate and colorectal cancer testing and reasons for testing using questions included in national surveys, the authors conducted a survey and reviewed the medical records for procedures performed among members of three health maintenance organizations (HMOs)

DESIGN:	Setting: member lists of Kaiser Permanente-Northern California (NC), Kaiser Permanente- Georgia (GA), and HealthPartners (HP), Minnesota Study design: cross-sectional Duration (mean follow-up): NA Overall study size (N enrolled/N analyzed): 3546

	Black men							White and other men						Women
Sample size:	363							847						920
Describe intervention

RECRUITMENT: (population-based, clinic-based, volunteer, other)	H.M.O. population based

INCLUSION CRITERIA:	Men aged 45 years and older and women aged 55 years and older as of September 1, 1999 who had been enrolled in the plan for at least 5 years were eligible for selection. Simple random sample for women. For men, an SRS for men at one site. At 2 sites, separate sample of black and white men recruited until all strata were filled. Also at one site, sample was drawn from members who had participated in a past survey.

EXCLUSION CRITERIA:	Screened out (statrum filled); language barrier, dead, out of network, phone disconnected, other

POPULATION CHARACTERISTICS:	Characteristics of study sample from 3 HMO’s
Mean age & range (years): Sex (% female):			Black men (n=363)					White and other men (n = 847)				Women (n = 920)
	Age (y)		%			(n)		%		(n)		%		(n)
	45 – 49		19.8			(72)		14.3		(121)		0.0		0
	50 – 59		41.0			149		38.6		327		23.3		(214)
	60 – 69		19.3			(70)		23.4		(198)		33.8		(311)
	70+		19.8			(72)		23.7		(201)		42.9		(394)
Race:	Ethnicity
	Hispanic		0.6			(2)		3.5		(30)		3.2		(29)
	Non-Hispanic		99.4			(361)		96.5		(816)		96.8		(888)
Other:	Education
	< HS		10.2			(37)		6.3		(53)		9.9		(91)
	HS/GED		22.6			(82)		17.9		(152)		27.1		(249)
	Some college/tech school		38.8			(141)		31.6		(268)		35.7		(328)
	College grad		28.4			(103)		44.2		(374)		27.2		(250)
	Marital status
	Married		78.0			(283)		82.4		(698)		55.8		(512)
	Unmarried		22.0			(80)		17.6		(149)		44.2		(406)
	Employment
	Employed		62.9			(227)		59.2		(499)		31.5		(288)
	Unemployed		33.8			(122)		39.3		(331)		62.0		(566)
	Retired/other		3.3			(12)		1.5		(13)		6.5		(59)
	Income
	<= $20,000		13.6			(46)		9.4		(72)		29.5		(236)
	$20,000 – $40,000		24.2			(82)		16.9		(130)		30.0		(240)
	$40,001 – $60,000		23.9			(81)		25.8		(199)		21.4		(171)
	> $60,000		38.3			(130)		47.9		(369)		19.0		(152)
	Health status
	Excellent/good		82.3			(298)		87.6		(741)		82.1		(754)
	Fair/poor		17.7			(64)		12.4		(105)		17.9		(164)
	Smoking status
	Current		17.9			(65)		10.0		(85)		7.5		(69)
	Former		46.6			(169)		49.9		(423)		36.2		(333)
	Never		35.5			(129)		40.0		(339)		56.3		(518)

Attrition/Drop-out (not available for Endpoint Measurement):	Response Rates (for survey): 4,833 members contacted 3,546 eligible 1,248 refused 2,298 completed
Adherence:	Response rate = 64.8% among those contacted and eligible
Contamination:	1181 not contacted (no answer, busy, answering service, exceeded call limit)

STATISTICAL ANALYSES:	The authors analyzed the data according to our sampling frame, that is, for all women, black men, and white and other men separately. Few participants were of races other than white or black (67 [3.1%] were of other races and 83 [3.9%] were multiracial). Therefore, non- Black men of other races and multiracial men were grouped together with white men. At HP, few men were black, and no statistics are presented for them. Participants with a history of prostate or colorectal cancer were excluded from the analyses. The final sample sizes were 363 black men, 847 white and other men, and 920 women. Respondent characteristics were summarized as frequencies and percentages. The authors calculated the percentages of respondents who reported having had their most recent DRE, PSA, or FOBT within 2 years before interview and sigmoidoscopy or colonoscopy within 5 years before interview. The percentages of respondents who had tests as determined by the medical record audit were similarly calculated; for FOBT, the authors counted only home kits to match with the survey question. On the basis of these time frames, the authors calculated the concordance (agreement) between the self-report and medical record audit for the tests. The authors also calculated agreement for endoscopy by combining sigmoidoscopy and colonoscopy. The authors calculated the kappa statistic by the method of Landis and Koch, which accounts for agreement expected by chance. Excellent agreement is defined as a kappa statistic greater than 0.75, fair to good agreement as 0.40 to 0.75, and poor agreement as less than 0.40. The authors also calculated the sensitivity and specificity of self-reports using the information from the medical records as the standard. Differences in sensitivity and specificity between health plans were determined with a pairwise test of proportions. The authors assessed the relationships between agreement and demographic characteristics (age, sex-race groups, and education) with polytomous regression models. The reference was agreement between self-reports and medical records and was compared to over- and underreporting. The authors also assessed the relationship with income, but because of the relatively large number of missing observations and no significant associations in any of the models, it was not included in the final models. All analyses were adjusted for study site. Further, the authors determined the reasons for testing and the concordance between the reasons reported by participants and the reasons recorded in the medical records. Tests were defined as screening tests when the reason for testing was screening, family history, or “part of exam/ doctor just did it.” Tests were defined as being performed for “other reasons” when the reasons were symptoms, problems, follow-up of an abnormal test, or other reasons. Finally, the authors evaluated whether and how frequently participants confused sigmoidoscopy and colonoscopy.

ASSESSMENT OF EXPOSURES AND POTENTIAL CONFOUNDERS:	Computer assited telephone interviews (CATI): The survey instrument elicited information on demographic and health characteristics, including a personal or family history of prostate or colorectal cancer, whether they had ever been tested for prostate or colorectal cancer and, if so, when they had the most recent tests (see Appendix for the questions asked). In addition, participants were asked the reasons for testing and the test results. The initial questions on prostate and colorectal cancer testing were based on the questions proposed for the Year 2000 National Health Interview Survey Cancer Control Supplement and Behavioral Risk Factor Surveillance System. The questionnaire is available from the authors upon request. Medical record reviews: Medical records were reviewed to determine whether any of the cancer tests included in the survey had been recorded within 5 years before the interview date. In addition, the dates and results of the tests were obtained. For each test recorded in the medical records, the authors ascertained the reason for or symptoms associated with the test by reviewing the records for up to 6 months before the test date but no more than 5 years before the survey (index) date. At each site, each page of all relevant medical records in the study time period was physically reviewed by a trained medical record analyst. Each provider note, all laboratory, radiology, endoscopy, and pharmacy information were abstracted using a standardized medical record abstraction form. Quality control procedures included a review of all abstraction forms for missing data and ambiguous responses and duplicate abstraction for 10% of the records. Data entry for all forms was conducted at Kaiser Permanente- NC with double key entry. Programs for internal consistency and validity were used to identify and correct errors of coding and data entry.

OUTCOME ASSESSMENT:	(see above)

RESULTS:

KQ2 - What factors influence the use of colorectal cancer screening?	NA

KQ3 - Which strategies are effective in increasing the appropriate use of colorectal cancer screening and follow-up?	NA

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	Percentage of respondents who received prostate or colorectal cancer tests according to self- report and medical record audits, 1999 - 2001
		Black men (n=363)					White + other men (n=847)				Women (n = 920)
		%			(n)		%		(n)		%		(n)
	FOBT*
	Survey	22.2			(79)		20.3		(169)		25.9		(234)
	Med recs	11.6			(42)		9.5		(80)		14.1		(129)
	Sig’oscpy**
	Survey	38.4			(138)		42.0		(352)		50.0		(447)
	Med rec	29.6			(107)		30.6		(258)		34.1		(313)
	C-scope**
	Survey	13.7			(49)		14.6		(121)		15.7		(140)
	Med rec	8.1			(29)		11.1		(94)		9.6		(88)
	End’scopy**
	Survey	44.4			(159)		49.9		(415)		58.6		(523)
	Med rec	34.4			(124)		37.8		(319)		39.8		(365)
	Note: endoscopy indicates sigmoidoscopy OR colonoscopy * test within past 2 years ** test within past 5 years CRC screening information: agreement between self-reports and medical record audits, 1999 - 2001
	procedure	Black men						White and other men					Women
			sens		spec	conc	K	n	sens	spec	conc	K	n	sens	spec	conc	K
	Kaiser Permanente-Georgia
	Fobt	187	0.89		0.86	0.86	0.57	293	0.90	0.85	0.86	0.56	310	0.89	0.82	0.84	0.62
	Sig	188	0.84		0.81	0.82	0.56	294	0.95	0.86	0.89	0.77	310	0.87	0.77	0.81	0.61
	Col	185	0.56		0.93	0.90	0.43	292	0.73	0.95	0.91	0.69	309	0.81	0.93	0.91	0.68
	End	186	0.87		0.83	0.84	0.64	294	0.96	0.88	0.92	0.83	311	0.95	0.73	0.83	0.67
	Kaiser Permanente – northern California
	Fobt	165	0.73		0.86	0.85	0.29	271	0.72	0.86	0.85	0.32	304	0.79	0.86	0.86	0.35
	Sig	166	0.83		0.77	0.80	0.58	271	0.80	0.77	0.78	0.53	301	0.83	0.76	0.78	0.55
	Col	165	0.67		0.88	0.87	0.36	270	0.80	0.93	0.92	0.60	300	0.95	0.94	0.94	0.65
	end	165	0.93		0.76	0.82	0.65	269	0.91	0.75	0.81	0.62	299	0.91	0.76	0.82	0.63
	HealthPartners, Minnesota
	Fobt	4	NA		NA	NA	NA	269	0.59	0.87	0.84	0.31	287	0.55	0.81	0.78	0.23
	Sig	4	NA		NA	NA	NA	270	0.77	0.67	0.70	0.38	280	0.86	0.53	0.63	0.31
	Col	4	NA		NA	NA	NA	267	0.69	0.90	0.89	0.38	280	0.73	0.87	0.86	0/30
	end	4	NA		NA	NA	NA	267	0.85	0.64	0.71	0.43	279	0.93	0.45	0.61	0.30
	Fobt = fecal occult blood test Sig = flexible sigmoidoscopy Col = colonoscopy End = endoscopy Predictors of agreement between self-reports and medical record audits of crc screening tests, generalized logits models, 1999 – 2001
	Predictor		Outcome			FOBT				Sig				Col
						OR		95% CI		OR		95% CI		OR		95% CI
	Age (y)
	50 – 59		Over			1.69		0.91, 3.13		2.13		1.20, 3.80		6.81		1.62, 28.58
			Under			1.22		0.34, 4.41		3.98		1.20, 13.19		6.86		0.90, 52.15
	60 – 69		Over			2.8		1.50, 5.25		3.07		1.70, 5.54		9.29		2.19, 39.42
			Under			1.9		0.51, 7.05		3.41		0.99, 11.72		9.95		1.27, 77.94
	70+		Over			2.78		1.46, 5.29		2.85		1.56, 5.19		14.36		3.38, 61.06
			Under			2.47		0.66, 9.31		4.06		1.18, 13.93		13.91		1.72, 111.45
	Sex/race
	White/other men		Over			0.92		0.62, 1.37		0.75		0.52, 1.10		0.51		0.31, 0.83
			Under			0.74		0.29, 1.90		0.82		0.45, 1.5		0.86		0.40, 1.85
	women		Over			0.9		0.60, 1.34		0.91		0.62, 1.32		0.44		0.27, 0.73
			Under			0.88		0.34, 2.24		0.79		0.43, 1.45		0.34		0.14, 0.81
	Education
	HS/GED		Over			0.87		0.54, 1.4		1.48		0.91, 2.39		1.17		0.64, 2.16
			Under			1.74		0.49, 6.19		1.42		0.63, 3.24		0.93		0.32, 2.73
	Some college / tech school		Over			0.73		0.46, 1.16		1.15		0.72, 1.84		0.84		0.46, 1.55
			Under			1.55		0.45, 5.40		1.07		0.48, 2.40		0.92		0.33, 2.56
	College grad		Over			0.85		0.54, 1.36		1.24		0.77, 2.00		0.77		0.41, 1.45
			Under			1.6		0.45, 5.70		1.09		0.48, 2.48		0.73		0.25, 2.10
	^aReference levels for predictors are: age = 40– 49; sex/race group = black men; education = < high school; all analyses are adjusted for site. ^bReference level for outcome variable is “agreement” between the survey and medical record. Over = overreport Under = underreport

QUALITY RATING:	Good

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?														NA

Were the drop-out or response rates acceptable (≤ 20%)? [If between 20% and 60%, check other and explain.]						X								Cooperation rate 68% Some concern about exclusion criteria (language barrier)

Were the differential drop-out or response rates acceptable (≤ 15%)?														NA

Were intervention/exposure measures valid, reliable, and equally applied?						X

Were the outcome assessors blinded to the intervention or exposure status of subjects?						X

Were outcome measures valid, reliable, and equally applied?						X

Does the analysis control for baseline differences?						X

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?						X

Were the statistical methods used to assess the abstracted outcomes appropriate?						X

Quality Rating (Good, Fair, or Poor): Good

STUDY:	Authors, ref ID: Haque et al.¹⁰³ Year of publication: 2005 Dates of data collection: Trial name: NA

OBJECTIVE OR AIM:	The goal of this study was to develop an automated data algorithm designed to distinguish screening and diagnostic endoscopy (sigmoidoscopy and colonoscopy) exams. The authors assessed the algorithm’s ability to correctly classify the exams using paper medical records as the “gold standard.”

DESIGN:	Setting: Kaiser Permanente Southern California (KPSC) cares for approximately 3 million members, of whom 13% are older than 50 years and targeted for colorectal cancer screening. Automated data tracks outpatient and inpatient care received. Study design: cross-sectional Duration (mean follow-up): Overall study size (N enrolled/N analyzed): Stratified random sample of 220 medical records reviewed.

	Had Colonoscopies									Had Sigmoidoscopies_110
Sample size:	110
Describe intervention:

RECRUITMENT: (population-based, clinic-based, volunteer, other)	HMO based (Kaiser Permanente)

INCLUSION CRITERIA:	Participants included all health plan members between the ages 50 and 70 years, who were continuously enrolled from 1998 to 2002, and completed an endoscopy during those years. Stratified random sample based on the algorithm’s classification. 110 FS, 30 classified as diagnostic and 80 as screening, and 110 COLON, 30 diagnostic and 80 screening.

EXCLUSION CRITERIA:	Participants with a history of colorectal cancer were excluded (N = 1972). Of the 220, 32 excluded due to mismatches in participants endoscopy dates.

POPULATION CHARACTERISTICS:	NA
Mean age & range (years):
Sex (% female):
Race:
Other:

Attrition/Drop-out (not available for endpoint measurement):	NA
Adherence:
Contamination:
Response Rates (e.g. for surveys):

STATISTICAL ANALYSES:	The authors conducted cross-tabulations between the algorithm and medical review classification to examine the sensitivity, specificity, and κ. The classification after medical record review was considered the gold standard. Sensitivity indicates the probability that a diagnostic endoscopy was classified as such by medical records review. Specificity indicates the probability that a nondiagnostic endoscopy was classified as screening. The κ indicates the overall agreement between the two sources.

ASSESSMENT OF EXPOSURES AND POTENTIAL CONFOUNDERS:	NA

OUTCOME ASSESSMENT:	Endoscopies were identified using International Classification of Disease (ICD 9 CM) and Current Procedural Technology-4 codes. In instances in which a participant completed multiple endoscopies in the 5-year period, the authors retrieved data for the first endoscopy. FOBT (due to poor sensitivity) and BE (due to infrequent use) were not included in the study. The algorithm used automated data to presumptively classify the endoscopies as diagnostic or screening. Endoscopies were classified as diagnostic if automated data included certain gastrointestinal conditions in the year prior to the exam, or signs or symptoms or a FOBT in the 45 days prior. The study gastroenterologist (KRM) identified the conditions and signs and symptoms likely to result in diagnostic endoscopies. All other endoscopies were classified into the screening group. Two trained abstractors reviewed medical records from 1997 to 2002 to confirm endoscopy use. The abstractors also assessed screening or diagnostic indications for the endoscopies, including the presence of gastrointestinal conditions or signs and symptoms. To minimize interrater variability, one abstractor reviewed all participants’ medical records classified as a diagnostic exam while the second reviewed all participants’ medical records classified as a screening exam. Abstractors classified the endoscopies as diagnostic if the exam was a follow-up to a previous abnormality or when clear-cut conditions or signs and symptoms were present, using the same list and time frames as the algorithm. All other endoscopies were classified as screening.

RESULTS:

KQ2 - What factors influence the use of colorectal cancer screening?	NA

KQ3 - Which strategies are effective in increasing the appropriate use of colorectal cancer screening and follow- up?	NA

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	The sensitivities for identifying diagnostic sigmoidoscopy and colonoscopy were 48.1% and 23.8%, respectively. The algorithm missed most of the diagnostic endoscopies. Conversely, the sensitivities for identifying screening sigmoidoscopy and colonoscopy were high (87.9% and 84.4%, respectively) but were associated with low specificities. Comparison of classification by automated algorithm versus medical record review
			Classification by medical record review
	Classification by automated algorithm		Diagnostic (n = 90)			Screening (n=98)		% sensitivity		% specificity				kappa
	Sigmoidoscopy
	Diagnostic		13			8		48.1 (13/27)		12.1 (8/66)				76.3
	Screening		14			58		87.9 (58/66)		51.9 (14/27)
	Total		27			66
	Colonoscopy
	Diagnostic		15			5		23.8 (15/63)		15.6 (5/32)				44.2
	Screening		48			27		84.4 (27/32)		76.2 (48/63)
	Total		63			32
	The authors conclude that studies relying solely on automated data overestimate screening rates if indication is not considered. They also conclude that the automated algorithm needs further improvements to better differentiate screening from diagnostic exams.

QUALITY RATING:	Fair

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?

Were the drop-out or response rates acceptable (≤ 20%)? [If between 20% and 60%, check other and explain.]

Were the differential drop-out or response rates acceptable (≤ 15%)?

Were intervention/exposure measures valid, reliable, and equally applied?						X								Algorithm developed by investigator

Were the outcome assessors blinded to the intervention or exposure status of subjects?										X

Were outcome measures valid, reliable, and equally applied?										X				Medical chart review—single person review. Little detail on training provided, no IRR assessment done

Does the analysis control for baseline differences?

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?

Were the statistical methods used to assess the abstracted outcomes appropriate?

Were data inputs valid?						X

Was an appropriate search strategy used to find data inputs?										X				Does not say how charts used were randomly selected

Were the calculations and statistical analyses adequate?						X

Were appropriate sensitivity analyses conducted (especially for any variables that were not based on data from literature)?

Other considerations:						X

Quality Rating (Good, Fair, or Poor): Fair

STUDY: Yield of claims data and surveys for determining cancer screening among health plan members	Authors, ref ID: Pignone et al ¹⁰⁴ Year of publication: 2009 Dates of data collection: Trial name: CHOICE

OBJECTIVE OR AIM:	To evaluate the independent and combined yield of claims and direct survey for identifying colorectal cancer screening among average risk health plan beneficiaries.

DESIGN:	Setting: 32 Primary care practices in Georgia, Florida taking part in a randomized trial of a CRC decision aid and practice-level academic detailing Study design: Observational Duration (mean followup): 2005-2007 Overall study size (N enrolled/N analyzed): 2558/1595 (responded to survey)

	Group 1
Sample size:	1595
Describe intervention:	none

RECRUITMENT: (population-based, clinic-based, volunteer, other)	Claims data from Aetna

INCLUSION CRITERIA:	Members with ages between 52 and 80 years whose primary care physicians had agreed to participate in the CHOICE study

EXCLUSION CRITERIA:	Individuals at increased risk for colorectal cancer (because the intervention was designed for average-risk patients) or with medical conditions that would limit their ability to participate in the study or who might not be considered reasonable candidates for screening. Above average-risk persons were defined as adults with a personal history of colorectal cancer or polyps, a known history of colorectal cancer or polyps in a first-degree relative, or a known history of inflammatory bowel disease also, dementia, chronic obstructive pulmonary disease, heart failure, coronary artery disease, current treatment for cancer or history of metastatic cancer, cirrhosis, upper or lower gastrointestinal bleeding, unintentional weight loss of >10% within 6 months, blindness, or uncorrectable hearing impairment. To be excluded, the individual had to have at least two claims with either a diagnosis or procedure code indicating that they had one of these additional conditions.

POPULATION CHARACTERISTICS:	Group 1
Mean age & range (years):	Age group 52-59 887 (69.90) 60-64 283 (22.30) 65-69 81 (6.38) 70-82 18 (1.42)
Sex (% female):	% female 60%
Race:	75% white/ 19% black/ 6% other
Other:

Attrition/Drop-out (not available for endpoint measurement):	Overall
Adherence:
Contamination:
Response Rates (e.g. for surveys):	Response rate was 62%

STATISTICAL ANALYSES:	Describe: Descriptive analyses, logistic regression and examination of confounders

ASSESSMENT OF EXPOSURES AND POTENTIAL CONFOUNDERS:	Yes

OUTCOME ASSESSMENT:	Outcome Measures: primary outcome was proportion of persons up to date with CRC screening Survey asked about individuals completion of CRC screening, whether even had and when tests were conducted Claims data on CRC screening, including FOBT (within 1 year), FS (5 years), colonoscopy (10 years) or BE (5 years), including most recent data for each type. Indication for the procedure was not specified.

RESULTS:

KQ2 - What factors influence the use of colorectal cancer screening?	NA

KQ3 - Which strategies are effective in increasing the appropriate use of colorectal cancer screening and followup?	NA

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	Outcomes: • Of 4,020 average-risk members identified, claims data indicated that 1,066 (27%) had recent colorectal cancer screening. Among the 1,269 average risk members with no evidence of screening by claims data who returned surveys, 498 (39%) reported being up-to-date with screening. • Combining claims data and survey data: • Prevalence of CRC screening, combining claims data plus self-reported data (and not including nonresponders to the survey): 47% • Prevalence of CRC screening, combining claims data plus self-reported data (assuming nonresponders were screened at the same rate as average-risk responders): 59%

QUALITY RATING:	Fair

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?														NA

Were the drop-out or response rates acceptable (≤20%)? [If between 20% and 60%, check other and explain.]						x								62% response rate

Were the differential drop-out or response rates acceptable (≤15%)?														Not known

Were intervention/exposure measures valid, reliable, and equally applied?														NA

Were the outcome assessors blinded to the intervention or exposure status of subjects?														NA

Were outcome measures valid, reliable, and equally applied?						x								However, screening vs. diagnostic procedures were not distinguished

Does the analysis control for baseline differences?														NA

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?														NA

Were the statistical methods used to assess the abstracted outcomes appropriate?						x

Quality Rating (Good, Fair, or Poor): Fair

STUDY: Data sources for measuring colorectal endoscopy use among Medicare enrollees Evaluation of claims, medical records, and self-report for measuring fecal occult blood testing among Medicare enrollees in fee for service	Authors, ref ID: Schenck ^105–106 Year of publication: 2007 and 2008 Dates of data collection: 1998–2002 Trial name:

OBJECTIVE OR AIM:	Comparison of different data sources used for measuring CRC testing

DESIGN:	Setting: Study design: Observational Duration (mean followup): Brief Overall study size (N enrolled/N analyzed): 936 eligible 561 analyzed

	Overall
Sample size:	561
Describe intervention: NA	endoscopy and FOBT

RECRUITMENT: (population-based, clinic-based, volunteer, other)	Population – NC Medicare patients from 10 different counties

INCLUSION CRITERIA:	North Carolina Medicare enrollees, African American or white, in fee for service Medicare, between ages 55 and 80 without history of CRC, who had responded to a 2002 survey.

EXCLUSION CRITERIA:	HMO coverage or gap in coverage

POPULATION CHARACTERISTICS:	Overall
Mean age & range (years):	50–64 10.5% 65–74 62.7% 75–80 26.8%
Sex (% female):	61% female
Race:	African American 23.5% White 76.5%

Attrition/Drop-out (not available for endpoint measurement):	Overall – NA
Adherence:
Contamination:
Response Rates (e.g. for surveys):	Response rate to survey not reported. Sample consisted of 1001 persons who had responded to the survey.

STATISTICAL ANALYSES:	Describe: descriptive statistics, report to record ratio to detect bias, concordance amongst sources

ASSESSMENT OF EXPOSURES AND POTENTIAL CONFOUNDERS:	No

OUTCOME ASSESSMENT:	Outcome Measures: • Sigmoidoscoy, colonoscopy and FOBT (both at home and in office) • Ever use of an endoscopic procedure measured by survey, and date of most recent (outcome of FS in last 4 years and colonscopy in last 5 years) • FOBT, distinguishing in office from home, and whether test was part of a check up or because of a problem, from survey • Medical record review (using a hierarchical algorithm to link the patient to a provider) to record date of endoscopic procedure, and whether screening or diagnostic; and dates of four most recent FOBTs • Claims (1/1998–12/2002) data using Medicare inpatient, outpatient, and physician claims. Screening and diagnostic codes were available.

RESULTS:

KQ2 - What factors influence the use of colorectal cancer screening?	Outcomes: NA

KQ3 - Which strategies are effective in increasing the appropriate use of colorectal cancer screening and followup?	Outcomes: NA

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	Outcomes: NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	Outcomes: %
						Self report				Claim				Medical record
	Sigmoidoscopy					22.8				21.6				15.2
	Colonoscopy					38.5				35.1				34.1
	Endoscopy					50.1				44.9				42.3
	In last 5 yrs					NR				NR
	FOBT- home													30.1
	FOBT office													31.4
	FOBT in last year					28.7				21.2				19.4
	FOBT in last 2 years					44.0				34.2				29.2
	Prevalence of endoscopy in the past year Overall: Survey 50.1 Administrative 44.9 Medical record review 42.3 By sociodemographic characteristics: Age 55–64; 65–74; 65–80 Survey 50.8; 52.4; 44.0 Administrative 35.6; 43.9; 50.7 Medical record review 32.2; 40.7; 50.0 All African Americans; all whites; all women, all men: Survey 40.9; 52.9; 46.8; 55.3 Administrative 41.7; 45.9; 43.6; 47.0 Medical record review 42.4; 42.2; 42.7; 41.6 Less than high school; high school diploma; more than high school: Survey 28.7; 46.9; 59.8 Administrative 39.4; 45.9; 45.8 Medical record review 38.3; 41.8; 43.6 Prevalence of FOBT in the past year Overall: Survey 28.7 Administrative 21.2 Medical record review: 19.4 By sociodemographic characteristics: Age 55–64; 65–74; 65–80 Survey 35.2; 27.9; 28.4 Administrative 19.3; 21.0; 23.6 Medical record review: 19.3; 19.8; 19.6 All African Americans; all whites; all women, all men Survey 32.0; 27.8; 30.6; 25.9 Administrative 18.8; 22.4; 25.5; 15.3 Medical record review: 12.5; 21.9; 21.7; 16.7 Less than high school; high school diploma; more than high school Survey 26.6; 26.0; 31.6 Administrative 20.2; 20.4; 22.8 Medical record review: 19.1; 16.3; 22.4 Measures of concordance for endoscopy use Administrative to medical record review Agreement: 95 (93–97) Kappa: 0.89 (0.81–0.98) Self-report to medical record review Agreement: 70 (66–73) Kappa: 0.39 (0.31–0.47) Self-report to administrative Agreement: 70 (66–74) Kappa: 0.40 (0.32–0.49) Agreement regarding test type (FS or colonoscopy) Claims to medical record review: 93 (88–97) Self-report to medical record review: 82 (75–89) Self-report to claims: 77 (70–85) Agreement regarding test purpose (screening or diagnostic): Administrative to medical record review: 52 (43–61) Self-report to medical record review: 65 (55–74) Self-report to administrative: 29 (20–36) Measures of concordance for FOBT Administrative to medical record review Agreement: 82 (79–85) Self-report to medical record review Agreement: 70 (66–74) Self-report to administrative Agreement: 67 (63–71) Sensitivity analyses included: excluding claims of FOBT on day of medical visit; including all medical record review of FOBTs (likely including in-office, single card FOBTs with digital rectal exams); did not appreciably change the measures

QUALITY RATING:	Good

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?														NA

Were the drop-out or response rates acceptable (≤20%)? [If between 20% and 60%, check other and explain.]														Survey response not recorded; possibility that non respondents to survey would be different in some way to alter agreement between a survey response and medical records/chart review

Were the differential drop-out or response rates acceptable (≤15%)?														NA

Were intervention/exposure measures valid, reliable, and equally applied?														NA

Were the outcome assessors blinded to the intervention or exposure status of subjects?														NA

Were outcome measures valid, reliable, and equally applied?						x

Does the analysis control for baseline differences?														NA

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?														NA

Were the statistical methods used to assess the abstracted outcomes appropriate?						x

Quality Rating (Good, Fair, or Poor): Good

STUDY:	Authors, ref ID: Schneider EC, et al.¹⁰⁷ Year: 2008 Trial Name:

RESEARCH OBJECTIVE OR AIM:	Evaluate quality measures by describing a field test of the colorectal cancer screening measure included in the Health Plan Employer Data and Information Set of the National Committee for Quality Assurance –

DESIGN:	Setting: 5 geographically dispersed healthcare plans that enrolled 189,193 considered eligible for CRC screening Study design: Observational Duration (mean follow-up): Overall study size (N enrolled/N analyzed): 1000 plus an additional random sample of 400 enrollees from the original health care plan cohort (3000 enrollees)

INTERVENTIONS:	NA
Sample size:
Describe intervention:

RECRUITMENT: (population-based, clinic-based, volunteer, other)	Each of the 5 enrolled health plans identified all enrollees > 51 y/o, sent this to RAND Corp for further analysis. Research staff at RAND Corp. randomly selected 1000 enrollees (200 per plan) who lacked evidence of CRC screening (based on administrative data). Then, medical record abstractors looked for evidence of CRC screening in these pts. Health plan also surveyed patients via mail. During a 6 week period, nonrespondents received 3 survey mailings + a reminder postcard and a final overnight mailing. Response rate was 48.1% (range 37.8% – 57.5%

INCLUSION CRITERIA:	> or = 51 y/o continuously enrolled for at least 2 years in one of the 5 health plans

EXCLUSION CRITERIA:	NR

POPULATION CHARACTERISTICS:	Survey respondents older, longer enrollment Survey respondents more likely than non respondents to have evidence of CRC screening
Mean age & range (years):
Sex (% female):
Ethnicity:
Other:

OUTCOME ASSESSMENT:						Plan A		Plan B		Plan C		Plan D		Plan E
	Age of plan, y					53		19		17		17		17
	Submitted cohort aged 51–80 and enrolled continuously for 2 years					65,241		68,659		28,564		3,561		23,168
	Age, mean, y					64.2		60.9		59.7		58.8		57.4
	Female, %					54.1%		53.0		51.4		52.8		51.4
	Enrollment, mo					73.3		NA		69.1		83.1		46.8
	Survey rspndts					N=1250
	Age, mean, y					65.9		60.4		60.9		59.4		56.8
	Female, %					58.9		51.7		50.2		54.7		51.3
	Enrollment, mo					76.5		NA		75.5		86.4		48.3
	Survey nonrespondents					N=1349
	Age, mean, y					64.6		60.3		59.9		58.8		56.7
	Female, %					54.2		50.7		52.0		51.8		49.3
	Enrollment,					81.8		NA		72.0		82.6		48.5
	The authors defined, for each data source (administrative, medical record, and survey results), the data elements needed to implement all of the measure specifications. The authors developed a list of outpatient diagnosis codes that represent a prior diagnosis of colorectal cancer, CPT codes related to an acceptable screening procedure, and historical CPT codes used within the previous 10 years. For the medical record method, the authors designed an abstraction tool to collect data on screening tests, clinical risk factors for colorectal cancer, and evidence of limited life expectancy and trained experienced nurses from each health care plan during a pilot test of the abstraction protocol using a common set of records. For the enrollee survey, the authors developed a sequence of questions (based on the Behavioral Risk Factor Surveillance System) that addressed each of the screening tests, the time frames in which they occurred, and a measure of risk status based on report of a family history of colorectal cancer. Data from all sources (administrative, survey, and medical record) were linked at the enrollee level to create a single analytic file. For each of the 4 colorectal screening tests, the authors compared across health care plans the percentage of sampled enrollees identified as screened based on administrative data and the percentage based on survey data. The authors could not determine the percentage screened by each test under the hybrid method because medical record abstractors were instructed to stop after identifying the occurrence of 1 of the tests. For the single measure that combined all tests, the authors calculated colorectal cancer screening performance scores based on administrative data only, survey data only, and combined administrative and medical record data (the hybrid method), weighing the medical record sample to represent the population from which they were drawn (unscreened based on administrative data).. Among the survey respondents, the authors assessed agreement of the sampled enrollees’ screening status according to the survey data and the hybrid data. The authors compared the rate of screening among survey respondents and nonrespondents using the hybrid estimation procedure.

RESULTS:

KQ4 - What are the current and projected capacities to deliver colorectal cancer screening and surveillance at the population level?	NA

KQ5 - What are the effective approaches for monitoring the use and quality of colorectal cancer screening?	Compared with survey non-respondents, respondents were older (60.4 vs. 59.4 years; P_.001) and had longer enrollment (73.3 vs. 67.6 months; P = .001), but the 2 groups had similar percentages of female participants (53.3% vs. 51.1%; P = .28). Survey respondents were more likely than non-respondents to have evidence of CRC screening (62.7% vs. 46.5%; p<0.001) The percentage of enrollees having specific tests varied between the administrative and survey data methods. In health care plans A and E, the percentages of enrollees screened by FOBT according to the 2 methods were similar; in health care plan B, the percentage based on survey data was nearly twice that based on administrative data; and in health care plan C, the percentage based on administrative data exceeded that based on survey data. For the procedural tests (flexible sigmoidoscopy, doublecontrast barium enema, and colonoscopy), the rates based on survey data were 2 to 3 times higher than the rates based on administrative data.
	Method					Plan A		Plan B		Plan C		Plan D		Plan E
	FOBT
	Administrative					23.6		15.0		31.1		NA		24.7
	Survey					25.4		26.3		20.5		21.8		25.1
	Flex Sig
	Administrative					14.2		17.9		18.4		15.3		15.4
	Survey					29.7		39.6		33.9		33.6		30.6
	COLON
	Administrative					12.8		12.1		9.4		10.5		14.2
	Survey					19.9		39.0		33.6		33.7		40.7
	Using the single measure of screening (based on any 1 of the 4 tests), the percentage of enrollees with evidence of colorectal cancer screening varied substantially depending on the data sources used. For 4 of the 5 health care plans, the hybrid method produced a higher screening rate than the administrative data method. The difference ranged from 0 (health care plan A) to 14.9 (health care plan B) percentage points. For all 5 health care plans, the survey data method produced a higher calculated screening rate than the other 2 methods.
	Method					Plan A		Plan B		Plan C		Plan D		Plan E
	Administrative					41.5 (41.1–41.9)		38.6 (38.2–38.9)		47.1 (46.5–47.6)		27.3 (25.8–28.7)		44.4 (43.8–45.1)
	Survey					53.2 (42.1–64.4)		69.7 (60.3–79.2)		55.0 (41.1–68.8)		62.1 (53.8–70.4)		66.2 (57.1–75.2)
	Hybrid					41.5 (41.1–41.9)		53.5 (48.5–56.8)		52.6 (48.3–56.8)		38.8 (34.3–43.4)		45.6 (44.0–47.2)

ANALYSIS:

ARE GROUPS COMPARABLE AT BASELINE:

ASCERTAINMENT METHODS ADEQUATE AND EQUALLY APPLIED:

STATISTICAL ANALYIS ADEQUATE:

ATTRITION:	NA
Dropout (not available for endpoint measurement):
Adherence in control group:
Contamination in control group:
Differential dropouts:

QUALITY RATING:	Fair

Quality Assessment-Internal Validity for Observational Studies

						Yes				No				Other (CD, NR, NA)

Were the groups similar at baseline regarding the most important prognostic indicators?

Were the drop-out or response rates acceptable (≤ 20%)? [If between 20% and 60%, check other and explain.]														RR varied among plans from 38% to 58%

Were the differential drop-out or response rates acceptable (≤ 15%)?

Were intervention/exposure measures valid, reliable, and equally applied?						X

Were the outcome assessors blinded to the intervention or exposure status of subjects?

Were outcome measures valid, reliable, and equally applied?						X								Questions Based on BRFSS

Does the analysis control for baseline differences?

Were important potential confounding and modifying variables taken into account in the design and analysis (e.g., through matching, stratification, or statistical adjustment)?

Were the statistical methods used to assess the abstracted outcomes appropriate?														Very specific purpose to this study to compare results of 3 different methods of estimating overall screening rates. Hence outcomes not directly comparable to other studies.

Quality Rating (Good, Fair, or Poor): Fair

From: Appendix C, Evidence Tables

Enhancing the Use and Quality of Colorectal Cancer Screening.

Evidence Reports/Technology Assessments, No. 190.

Holden DJ, Harris R, Porterfield DS, et al.

Rockville (MD): Agency for Healthcare Research and Quality (US); 2010 Feb.

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.