BACKGROUND:

The validity of quality of care assessments relies upon data quality, yet little is known about the relative completeness and validity of data sources for evaluating the quality of care.

OBJECTIVES:

We evaluated concordance between ambulatory medical record and patient survey data. Levels of concordance, variations by type of item, sources of disagreement between data sources, and implications for quality of care assessment efforts are discussed.

DESIGN AND SUBJECTS:

This was an observational study that included 1270 patients sampled from 39 West Coast medical organizations with at least 1 of the following: diabetes, ischemic heart disease, asthma or chronic obstructive pulmonary disease, or low back pain.

MEASURES:

Items from both data sources were grouped into 4 conceptual domains: diagnosis, clinical services delivered, counseling and referral, and medication use. We present total agreement, kappa, sensitivity, and specificity at the item and domain-levels and for all items combined.

RESULTS:

We found good concordance between survey and medical records overall, but there was substantial variation within and across domains. The worst concordance was in the counseling and referrals domain, the best in the medication use domain. Patients were able to report with good sensitivity on memorable items.

CONCLUSIONS:

Quality ratings are likely to vary in differing directions, depending on the data source used. The most appropriate data source for analyses of components of and overall quality of care must be considered in light of study objectives and resources. We recommend data collection from multiple sources to most accurately portray the patient and provider experience of medical care.