BACKGROUND: Hospital discharge diagnoses, coded by use of the International Classification of Diseases, 9th Revision, Clinical Modification (ICD-9-CM), increasingly determine reimbursement and support quality monitoring. Prior studies of coding validity have investigated whether coding guidelines were met, not whether the clinical condition was actually present. OBJECTIVE: To determine whether clinical evidence in medical records confirms selected ICD-9-CM discharge diagnoses coded by hospitals. RESEARCH DESIGN AND SUBJECTS: Retrospective record review of 485 randomly sampled 1994 hospitalizations of elderly Medicare beneficiaries in Califomia and Connecticut. MAIN OUTCOME MEASURE: Proportion of patients with specified ICD-9-CM codes representing potential complications who had clinical evidence confirming the coded condition. RESULTS: Clinical evidence supported most postoperative acute myocardial infarction diagnoses, but fewer than 60% of other diagnoses had confirmatory clinical evidence by explicit clinical criteria; 30% of medical and 19% of surgical patients lacked objective confirmatory evidence in the medical record. Across 11 surgical and 2 medical complications, objective clinical criteria or physicians' notes supported the coded diagnosis in >90% of patients for 2 complications, 80% to 90% of patients for 4 complications, 70% to <80% of patients for 5 complications, and <70% for 2 complications. For some complications (postoperative pneumonia, aspiration pneumonia, and hemorrhage or hematoma), a large fraction of patients had only a physician's note reporting the complication. CONCLUSIONS: Our findings raise questions about whether the clinical conditions represented by ICD-9-CM codes used by the Complications Screening Program were in fact always present. These findings highlight concerns about the clinical validity of using ICD-9-CM codes for quality monitoring.