This study uses national Medicare data as well as data that were abstracted to calibrate the Medicare Mortality Predictor System to assess the usefulness of a risk adjustment system in interpreting hospital mortality rates. The majority of variation in annual hospital death rates for the four conditions studied (stroke, pneumonia, myocardial infarction, and congestive heart failure) is chance variability that results from the relatively small numbers of patients treated in most hospitals in a year. For hospitals in the highest and lowest quartiles of observed death rates, the difference between observed rates and those predicted by the Medicare Mortality Predictor System is not quite on third smaller than the difference between observed rates and unadjusted national rates. Risk adjustment methods do not show whether the unexplained difference in mortality rates results from differences in effectiveness of care or unmeasured differences in patient risk at the time of admission. Risk-adjusted mortality rates, therefore, should be supplemented by review of the actual care rendered before conclusions are drawn regarding effectiveness of care.