NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Dudley RA, Frolich A, Robinowitz DL, et al. Strategies To Support Quality-based Purchasing: A Review of the Evidence. Rockville (MD): Agency for Healthcare Research and Quality (US); 2004 Jul. (Technical Reviews, No. 10.)

Cover of Strategies To Support Quality-based Purchasing

Strategies To Support Quality-based Purchasing: A Review of the Evidence.

Show details

Appendix BGeneral Approach to Simulations

The algorithm for each simulated scenario is as follows:

  1. Create a hypothetical hospital world based on input parameters using data available from the real world. These models contain either two or three homogenous groups of hospitals each with a defined level of hospital performance. This model is the world of true hospitals, or the gold standard of the model.
    Our hypothetical model is somewhat conceptually different from Thomas and Hofer's. Instead of using the likelihood of receiving poor or good quality care, we differentiated hospitals based on the overall level of care provided to all patients. A good hospital may have processes or personnel in place to provide better quality care to each of its patients, not just to limit poor care to fewer of its patients. This assumption allows us to build a world view identical to Thomas and Hofer, but to start deeper in their model, at the level of probability of death in each hypothetical hospital group (without deriving these values from their assumptions outlined above).
  2. Apply a grading function to a set of hospital outcomes. In our simulation outlier cutoffs, or “trim points,” were used to label outcomes as “poor,” or “good,” or in models with three categories, “superior.” The value of the trim point is estimated by assuming that the observed mortality risk outcomes assume a normal distribution around the mean mortality rate of the hospitals. The trim point(s) are set such that a given percent of the mortality outcomes of the population of hospitals will fall above or below the respective poor and superior trim points. Other possible grading functions could use arbitrary trim points (for absolute standards of quality), trim points based on reference populations, or trim points based on other distributional assumptions.
    Note that the Thomas and Hofer evaluation function assumes that the overall distribution - that which can be observed, is equivalent to a normal distribution around the mean hospital probability of death, with standard deviation defined using the number of patients at each hospital. In reality, the sum of the “good” and “poor” distributions - the solid line in figure 2, is actually a right skewed distribution, due to the larger standard deviation of the “poor” sub-group, as a function of the higher probability of mortality in this subgroup, as calculated with the following equation: std_dev of poor group = Squareroot (prob_death * (1 - prob_death)/num_patients_per_hospital). Note also that these distributions are not truly normal, as they terminate at 0.0 (i.e. there is no negative probability of death).
  3. Assess the performance of the evaluation system - either via sensitivity and specificity (i.e. how likely is the system to correctly label poor quality hospitals as “poor” and superior quality hospitals as “superior”) or predictive values (i.e. given a grade of “superior,” how likely is a hospital actually to be of superior quality?). The former measure is of most concern to hospitals, concerned about being mislabeled, while the accuracy of predictive values tells consumers, purchasers, and other policymakers how much to trust the grades assigned. The perfect evaluation system would label each hospital according to the true world group to which it belongs.
    This step is repeated for a given grading function over several possible hypothetical hospital worlds (see step 1) to test the robustness of the evaluation system. Results from the representative scenarios are discussed in Section 3.

The models were produced using Microsoft Excel with statistical functions and Visual Basic for Applications, 2003. Each parameter was either entered by hand, or derived using a recreation of the Thomas and Hofer model or from empiric data as described above. For each hospital group, the chance of each grade was determined using the NORMDIST function, which given a mean (in this case, the mortality risk as defined for the group), standard deviation (calculated using the group's mortality probability and number of patients per hospital), and a trim point (the trim point as defined in the approach to evaluation and labeling, based on the observed, total distribution of hospital mean mortality), returns the probability of selecting an outcome that exceeds the trim point, assuming a normal distribution based on the mean and standard deviation supplied. This corresponds to the area under the hospital group's curve that is to extreme side of the trim point line.


  • PubReader
  • Print View
  • Cite this Page
  • PDF version of this title (1.7M)

Recent Activity

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

See more...