A comparison of propensity score-based approaches to health service evaluation: a case study of a preoperative physician-led clinic for high-risk surgical patients

Rationale, aims and objectives: A physician-led clinic for the preoperative optimization and management of high-risk surgical patients was implemented in a South Australian public hospital in 2008. This study aimed to estimate the costs and effects of the clinic using a mixed retrospective and prospective observational study design.

Method: Alternative propensity score estimation methods were applied to retrospective routinely collected administrative and clinical data, using weighted and matched cohorts. Supplementary survey-based prospective data were collected to inform the analysis of the retrospective data and reduce potential unmeasured confounding.

Results: Using weighted cohorts, clinic patients had a significantly longer mean length of stay and higher mean cost. With the matched cohorts, reducing the calliper width resulted in a shorter mean length of stay in the clinic group, but the costs remained significantly higher. The prospective data indicated potential unmeasured confounding in all analyses other than in the most tightly matched cohorts.

Conclusions: The application of alternative propensity-based approaches to a large sample of retrospective data, supplemented with a smaller sample of prospective data, informed a pragmatic approach to reducing potential observed and unmeasured confounding in an evaluation of a physician-led preoperative clinic. The need to generate tightly matched cohorts to reduce the potential for unmeasured confounding indicates that significant uncertainty remains around the effects of the clinic. This study illustrates the value of mixed retrospective and prospective observational study designs but also underlines the need to prospectively plan for the evaluation of costs and effects alongside the implementation of significant service innovations.