Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative

Kathryn Pluta; Sarah D Hohl; Heather D'Angelo; Jamie S Ostroff; Donna Shelley; Yasmin Asvat; Li-Shiun Chen; K Michael Cummings; Neely Dahl; Andrew T Day; Linda Fleisher; Adam O Goldstein; Rashelle Hayes; Brian Hitsman; Deborah Hudson Buckles; Andrea C King; Cho Y Lam; Katie Lenhoff; Arnold H Levinson; Mara Minion; Cary Presant; Judith J Prochaska; Kimberly Shoenbill; Vani Simmons; Kathryn Taylor; Hilary Tindle; Elisa Tong; Justin S White; Kara P Wiseman; Graham W Warren; Timothy B Baker; Betsy Rolland; Michael C Fiore; Ramzi G Salloum

doi:10.1186/s43058-023-00433-3

Data envelopment analysis to evaluate the efficiency of tobacco treatment programs in the NCI Moonshot Cancer Center Cessation Initiative

Implement Sci Commun. 2023 May 11;4(1):50. doi: 10.1186/s43058-023-00433-3.

Authors

Kathryn Pluta¹, Sarah D Hohl^{2

3}, Heather D'Angelo⁴, Jamie S Ostroff⁵, Donna Shelley⁶, Yasmin Asvat⁷, Li-Shiun Chen⁸, K Michael Cummings⁹, Neely Dahl¹⁰, Andrew T Day¹¹, Linda Fleisher¹², Adam O Goldstein¹³, Rashelle Hayes¹⁴, Brian Hitsman¹⁵, Deborah Hudson Buckles¹⁶, Andrea C King¹⁷, Cho Y Lam¹⁸, Katie Lenhoff¹⁹, Arnold H Levinson²⁰, Mara Minion², Cary Presant²¹, Judith J Prochaska²², Kimberly Shoenbill¹³, Vani Simmons²³, Kathryn Taylor²⁴, Hilary Tindle²⁵, Elisa Tong²⁶, Justin S White²⁷, Kara P Wiseman¹⁰, Graham W Warren⁹, Timothy B Baker³, Betsy Rolland^{2

28}, Michael C Fiore^{2

3}, Ramzi G Salloum²⁹

Affiliations

¹ Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine, 2004 Mowry Rd, Gainesville, FL, 32610, USA.
² University of Wisconsin Carbone Cancer Center, 600 Highland Ave, Madison, WI, 53705, USA.
³ School of Medicine and Public Health, University of Wisconsin, 750 Highland Ave, Madison, WI, 53705, USA.
⁴ National Cancer Institute, 9609 Medical Center Dr, Rockville, MD, 20850, USA.
⁵ Memorial Sloan Kettering Cancer Center, 1275 York Ave, New York, NY, 10065, USA.
⁶ New York University School of Global Public Health, 708 Broadway, New York, NY, 10003, USA.
⁷ Rush University Medical Center and Rush Cancer Center, 1725 W Harrison St, Suite 1010, Chicago, IL, 60612, USA.
⁸ Washington University Siteman Cancer Center, 4921 Parkview Pl, St. Louis, MO, 63110, USA.
⁹ Medical University of South Carolina, 171 Ashley Ave, Charleston, SC, 29425, USA.
¹⁰ University of Virginia Comprehensive Cancer Center, 1240 Lee St, Charlottesville, VA, 22903, USA.
¹¹ University of Texas Southwestern Medical Center, 5323 Harry Hines Blvd, Dallas, TX, 75390, USA.
¹² Fox Chase Cancer Center, 333 Cottman Ave, Philadelphia, PA, 19111, USA.
¹³ University of North Carolina Lineberger Cancer Center, 450 West Dr, Chapel Hill, NC, 27599, USA.
¹⁴ Virginia Commonwealth University Department of Psychiatry, 501 N 2Nd St, Suite 400B, Richmond, VA, 23219, USA.
¹⁵ Northwestern University Feinberg School of Medicine and Lurie Comprehensive Cancer Center of Northwestern University, 420 E Superior St, Chicago, IL, 60611, USA.
¹⁶ Indiana University Simon Comprehensive Cancer Center, 535 Barnhill Dr, Indianapolis, IN, USA.
¹⁷ University of Chicago Medicine Comprehensive Cancer Center, 5758 S Maryland Dr, Chicago, IL, 60637, USA.
¹⁸ Huntsman Cancer Institute, University of Utah, 1950 Circle of Hope Dr, Salt Lake City, UT, 84112, USA.
¹⁹ One Medical Center Drive, Dartmouth-Hitchcock Norris Cotton Cancer Center, Lebanon, NH, 03756, USA.
²⁰ University of Colorado Comprehensive Cancer Center, 1665 North Aurora Court, Aurora, 200480045, USA.
²¹ City of Hope Comprehensive Cancer Center and Beckman Research Institute, 1500 E Duarte Rd, Duarte, CA, 91010, USA.
²² Stanford Cancer Institute, Stanford University, 265 Campus Dr, Ste G2103, Stanford, CA, 94305, USA.
²³ H. Lee Moffitt Cancer Center, 3011 Holly Dr, Tampa, FL, 33612, USA.
²⁴ Georgetown University Lombardi Comprehensive Cancer Center, 3800 Reservoir Rd NW, Washington, DC, 20007, USA.
²⁵ Vanderbilt University Medical Center Vanderbilt-Ingram Cancer Center, 2220 Pierce Ave, Nashville, TN, 37232, USA.
²⁶ University of California Davis Comprehensive Cancer Center, 2279 45Th St, Sacramento, CA, 95817, USA.
²⁷ Philip R. Lee Institute for Health Policy Studies, University of California San Francisco, 490 Illinois St, Floor 7, San Francisco, CA, 94158, USA.
²⁸ University of Wisconsin Institute for Clinical and Translational Research, 750 Highland Ave, Madison, WI, 53705, USA.
²⁹ Department of Health Outcomes and Biomedical Informatics, University of Florida College of Medicine, 2004 Mowry Rd, Gainesville, FL, 32610, USA. rsalloum@ufl.edu.

Abstract

Background: The Cancer Center Cessation Initiative (C3I) is a National Cancer Institute (NCI) Cancer Moonshot Program that supports NCI-designated cancer centers developing tobacco treatment programs for oncology patients who smoke. C3I-funded centers implement evidence-based programs that offer various smoking cessation treatment components (e.g., counseling, Quitline referrals, access to medications). While evaluation of implementation outcomes in C3I is guided by evaluation of reach and effectiveness (via RE-AIM), little is known about technical efficiency-i.e., how inputs (e.g., program costs, staff time) influence implementation outcomes (e.g., reach, effectiveness). This study demonstrates the application of data envelopment analysis (DEA) as an implementation science tool to evaluate technical efficiency of C3I programs and advance prioritization of implementation resources.

Methods: DEA is a linear programming technique widely used in economics and engineering for assessing relative performance of production units. Using data from 16 C3I-funded centers reported in 2020, we applied input-oriented DEA to model technical efficiency (i.e., proportion of observed outcomes to benchmarked outcomes for given input levels). The primary models used the constant returns-to-scale specification and featured cost-per-participant, total full-time equivalent (FTE) effort, and tobacco treatment specialist effort as model inputs and reach and effectiveness (quit rates) as outcomes.

Results: In the DEA model featuring cost-per-participant (input) and reach/effectiveness (outcomes), average constant returns-to-scale technical efficiency was 25.66 (SD = 24.56). When stratified by program characteristics, technical efficiency was higher among programs in cohort 1 (M = 29.15, SD = 28.65, n = 11) vs. cohort 2 (M = 17.99, SD = 10.16, n = 5), with point-of-care (M = 33.90, SD = 28.63, n = 9) vs. no point-of-care services (M = 15.59, SD = 14.31, n = 7), larger (M = 33.63, SD = 30.38, n = 8) vs. smaller center size (M = 17.70, SD = 15.00, n = 8), and higher (M = 29.65, SD = 30.99, n = 8) vs. lower smoking prevalence (M = 21.67, SD = 17.21, n = 8).

Conclusion: Most C3I programs assessed were technically inefficient relative to the most efficient center benchmark and may be improved by optimizing the use of inputs (e.g., cost-per-participant) relative to program outcomes (e.g., reach, effectiveness). This study demonstrates the appropriateness and feasibility of using DEA to evaluate the relative performance of evidence-based programs.

Keywords: Cancer; Data envelopment analysis; Efficiency; Implementation costs; Implementation science; Program performance; Smoking cessation; Tobacco treatment.

Abstract

Grants and funding