Derivation and Validation of a Clinical Prediction Score to Identify the Isolation of Pseudomonas in Pneumonia

ABSTRACT Given the focus of existing clinical prediction scores on identifying drug-resistant pathogens as a whole, the application to individual pathogens and other institutions may yield weaker performance. This study aimed to develop a locally derived clinical prediction model for Pseudomonas-mediated pneumonia. This retrospective study included patients ≥18 years of age who were admitted to an academic medical center between 1 July 2010 and 31 July 2020 with a CDC National Healthcare Safety Network confirmed pneumonia diagnosis and were receiving antimicrobials during the index encounter, with a positive respiratory culture. Cystic fibrosis patients were excluded. Logistic regression analysis identified risk factors associated with the isolation of Pseudomonas aeruginosa from respiratory cultures within the derivation cohort (n = 186), which were weighted to generate a prediction score that was applied to the derivation and internal validation (n = 95) cohorts. A total of 281 patients met the inclusion criteria. Five predictor variables were identified, namely, tracheostomy status (4 points), chronic obstructive pulmonary disease (5 points), enteral nutrition (9 points), chronic steroid use (11 points), and Pseudomonas aeruginosa isolation from any culture in the prior 6 months (14 points). At a score of >11, the prediction score demonstrated a sensitivity of 52.4% (95% confidence interval [CI], 36.4 to 68.0%) and a specificity of 84.9% (95% CI, 72.4 to 93.35%) in the validation cohort. Score accuracy was 70.5% (95% CI, 60.3 to 79.4%), and the area under the receiver operating characteristic curve (AUROC) was 0.77 (95% CI, 0.68 to 0.87) in the validation cohort. A prediction score for identifying Pseudomonas aeruginosa in pneumonia was derived, which may have the potential to decrease the use of broad-spectrum antibiotics. Validation with larger and external cohorts is necessary. IMPORTANCE In this study, we aimed to develop a locally derived clinical prediction model for Pseudomonas-mediated pneumonia. Utilizing a locally validated prediction score may help direct therapeutic management and be generalizable to other clinical settings and similar populations for the selection of appropriate antimicrobial coverage when data are lacking. Our study highlights a unique patient population, including immunocompromised, structural lung disease, and transplant patients. Five predictor variables were identified, namely, tracheostomy status, chronic obstructive pulmonary disease, enteral nutrition, chronic steroid use, and Pseudomonas aeruginosa isolation from any culture in the prior 6 months. A prediction score for identifying Pseudomonas aeruginosa in pneumonia was derived, which may have the potential to decrease the use of broad-spectrum antibiotics, although validation with larger and external cohorts is necessary.

Page 5 Lines 118-120: May consider revising the context here of the problem leading to the objective of the study as still somewhat opaque as to why only developing for Pseudomonas. Perhaps as avoidance of MRSA therapy can be driven by guideline-supported testing of nasal surveillance for this determination? Thus practically, could drive to CAP coverage if no Pseudomonas per risk score and negative nasal swab (and presumably low CRE/ESBL setting)? Even the developers of DRIP have more or less followed this paradigm so perhaps reasonable argument ( Page 7 Lines 159-169: For all time-varying potential predictors (e.g. mechanical ventilation), please clarify they were obtained before or at time of index culture. If they were past this time point, would exclude from model as would be unavailable for prediction till later in patient encounter.
Page 9 Lines 198-202: Again, as recommended per TRIPOD, would include the full regression model (in addition to current risk score model) including the beta coefficients and intercept.
Page 10 Lines 28-31 and Pages 12-13 Lines 284-290: How was NPV and PPV derived if this was a case control study and therefore prevalence of the pathogen is not derivable from these patients? Do you have the prevalence from other data within your system or was assumed from literature?
Page 10 Lines 228-231: "the scoring model improved the accuracy" compared to what? What was it improved from? As no comparison was made (e.g. to say clinical judgement), would consider revising verbiage. Similarly, on page 12 Lines 272-277, would avoid "prevalence" verbiage given this is a case control study.
Reviewer #2 (Comments for the Author): The authors present a single-center retrospective study to identify their center's local risk factors for pneumonia due to Pseudomonas. They acknowledge that other risk scores have been developed but do not perform as well outside of the center at which they were derived. To this end, they sought to look at their population to determine risk factors for Pseudomonas amongst all those admitted for CAP/HAP/VAP. One strength of the study is the use of the standardized CDC definition of pneumonia and microbiologic diagnosis in all included patients. One weakness, which they acknowledge, is the heterogeneous inclusion of CAP/HCAP/VAP/HAP cases in the score.
2. Line 144: patients selected for the control group were not selected "at random" if every 3rd non-Pseudomonas culture was selected 3. How were COPD, emphysema, bronchiolitis and asthma defined? There is a lot of overlap between these entities. Since COPD was part of the final score, would suggest adding more clarity surrounding how these were defined.
4. Most of the predictive weight in the score comes from a prior positive Pseudomonas culture. In practice, empiric Pseudomonas coverage would be given to these patients without calculating a risk score. The patients without prior culture are the ones where a prediction score would be useful. Would recommend presenting an alternative model that does NOT include this variable, along with performance characteristics.
In the introduction, the authors correctly state that a limitation of prior studies is lack of availability to other sites, as mentioned above. Here, they present a study that can be potentially useful at their own site, and rightly point out that the generalizability needs to be determined.

Preparing Revision Guidelines
To submit your modified manuscript, log onto the eJP submission site at https://spectrum.msubmit.net/cgi-bin/main.plex. Go to Author Tasks and click the appropriate manuscript title to begin the revision process. The information that you entered when you first submitted the paper will be displayed. Please update the information as necessary. Here are a few examples of required updates that authors must address: • Point-by-point responses to the issues raised by the reviewers in a file named "Response to Reviewers," NOT IN YOUR COVER LETTER. • Upload a compare copy of the manuscript (without figures) as a "Marked-Up Manuscript" file. • Each figure must be uploaded as a separate file, and any multipanel figures must be assembled into one file. For complete guidelines on revision requirements, please see the journal Submission and Review Process requirements at https://journals.asm.org/journal/Spectrum/submission-review-process. Submissions of a paper that does not conform to Microbiology Spectrum guidelines will delay acceptance of your manuscript. " Please return the manuscript within 60 days; if you cannot complete the modification within this time period, please contact me. If you do not wish to modify the manuscript and prefer to submit it to another journal, please notify me of your decision immediately so that the manuscript may be formally withdrawn from consideration by Microbiology Spectrum.
If your manuscript is accepted for publication, you will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
Corresponding authors may join or renew ASM membership to obtain discounts on publication fees. Need to upgrade your membership level? Please contact Customer Service at Service@asmusa.org.
Thank you for submitting your paper to Microbiology Spectrum. Thank you for your comprehensive review of our manuscript titled "Derivation and validation of a clinical prediction score to identify the isolation of Pseudomonas in pneumonia". We appreciate the thoughtful editorial and reviewer feedback. In response to the reviewer comments, we have updated our manuscript to add clarity. Please find enclosed a copy with yellow highlighted changes and a clean version of the manuscript. Our responses to reviewer comments are noted below.

Reviewer #1
Overall: The present study is a derivation and validation study of a clinical prediction model for pseudomonas as etiology of pneumonia among CAP, HAP, and VAP patients. Overall, the study found moderate discrimination performance for the classification of pseudomonas etiology, yield potential benefit in driving appropriate empiric antimicrobial therapy. These data are informative and interesting. I applaud the efforts of the authors in their work. However, there are several issues that should be addressed before publishing.

1.) Title: Would consider indicating the validation analysis as well in the title
Response: We updated the title to reflect the reviewer's comment (Page 1). To reflect the intent of the reviewer's comment, we also updated Page 5, Line 117 with "and validate". Response: We appreciate the reviewer's suggestion to include the TRIPOD checklist as part of the manuscript. To address specifics, please find below the checklist with our response to items that were absent (noted in red font) in the original manuscript. Please note, we did not include the checklist within the body of the manuscript. In lieu of the entire checklist, we have added a statement within the Methods section to report that we followed the checklist and added the above reviewer referenced article (Page 9, Lines 196-198).
Regarding sample size justification, we have included further details to provide transparency in how we arrived at the final sample of patients used to generate the prediction model (Page 7, Lines 148-160). In short, the final number of patients included was determined by the number of patients meeting inclusion criteria of documented Pseudomonas aeruginosa and CDC defined pneumonia with subsequent screening of controls to ensure a balance of cases to controls of approximately 1:1. Title  1 Identify the study as developing and/or validating a multivariable prediction model, the target population, and the outcome to be predicted. Validation analysis was added to the title as recommended.

Introduction
Background and 3a Explain the medical context (including whether diagnostic 4-5 Objectives or prognostic) and rationale for developing or validating the multivariable prediction model, including references to existing models. 3b Specify the objectives, including whether the study describes the development or validation of the model or both. 5

Source of Data 4a
Describe the study design or source of data (e.g., randomized trial, cohort, or registry data), separately for the development and validation data sets, if applicable.

6-7 4b
Specify the key study dates, including start of accrual; end of accrual; and, if applicable, end of follow-up.

6-7
Participants 5a Specify key elements of the study setting (e.g., primary care, secondary care, general population) including number and location of centres.

6-7 5b
Describe eligibility criteria for participants. 6-7 5c Give details of treatments received, if relevant. 6-7 Outcome 6a Clearly define the outcome that is predicted by the prediction model, including how and when assessed.

6b
Report any actions to blind assessment of the outcome to be predicted.

N/A Predictors 7a
Clearly define all predictors used in developing or validating the multivariable prediction model, including how and when they were measured. Statements were added to clarify that time-varying potential predictors were obtained prior to or at the time of index culture collection as recommended.

13-14
Interpretation 19a For validation, discuss the results with reference to performance in the development data, and any other validation data.

11-12
19b Give an overall interpretation of the results, considering objectives, limitations, and results from similar studies, and other relevant evidence.

13-14
Implications 20 Discuss the potential clinical use of the model and implications for future research. In the introduction, the authors correctly state that a limitation of prior studies is lack of availability to other sites, as mentioned above. Here, they present a study that can be potentially useful at their own site, and rightly point out that the generalizability needs to be determined.

Response:
We appreciate the perspective of the reviewer and hope our study can further guide additional studies to determine the generalizability of our clinical prediction model. Your manuscript has been accepted, and I am forwarding it to the ASM Journals Department for publication. You will be notified when your proofs are ready to be viewed.
The ASM Journals program strives for constant improvement in our submission and publication process. Please tell us how we can improve your experience by taking this quick Author Survey.
As an open-access publication, Spectrum receives no financial support from paid subscriptions and depends on authors' prompt payment of publication fees as soon as their articles are accepted. You will be contacted separately about payment when the proofs are issued; please follow the instructions in that e-mail. Arrangements for payment must be made before your article is published. For a complete list of Publication Fees, including supplemental material costs, please visit our website.
ASM policy requires that data be available to the public upon online posting of the article, so please verify all links to sequence records, if present, and make sure that each number retrieves the full record of the data. If a new accession number is not linked or a link is broken, provide production staff with the correct URL for the record. If the accession numbers for new data are not publicly accessible before the expected online posting of the article, publication of your article may be delayed; please contact the ASM production staff immediately with the expected release date.