Derivation and implementation of a machine learning approach to prompt serious illness conversations among outpatients with cancer.

Authors

Ravi Parikh

Ravi Bharat Parikh

University of Pennsylvania, Philadelphia, PA

Ravi Bharat Parikh, Chris Manz, Corey Chivers, Susan B Regli, Jennifer Braun, Joshua Adam Jones, Ronac Mamtani, Michael Draugelis, Justin E. Bekelman, Amol S. Navathe, Mitesh S. Patel, Lawrence N. Shulman, Lynn Schuchter, Nina O'Connor

Organizations

University of Pennsylvania, Philadelphia, PA, Abramson Cancer Center, Philadelphia, PA, University of Pennsylvania, Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA

Research Funding

Other
Penn Center for Precision Medicine Accelerator Fund, U.S. National Institutes of Health.
Background: Machine learning (ML) algorithms can accurately identify patients with cancer at risk of short-term mortality and facilitate timely conversations about treatment and end-of-life preferences. We developed, validated, and implemented a ML algorithm to predict mortality in a general oncology setting, using electronic health record (EHR) data prior to a clinic visit.

Methods: Our cohort consisted of patients aged ≥18 years who had an encounter in outpatient oncology practices within a large academic health system between February 1st and July 1st, 2016. We randomly split the sample into training (70%) and validation (30%) cohorts at the patient-encounter level. We trained three ML algorithms to predict 180-day mortality and describe performance in the holdout validation cohort. From October 2018 to February 2019, we used the best-performing algorithm to generate weekly lists of high-risk patients at a single community oncology practice and studied the impact on rates of documented serious illness conversations (SICs).

Results: Among 62,377 encounters used to train the algorithms, 7.4% involved a patient who died within 180 days. Gradient boosting and/or random forest outperformed logistic regression in all metrics (Table), and the gradient boosting model had superior discrimination and calibration. In the gradient boosting model, observed 180-day mortality was 45.5% (95% CI 39.0-52.3%) in the high-risk group vs. 3.3% (95% CI 2.9-3.7%) in the low-risk group. In a survey of oncology clinicians, 59% of patients flagged as high-risk were appropriate for a serious illness conversation in the upcoming week (response rate 52%). Five months after implementing the intervention, average monthly documented SICs increased by 23% (31.7 to 39).

Conclusions: A ML algorithm based on EHR data accurately identified patients with cancer at risk of short-term mortality, was concordant with oncologists’ assessments, and was associated with more SICs.

Table

Area under the curve

Negative predictive value

Positive predictive value

Sensitivity

Specificity

Logistic regression

0.86

0.96

0.44

0.22

0.99

Random forest

0.86

0.96

0.48

0.19

0.99

Gradient boosting

0.87

0.97

0.44

0.27

0.98

Disclaimer

This material on this page is ©2024 American Society of Clinical Oncology, all rights reserved. Licensing available upon request. For more information, please contact licensing@asco.org

Abstract Details

Meeting

2019 Supportive Care in Oncology Symposium

Session Type

Oral Abstract Session

Session Title

Oral Abstract Session A

Track

Prognostication ,Coordination and Continuity of Care,Caregiver Support

Sub Track

Prognostication

Citation

J Clin Oncol 37, 2019 (suppl 31; abstr 131)

DOI

10.1200/JCO.2019.37.31_suppl.131

Abstract #

131

Abstract Disclosures

Similar Abstracts

First Author: Ravi Bharat Parikh

First Author: Chris Manz

First Author: Brandon Butler

First Author: Prathamesh Parchure