Implementing clinical decision support for oncology advanced care planning: A systems engineering framework to optimize the usability and utility of a machine learning predictive model in clinical practice.

Authors

null

Teja Ganta

Tisch Cancer Institute, Division of Hematology and Medical Oncology, Icahn School of Medicine at Mount Sinai, New York, NY

Teja Ganta, Stephanie Lehrman, Rachel Pappalardo, Madalene Crow, Meagan Will, Mark Liu, Robbie Freeman, Arash Kia, Prathamesh Parchure, Alla Keyzner, Madhu Mazumdar, Aarti Sonia Bhardwaj, Cardinale B. Smith

Organizations

Tisch Cancer Institute, Division of Hematology and Medical Oncology, Icahn School of Medicine at Mount Sinai, New York, NY, Institute for Healthcare Delivery Science, Icahn School of Medicine at Mount Sinai, New York, NY, Health System Operations, Mount Sinai Health System, New York, NY, Tisch Cancer Institute, Icahn School of Medicine at Mount Sinai, New York, NY, Institute for Healthcare Delivery Science, Tisch Cancer Institute, Department of Population Health Science and Policy, Icahn School of Medicine at Mount Sinai, New York, NY, Division of Hematology and Medical Oncology, Tisch Cancer Institute at Mount Sinai, New York, NY

Research Funding

No funding received
None

Background: Machine learning models are well-positioned to transform cancer care delivery by providing oncologists with more accurate or accessible information to augment clinical decisions. Many machine learning projects, however, focus on model accuracy without considering the impact of using the model in real-world settings and rarely carry forward to clinical implementation. We present a human-centered systems engineering approach to address clinical problems with workflow interventions utilizing machine learning algorithms. Methods: We aimed to develop a mortality predictive tool, using a Random Forest algorithm, to identify oncology patients at high risk of death within 30 days to move advance care planning (ACP) discussions earlier in the illness trajectory. First, a project sponsor defined the clinical need and requirements of an intervention. The data scientists developed the predictive algorithm using data available in the electronic health record (EHR). A multidisciplinary workgroup was assembled including oncology physicians, advanced practice providers, nurses, social workers, chaplain, clinical informaticists, and data scientists. Meeting bi-monthly, the group utilized human-centered design (HCD) methods to understand clinical workflows and identify points of intervention. The workgroup completed a workflow redesign workshop, a 90-minute facilitated group discussion, to integrate the model in a future state workflow. An EHR (Epic) analyst built the user interface to support the intervention per the group’s requirements. The workflow was piloted in thoracic oncology and bone marrow transplant with plans to scale to other cancer clinics. Results: Our predictive model performance on test data was acceptable (sensitivity 75%, specificity 75%, F-1 score 0.71, AUC 0.82). The workgroup identified a “quality of life coordinator” who: reviews an EHR report of patients scheduled in the upcoming 7 days who have a high risk of 30-day mortality; works with the oncology team to determine ACP clinical appropriateness; documents the need for ACP; identifies potential referrals to supportive oncology, social work, or chaplain; and coordinates the oncology appointment. The oncologist receives a reminder on the day of the patient’s scheduled visit. Conclusions: This workgroup is a viable approach that can be replicated at institutions to address clinical needs and realize the full potential of machine learning models in healthcare. The next steps for this project are to address end-user feedback from the pilot, expand the intervention to other cancer disease groups, and track clinical metrics.

Disclaimer

This material on this page is ©2024 American Society of Clinical Oncology, all rights reserved. Licensing available upon request. For more information, please contact licensing@asco.org

Abstract Details

Meeting

2021 ASCO Quality Care Symposium

Session Type

Poster Session

Session Title

Poster Session B: Patient Experience; Quality, Safety, and Implementation Science; Technology and Innovation in Quality of Care

Track

Technology and Innovation in Quality of Care,Patient Experience,Quality, Safety, and Implementation Science,Cost, Value, and Policy,Health Care Access, Equity, and Disparities

Sub Track

Use of IT/Analytics to Improve Quality

Citation

J Clin Oncol 39, 2021 (suppl 28; abstr 330)

DOI

10.1200/JCO.2020.39.28_suppl.330

Abstract #

330

Poster Bd #

F8

Abstract Disclosures

Similar Abstracts

First Author: Ravi Bharat Parikh

First Author: Thomas J Roberts

Abstract

2023 ASCO Quality Care Symposium

Use of EMR data to validate performance of machine learning prognostic models for patients with cancer.

First Author: Tyler Raclin