Background: Advance care planning (ACP) can help patients understand and share their long-term health goals and values, and in turn facilitate the delivery of medical care consistent with those goals and values. However, in a survey of Californians about attitudes towards death and dying, while almost 80% of respondents indicated that they would like to talk to their doctor about end-of-life care, only 7% had ever had a doctor bring up the subject. To address this problem, the Stanford inpatient oncology service implemented an AI-enabled workflow to facilitate timely identification of patients for ACP conversations before discharge. While the workflow implementation has thus far been successful, less is known of the human factors from patients and providers that influence the feasibility and acceptability of this workflow. Methods: To gather perspectives on the AI-enabled tool, we used qualitative research methods to conduct semi-structured interviews with 15 patients and 15 providers. Patients were over the age of 18 and had been recently admitted to the Stanford Inpatient Oncology Service. Providers were over the age of 18 and worked within the Stanford Inpatient Oncology Service in one of four roles: attending oncologist, advanced practice provider, clinical dietician, occupational therapist. Interviews were conducted virtually via Zoom, recorded and transcribed for subsequent thematic analysis. Once transcribed, interviews were qualitatively coded using an inductive approach. Results: Patients and providers affirm the positive potential for AI implementation in healthcare settings, but emphasize the necessity of maintaining human-centered models of care which prioritize strong interpersonal relationships. Providers observe relatively minimal impacts of the AI-enabled tool on existing advance care planning workflows/practices, but express optimism surrounding the implementation of AI tools in inpatient oncology more generally. Patients exhibit similar levels of optimism and a general openness toward AI applications in healthcare settings, but highlight the need for transparency and informed consent throughout the processes of AI development and implementation. Conclusions: This study provides novel insights on the impacts of AI-enabled workflows in inpatient oncology care, as well as patient and provider perspectives concerning the implementation of AI technology in healthcare more broadly. As AI is increasingly developed with clinical care applications in mind, these insights provide valuable foundations for successful implementation.