Background: Computed tomography (CT) and H&E whole-slide images (WSI) have been found to carry rich prognostic information for patients diagnosed with head and neck squamous cell carcinoma (HNSCC). However, most machine learning models aimed for outcome prediction only took advantage of single image modality. In this work, we developed and validated a prognostic machine learning method incorporating both CT and WSI to predict overall survival in HNSCC patients. Methods: Matched radiographic CT scans and digitized WSI were acquired from the Cleveland Clinic for 167 HNSCC patients, including 120 HPV-associated oropharyngeal cancer and 47 laryngeal cancer. Both primary tumor and the largest suspicious lymph node were annotated on CT scans and primary tumor was delineated on H&E WSI. We split the dataset into training and validation set using a 7:3 ratio, which resulted in 119 patients in the training set and 48 for hold-out validation. We applied a machine learning model (M_ML) using both CT and WSI as input to perform end-to-end predictions of overall survival. We used the harrell’s concordance index (C-index) to evaluate the prognostic performance. Finally, we performed the multivariable cox proportional hazard analysis adjusting for clinicopathological variables (i.e. age, gender, smoking pack-year [PY], AJCC 7^th edition overall stage, and treatment modality) to validate the independent prognostic significance of the model. Results: The combined machine learning model M_ML (C-index = 0.81) outperformed the model using CT images alone (C-index = 0.63) and WSI alone (C-index = 0.64) on the validation set. In multivariable analysis, M_ML is still statistically significant accounting for clinicopathologic factors (p = 0.0007). Conclusions: This pilot study shows that a multi-omic machine learning model utilizing both radiographic CT and digitized WSI can predict HNSCC overall survival and outperforms models using only a single modality.

Multivariable cox proportional hazard analysis accounting for clinicopathologic factors.