Background: Immunotherapy (IO) is the standard of care in 1L stage IV non-small cell lung cancer (NSCLC) cases that are not eligible for targeted therapies. Today, a level of PD-L1 expression of 50% or above is the only standard predictive biomarker for IO efficacy as monotherapy. However, a majority of patients fail to respond to the treatment and are exposed to potentially severe immune-related adverse events. There is thus an urgent need to discover new predictive signatures of response to IO in such setting. Methods: A retrospective 1-year cohort of 63 patients with advanced NSCLC, PD-L1 expression > 50%, and treated with 1L pembrolizumab monotherapy was analyzed to develop a machine learning-based algorithm predictive of response to immunotherapy. Multimodal baseline data were collected including clinical, biological, pathology, molecular, baseline CT scan data and clinical outcomes status (objective response at first evaluation, PFS, OS). For each patient, thoracic tumors were segmented in 3D by both an experimented pneumologist and a radiologist using the SOPHiA DDM for Radiomics platform. Radiomics features were then extracted following the IBSI standards and combined with the other data modalities. A filter-based variable selection method was applied before testing several machine learning algorithms to obtain an individual prediction of the response at first evaluation. The optimization criterion was the Area Under the ROC Curve (AUC). Due to the small size of the cohort, a nested leave-pair-out cross-validation was used to properly estimate the model performance. Results: A logistic regression algorithm reached an AUC of 0.85, a precision of 83%, a sensitivity of 77% and a specificity of 74% for predicting response at first evaluation. The algorithm was able to correctly predict 21 progressions among 28 observed and 27 disease controls among 35 observed. Features with highest weight were representative of the full scope of data modalities included in the model, highlighting the importance of a truly multimodal analysis. Indeed, withdrawing any specific data modality (e.g., radiomics features), led to a decrease of at least ̃10% of the AUC. Patients were then stratified into two groups, progression versus disease control, based upon their predicted response status at first evaluation. These two groups displayed a statistically significant difference in PFS (p < 0.001), suggesting that baseline multimodal data analysis could help predict long-term outcomes. Conclusions: This proof-of-concept study suggests that machine learning applied to baseline multi-modal data can help predict response to IO at the individual patient level, as well as stratify patients to inform long-term outcomes. This algorithm is being improved and validated through a large real-world multicentric international observational study including more than 4000 patients (DEEP-Lung-IV study, NCT04994795).