Background: Overall survival of patients with metastatic non-small cell lung cancer (NSCLC) has increased with the use of anti-PD-1 immune checkpoint inhibitors. However, the duration of response remains highly variable between patients, and only 20-30% of patients are alive at 2 years. Thus, new biomarkers for predicting response to treatment and patient outcomes are still needed to guide therapeutic decision. In this study, we retrospectively investigated multimodal approaches that might improve the limited predictive power of clinical data. Methods: We studied a cohort of 317 patients with metastatic NSCLC treated with first-line immune checkpoint inhibitors alone or combined with platinum-based chemotherapy. Clinical data were collected for all patients, pathological slides (HES and PD-L1 staining) and baseline 18-FDG PET/CT scans were available in 237 and 130 patients respectively. An automatic cell type detection algorithm was applied to each pathological slide and pathomic features were extracted from the resulting annotations. After semi-automated segmentation of all tumor foci in the PET/CT scans, radiomic features were calculated for each tumor lesion and aggregated across all the lesions of each patient. Prognostic models were built using random forest and XGboost classifiers to predict patient survival at 12 months based on 1) features from single modalities (clinical, pathomic, or radiomic), 2) features from multiple modalities, where early fusion and late fusion strategies were investigated. The models were trained and tested with cross-validation and their performances were established using the area under the ROC curve (AUC) computed on the same 88 test patients for whom all the modalities were available. Results: Unimodal strategies yielded AUC of 0.62 ± 0.08 (1 std), 0.64 ± 0.07, 0.59 ± 0.08 for clinical, radiomic and pathomic features respectively. With late fusion, bimodal models consistently outperformed the clinical model, with the combination of radiomic and clinical features giving the best performance (AUC = 0.67 ± 0.07). The trimodal model outperformed all other modality combinations with an AUC of 0.69 ± 0.07; in particular, it was significantly superior to the clinical model (p-value < 0.001, paired t-test). The early fusion experiments confirmed the superiority of every bimodal approach over the clinical model. However, the trimodal model did not outperform the best bimodal model with early fusion. Validation will be performed on independent cohorts from external centers. Conclusions: Our study highlighted the potential of multimodal approaches for predicting the outcome of metastatic NSCLC patients treated with immunotherapy. Models integrating medical images and pathological slides usually collected from routine care outperformed a model trained on clinical data alone.