Background: Real-world evidence (RWE) is increasingly important for discovery and may be an opportunity for regulatory approval. Effective use of RWE relies on determining treatment-specific outcomes, such as overall response rate (ORR) and progression-free survival (PFS), which are challenging to accurately evaluate retrospectively and at scale. We hypothesized the use of machine learning of text radiology reports from patients with NSCLC treated with PD-1 blockade could be used to train a model that estimates RECIST-defined outcomes. Methods: 2753 imaging reports from 453 patients with advanced NSCLC treated with PD-1 blockade were collected and separated into independent training (80%, n = 362) and validation (20%, n = 92) cohorts. Reports were limited to interval of PD-1 blockade. RECIST reads performed by thoracic radiologists on all patients served as “gold standard” to determine ORR, occurrence of, and date of progression. Baseline reports were compared to all follow up reports to determine machine-learning RECIST (ml-RECIST). A four layers neural-network model for classification was proposed to solve the three above tasks. Results: In the training cohort, ml-RECIST best estimated ORR by RECIST (accuracy CR/PR 84%, SD 82%, POD 91%). ml-RECIST estimated PFS by RECIST accurately predicting progression occurred at any time (86%) and exact progression date (65%). Date of progression was closely correlated (Pearson’s r coefficient = 0.91, 95% CI:0.89-0.94, p < 0.001) in patients determined to have progressed by both methods. Similar accuracy of ml-RECIST was observed in the validation cohort (accuracy CR/PR 84%, SD 80%, POD 90%; progression occurred 86%, progression date 72%). Accuracy was consistent when RECIST reads were performed prospectively as part of clinical trials vs retrospectively for standard of care treatment (e.g. CR/PR 82% vs 88%, respectively). ml-RECIST-defined response similarly determined improvement in overall survival compared to RECIST (HR = 0.19, p < 0.001 vs HR = 0.26, p < 0.001 respectively). Conclusions: Machine learning-RECIST ("ml-RECIST") accurately estimates outcomes using imaging text reports. ml-RECIST may be tool to determine outcomes expeditiously and at scale for use in RWE studies, enabling more useful and reliable applications of large clinical databases.