Background: The current standard of care for men with intermediate- and high-risk localized prostate cancer treated with radiotherapy (RT) is the addition of androgen deprivation therapy (ADT). Presently, there are no validated predictive biomarkers to guide ADT use or duration in such men. Herein, we train and validate the first predictive biomarker for ADT use in prostate cancer using multiple phase III NRG Oncology randomized trials. Methods: Pre-treatment biopsy slides were digitized from five phase III NRG Oncology randomized trials of men receiving RT with or without ADT. The training set to develop the artificial intelligence (AI)-derived predictive biomarker included NRG/RTOG 9202, 9413, 9910, and 0126, and was trained to predict distant metastasis (DM). A multimodal deep learning architecture was developed to learn from both clinicopathologic and digital imaging histopathology data and identify differential outcomes by treatment type. After the model was locked, an independent biostatistician performed validation on NRG/RTOG 9408, a phase III randomized trial of RT +/- 4 months of ADT. The DM rates were calculated using cumulative incidence functions in biomarker positive and negative groups, and biomarker-treatment interaction was assessed using Fine-Gray regression such that death without DM was treated as a competing event. Results: Clinical and histopathological data was available for 5,654 of 7,957 eligible patients (71.1%). The training cohort included 3,935 patients and had a median follow-up of 13.6 years (IQR [10.2, 17.7]). After the AI-derived predictive ADT classifier was trained, it was validated in NRG/RTOG 9408 (n = 1719, median follow-up 17.6 years, IQR [15.0, 19.7]). In the NRG/RTOG 9408 validation cohort that had digital histopathology data, ADT significantly improved DM (HR 0.62, 95% CI [0.44, 0.87], p = 0.006), consistent with the published trial results. The biomarker-treatment interaction was significant (p-value = 0.0021). In patients with AI-biomarker positive disease (n = 673, 39%), ADT had a greater benefit compared to RT alone (HR 0.33, 95% CI [0.19, 0.57], p < 0.001). In the biomarker negative subgroup (n = 1046, 61%), the addition of ADT did not improve outcomes over RT alone (HR 1.00, 95% CI [0.64, 1.57], p = 0.99). The 15-year DM rate difference between RT versus RT+ADT in the biomarker negative group was 0.3%, vs biomarker positive group 9.4%. Conclusions: We have successfully validated in a phase III randomized trial the first predictive biomarker of ADT benefit with RT in localized intermediate risk prostate cancer using a novel AI-derived digital pathology-based platform. This AI-derived predictive biomarker demonstrates that a majority of patients treated with RT on NRG/RTOG 9408 did not require ADT and could have avoided the associated costs and side effects of this treatment.