Background: Trimodality therapy consisting of chemoradiation followed by surgery (CRT+S) is considered optimal therapy for locally advanced adenocarcinoma (ACA) of the esophagus, yet the morbidity of surgery is substantial. We used CT-derived body composition and age to categorize patients into perioperative surgical mortality risk groups. Among patients with locally advanced esophageal ACA who were treated with CRT, the benefit of surgery (CRT+S) was examined in low-risk and high-risk patients. Methods: The association of perioperative mortality with age and skeletal muscle gauge (SMG) was estimated using a group of 290 patients undergoing esophagectomy of various histologies between 2010 and 2022. CT scans were analyzed to calculate SMG as the product of skeletal muscle area and skeletal muscle density. Logistic regression models were used to evaluate the association of age and SMG with 90-day surgical mortality. Patients were classified based on their predicted probability of 90 day mortality into low and high risk groups, where high risk was defined as the highest-risk 25% of patients. This model was then applied to the study cohort of patients with ACA. Log-rank tests were used to compare overall survival by treatment (CRT vs CRT+S) in the high risk and low risk groups. Results: Of 316 patients with locally advanced ACA treated with CRT, 254 underwent surgery (CRT+S). 90-day post operative mortality in the CRT+S group was 5.1%. 74 patients were categorized as high-risk and 242 as low-risk. Among low-risk patients, 213 (88.0%) were treated with surgery (CRT+S) and had a significantly longer overall survival compared to CRT (p<0.0001), with median survival of 40.6mo (95% CI 32.3, 55.5) in the CRT+S group and 17.3mo (95% CI 13.3, 36.4) in the CRT group. Among 74 high risk patients, 41 (55.4%) were treated with surgery. We did not find a difference in overall survival between the surgical group (CRT+S) and the group treated with CRT alone (p=0.34). The median survival was 27.4mo (95% CI 15.8, 32.9) in the CRT group compared to 23.0mo (95% CI 14.7, 54.4) in the CRT+S group. Conclusions: Our retrospective study demonstrated that for low-risk patients with adenocarcinoma of the esophagus, there is a survival benefit to the addition of surgery. Conversely, we failed to detect a survival benefit to the addition of surgery in the highest-risk quartile of patients. Measurement of body composition may help identify a high-risk subset of patients with locally advanced adenocarcinoma of the esophagus who may benefit from forgoing surgery after chemoradiation.