Background: Colorectal cancer was the most prevalent malignant disease in developed country and cause 45% mortality rate. Rectal cancer accounted 40% of this disease and is difficult to treat due to its anatomical position. Preoperative neoadjuvant radio-chemotherapy for local advanced rectal cancer can increase circumferential resection margin (CRM) which lead to lower local recurrence rate. MRI was suggested as staging methods for CRM status and nodal evaluation. However, MRI resources were much less than CT scans. CT scans was prevalent and can provide substantial agreement in CRM evaluation under experienced specialist. This study tries to use deep learning to detect local advanced rectal cancer in CT images at initiation of treatment as advisement for neoadjuvant chemoradiotherapy. Methods: From 2010.10.1~2021.12.31, rectal cancer patients with cT3-4 lesion was included. Collect high quality CT images with DICOM files in tumor segment. cT1-2, low rectal lesions, non-CRC cases were not included. Non-contrast and artificial defect images were also excluded. CT images were labeled as” diseased “ when CRM were threatened ( < 2mm). All images were labeled according to judgment of 2 specialist. The data were separated into 2 parts. One for AI model training and testing, another for external validation. The training testing dataset was achieved by deep learning neuro network and evaluating model accuracy performance. Then the model was applied into external validation dataset for real-world testing, evaluating coherent rate between AI and the Dr. decision. Furthermore, to see the cancer survival outcomes according to AI model prediction results. Results: From 2010.10.1~2022.12.31, 1256 rectal cancer cases images were included. 706 cases (889 CT series, 5958 images) with cT3-4 lesions and high quality DICOM files were eligible. 620 cases (803 series, 5318 images) with cT3-4 lesions were classified in training validation set. 1907 (35.9%) images were diseased, 3411 (64.1%) images were normal. 86 clinical stage 2-3 cases (640 images) directly underwent resection during 2014.12.1-2017.5.31 were classified as external validation group. For recognizing CRM+ local advanced rectal cancer by EfficientNetB0, the algorithm achieved an area under the receiver operating characteristic curve (AUC): 0.91 (86.1% accuracy). For External validation set, the sensitivity for AI to detect CRM+ by Dr was 80%; 88% (by images; by cases), the specificity was 68%; 66% (by images; by cases). The survival curve (local recurrence rate, DFS, OS) was also superior in AI- group than AI+ group which was comparable with the results of Dr. Conclusions: Deep Learning can detect local advanced rectal cancer in CT images and prediction results also implicate poor disease outcome.