Background: The prognostic significance of perineural and/or lymphovascular invasion (PLVI) and its relationship with tumor regression grade (TRG) in patients with locally advanced rectal cancer (LARC) treated with neoadjuvant chemoradiotherapy (CRT) and surgery. Methods: A total of 324 patients with LARC treated with CRT were operated on between January 1992 and June 2007. Tumors were graded using a quantitative 5-grade TRG classification, and the presence of PLVI was studied histologically. Results: At a median follow-up of 79.0 months (range 3–250 months), a total of 80 patients (24.7%) relapsed. The observed 5- and 10-year overall survival (OS) was 83.2% and 74.9% respectively. The 5- and 10-year disease-free survival (DFS) was 75.1% and 71.4%, respectively. A significant correlation was found between the TRG and survival (log rank, p<0.001). The 10-year OS and DFS was 32.7% and 31.8% for grade 1; 63.8% and 58.6% for grade 2; 75.0% and 70.4% for grade 3; 90.4% and 88.4% for grade 3+, and 96.0% and 97.1% for grade 4. In patients with PLVI, the TRG had no impact on survival. When excluding patients with PLVI, TRG was an independent prognostic factor for OS and DFS. Conclusions: The presence of PLVI is a more powerful prognostic factor than TRG in LARC patients treated with neoadjuvant CRT followed by surgery. PLVI denotes an aggressive phenotype, suggesting that these patients may benefit from adjuvant systemic therapy.