Background: Colorectal cancer (CRC) is the third most prevalent and the second deadliest cancer globally, accounting for an estimated 1,926,000 new cases and 903,800 deaths in the year 2022 alone. It's also established that chronic inflammatory conditions, such as Crohn's disease and ulcerative colitis, elevate the risk of developing CRC. Therefore, a timely CRC diagnosis for patients with colorectal disease can significantly improve their prognosis and therefore reduce cancer-related mortalities. Methods: In this study, we enrolled a total of 167 patients diagnosed with CRC (stages I: 43, II: 55, III: 56, IV: 13) and 217 patients with benign colorectal disease (BCD), from five participating hospitals. All participants underwent colonoscopy procedures, and their CRC or BCD status was pathologically confirmed. The development of the cancer detection model was based on a training cohort of 98 CRC patients and 131 BCD patients from three hospitals, and the evaluation of the model's performance was carried out on an independent test cohort comprising 69 CRC patients and 96 BCD patients from the remaining two hospitals. The predictive model leveraged cell-free fragmentomics profiling, employing low-pass whole-genome sequencing (WGS) on pre-operative plasma samples to identify CRC. Results: The early detection model exhibited remarkable efficacy in differentiating CRC patients from those with BCD, achieving an Area Under the Curve (AUC) of 0.930 in the training cohort and a nearly equivalent AUC of 0.926 in the independent test cohort. By applying the same cutoff determined in the training cohort, our model demonstrated a high sensitivity (91.8% in the training cohort and 91.3% in the test cohort) and a satisfactory specificity (78.6% in the training cohort and 82.3% in the test cohort). Notably, the model showed consistent and robust performance across various CRC characteristics, including tumor location, histology subtypes, grades, and mismatch repair deficiency status in both cohorts. Moreover, our model outperformed traditional CRC screening methods such as the fecal immunochemical test, carcinoembryonic antigen (CEA), and CA19-9, which achieved AUCs of 0.742, 0.720, and 0.543, respectively. Further validation was conducted using an additional cohort of 31 patients with advanced colorectal adenoma (advCRA), where the model achieved an AUC of 0.846 and a 66.7% sensitivity for distinguishing advCRA from BCD. Conclusions: The study introduces a non-invasive, cell-free fragmentomics assay that incorporates low-pass WGS and machine learning algorithms, demonstrating significant clinical potential in accurately distinguishing CRC and BCD patients. Our innovative approach offers a promising alternative to traditional screening methods that can improve early CRC detection, enhance CRC patient outcomes, and reduce mortality rates.