Background: Prostate cancer affects millions of men globally and biochemical recurrence (BCR) is an important indicator of its progression. With the increasing use of computational techniques, there has been a growing interest in utilizing whole slide images (WSIs) of H&E stained prostate tissue to evaluate the patients’ clinical risks. Our study aimed to develop a novel method for predicting BCR by performing clustering on features extracted from deep learning models, thereby capturing morphological characteristics. Methods: The WSIs and clinical information were collected from The Cancer Genome Atlas (TCGA, from 2000 to 2013) dataset, for 321 patients (43 patients with BCR) who were not treated with any adjuvant therapies, with 2.5 years of median follow-up time. 356 collected WSIs were split into 1024 x 1024 patches at 5x magnification for the deep learning-based analysis. We used an ImageNet-pretrained deep learning-based clustering model (SwAV) to extract features from each patch and applied additional K-means clustering to assign the patches into one of 16 clusters. We tested both the cosine similarity and Euclidean distance as the clustering measure. After excluding the dissimilar patches under specific thresholds, the proportion of each cluster was calculated per slide to describe the distribution of morphological patterns as a 16-dimensional vector. To evaluate the predictability, we trained Cox models with the proportion vectors and assessed their performance via the concordance index (c-index). We repeated a stratified 5-fold cross-validation process 100 times to achieve robustness. As a reference, we conducted the same experiment with a BCR prediction nomogram published by MSKCC. Moreover, with the median proportion of each cluster, we separated two patient groups, drawed Kaplan-Meier curves, performed the log-rank test for significance, and we obtained the hazard ratio (HR) of each cluster from the trained Cox model of each fold. Results: We explored several thresholds to find ones yielding the best performance. When using the similarity, the best c-index was 0.704 with the threshold value of 0.25. With the distance value of 0.60, the c-index was 0.701. When using the MSKCC nomogram that incorporates clinical features, the average c-index was 0.717. The table contains the results of statistical analysis for selected clusterings that were significant in the log-rank test. Particularly in cluster B, the patches showed high-risk patterns including perineural invasion, high grade cancer, and necrosis in agreement with the high HR. Conclusions: Our results suggest that the histomorphological pattern profiles depicted by the deep learning model have the potential for effective risk stratification in prostate cancer.