Background: PSMA positron emission tomography (PET) has a higher accuracy than computed tomography (CT) and bone scans to stage patients with prostate cancer. However, we do not understand how to apply clinical trial data based on conventional imaging to patients staged using PSMA PET. Therefore, we aim to evaluate the ability of bone scans to detect osseous metastases using PSMA PET as a reference standard. Methods: In this multicenter retrospective diagnostic study, 167 patients with prostate cancer, who were imaged with bone scan and PSMA PET performed within 100 days were included for analysis. Each study was interpreted by three blinded readers, and the results of the PSMA PET were used as the reference standard. Endpoints were positive predictive value (PPV), negative predictive value (NPV) and specificity for bone scans. Additionally, inter-reader reproducibility, positivity rate, uptake on PSMA PET, and number of lesions were evaluated. Results: 167 patients were included in the study, with 77 at initial staging, 60 in the BCR/CSPC setting and 30 in the CRPC setting. BS findings for metastatic disease validated by PSMA PET were tabulated. In all patients, the PPV, NPV and specificity for bone scans were 0.73 [0.61,0.82], 0.82 [0.74,0.88] and 0.82 [0.74,0.88]. In patients at initial staging, the PPV, NPV and specificity for bone scans were 0.43 [0.26,0.63], 0.94 [0.85,0.98], and 0.80 [0.68,0.88]. At initial staging 13/23 (57%) of positive bone scans were false positive. Inter-reader agreement for bone disease was moderate for bone scans (Fleiss k, 0.51) and substantial for the PSMA PET reference standard (Fleiss k, 0.80). Conclusions: In this multicenter retrospective study, the PPV of bone scans was low in patients at initial staging with majority of positive bone scans being false positives. This suggests that a large proportion of patients considered low volume metastatic by bone scan actually have localized disease.