Failure rates after first-line treatment of localized prostate cancer (PCa) treatment remain high. Improvements to patient selection and identification of at-risk patients are central to reducing mortality. We aimed to determine if cancer aggressiveness correlates with androgen levels in patients undergoing radical prostatectomy for localized PCa. We performed a prospective, multicenter cohort study between June 2013 and June 2016, involving men with localized PCa scheduled to undergo radical prostatectomy. Clinical and hormonal patient data (testosterone deficiency, defined by total testosterone (TT) levels < 300 ng/dL and/or bioavailable testosterone (BT) levels < 80 ng/dL) were prospectively collected, along with pathological assessment of preoperative biopsy and subsequent radical prostatectomy specimens, using predominant Gleason pattern (prdGP) 3/4 grading. Of 1343 patients analyzed, 912 (68%) had prdGP3 PCa and 431 (32%) had high-grade (prdGP4, i.e., ISUP ≥ 3) disease on prostatectomy specimens. Only moderate concordance in prdGP scores between prostate biopsies and prostatectomy specimens was found. Compared with patients with prdGP3 tumors (i.e., ISUP ≤ 2), significantly more patients with prdGP4 cancers had demonstrable hypogonadism, characterized either by BT levels (17.4% vs. 10.7%, p < 0.001) or TT levels (14.2% vs. 9.7%, p = 0.020). BT levels were also lower in patients with prdGP4 tumors compared to those with prdGP3 disease. Testosterone deficiency (defined by TT and/or BT levels) was independently associated with higher PCa aggressiveness. BT is a predictive factor for prdGP4 disease, and evaluating both TT and BT to define hypogonadism is valuable in preoperative assessment of PCa (AndroCan Trial: NCT02235142).