Currently used tumor markers for early diagnosis of prostate cancer (PCa) are often lacking sufficient specificity and sensitivity. Therefore, the diagnostic potential of selected microRNAs in comparison to serum PSA levels and PSA density (PSAD) was explored. A panel of 12 PCa-associated microRNAs was quantified by qPCR in urinary sediments from 50 patients with suspected PCa undergoing prostate biopsy, whereupon PCa was detected in 26 patients. Receiver operating characteristic (ROC) curve analyses revealed a potential for non-invasive urine-based PCa detection for miR-16 (AUC = 0.744, p = 0.012; accuracy = 76%) and miR-195 (AUC = 0.729, p = 0.017; accuracy = 70%). While serum PSA showed an insufficient diagnostic value (AUC = 0.564, p = 0.656; accuracy = 50%) in the present cohort, PSAD displayed an adequate diagnostic performance (AUC = 0.708, p = 0.031; accuracy = 70%). Noteworthy, the combination of PSAD with the best candidates miR-16 and miR-195 either individually or simultaneously improved the diagnostic power (AUC = 0.801–0.849, p < 0.05; accuracy = 76–90%). In the sub-group of patients with PSA ≤ 10 ng/mL ( n = 34), an inadequate diagnostic power of PSAD alone (AUC = 0.595, p = 0.524; accuracy = 68%) was markedly surpassed by miR-16 and miR-195 individually as well as by their combination with PSAD (AUC = 0.772–0.882, p < 0.05; accuracy = 74–85%). These findings further highlight the potential of urinary microRNAs as molecular markers with high clinical performance. Overall, these results need to be validated in a larger patient cohort.