Introduction Considering the increase in the number of patients with diabetes, the quality of diabetes care provided by general practitioners (GP) is critical for preventing complications. We performed a nationwide survey to determine whether the diabetic management provided to patients with type 2 diabetes mellitus (T2DM) by Japanese practitioners is appropriate. Methods We randomly selected 463 clinics throughout Japan; 8070 patients with T2DM (6525 and 1545 under the care of GP and specialists [SP], respectively) were enrolled. We obtained information on hemoglobin A1c (HbA1c) levels, age, height, body weight, diabetes type and treatment modality, blood pressure (BP), and hypertension or dyslipidemia from each patient. Additionally, we surveyed the collaborations among physicians. Results The median HbA1c level of patients treated by GP was lower than that of patients treated by SP (6.8 [6.2–7.3], median [interquartile range] vs. 6.9 [6.5–7.5], p < 0.0001). The percentage of patients receiving insulin therapy was also higher (23.8%) among patients treated by SP than among those treated by GP (8.6%). Patients not receiving insulin therapy showed lower median HbA1c levels than those receiving insulin therapy, irrespective of the care provider. The mean body mass index of patients with HbA1c levels < 6.9% or > 9.0% cared for by SP was lower than that of those cared for by GP. The rate of target BP (< 140/90 mmHg) achievement was 73.2% and 73.3% among patients with T2DM and hypertension cared for by GP and SP, respectively. Furthermore, 88.2% of GP reported that consulting with SP was easy. Conclusion The present study clearly demonstrated that many patients with T2DM are appropriately cared for by general practitioners instead of diabetes specialists in Japan, although the number of diabetes specialists is insufficient to cover all patients with diabetes.