Background Diabetes disproportionately affects Latinos. However, examining Latinos as one group obscures important intra-group differences. This study examined how generational status, duration of US residence, and language preference are associated with diabetes prevalence and to what extent these explain the higher prevalence among Latinos. Methods We determined nativity, duration of US residence, language preference, and diabetes prevalence among 11 817 Latino, 6109 black, and 52 184 white participants in the California Men's Health Study. We combined generational status and residence duration into a single migration status variable with levels: ≥ third generation; second generation; and immigrant living in the US for > 25, 16-25, 11-15, or ≤ 10 years. Language preference was defined as language in which the participant took the survey. Logistic regression models were specified to assess the associations of dependent variables with prevalent diabetes. Results Diabetes prevalence was 22%, 23%, and 11% among Latinos, blacks, and whites, respectively. In age-adjusted models, we observed a gradient of risk of diabetes by migration status among Latinos. Further adjustment for socioeconomic status, obesity and health behaviors only partially attenuated this gradient. Language preference was a weak predictor of prevalent diabetes in some models and not significant in others. In multivariate models, we found that odds of diabetes were higher among US-born Latinos than US-born blacks. Conclusion Generational status and residence duration were associated with diabetes prevalence among middle-aged Latino men in California. As the Latino population grows, the burden of diabetes-associated disease is likely to increase and demands public health attention.