Although diet is an important factor influencing gut microbiota, there are very few studies regarding that relationship in Japanese people. Here, we analyzed the relationship between habitual dietary intake surveyed by food frequency questionnaire and the quantitative features of gut bacteria by quantitative PCR and next generation sequencer in 354 healthy Japanese adults. The α-diversity of gut microbiota was positively correlated with the intake of mushrooms and beans and negatively correlated with the intake of grains. The β-diversity was significantly associated with the intake of fruits, mushrooms, seaweeds, seafoods, and alcoholic beverages. Multiple linear regression analysis of the relationship between food groups associated with the diversity of gut microbiota and the number of gut bacteria at the genus level found 24 significant associations, including a positive association between alcoholic beverages and the number of Fusobacterium . These results support that habitual dietary intake influenced the diversity of gut microbiota and was strongly associated with the number of specific gut bacteria. These results will help us to understand the complex relationship between habitual diet and gut microbiota of the Japanese.