Fungal diversity within plant roots is affected by several factors such as dispersal limitation, habitat filtering, and plant host preference. Given the differences in life style between symbiotic and non-symbiotic fungi, the main factors affecting these two groups of fungi may be different. We assessed the diversity of root associated fungi of Rhododendron decorum using internal transcribed spacer (ITS) sequencing and terminal restriction fragment length polymorphism (T-RFLP) analysis, and our aim was to evaluate the role of different factors in structuring ericoid mycorrhizal (ERM) and non-ericoid mycorrhizal (NEM) fungal communities. Thirty-five fungal operational taxonomic units (OTUs) were found in roots of R. decorum, of which 25 were putative ERM fungal species. Of the two main groups of known ERM, helotialean fungi were more abundant and common than sebacinalean species. Geographic and host patterning of the fungal assemblages were different for ERM and NEM. The distribution of putative ERM fungal terminal restriction fragments (TRFs) showed that there were more common species within ERM than in the NEM fungal assemblages. Results of Mantel tests indicated that the composition of NEM fungal assemblages correlated with geographic parameters while ERM fungal assemblages lacked a significant geographic pattern and instead were correlated with host genotype. Redundancy analysis (RDA) showed that the NEM fungal assemblages were significantly correlated with latitude, longitude, elevation, mean annual precipitation (MAP), and axis 2 of a host-genetic principle component analysis (PCA), while ERM fungal assemblages correlated only with latitude and axis 1 of the host-genetic PCA. We conclude that ERM and NEM assemblages are affected by different factors, with the host genetic composition more important for ERM and geographic factors more important for NEM assemblages. Our results contribute to understanding the roles of dispersal limitation, abiotic factors and biotic interactions in structuring fungal communities in plant roots.