Impairments in language and communication are core features of autism spectrum disorder (ASD). The anatomy of critical language areas has been studied in ASD with inconsistent findings. We used MRI to measure gray matter volume and asymmetry of Heschl’s gyrus, planum temporale, pars triangularis, and pars opercularis in 40 children and adolescents with ASD and 40 typically developing individuals, each divided into younger (7–11 years) and older (12–19 years) cohorts. The older group had larger left planum temporale volume and stronger leftward asymmetry than the younger group, regardless of diagnosis. The pars triangularis and opercularis together were larger in ASD than controls. Correlations between frontal language areas with language and symptom severity scores were significant in younger ASD children. Results suggest similar developmental changes in planum temporale anatomy in both groups, but group differences in pars triangularis and opercularis that may be related to language abilities and autism symptom severity.