Neural compensatory mechanisms associated with broad cognitive abilities have been studied. However, those associated with specific cognitive subdomains (e.g., verbal fluency) remain to be investigated in healthy aging. Here, we delineate: (a) neural substrates of verbal (phonemic) fluency, and (b) compensatory mechanisms mediating the association between these neural substrates and phonemic fluency. We analyzed resting-state functional magnetic resonance imaging from 133 right-handed, cognitively normal individuals who underwent the Controlled Oral Word Association Test (COWAT) to record their phonemic fluency. We evaluated functional connectivity in an established and extended language network comprising Wernicke, Broca, thalamic and anti-correlated modules. (a) We conducted voxel-wise multiple linear regression to identify the brain areas associated with phonemic fluency. (b) We used mediation effects of cognitive reserve, measured by the Wechsler Adult Intelligence Scale—Information subtest, upon the association between functional connectivity and phonemic fluency tested to investigate compensation. We found that: (a) Greater functional connectivity between the Wernicke module and brain areas within the anti-correlated module was associated with better performance in phonemic fluency, (b) Cognitive reserve was an unlikely mediator in younger adults. In contrast, cognitive reserve was a partial mediator of the association between functional connectivity and phonemic fluency in older adults, likely representing compensation to counter the effect of aging. We conclude that in healthy aging, higher performance in phonemic fluency at older ages could be attributed to greater functional connectivity partially facilitated by higher cognitive reserve, presumably reflecting compensatory mechanisms to minimize the effect of aging.