Although the association between stress and poor reproductive health is well established, this association has not been examined from a life course perspective. Using data from the National Longitudinal Survey of Youth 1997 cohort (N = 1652), we fit logistic regression models to test the association between stressful life events (SLEs) (e.g., death of a close relative, victim of a violent crime) during childhood, adolescence, and early adulthood and later experiences of infertility (inability to achieve pregnancy after 12 months of intercourse without contraception) reported by female respondents. Because reactions to SLEs may be moderated by different family life experiences, we stratified responses by maternal responsiveness (based on the Conger and Elder Parent-Youth Relationship scale) in adolescence. After adjusting for demographic and environmental factors, in comparison to respondents with one or zero SLEs, those with 3 SLEs and ≥ 4 SLEs had 1.68 (1.16, 2.42) and 1.88 (1.38, 2.57) times higher odds of infertility, respectively. Respondents with low maternal responsiveness had higher odds of infertility that increased in a dose-response manner. Among respondents with high maternal responsiveness, only those experiencing four or more SLEs had an elevated risk of infertility (aOR = 1.53; 1.05, 2.25). In this novel investigation, we demonstrate a temporal association between the experience of SLEs and self-reported infertility. This association varies by maternal responsiveness in adolescence, highlighting the importance of maternal behavior toward children in mitigating harms associated with stress over the life course.