Background Prospective studies linking social factors to long term patterns of physical activity are lacking. In this 22 year longitudinal study, we seek to identify long term patterns of involvement in leisure time physical activity (LTPA), and explore socioeconomic and demographic predictors of distinct LTPA trajectories. Methods Among 2102 individuals aged 18–60 years in 1981 who participated in the 1981 Canada Fitness Survey/1988 Campbell's Survey of Well-Being, 1186 (56.4%) completed questionnaires for the 2002/04 follow-up. Complete data on LTPA at all 3 surveys were available for 884 participants. Latent class growth analysis was used to identify major classes of LTPA trajectories; predictors of class membership were identified using polytomous logistic regression. Results Four latent classes were identified: inactive, increasers, active, and decreasers (53%, 26%, 12%, and 9% of participants, respectively). Women, older participants, those with lower household income, and with lower educational attainment, were significantly less likely to follow active (Vs. inactive) trajectories of LTPA. Disadvantaged groups with respect to education and income were also significantly more likely to follow decreasing (Vs. active) trajectories. Conclusion There is a need for continued efforts to increase overall population levels of LTPA, particularly among socially disadvantaged groups with respect to income and education, who are most likely to experience unfavorable trajectories of LTPA.