Objectives The relationship between lipid levels and arterial stiffness remains controversial. Therefore, we aimed to determine the relationship between lipid profiles and brachial–ankle pulse wave velocity (baPWV) as an indicator of arterial stiffness. Methods A total of 909 participants aged 24 to 84 years were stratified into four baPWV quartiles in our study. Serum lipids, baPWV, and other variables of the participants were measured. Univariable and multivariable linear regression analyses were used to examine the relationships between lipid parameters and baPWV. Results The highest baPWV quartile group had higher aspartate transaminase, alanine aminotransferase, plasma glucose, total cholesterol, triglyceride (TG), and low-density lipoprotein cholesterol levels and maximum ankle–brachial index, and lower high-density lipoprotein cholesterol levels and estimated glomerular filtration rate. Univariate regression analysis showed that total cholesterol, TG, and low-density lipoprotein cholesterol levels were positively related and high-density lipoprotein cholesterol levels were negatively related to baPWV. After adjusting for age, body mass index, smoking status, aspartate transaminase, alanine aminotransferase, plasma glucose, and estimated glomerular filtration rate, only TG levels were correlated with baPWV (β = 0.075). Conclusions Four lipid variates are associated with arterial stiffness, and TG levels are positively related to arterial stiffness, independent of cardiovascular risks and liver function.