Background: The hemoglobin glycation index (HGI) has been proposed as a marker to quantify inter-individual variation in hemoglobin glycosylation. However, whether HGI is associated with an increased risk of diabetic complications independent of glycated hemoglobin (HbA1c) remains unclear. This meta-analysis aimed to determine the association between HGI and the risk of all cause mortality and composite cardiovascular disease (CVD). Methods: PubMed, and EMBASE databases were searched for related studies up to March 31, 2021. Observational studies reported associations between HGI levels and composite CVD and all cause mortality were included for meta-analysis. A random effect model was used to calculate the hazard ratios (HRs) and 95% confidence intervals (CI) for higher HGI. Results: A total of five studies, comprising 22,035 patients with type two diabetes mellitus were included for analysis. The median follow-up duration was 5.0 years. After adjusted for multiple conventional cardiovascular risk factors, an increased level of HGI was associated with a higher risk of composite CVD (per 1 SD increment: HR = 1.14, 95% CI = 1.04–1.26) and all cause mortality (per 1 SD increment: HR = 1.18, 95% CI = 1.05–1.32). However, when further adjusted for HbA1c, the association between HGI and risk of composite CVD (per 1 SD increment of HGI: HR = 1.01, 95% CI = 0.93–1.10) and all cause mortality (per 1 SD increment of HGI: HR = 1.03, 95% CI = 0.96–1.10) became insignificant. Conclusions: High HGI was associated with an increased risk of composite CVD and all cause mortality after adjustment for multiple conventional cardiovascular risk factors. However, the association was mainly mediating by the level of HbA1c.