Bauxite residues (BRs) are highly alkaline wastes generated during alumina production from bauxite ore. Billions of tons have been accumulating worldwide for more than 100 years, they are stored in various forms, and pose environmental and societal issues. At the same time, BRs are promising secondary sources for the production of critical metals including rare earth elements (REEs). However, knowledge on REE speciation is lacking, and is consequently an obstacle to the development of large-scale extraction process. This study analyses the influence of origin of the bauxite ore (lateritic or karstic), the storage conditions and storage time on the properties of ten BR samples, with a particular focus on the speciation of yttrium, which is used as a proxy to identify the behaviour of heavy REE. A multi-scale approach linked yttrium speciation and the origin of the bauxite ore whereas no major variation was observed as a function of storage conditions or ageing of the BRs. Yttrium is mainly found in the form of xenotime phosphate particles in BRs of lateritic origin, while in karstic BRs, the majority of yttrium is probably adsorbed or incorporated into other minerals including iron oxyhydroxide and hydroxyapatite minerals.