Abstract The coupled lateral–torsional dynamics of parallel rotor-bearing systems has been intensively investigated. However, little attention has been paid to the analysis of coupled vibrations of angled rotor–bearing systems so that the torsional and the lateral vibrations of those systems are usually analyzed separately. In this paper, the coupled axial–lateral–torsional dynamics of a rotor–bearing system geared by bevel gears is studied. The meshing of two spur bevel gears is analyzed on the basis of a pair of virtual cylindrical gears, and thereafter the constraint condition describing the relationship between the generalized displacements of bevel gears is derived under some assumptions. The coupled dynamic model is established by using Lagrange's equation under this constraint condition. The numerical results of a number of case studies show that the critical speeds of the coupled model are different from those of the uncoupled model both in values and modes, and the threshold speed of stability is fairly less than that of the uncoupled model. The effects of system parameters, such as the pitch cone angles, on the coupling behavior are also discussed.