Abstract The stress-optic response of glass refers to the birefringence induced by application of anisotropic stress. We showed in an earlier publication [M. Guignard, L. Albrecht, J.W. Zwanziger, Chem. Mater. 19 (2007) 286] that this effect may be predicted based on the combination of bond metallicity and coordination number in the glass, and from the combination of these effects a variety of zero stress-optic glasses can be generated. In the present work we study in detail the relation of the structure and the bond metallicity to the stress-optic response in three glasses, two of which show zero and negative stress-optic response while the other shows only positive response. In particular we show by solid-state nuclear magnetic resonance spectroscopy that barium and lead phosphate glasses are much more similar in structure than tin phosphate glass, but that the lead and tin glasses have similar stress-optic response. The reason is the similar bond metallicity, which we explore by first-principles calculations, and the low metal ion coordination numbers that the lead and tin glasses exhibit.