Minor mergers are thought to be responsible for the size growth of quiescent field galaxies with decreasing redshift. We test this hypothesis using the cluster environment as a laboratory. Satellite galaxies in clusters move at high velocities, making mergers between them rare. The stellar mass-size relation in 10 clusters and in the field is measured and compared at z similar to 1. Our cluster sample contains 344 spectroscopically confirmed cluster members with Gemini/Gemini Multi-Object Spectrographs and 182 confirmed with Hubble Space Telescope/Wide Field Camera 3 G141 grism spectroscopy. On average, quiescent and star-forming cluster galaxies are smaller than their field counterparts by (0.08 +/- 0.04) and (0.07 +/- 0.01) dex, respectively. These size offsets are consistent with the average sizes of quiescent and star-forming field galaxies between 1.2 <= z <= 1.5, implying the cluster environment has inhibited size growth between this period and z similar to 1. The negligible differences measured between the z similar to 0 field and cluster quiescent mass-size relations in other works imply that the average size of quiescent cluster galaxies must rise with decreasing redshift. Using a toy model, we show that the disappearance of the compact cluster galaxies might be explained if, on average, similar to 40 per cent of them merge with their brightest cluster galaxies (BCGs) and similar to 60 per cent are tidally destroyed into the intracluster light (ICL) between 0 <= z <= 1. This is in agreement with the observed stellar mass growth of BCGs between 0 <= z <= 1 and the observed ICL stellar mass fraction at z similar to 0. Our results support minor mergers as the cause for the size growth in quiescent field galaxies, with cluster-specific processes responsible for the similarity between the field and cluster quiescent mass-size relations at low redshift.