Abstract A quantitative genetic analysis was performed to assess the suitability of automated image analysis of cutlets as a selection tool to genetically improve flesh composition and colour in large rainbow trout. Fish were reared on two diets with different lipid and protein content to assess the robustness of the image analysis method across different nutritional environments, and the strength of potential genotype-by-diet interactions. Chops were scanned and digitally analysed for colour (chroma, hue, and lightness) and for areas of lipid stripes, dorsal lipid, red muscle and white muscle. Percents of lipid and white muscle area to whole chop area were compared to percent chop lipid and protein recorded by chemical analysis. The results showed that on both diets, percent chop lipid and percent white muscle area displayed high phenotypic (≥ 0.42) and genetic correlations (≥ 0.70) with the respective chemical lipid and protein percents. Moreover, on both diets, the lipid area percents had moderate to high heritabilities ( h 2 = 0.29–0.70) that were of the same magnitude or higher compared to the previously published estimates for chemically analysed lipid traits. These results confirm that regardless of the diet composition, true lipid and protein deposition can be genetically improved by selecting for the respective image analysis traits. Muscle lightness, hue (tint of colour) and chroma (saturation level of colour) displayed moderate to high heritabilities ( h 2 = 0.32–0.46), and phenotypic and genetic correlations between these traits were favourable. Muscle colour can thus be effectively improved by selection. Some genetic constraints for breeding efforts were identified, in terms of lipid deposited at different locations within the body correlating only weakly. Genetic correlations between diets were strongly positive (≥ 0.85), revealing only weak re-ranking of families across the diet treatments.