While the impact of dietary cholesterol on the progression of atherosclerosis has probably been overestimated, increasing evidence suggests that dietary cholesterol might favor the transition from blunt steatosis to non-alcoholic steatohepatitis (NASH), especially in combination with high fat diets. It is poorly understood how cholesterol alone or in combination with other dietary lipid components contributes to the development of lipotoxicity. The current study demonstrated that liver damage caused by dietary cholesterol in mice was strongly enhanced by a high fat diet containing soybean oil-derived ω6-poly-unsaturated fatty acids (ω6-PUFA), but not by a lard-based high fat diet containing mainly saturated fatty acids. In contrast to the lard-based diet the soybean oil-based diet augmented cholesterol accumulation in hepatocytes, presumably by impairing cholesterol-eliminating pathways. The soybean oil-based diet enhanced cholesterol-induced mitochondrial damage and amplified the ensuing oxidative stress, probably by peroxidation of poly-unsaturated fatty acids. This resulted in hepatocyte death, recruitment of inflammatory cells, and fibrosis, and caused a transition from steatosis to NASH, doubling the NASH activity score. Thus, the recommendation to reduce cholesterol intake, in particular in diets rich in ω6-PUFA, although not necessary to reduce the risk of atherosclerosis, might be sensible for patients suffering from non-alcoholic fatty liver disease.