Objective. Material decomposition (MD) evaluates the elemental composition of human tissues and organs via computed tomography (CT) and is indispensable in correlating anatomical images with functional ones. A major issue in MD is inaccurate elemental information about the real human body. To overcome this problem, we developed a virtual CT system model, by which various reconstructed images can be generated based on ICRP110 human phantoms with information about six major elements (H, C, N, O, P, and Ca). Approach. We generated CT datasets labelled with accurate elemental information using the proposed generative CT model and trained a deep learning (DL)-based model to estimate the material distribution with the ICRP110 based human phantom as well as the digital Shepp–Logan phantom. The accuracy in quad-, dual-, and single-energy CT cases was investigated. The influence of beam-hardening artefacts, noise, and spectrum variations were analysed with testing datasets including elemental density and anatomical shape variations. Main results. The results indicated that this DL approach can realise precise MD, even with single-energy CT images. Moreover, noise, beam-hardening artefacts, and spectrum variations were shown to have minimal impact on the MD. Significance. Present results suggest that the difficulty to prepare a large CT database can be solved by introducing the virtual CT system and the proposed technique can be applied to clinical radiodiagnosis and radiotherapy.