Abstract This paper presents a principal component analysis (PCA) based data compression method for the image-base relighting (IBL) technology, which needs tremendous reference images to produce high quality rendering. The method contains two main steps, eigen-image based representation and eigen-image compression. We extract eigen-images by the cascade recursive least squared (CRLS) networks based PCA due to the large data dimension. By keeping only a few important eigen-images, which are enough to describe the IBL data set, the data size can be drastically reduced. To further reduce the data size, we use the embedded zero wavelet (EZW) approach to compress those retained eigen-images, and use uniform quantization plus arithmetic coding to compress the representing coefficients. Simulation results demonstrate that our approach is superior to that of compressing reference images separately with JPEG or EZW.