Abstract Inter-frame dependencies are usually ignored in video encoder coding parameter selection. This gives a non-optimal solution and degrades the compression performance. A mathematical model to estimate the importance of each pixel on the reconstructed video quality, called PixelRank, is developed in this paper. Theoretical analysis on the parameters used for PixelRank score calculation dealing with the video coding optimization problem is also given. The PixelRank algorithm tracks the importance of each pixel and distributes the PixelRank scores. With the PixelRank scores for all the pixels, MB-based quantization parameters are adjusted accordingly. Based on this technique, the rate can be allocated more accurately according to the importance of the pixels, thus achieving better overall rate-distortion performance. Compared to the non-optimized scheme in H.264/AVC, the proposed scheme can reduce 13.53% of the average bitrate and up to 25.17% of bitrate in the simulations.