Abstract In this paper, we consider a kind of innovational model for fractional Brownian motions (fBm), which is different from those defined by Mandelbrot and van Ness [SIAM Rev. 10 (4) (1968) 422], Barton and Vincent Poor [IEEE Trans. Informat. Theory 34 (5) (1988) 943] and Decreusefond and Üstünel [Potential Anal. 10 (1999) 177]; and give the whitening filter formula for the self-similar process BH0(t) by means of fractional differential–integral technique.