Abstract This paper has been established based on harmonic detection and compensation on power system. The present work deals with two powerful methods in harmonic detection field of digital signal processing. The first one is the windowing technique and the other one is wavelet transform. A traditional synchronous fundamental dq-frame algorithm is modified in filtering process. As an improvement, the Fourier-based low pass filter is replaced with a windowing–wavelet method. The adopted windowing aspect is Hamming window to reduce traditional rectangular window effete in frequency domain. In addition, the applied mother wavelet is selected in terms of its effectiveness on transient response, low overshot and low oscillation at frequency domain. Due to these concepts, the db8 is selected as mother wavelet. In harmonic elimination procedure, the detected harmonic is injected using a shunt active filter based on predictive current control technique. The presented method has been verified with MATLAB–SIMULINK model and wavelet toolbox. The results confirm that this method is superior to traditional FFT based and wavelet based filters because of fast transient response and small negligible sidelobes. As a consequence this technique is able to reduce power system harmonic.