In this paper, the problem of fitting the exploratory factor analysis (EFA) model to data matrices with more variables than observations is reconsidered. A new algorithm named ‘zig-zag EFA’ is introduced for the simultaneous least squares estimation of all EFA model unknowns. As in principal component analysis, zig-zag EFA is based on the singular value decomposition of data matrices. Another advantage of the proposed computational routine is that it facilitates the estimation of both common and unique factor scores. Applications to both real and artificial data illustrate the algorithm and the EFA solutions.