Underdetermined reverberant audio-source separation is an important issue in speech and audio processing. To solve this problem, many separation algorithms have been proposed, in which model parameter estimation is performed in the time–frequency domain, leading to permutation ambiguity and poor separation performance. Additionally, in the existing expectation–maximization (EM) algorithms, one of the crucial problem is that updating the model parameters at each iterative step is time-consuming. In this paper, we present an improved EM algorithm that combines nonnegative matrix factorization (NMF) and time differences of arrival (TDOA) estimation, avoiding the time consumption by properly selecting initial values of the EM algorithm. In the proposed algorithm, NMF source model is used to avoid the permutation ambiguity problem, and acoustic localization can be achieved by transforming the TDOA. Then, model parameters are updated to obtain better separation results. Finally, the source signals are separated using Wiener filters. The experimental results show that compared with existing blind separation methods, the proposed algorithm achieves better performance on source separation.