In this paper, we mainly focus on the penalized maximum likelihood estimation of the high-dimensional approximate factor model. Since the current estimation procedure can not guarantee the positive definiteness of the error covariance matrix, by reformulating the estimation of error covariance matrix and based on the lagrangian duality, we propose an accelerated proximal gradient (APG) algorithm to give a positive definite estimate of the error covariance matrix. Combined the APG algorithm with EM method, a new estimation procedure is proposed to estimate the high-dimensional approximate factor model. The new method not only gives positive definite estimate of error covariance matrix but also improves the efficiency of estimation for the high-dimensional approximate factor model. Although the proposed algorithm can not guarantee a global unique solution, it enjoys a desirable non-increasing property. The efficiency of the new algorithm on estimation and forecasting is also investigated via simulation and real data analysis.