Background Although the number of positive lymph nodes (LN), the total number of resected LN, and the proportion of positive LN have been reported to be associated with survival in patients with esophageal cancer, little is known about the prognostic impact of the number of negative LN. Methods A retrospective review of 754 patients receiving transthoracic esophagectomy for cancer between January 1995 and September 2011 was performed. The prognostic impact of the number of negative LN was analyzed. Log rank testing was used to compare survival curves, and Cox regression analysis was performed to identify significant prognostic factors. Results A total of 707 patients were included. The mean follow-up time was 28.4 ± 30.9 months. The 5-year overall survival for the entire cohort was 30.1%. Patients with a high number of negative LN (≥19) had better overall survival than patients with a low number of negative LN (5-year survival rate 33.4% versus 26.4%, p = 0.005). Stratified analysis showed that the impact of the number of negative LN was significant in patients with T3/4 (p = 0.027) and node-positive (p = 0.002) esophageal cancers but not in patients with less advanced tumors. Multivariate Cox regression analysis demonstrated that the number of negative LN (in addition to age, sex, T stage, N stage, tumor length, and surgical approach) was an independent prognostic factor. Conclusions A higher number of negative LN is associated with better overall survival of esophageal cancer patients after resection. The correlation of a high number of negative LN (≥19) with survival was more prominent in patients with advanced (T3/4 stage, node-positive) tumors.