The current COVID-19 pandemic has led to a devastating impact across the world. Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) (the virus causing COVID-19) is known to use the receptor-binding domain (RBD) at viral surface spike (S) protein to interact with the angiotensin-converting enzyme 2 (ACE2) receptor expressed on many human cell types. The RBD-ACE2 interaction is a crucial step to mediate the host cell entry of SARS-CoV-2. Recent studies indicate that the ACE2 interaction with the SARS-CoV-2 S protein has a higher affinity than its binding with the structurally identical S protein of SARS-CoV-1, the virus causing the 2002–2004 SARS outbreak. However, the biophysical mechanism behind such binding affinity difference is unclear. This study utilizes combined single-molecule force spectroscopy and steered molecular dynamics (SMD) simulation approaches to quantify the specific interactions between SARS-CoV-2 or SARS-CoV-1 RBD and ACE2. Depending on the loading rates, the unbinding forces between SARS-CoV-2 RBD and ACE2 range from 70 to 105 pN and are 30–40% higher than those of SARS-CoV-1 RBD and ACE2 under similar loading rates. SMD results indicate that SARS-CoV-2 RBD interacts with the N-linked glycan on Asn90 of ACE2. This interaction is mostly absent in the SARS-CoV-1 RBD-ACE2 complex. During the SMD simulations, the extra RBD-N-glycan interaction contributes to a greater force and prolonged interaction lifetime. The observation is confirmed by our experimental force spectroscopy study. After removing N-linked glycans on ACE2, its mechanical binding strength with SARS-CoV-2 RBD decreases to a similar level of the SARS-CoV-1 RBD-ACE2 interaction. Together, the study uncovers the mechanism behind the difference in ACE2 binding between SARS-CoV-2 and SARS-CoV-1 and could help develop new strategies to block SARS-CoV-2 entry.