Despite evidence for liquid water at the surface of Mars during the Noachian epoch, the temperature of early aqueous environments has been impossible to establish, raising questions of whether the surface of Mars was ever warmer than today. We address this problem by determining the precipitation temperature of secondary carbonate minerals preserved in the oldest known sample of Mars’ crust—the approximately 4.1 billion-year-old meteorite Allan Hills 84001 (ALH84001). The formation environment of these carbonates, which are constrained to be slightly younger than the crystallization age of the rock (i.e., 3.9 to 4.0 billion years), has been poorly understood, hindering insight into the hydrologic and carbon cycles of earliest Mars. Using “clumped” isotope thermometry we find that the carbonates in ALH84001 precipitated at a temperature of approximately 18 °C, with water and carbon dioxide derived from the ancient Martian atmosphere. Furthermore, covarying carbonate carbon and oxygen isotope ratios are constrained to have formed at constant, low temperatures, pointing to deposition from a gradually evaporating, subsurface water body—likely a shallow aquifer (meters to tens of meters below the surface). Despite the mild temperatures, the apparently ephemeral nature of water in this environment leaves open the question of its habitability.