Current approaches to mathematical cognition divide into two major camps. Cognitive studies try to render mathematical intuition—the faculty that gives us immediate and authoritative knowledge of mathematics—respectable on scientific grounds. Cultural studies, on the other hand, regard mathematics as a form of cultural achievement, like literature or architecture. Both positions have their own shortcomings. While cognitive approaches are limited in scope and fail to account for complex mathematical developments, cultural approaches are short of detailed answers as to what enables us to participate in a common mathematical practice. This situation evinces a need to balance a cognitive perspective on mathematical culture against a cultural perspective on mathematical cognition. According to current behavioral and neuropsychological evidence, the complex, uniquely human, culture-specific mathematical skills exhibited by human adults rest on a set of psychological and neural mechanisms that (a) are shared by other animals, and (b) emerge early in human development, continue to function throughout the lifespan, and thus are common to infants, children and adults. It has been proposed that these common and evolutionary ancient mechanisms account for humans’ basic “number sense” and form the building blocks for the development of more sophisticated numerical skills. Indeed, infants leave animals far behind in their numerical sophistication. What boosts this developmental difference? How do human beings acquire mathematical concepts such as the concept of natural number? First, I specify the representations that are the building blocks for the target concepts. Second, I describe how the target concepts differ from these basic representations. And finally, I characterize the learning mechanisms that enable the construction of the target concepts out of those prior representations. I argue that the power of the resulting conceptual system derives from the combination and integration of previously distinct representational systems, capitalizing on the human capacity for creating and using external symbols: human beings can only develop their distinct conceptual abilities due to their original embeddedness in both the physical world and, most importantly, in a rich milieu of cultural resources. Thus, an important developmental source of number representations, in addition to the preverbal systems mentioned above, is the representation of numbers within natural language.