The influence of genes on cortical structures has been assessed through various phenotypes. The sulcal pits, which are the putative first cortical folds, have for long been assumed to be under tight genetic control, but this was never quantified. We estimated the pit depth heritability in various brain regions using the high quality and large sample size of the Human Connectome Project pedigree cohort. Analysis of additive genetic variance indicated that their heritability ranges between 0.2 and 0.5 and displays a regional genetic control with an overall symmetric pattern between hemispheres. However, a noticeable asymmetry of heritability estimates is observed in the superior temporal sulcus and could thus be related to language lateralization. The heritability range estimated in this study reinforces the idea that cortical shape is determined primarily by nongenetic factors, which is consistent with the important increase of cortical folding from birth to adult life and thus predominantly constrained by environmental factors. Nevertheless, the genetic cues, implicated with various local levels of heritability in the formation of sulcal pits, play a fundamental role in the normal gyral pattern development. Quantifying their influence and identifying the underlying genetic variants would provide insight into neurodevelopmental disorders.