Abstract Mammographic risk assessment is becoming increasingly important in decision making in screening mammography and computer aided diagnosis systems. Strong evidence shows that characteristic patterns of breast tissue as seen in mammography, referred to as mammographic parenchymal patterns, provide crucial information about breast cancer risk. Quantitative evaluation of the characteristic mixture of breast tissue can be used both for the estimation of mammographic risk assessment, as well as for quantifying the change of the relative proportion of different breast tissue patterns. This paper investigates mammographic image segmentation based on geometric moments, and prior information of the mammographic building blocks as described by Tabár tissue modelling. The segmentation methodology presented here consists of five distinct steps: (1) feature extraction using mammographic patches, (2) deriving local image properties, (3) feature transformation, (4) mammographic building block based model generation by clustering, and (5) model driven segmentation. The Mammographic Image Analysis Society database was used to facilitate the quantitative and qualitative evaluation, with respect to mammographic risk assessment, based on both Tabár and Breast Imaging Reporting And Data System schemes. Classification accuracies of 71% and 79% were achieved in the corresponding low and high risk categories for Tabár and Breast Imaging Reporting And Data System schemes, respectively. Visual assessment indicates that the proposed segmentation approach can produce consistent and realistic segmentation results, with respect to breast anatomy and Tabár tissue modelling. For screening mammography and computer aided diagnosis, the proposed mammographic segmentation approach is useful in aiding radiologists’ estimation of breast cancer risk and treatment planning prior to biopsies.