Abstract A relational matching approach for imagery-to-GIS data is presented. This method applies image aspect interpretation and geospatial data mining techniques to realize their integration. Three-dimensional (3D) primitives, standing for houses, are chosen and their projections are represented by these aspects. The hierarchy aspect graphs are constructed to represent their interconnected relations. In this connection arcs are described by attribute data via formulated coding regulations. The nodes of the graph represent image features and their attributes can contain measurements on these features. The arcs of the graph represent relations between features and their attributes can contain measurements on spatial relations. Data mining is used to discover the semantic relationship of these primitives. Aerial images are interpreted via these aspects and geospatial data mining. Our experimental results demonstrate that the method presented is capable of effectively interpreting aerial images and extracting high accuracy from the DBM (digital building model) at a rate of 87%.