Recent literature demonstrates the contribution of short food supply chains (SFSC) to regional economies and sustainable food systems, and acknowledges their role as drivers for sustainable development. Moreover, different types of SFSC have been supported by urban food policies (UFP) over the few last years and actors from the food chain became part of new institutional settings for urban food policies. However, evidence from the sustainability impact assessment (SIA) of these SFSC in urban contexts is limited. Our paper presents an approach for the development of an SIA framework for different SFSC types. In addition, a practical application of the tool in four metropolitan regions (Berlin, London, Ljubljana and Nairobi) is tested. The conceptual development of the SIA framework is based on an in-depth analysis of existing SIA frameworks and methods for assessing the sustainability of agriculture, food chains and food systems and adapted to the specific needs of analysing SFSC in metropolitan regions. The operational value and utility of the framework and the tool were tested with social and natural scientists and local stakeholders. The assessment results demonstrate that SFSC seems to be more sustainable than the baseline (long global food chains) in the social dimension, but also feature specific strengths and weaknesses concerning their economic and environmental sustainability. This might give an indication for regional adjusted strategies and food chain innovations that improve the sustainability performance may be required. We found the tool useful for framing the dialogue between food chain actors, consumers and policy, because it makes benefits and trade-offs of the chain types operating in an urban-rural context more visible and communicable. The online version contains supplementary material available at 10.1007/s10460-021-10288-w. © The Author(s) 2022.