Populist radical right (PRR) parties are more successful in some areas than others, which has fuelled the study of the role of context in shaping such support. However, on closer observation it becomes apparent that they thrive in areas that are actually quite different in their outlook. In this paper, which is based on a unique four-country comparative dataset of geocoded survey and contextual data, we argue and show that context does matter. However, we argue and show our understanding should be enhanced in two ways. First, by acknowledging that contextual effects are conditional, given that it has been demonstrated that some citizens are less likely to be affected by local grievances than others. Second, by studying the mechanisms linking contextual factors to political behaviour. We hypothesize that contextual developments, such as rising levels of immigration, might generate a generalized sense of local decline, which might in turn be translated into a broader set of grievances producing nativism and political discontent. To test these hypotheses, we rely on a fine-grained geocoded survey especially collected for these purposes in France, Germany, UK and the Netherlands, and which is also linked to contextual data. This dataset allows us to test how and among whom PRR support is predicted immigrant presence, the level of unemployment, and demographic decline.