Using fMRI, we measured brain activity while participants viewed photographs in which one person posed a potential threat to another. Visuospatial areas, specifically temporal-occipital junction, extrastriate, and fusiform cortices and right superior parietal lobe (BA7), responded when a threatening person was close to the personal space of another. Strikingly, this selectivity was absent when the people were further apart. Furthermore, posterior parietal areas, which code the space surrounding one's own body, responded when the individual was close to the other person's body space, regardless of whether he appeared threatening. We suggest that the spatial dimension of social interactions contributes to an observer's understanding of potentially dangerous social situations, and that higher level visual cortices play a role in distinguishing social categories based on a person's features.