Background To segregate luminance-related, face-related and non-specific components involved in spatio-temporal dynamics of cortical activations to a face stimulus, we recorded cortical responses to face appearance (Onset), disappearance (Offset), and change (Change) using magnetoencephalography. Results Activity in and around the primary visual cortex (V1/V2) showed luminance-dependent behavior. Any of the three events evoked activity in the middle occipital gyrus (MOG) at 150 ms and temporo-parietal junction (TPJ) at 250 ms after the onset of each event. Onset and Change activated the fusiform gyrus (FG), while Offset did not. This FG activation showed a triphasic waveform, consistent with results of intracranial recordings in humans. Conclusion Analysis employed in this study successfully segregated four different elements involved in the spatio-temporal dynamics of cortical activations in response to a face stimulus. The results show the responses of MOG and TPJ to be associated with non-specific processes, such as the detection of abrupt changes or exogenous attention. Activity in FG corresponds to a face-specific response recorded by intracranial studies, and that in V1/V2 is related to a change in luminance.