Although a large number of neuropsychological and imaging studies have demonstrated that the medial temporal lobe (MTL) plays an important role in human memory, there are few data regarding the activity of neurons involved in this process. The MTL receives massive inputs from visual cortical areas, and evidence over the last decade has consistently shown that MTL neurons respond selectively to complex visual stimuli. Here, we focus on how the activity patterns of these cells might reflect the transformation of visual percepts into long-term memories. Given the very sparse and abstract representation of visual information by these neurons, they could in principle be considered as ‘grandmother cells’. However, we give several arguments that make such an extreme interpretation unlikely.