It is a key assumption of the perception/action model that the dorsal stream relies on current visual input and does not store visual information over an extended period of time. Consequently, it is expected that action which is guided by memorized visual information, so-called delayed action, cannot use information from the dorsal stream but must rely instead on the ventral stream input. However, it is currently unclear how the information from the ventral stream can be used to guide an action. This issue is particularly challenging given that the perception/action model also assumes that ventral stream input is not particularly useful for guiding actions since the information it provides is coded relative to the visual scene and not relative to the observer. We describe two possible solutions to this problem and suggest that they can be tested using the prism adaptation paradigm. Subjects in our study were adapted to optical prisms using either an immediate or a delayed pointing task. In both cases, subjects showed the typical post-exposure negative aftereffect. Moreover, there was almost complete transfer of the aftereffect between immediate and delayed pointing. This is particularly surprising given the long history of findings showing little transfer between motor tasks for which separate neural representations are assumed. In this context our findings suggest a substantial overlap in the visuomotor transformation processes used for immediate and delayed pointing.