Several approaches exist to ascertain the connectivity of the brain, and these approaches lead to markedly different topologies, often incompatible with each other. Specifically, recent single-cell recording results seem incompatible with current structural connectivity models. We present a novel method that combines anatomical and temporal constraints to generate biologically plausible connectivity patterns of the visual system of the macaque monkey. Our method takes structural connectivity data from the CoCoMac database and recent single-cell recording data as input and employs an optimization technique to arrive at a new connectivity pattern of the visual system that is in agreement with both types of experimental data. The new connectivity pattern yields a revised model that has fewer levels than current models. In addition, it introduces subcortical–cortical connections. We show that these connections are essential for explaining latency data, are consistent with our current knowledge of the structural connectivity of the visual system, and might explain recent functional imaging results in humans. Furthermore we show that the revised model is not underconstrained like previous models and can be extended to include newer data and other kinds of data. We conclude that the revised model of the connectivity of the visual system reflects current knowledge on the structure and function of the visual system and addresses some of the limitations of previous models.