Eye movements towards a new target can be guided or disrupted by input from multiple modalities. The degree of oculomotor competition evoked by a distractor depends on both distractor and target properties, such as distractor salience or certainty regarding the target location. The ability to localize the target is particularly important when studying saccades made towards auditory targets, since determination of elevation and azimuth of a sound are based on different processes, and these processes may be affected independently by a distractor. We investigated the effects of a visual distractor on saccadic eye movements made to an auditory target in a two-dimensional plane. Results showed that the competition evoked by a vertical visual distractor was stronger compared with a horizontal visual distractor. The eye movements that were not captured by the vertical visual distractor were still influenced by it: a deviation of endpoints was seen in the direction of the visual distractor. Furthermore, the interference evoked by a high-contrast visual distractor was stronger compared with low-contrast visual stimuli, which was reflected by a faster initiation of an eye movement towards the high-contrast visual distractor and a stronger shift of endpoints in the direction of the high-contrast visual distractor. Together, these findings show that the influence of a visual distractor on aurally guided eye movements depends strongly on its location relative to the target, and to a lesser extent, on stimulus contrast.