In multiple reversal learning, animals trained to discriminate a reinforced from a non-reinforced stimulus are subjected to various, successive reversals of stimulus contingencies (e.g. A+ vs. B−, A− vs. B+, A+ vs. B−). This protocol is useful to determine whether or not animals “learn to learn” and solve successive discriminations faster (or with fewer errors) with increasing reversal experience. Here we used the olfactory conditioning of proboscis extension reflex to study how honeybees Apis mellifera perform in a multiple reversal task. Our experiment contemplated four consecutive differential conditioning phases involving the same odors (A+ vs. B− to A− vs. B+ to A+ vs. B− to A− vs. B+). We show that bees in which the weight of reinforced or non-reinforced stimuli was similar mastered the multiple olfactory reversals. Bees which failed the task exhibited asymmetric responses to reinforced and non-reinforced stimuli, thus being unable to rapidly reverse stimulus contingencies. Efficient reversers did not improve their successive discriminations but rather tended to generalize their choice to both odors at the end of conditioning. As a consequence, both discrimination and reversal efficiency decreased along experimental phases. This result invalidates a learning-to-learn effect and indicates that bees do not only respond to the actual stimulus contingencies but rather combine these with an average of past experiences with the same stimuli.