Reinforcement learning (RL), including multi-armed bandits, is a branch of machine learning that deals with the problem of sequential decision-making in uncertain and unknown environments through learning by practice. While best known for being the core of the artificial intelligence (AI) world's best Go game player, RL has a vast range of potential applications. RL may help to address some of the criticisms leveled against crop management decision support systems (DSS): it is an interactive, geared towards action, contextual tool to evaluate series of crop operations faced with uncertainties. A review of RL use for crop management DSS reveals a limited number of contributions. We profile key prospects for a human-centered, real-world, interactive RL-based system to face tomorrow's agricultural decisions, and theoretical and ongoing practical challenges that may explain its current low uptake. We argue that a joint research effort from the RL and agronomy communities is necessary to explore RL's full potential.