Charge transport through a nanoscale junction coupled to two macroscopic electrodes is investigated for the situation when bound states are present. We provide numerical evidence that bound states give rise to persistent, non-decaying current oscillations in the junction. We also show that the amplitude of these oscillations can exhibit a strong dependence on the history of the applied potential as well as on the initial equilibrium configuration. Our simulations allow for a quantitative investigation of several transient features. We also discuss the existence of different time-scales and address their microscopic origin.