Light-shining-through-a-wall experiments represent a new experimental approach in the search for undiscovered elementary particles not accessible with accelerator based experiments. The next generation of these experiments, such as ALPS II, require high finesse, long baseline optical cavities with fast length control. In this paper we report on a length stabilization control loop used to keep a 9.2 m cavity resonant. The finesse of this cavity was measured to be 101,300 ±500 for 1064 nm light. Fluctuations in the differential cavity length as seen with 1064 nm and 532 nm light were measured. Such fluctuations are of high relevance, since 532 nm light will be used to sense the length of the ALPS II regeneration cavity. Limiting noise sources and different control strategies are discussed, in order to fulfill the length stability requirements for ALPS II.