[abridged] Radiation pressure on dust grains may be an important mechanism in driving winds in a wide variety of astrophysical systems. However, the efficiency of the coupling between the radiation field and the dusty gas is poorly understood in environments characterized by high optical depths. We present a series of idealized numerical experiments, performed with the radiation-hydrodynamic code ORION, in which we study the dynamics of such winds and quantify their properties. We find that, after wind acceleration begins, radiation Rayleigh-Taylor instability forces the gas into a configuration that reduces the rate of momentum transfer from the radiation field to the gas by a factor ~ 10 - 100 compared to an estimate based on the optical depth at the base of the atmosphere; instead, the rate of momentum transfer from a driving radiation field of luminosity L to the gas is roughly L/c multiplied by one plus half the optical depth evaluated using the photospheric temperature, which is far smaller than the optical depth one would obtain using the interior temperature. When we apply our results to conditions appropriate to ULIRGs and star clusters, we find that the asymptotic wind momentum flux from such objects should not significantly exceed that carried by the direct radiation field, L/c. This result constrains the expected mass loss rates from systems that exceed the Eddington limit to be of order the so-called "single-scattering" limit, and not significantly higher. We present an approximate fitting formula for the rate of momentum transfer from radiation to dusty gas through which it passes, which is suitable for implementation in sub-grid models of galaxy formation. Finally, we provide a first map of the column density distribution of gas in a radiatively-driven wind as a function of velocity, and velocity dispersion.