Abstract This paper presents a study on the application of a laser Doppler vibrometer (LDV) for structural health monitoring using ambient vibration. The work covers three important issues namely, data acquisition, system identification, and structural damage detection. A two-laser system is employed for data acquisition using ambient response of the structure. Modal parameters are estimated by the eigensystem realization algorithm, after first deriving the impulse response functions from both laser responses using the Natural Excitation Technique. In damage detection, a new matrix-updating-based method is proposed. The essential feature of this method is the non-iterative solving technique of inverse problem, which allows damage to be located and quantified by employing the modal parameters obtained before and after damage. Numerical simulation and laboratory-scaled experiments using bolted lap joint plate demonstrate that the proposed technique can detect locations and magnitude of damage with incomplete modal information.