We report an improved dynamic determination of the Casimir pressure between two plane plates obtained using a micromachined torsional oscillator. The main improvements in the current experiment are a significant suppression of the surface roughness of the Au layers deposited on the interacting surfaces, and a decrease in the experimental error in the measurement of the absolute separation. A metrological analysis of all data permitted us to determine both the random and systematic errors, and to find the total experimental error as a function of separation at the 95% confidence level. In contrast to all previous experiments on the Casimir effect, our smallest experimental error ($\sim 0.5$%) is achieved over a wide separation range. The theoretical Casimir pressures in the experimental configuration were calculated by the use of four theoretical approaches suggested in the literature. All corrections to the Casimir force were calculated or estimated. All theoretical errors were analyzed and combined to obtain the total theoretical error at the 95% confidence level. Finally, the confidence interval for the differences between theoretical and experimental pressures was obtained as a function of separation. Our measurements are found to be consistent with two theoretical approaches utilizing the plasma model and the surface impedance over the entire measurement region. Two other approaches to the thermal Casimir force, utilizing the Drude model or a special prescription for the determination of the zero-frequency contribution to the Lifshitz formula, are excluded on the basis of our measurements at the 99% and 95% confidence levels, respectively. Finally, constraints on Yukawa-type hypothetical interactions are strengthened by up to a factor of 20 in a wide interaction range.