Bootstrap tests are tests for which the significance level is calculated by some sort of bootstrap procedure, which may be parametric or nonparametric. We provide a theoretical framework in which to study the size distortions of bootstrap P values. We show that, in many circumstances, the size distortion of a bootstrap test will be one whole order of magnitude smaller than that of the corresponding asymptotic test. We also show that, at least in the parametric case, the magnitude of the distortion will depend on the shape of what we call the P value function. Monte Carlo results are presented for the case of nonnested hypothesis tests. These results confirm and illustrate the utility of our theoretical results, and they also suggest that bootstrap tests may often work extremely well in practice.