Limited psychometric information is available to guide best practices for measuring youth irritability. This report compares performance of irritability measures using item response theory (IRT). Study 1 used a sample of 482 early adolescents and compared the parent- and youth-report affective reactivity index (ARI) and irritability factors derived from the parent-report Child Behavior Checklist (CBCL) and clinician-administered Kiddie Schedule for Affective Disorders (K-SADS). Study 2 combined data from three childhood samples (N = 811) and compared performance of the parent-report ARI and CBCL and the clinician-administered Preschool Age Psychiatric Assessment (PAPA). The ARI emerged as the best measure of childhood irritability across the developmental periods, while the CBCL and K-SADS provided an adequate amount of information in early adolescents. No measure reliably assessed irritability at modest severity levels. Using IRT across large pools of developmental samples and measures is needed to guide the field in the measurement of youth irritability.