Affordable Access

Access to the full text

Judging the judges: evaluating the accuracy and national bias of international gymnastics judges

Authors
  • Heiniger, Sandro1
  • Mercier, Hugues2
  • 1 University of St.Gallen, Switzerland , (Switzerland)
  • 2 University of Applied Sciences and Arts of Western Switzerland, Switzerland , (Switzerland)
Type
Published Article
Journal
Journal of Quantitative Analysis in Sports
Publisher
De Gruyter
Publication Date
Oct 19, 2021
Volume
17
Issue
4
Pages
289–305
Identifiers
DOI: 10.1515/jqas-2019-0113
Source
De Gruyter
Keywords
Disciplines
  • Research article
License
Yellow

Abstract

We design, describe and implement a statistical engine to analyze the performance of gymnastics judges with three objectives: (1) provide constructive feedback to judges, executive committees and national federations; (2) assign the best judges to the most important competitions; (3) detect bias and persistent misjudging. Judging a gymnastics routine is a random process, and we model this process using heteroscedastic random variables. The developed marking score scales the difference between the mark of a judge and the true performance level of a gymnast as a function of the intrinsic judging error variability estimated from historical data for each apparatus. This dependence between judging variability and performance quality has never been properly studied. We leverage the intrinsic judging error variability and the marking score to detect outlier marks and study the national bias of judges favoring athletes of the same nationality. We also study ranking scores assessing to what extent judges rate gymnasts in the correct order. Our main observation is that there are significant differences between the best and worst judges, both in terms of accuracy and national bias. The insights from this work have led to recommendations and rule changes at the Fédération Internationale de Gymnastique.

Report this publication

Statistics

Seen <100 times