A cross-sectional observational study with repeated observations was conducted on 16 Danish dairy farms to quantify the influence of observer, parity, time (stage in lactation) and farm on variables routinely selected for inclusion in clinical protocols, thereby to enable a more valid comparison of udder health between different herds. During 12 months, participating herds were visited 5 times by project technicians, who examined 20 cows and scored the selected clinical variables. The estimates of effect on variables were derived from a random regression model procedure. Statistical analyses revealed that, although estimates for occurrence of several the variables, e.g. degree of oedema, varied significantly between observers, the effects on many of these estimates were similar in size. Almost all estimates for occurrences of variables were significantly affected either parity and lactation stage, or by both e.g. udder tissue consistency. Some variables, e.g. mange, had high estimates for the farm component, and others e.g. teat skin quality had a high individual component. Several of the variables, e.g. wounds on warts, had a high residual component indicating that a there still was a major part of the variation in data, which was unexplained. It was concluded that most of the variables were relevant for implementation in herd health management, but that adjustments need to be made to improve reliability.