Universal screening for preterm delivery by adding transvaginal ultrasound measurement of cervical length to routine second trimester ultrasound has been proposed. The aim is to estimate inter- and intraobserver agreement and reliability of second trimester transvaginal ultrasound measurements of cervical length performed by specially trained midwife sonographers. This is a prospective reliability and agreement study performed in seven Swedish ultrasound centers. In total, 18 midwife sonographers specially trained to perform ultrasound measurements of cervical length and 286 women in the second trimester were included. In each center, two midwife sonographers measured cervical length a few minutes apart in the same woman, the number of women examined per examiner pair varying between 24 and 30 (LIVE study). Sixteen midwife sonographers measured cervical length twice ≥2 months apart on 93 video clips (CLIPS study). The main outcome measures were mean difference, limits of agreement, intraclass correlation coefficient, intra-individual standard deviation, repeatability, Cohen's kappa and Fleiss kappa. The limits of agreement and intraclass correlation coefficient of the best examiner pair in the LIVE study were -4.06 to 4.72 mm and 0.91, and those of the poorest were -11.11 to 11.39 mm and 0.31. In the CLIPS study, median (range) intra-individual standard deviation was 2.14 mm (1.40-3.46), repeatability 5.93 mm (3.88-9.58), intraclass correlation coefficient 0.84 (0.66-0.94). Median (range) interobserver agreement for cervical length ≤25 mm in the CLIPS study was 94.6% (84.9%-98.9%) and Cohen's kappa 0.56 (0.12-0.92), median (range) intraobserver agreement was 95.2% (87.1%-98.9%) and Cohen's kappa 0.68 (0.27-0.93). Agreement and reliability of cervical length measurements differed substantially between examiner pairs and examiners. If cervical length measurements are used to guide management there is potential for both over- and under-treatment. Uniform training and rigorous supervision and quality control are advised. © 2020 The Authors. Acta Obstetricia et Gynecologica Scandinavica published by John Wiley & Sons Ltd on behalf of Nordic Federation of Societies of Obstetrics and Gynecology (NFOG).