Affordable Access

deepdyve-link
Publisher Website

Calibration of medical diagnostic classifier scores to the probability of disease.

Authors
  • Chen, Weijie1
  • Sahiner, Berkman1
  • Samuelson, Frank1
  • Pezeshk, Aria1
  • Petrick, Nicholas1
  • 1 Office of Science and Engineering Laboratories, Center for Devices and Radiological Health, Food and Drug Administration, Silver Spring, USA.
Type
Published Article
Journal
Statistical Methods in Medical Research
Publisher
SAGE Publications
Publication Date
May 01, 2018
Volume
27
Issue
5
Pages
1394–1409
Identifiers
DOI: 10.1177/0962280216661371
PMID: 27507287
Source
Medline
Keywords
License
Unknown

Abstract

Scores produced by statistical classifiers in many clinical decision support systems and other medical diagnostic devices are generally on an arbitrary scale, so the clinical meaning of these scores is unclear. Calibration of classifier scores to a meaningful scale such as the probability of disease is potentially useful when such scores are used by a physician. In this work, we investigated three methods (parametric, semi-parametric, and non-parametric) for calibrating classifier scores to the probability of disease scale and developed uncertainty estimation techniques for these methods. We showed that classifier scores on arbitrary scales can be calibrated to the probability of disease scale without affecting their discrimination performance. With a finite dataset to train the calibration function, it is important to accompany the probability estimate with its confidence interval. Our simulations indicate that, when a dataset used for finding the transformation for calibration is also used for estimating the performance of calibration, the resubstitution bias exists for a performance metric involving the truth states in evaluating the calibration performance. However, the bias is small for the parametric and semi-parametric methods when the sample size is moderate to large (>100 per class).

Report this publication

Statistics

Seen <100 times