Affordable Access

Access to the full text

On the optimistic performance evaluation of newly introduced bioinformatic methods

Authors
  • Buchka, Stefan1
  • Hapfelmeier, Alexander2, 2
  • Gardner, Paul P.3
  • Wilson, Rory4
  • Boulesteix, Anne-Laure1
  • 1 LMU, Munich, Germany , Munich (Germany)
  • 2 TUM, Munich, Germany , Munich (Germany)
  • 3 University of Otago, Otago, New Zealand , Otago (New Zealand)
  • 4 German Research Center for Environmental Health, Neuherberg, Germany , Neuherberg (Germany)
Type
Published Article
Publication Date
May 11, 2021
Volume
22
Issue
1
Identifiers
DOI: 10.1186/s13059-021-02365-4
Source
Springer Nature
Keywords
License
Green

Abstract

Most research articles presenting new data analysis methods claim that “the new method performs better than existing methods,” but the veracity of such statements is questionable. Our manuscript discusses and illustrates consequences of the optimistic bias occurring during the evaluation of novel data analysis methods, that is, all biases resulting from, for example, selection of datasets or competing methods, better ability to fix bugs in a preferred method, and selective reporting of method variants. We quantitatively investigate this bias using an example from epigenetic analysis: normalization methods for data generated by the Illumina HumanMethylation450K BeadChip microarray.

Report this publication

Statistics

Seen <100 times