Affordable Access

Access to the full text

Taxonomy based performance metrics for evaluating taxonomic assignment methods

Authors
  • Chen, Chung-Yen1
  • Tang, Sen-Lin2
  • Chou, Seng-Cho T.1
  • 1 National Taiwan University, Department of Information Management, Taipei, 106, Taiwan , Taipei (Taiwan)
  • 2 Biodiversity Research Center, Academia Sinica, Taipei, 115, Taiwan , Taipei (Taiwan)
Type
Published Article
Journal
BMC Bioinformatics
Publisher
Springer (Biomed Central Ltd.)
Publication Date
Jun 11, 2019
Volume
20
Issue
1
Identifiers
DOI: 10.1186/s12859-019-2896-0
Source
Springer Nature
Keywords
License
Green

Abstract

BackgroundMetagenomics experiments often make inferences about microbial communities by sequencing 16S and 18S rRNA, and taxonomic assignment is a fundamental step in such studies. This paper addresses the weaknesses in two types of metrics commonly used by previous studies for measuring the performance of existing taxonomic assignment methods: Sequence count based metrics and Binary error measurement. These metrics made performance evaluation results biased, less informative and mutually incomparable.ResultsWe investigated weaknesses in two types of metrics and proposed new performance metrics including Average Taxonomy Distance (ATD) and ATD_by_Taxa, together with the visualized ATD plot.ConclusionsBy comparing the evaluation results from four popular taxonomic assignment methods across three test data sets, we found the new metrics more robust, informative and comparable.

Report this publication

Statistics

Seen <100 times