Affordable Access

Access to the full text

Comparison of different ML methods concerning prediction quality, domain adaptation and robustness

Authors
  • Goodarzi, Payman1
  • Schütze, Andreas1
  • Schneider, Tizian1
  • 1 Lab for Measurement Technology, Germany , (Germany)
Type
Published Article
Journal
tm - Technisches Messen
Publisher
De Gruyter Oldenbourg
Publication Date
Feb 25, 2022
Volume
89
Issue
4
Pages
224–239
Identifiers
DOI: 10.1515/teme-2021-0129
Source
De Gruyter
Keywords
Disciplines
  • Beiträge
License
Yellow

Abstract

Nowadays machine learning methods and data-driven models have been used widely in different fields including computer vision, biomedicine, and condition monitoring. However, these models show performance degradation when meeting real-life situations. Domain or dataset shift or out-of-distribution (OOD) prediction is mentioned as the reason for this problem. Especially in industrial condition monitoring, it is not clear when we should be concerned about domain shift and which methods are more robust against this problem. In this paper prediction results are compared for a conventional machine learning workflow based on feature extraction, selection, and classification/regression (FESC/R) and deep neural networks on two publicly available industrial datasets. We show that it is possible to visualize the possible shift in domain using feature extraction and principal component analysis. Also, experimental competition shows that the cross-domain validated results of FESC/R are comparable to the reported state-of-the-art methods. Finally, we show that the results for simple randomly selected validation sets do not correctly represent the model performance in real-world applications.

Report this publication

Statistics

Seen <100 times