Affordable Access

Publisher Website

A model-based evaluation of data quality activities in KDD

Authors
Journal
Information Processing & Management
0306-4573
Publisher
Elsevier
Identifiers
DOI: 10.1016/j.ipm.2014.07.007
Keywords
  • Data Quality
  • Data Cleansing
  • Model Checking
  • Real-Life Application

Abstract

Abstract We live in the Information Age, where most of the personal, business, and administrative data are collected and managed electronically. However, poor data quality may affect the effectiveness of knowledge discovery processes, thus making the development of the data improvement steps a significant concern. In this paper we propose the Multidimensional Robust Data Quality Analysis, a domain-independent technique aimed to improve data quality by evaluating the effectiveness of a black-box cleansing function. Here, the proposed approach has been realized through model checking techniques and then applied on a weakly structured dataset describing the working careers of millions of people. Our experimental outcomes show the effectiveness of our model-based approach for data quality as they provide a fine-grained analysis of both the source dataset and the cleansing procedures, enabling domain experts to identify the most relevant quality issues as well as the action points for improving the cleansing activities. Finally, an anonymized version of the dataset and the analysis results have been made publicly available to the community.

There are no comments yet on this publication. Be the first to share your thoughts.