Affordable Access

Publisher Website

A comparative study of TF*IDF, LSI and multi-words for text classification

Authors
Journal
Expert Systems with Applications
0957-4174
Publisher
Elsevier
Publication Date
Volume
38
Issue
3
Identifiers
DOI: 10.1016/j.eswa.2010.08.066
Keywords
  • Text Representation
  • Tf*Idf
  • Lsi
  • Multi-Word
  • Text Classification
  • Information Retrieval
  • Text Categorization
Disciplines
  • Computer Science
  • Linguistics

Abstract

Abstract One of the main themes in text mining is text representation, which is fundamental and indispensable for text-based intellegent information processing. Generally, text representation inludes two tasks: indexing and weighting. This paper has comparatively studied TF*IDF, LSI and multi-word for text representation. We used a Chinese and an English document collection to respectively evaluate the three methods in information retreival and text categorization. Experimental results have demonstrated that in text categorization, LSI has better performance than other methods in both document collections. Also, LSI has produced the best performance in retrieving English documents. This outcome has shown that LSI has both favorable semantic and statistical quality and is different with the claim that LSI can not produce discriminative power for indexing.

There are no comments yet on this publication. Be the first to share your thoughts.