Affordable Access

deepdyve-link
Publisher Website

Measuring and Predicting Tag Importance for Image Retrieval.

Authors
  • Li, Shangwen
  • Purushotham, Sanjay
  • Chen, Chen
  • Ren, Yuzhuo
  • Kuo, C-C Jay
Type
Published Article
Journal
IEEE transactions on pattern analysis and machine intelligence
Publication Date
Dec 01, 2017
Volume
39
Issue
12
Pages
2423–2436
Identifiers
DOI: 10.1109/TPAMI.2017.2651818
PMID: 28092521
Source
Medline
License
Unknown

Abstract

Textual data such as tags, sentence descriptions are combined with visual cues to reduce the semantic gap for image retrieval applications in today's Multimodal Image Retrieval (MIR) systems. However, all tags are treated as equally important in these systems, which may result in misalignment between visual and textual modalities during MIR training. This will further lead to degenerated retrieval performance at query time. To address this issue, we investigate the problem of tag importance prediction, where the goal is to automatically predict the tag importance and use it in image retrieval. To achieve this, we first propose a method to measure the relative importance of object and scene tags from image sentence descriptions. Using this as the ground truth, we present a tag importance prediction model to jointly exploit visual, semantic and context cues. The Structural Support Vector Machine (SSVM) formulation is adopted to ensure efficient training of the prediction model. Then, the Canonical Correlation Analysis (CCA) is employed to learn the relation between the image visual feature and tag importance to obtain robust retrieval performance. Experimental results on three real-world datasets show a significant performance improvement of the proposed MIR with Tag Importance Prediction (MIR/TIP) system over other MIR systems.

Report this publication

Statistics

Seen <100 times