Affordable Access

Access to the full text

A survey on deep neural network-based image captioning

Authors
  • Liu, Xiaoxiao1
  • Xu, Qingyang1
  • Wang, Ning2
  • 1 Shandong University, School of Mechanical, Electrical and Information Engineering, Weihai, 264209, People’s Republic of China , Weihai (China)
  • 2 Dalian Maritime University, Marine Engineering College, Dalian, 116026, People’s Republic of China , Dalian (China)
Type
Published Article
Journal
The Visual Computer
Publisher
Springer Berlin Heidelberg
Publication Date
Jun 09, 2018
Volume
35
Issue
3
Pages
445–470
Identifiers
DOI: 10.1007/s00371-018-1566-y
Source
Springer Nature
Keywords
License
Yellow

Abstract

Image captioning is a hot topic of image understanding, and it is composed of two natural parts (“look” and “language expression”) which correspond to the two most important fields of artificial intelligence (“machine vision” and “natural language processing”). With the development of deep neural networks and better labeling database, the image captioning techniques have developed quickly. In this survey, the image captioning approaches and improvements based on deep neural network are introduced, including the characteristics of the specific techniques. The early image captioning approach based on deep neural network is the retrieval-based method. The retrieval method makes use of a searching technique to find an appropriate image description. The template-based method separates the image captioning process into object detection and sentence generation. Recently, end-to-end learning-based image captioning method has been verified effective at image captioning. The end-to-end learning techniques can generate more flexible and fluent sentence. In this survey, the image captioning methods are reviewed in detail. Furthermore, some remaining challenges are discussed.

Report this publication

Statistics

Seen <100 times