Affordable Access

Access to the full text

Wavelet Domain Generative Adversarial Network for Multi-scale Face Hallucination

Authors
  • Huang, Huaibo1, 2, 3, 4
  • He, Ran1, 2, 3, 4
  • Sun, Zhenan1, 2, 3, 4
  • Tan, Tieniu1, 2, 3, 4
  • 1 University of Chinese Academy of Sciences, School of Artificial Intelligence, Beijing, China , Beijing (China)
  • 2 CASIA, Center for Research on Intelligent Perception and Computing, Beijing, China , Beijing (China)
  • 3 CASIA, National Laboratory of Pattern Recognition, Beijing, China , Beijing (China)
  • 4 CAS, Center for Excellence in Brain Science and Intelligence Technology, Beijing, China , Beijing (China)
Type
Published Article
Journal
International Journal of Computer Vision
Publisher
Springer-Verlag
Publication Date
Feb 12, 2019
Volume
127
Issue
6-7
Pages
763–784
Identifiers
DOI: 10.1007/s11263-019-01154-8
Source
Springer Nature
Keywords
License
Yellow

Abstract

Most modern face hallucination methods resort to convolutional neural networks (CNN) to infer high-resolution (HR) face images. However, when dealing with very low-resolution (LR) images, these CNN based methods tend to produce over-smoothed outputs. To address this challenge, this paper proposes a wavelet-domain generative adversarial method that can ultra-resolve a very low-resolution (like 16×16\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$16\times 16$$\end{document} or even 8×8\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$8\times 8$$\end{document}) face image to its larger version of multiple upscaling factors (2×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$2\times $$\end{document} to 16×\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$16\times $$\end{document}) in a unified framework. Different from the most existing studies that hallucinate faces in image pixel domain, our method firstly learns to predict the wavelet information of HR face images from its corresponding LR inputs before image-level super-resolution. To capture both global topology information and local texture details of human faces, a flexible and extensible generative adversarial network is designed with three types of losses: (1) wavelet reconstruction loss aims to push wavelets closer with the ground-truth; (2) wavelet adversarial loss aims to generate realistic wavelets; (3) identity preserving loss aims to help identity information recovery. Extensive experiments demonstrate that the presented approach not only achieves more appealing results both quantitatively and qualitatively than state-of-the-art face hallucination methods, but also can significantly improve identification accuracy for low-resolution face images captured in the wild.

Report this publication

Statistics

Seen <100 times