Affordable Access

Controlling Utterance Length in NMT-based Word Segmentation with Attention

Authors
  • Godard, Pierre
  • Besacier, Laurent
  • Yvon, François
Publication Date
Nov 02, 2019
Source
HAL-ENAC
Keywords
Language
English
License
Unknown
External links

Abstract

One of the basic tasks of computational language documentation (CLD) is to identifyword boundaries in an unsegmented phonemic stream. While several unsupervisedmonolingual word segmentation algorithms exist in the literature,they are challenged in real-world CLD settings by the small amount of availabledata. A possible remedy is to take advantage of glosses or translation in a foreign,well-resourced, language, which often exist for such data. In this paper, we explore and compareways to exploit neural machine translation models to perform unsupervised boundary detection with bilingual information, notably introducing a new loss function for jointly learning alignment and segmentation. We experiment with an actual under-resourced language, Mboshi, and show that these techniques can effectively control the output segmentation length.

Report this publication

Statistics

Seen <100 times