How2: A large-scale dataset for multimodal language understanding
File(s)1811.00347v2.pdf (4.69 MB)
Working paper
Author(s)
Type
Working Paper
Abstract
In this paper, we introduce How2, a multimodal collection of instructional
videos with English subtitles and crowdsourced Portuguese translations. We also
present integrated sequence-to-sequence baselines for machine translation,
automatic speech recognition, spoken language translation, and multimodal
summarization. By making available data and code for several multimodal natural
language tasks, we hope to stimulate more research on these and similar
challenges, to obtain a deeper understanding of multimodality in language
processing.
videos with English subtitles and crowdsourced Portuguese translations. We also
present integrated sequence-to-sequence baselines for machine translation,
automatic speech recognition, spoken language translation, and multimodal
summarization. By making available data and code for several multimodal natural
language tasks, we hope to stimulate more research on these and similar
challenges, to obtain a deeper understanding of multimodality in language
processing.
Date Issued
2018-12-07
Citation
2018
Identifier
http://arxiv.org/abs/1811.00347v2
Subjects
cs.CL
cs.CL