输入:通过输入和动态规划的序列建模（CS SD）

• 2020 年 3 月 17 日
• 笔记

提出了一种通过输入迭代生成输出序列的神经序列模型——输入器。输入是一个迭代生成模型，只需要与输入或输出标记的数量无关的固定数量的生成步骤。输入端可以被训练成在输入和输出序列以及所有可能的生成顺序之间的所有可能的对齐上近似地边缘化。提出了一种易于处理的动态规划训练算法，该算法给出了对数边际似然的一个下界。当应用于端到端语音识别时，输入器的性能优于先前的非自回归模型，并取得了与自回归模型相竞争的结果。在LibriSpeech测试-其他，输入达到11.1 WER，优于CTC的13.0 WER和seq2seq的12.5 WER。

原文题目：Imputer: Sequence Modelling via Imputation and Dynamic Programming

原文：This paper presents the Imputer, a neural sequence model that generates output sequences iteratively via imputations. The Imputer is an iterative generative model, requiring only a constant number of generation steps independent of the number of input or output tokens. The Imputer can be trained to approximately marginalize over all possible alignments between the input and output sequences, and all possible generation orders. We present a tractable dynamic programming training algorithm, which yields a lower bound on the log marginal likelihood. When applied to end-to-end speech recognition, the Imputer outperforms prior non-autoregressive models and achieves competitive results to autoregressive models. On LibriSpeech test-other, the Imputer achieves 11.1 WER, outperforming CTC at 13.0 WER and seq2seq at 12.5 WER.

原文作者：William Chan, Chitwan Saharia, Geoffrey Hinton, Mohammad Norouzi, Navdeep Jaitly

原文地址：http://cn.arxiv.org/abs/2002.08926