利用迭代细化进行依存关系语法分析的递归非自回归图到图转换器

我们提出了一种通过非自回归图到图转换器的递归应用程序对任意图进行迭代细化的递归非自回归图到图转换器(RNG-Tr)。虽然之前自回归图预测中已经使用了newcite{mohammadshahi2019graphtograph}的图到图转换器,但现在我们主要用该转换器在之前同一个图的预测数据基础上独立预测该图的所有边缘。我们利用经BERT cite{devlin2018bert}预先训练的细化模型在多个依存语料库上对RNG-Tr的能力和效力进行了验证。除此之外,我们还介绍一个与我们的细化模型相似的非递归解析器,即依存性BERT(DepBERT)。RNG-Tr能够提高通用依存性树库以及英语和中文Penn树库中13种语言的各种初级解析器精确度,甚至改善通过DepBERT获得的最新先进结果,明显改善所有受测语料库的现代化水平。

原文标题:Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement

We propose the Recursive Non-autoregressive Graph-to-graph Transformer architecture (RNG-Tr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing. The Graph-to-Graph Transformer architecture of newcite{mohammadshahi2019graphtograph} has previously been used for autoregressive graph prediction, but here we use it to predict all edges of the graph independently, conditioned on a previous prediction of the same graph. We demonstrate the power and effectiveness of RNG-Tr on several dependency corpora, using a refinement model pre-trained with BERT cite{devlin2018bert}. We also introduce Dependency BERT (DepBERT), a non-recursive parser similar to our refinement model. RNG-Tr is able to improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks and the English and Chinese Penn Treebanks, even improving over the new state-of-the-art results achieved by DepBERT, significantly improving the state-of-the-art for all corpora tested.

原文作者: Alireza Mohammadshahi, James Henderson

原文链接:https://arxiv.org/abs/2003.13118