利用迭代細化進行依存關係語法分析的遞歸非自回歸圖到圖轉換器
- 2020 年 4 月 3 日
- 筆記
我們提出了一種通過非自回歸圖到圖轉換器的遞歸應用程式對任意圖進行迭代細化的遞歸非自回歸圖到圖轉換器（RNG-Tr）。雖然之前自回歸圖預測中已經使用了newcite{mohammadshahi2019graphtograph}的圖到圖轉換器，但現在我們主要用該轉換器在之前同一個圖的預測數據基礎上獨立預測該圖的所有邊緣。我們利用經BERT cite{devlin2018bert}預先訓練的細化模型在多個依存語料庫上對RNG-Tr的能力和效力進行了驗證。除此之外，我們還介紹一個與我們的細化模型相似的非遞歸解析器，即依存性BERT（DepBERT）。RNG-Tr能夠提高通用依存性樹庫以及英語和中文Penn樹庫中13種語言的各種初級解析器精確度，甚至改善通過DepBERT獲得的最新先進結果，明顯改善所有受測語料庫的現代化水平。
原文標題：Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement
We propose the Recursive Non-autoregressive Graph-to-graph Transformer architecture (RNG-Tr) for the iterative refinement of arbitrary graphs through the recursive application of a non-autoregressive Graph-to-Graph Transformer and apply it to syntactic dependency parsing. The Graph-to-Graph Transformer architecture of newcite{mohammadshahi2019graphtograph} has previously been used for autoregressive graph prediction, but here we use it to predict all edges of the graph independently, conditioned on a previous prediction of the same graph. We demonstrate the power and effectiveness of RNG-Tr on several dependency corpora, using a refinement model pre-trained with BERT cite{devlin2018bert}. We also introduce Dependency BERT (DepBERT), a non-recursive parser similar to our refinement model. RNG-Tr is able to improve the accuracy of a variety of initial parsers on 13 languages from the Universal Dependencies Treebanks and the English and Chinese Penn Treebanks, even improving over the new state-of-the-art results achieved by DepBERT, significantly improving the state-of-the-art for all corpora tested.
原文作者： Alireza Mohammadshahi, James Henderson
原文鏈接：https://arxiv.org/abs/2003.13118