兩分鐘論文是原發佈於YouTube上的一個輕鬆有趣的AI論文學習欄目，講解人語速較慢，發音清晰，無論是作為聽力練習還是日常學習，都是不錯的材料選擇。

字幕組製作了雙語字幕影片，任何人都可以免費觀看，同時我們歡迎大家加入AI交流群，一起分享心得~（文末二維碼掃碼進群）

論文標題：Language Models are Few-Shot Learners

論文地址：//arxiv.org/abs/2005.14165v2

摘要

Recent work has demonstrated substantial gains on many NLP tasks and benchmarks by pre-training on a large corpus of text followed by fine-tuning on a specific task. While typically task-agnostic in architecture, this method still requires task-specific fine-tuning datasets of thousands or tens of thousands of examples. By contrast, humans can generally perform a new language task from only a few examples or from simple instructions - something which current NLP systems still largely struggle to do. Here we show that scaling up language models greatly improves task-agnostic, few-shot performance, sometimes even reaching competitiveness with prior state-of-the-art fine-tuning approaches. Specifically, we train GPT-3, an autoregressive language model with 175 billion parameters, 10x more than any previous non-sparse language model, and test its performance in the few-shot setting. For all tasks, GPT-3 is applied without any gradient updates or fine-tuning, with tasks and few-shot demonstrations specified purely via text interaction with the model. GPT-3 achieves strong performance on many NLP datasets, including translation, question-answering, and cloze tasks, as well as several tasks that require on-the-fly reasoning or domain adaptation, such as unscrambling words, using a novel word in a sentence, or performing 3-digit arithmetic. At the same time, we also identify some datasets where GPT-3's few-shot learning still struggles, as well as some datasets where GPT-3 faces methodological issues related to training on large web corpora. Finally, we find that GPT-3 can generate samples of news articles which human evaluators have difficulty distinguishing from articles written by humans. We discuss broader societal impacts of this finding and of GPT-3 in general.

最近的工作表明，通過對大量文本進行預訓練，再對特定任務進行微調，許多NLP任務和基準方面取得了重大進展。儘管在架構上通常採取task-agnostic策略，但此方法仍需要有成千上萬樣本的，task-specific策略下的微調數據集。相比之下，人類通常只能通過幾個例子或簡單的指令來完成一項新的語言任務——而目前的NLP系統在很大程度上仍然難以做到這一點。在這裡，我們展示了擴展語言模型可以極大地提升task-agnostic、少樣本性能，有時甚至可以通過現有的最新微調方法進一步提高競爭力。具體來說，我們訓練了GPT-3（一種具有1750億個參數的自回歸語言模型），這比以前的任何非稀疏語言模型都多10倍，並在嘗試多次測試其性能。對於所有的任務，應用GPT-3時無需進行任何梯度更新或微調，僅通過與模型的文本交互指定任務和少量演示即可。GPT-3在許多NLP數據集上表現出了出色的性能，包括翻譯、問答和完形填空，以及一些需要動態推理或領域適應的任務，如解讀單詞、在句子中使用新單詞或執行三位數運算。同時，我們還用它識別一些數據集，其中GPT-3的少樣本學習仍然很困難，尤其是與大型Web語料庫訓練有關的方法論問題相關的數據集。最後，我們發現GPT-3生成的新聞樣本，評估人員很難將其與人類撰寫的文章區分開，我們將討論這一發現以及GPT-3的廣泛社會影響。

論文講解雙語影片
（點擊跳轉播放地址）

回復字幕君：「AI交流群」即可