不成對圖像到圖像轉換轉向終身監督模式

• 2020 年 4 月 3 日
• 筆記

不成對圖像到圖像轉換（I2IT）任務通常存在缺乏數據的問題，這也是自監督學習（SSL）存在的問題。而SSL最近十分流行，在解決問題上也是分成功。使用例如旋轉預測或生成着色等方法，能夠使SSL在低數據集中具有更強的代表性。但將這些任務與I2IT一起訓練會產生計算難題，因為模型規模和任務數量都會變多。

在另一方面，按順序逐個學習又會導致之前學習任務人的大量遺忘。為了緩解這一問題，我們引進了終生監督（LiSS）模式，來預訓練一種 I2IT模型（例如：CycleGAN），進行一系列的自監督學習輔助任務。通過保持之前編碼器的指數移動平均數並提取積累的知識，我們可以在許多任務中保持驗證性能，同時沒有使用持續學習中常用的重放、參數隔離或再訓練技術。

本研究結果證明使用LiSS訓練的模型的表現比遺忘任務中表現更好，而且能比CycleGAN基線更有效地區分誤差和實體糾纏（當兩個實體十分接近時）。

原文標題：Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation

Unpaired Image-to-Image Translation (I2IT) tasks often suffer from lack of data, a problem which self-supervised learning (SSL) has recently been very popular and successful at tackling. Leveraging auxiliary tasks such as rotation prediction or generative colorization, SSL can produce better and more robust representations in a low data regime. Training such tasks along an I2IT task is however computationally intractable as model size and the number of task grow.

On the other hand, learning sequentially could incur catastrophic forgetting of previously learned tasks. To alleviate this, we introduce Lifelong Self-Supervision (LiSS) as a way to pre-train an I2IT model (e.g., CycleGAN) on a set of self-supervised auxiliary tasks. By keeping an exponential moving average of past encoders and distilling the accumulated knowledge, we are able to maintain the network's validation performance on a number of tasks without any form of replay, parameter isolation or retraining techniques typically used in continual learning.

We show that models trained with LiSS perform better on past tasks, while also being more robust than the CycleGAN baseline to color bias and entity entanglement (when two entities are very close).

原文鏈接：https://arxiv.org/abs/2004.00161

原文作者：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)