不成对图像到图像转换转向终身监督模式

• 2020 年 4 月 3 日
• 笔记

不成对图像到图像转换（I2IT）任务通常存在缺乏数据的问题，这也是自监督学习（SSL）存在的问题。而SSL最近十分流行，在解决问题上也是分成功。使用例如旋转预测或生成着色等方法，能够使SSL在低数据集中具有更强的代表性。但将这些任务与I2IT一起训练会产生计算难题，因为模型规模和任务数量都会变多。

在另一方面，按顺序逐个学习又会导致之前学习任务人的大量遗忘。为了缓解这一问题，我们引进了终生监督（LiSS）模式，来预训练一种 I2IT模型（例如：CycleGAN），进行一系列的自监督学习辅助任务。通过保持之前编码器的指数移动平均数并提取积累的知识，我们可以在许多任务中保持验证性能，同时没有使用持续学习中常用的重放、参数隔离或再训练技术。

本研究结果证明使用LiSS训练的模型的表现比遗忘任务中表现更好，而且能比CycleGAN基线更有效地区分误差和实体纠缠（当两个实体十分接近时）。

原文标题：Towards Lifelong Self-Supervision For Unpaired Image-to-Image Translation

Unpaired Image-to-Image Translation (I2IT) tasks often suffer from lack of data, a problem which self-supervised learning (SSL) has recently been very popular and successful at tackling. Leveraging auxiliary tasks such as rotation prediction or generative colorization, SSL can produce better and more robust representations in a low data regime. Training such tasks along an I2IT task is however computationally intractable as model size and the number of task grow.

On the other hand, learning sequentially could incur catastrophic forgetting of previously learned tasks. To alleviate this, we introduce Lifelong Self-Supervision (LiSS) as a way to pre-train an I2IT model (e.g., CycleGAN) on a set of self-supervised auxiliary tasks. By keeping an exponential moving average of past encoders and distilling the accumulated knowledge, we are able to maintain the network's validation performance on a number of tasks without any form of replay, parameter isolation or retraining techniques typically used in continual learning.

We show that models trained with LiSS perform better on past tasks, while also being more robust than the CycleGAN baseline to color bias and entity entanglement (when two entities are very close).

原文链接：https://arxiv.org/abs/2004.00161

原文作者：Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG); Image and Video Processing (eess.IV)