使用模拟近似乘子的深度学习训练（performance）

• 2020 年 1 月 6 日
• 筆記

本文通过仿真提出了如何利用近似乘子提高卷积神经网络(CNNs)的训练性能。与精确乘法器相比，近似乘法器在速度、功率和面积方面具有更好的性能。然而，近似乘数有一个不准确性，这被定义为平均相对误差(MRE)。为了评估近似乘法器在提高CNN训练性能方面的适用性，本文模拟了近似乘法器误差对CNN训练的影响。本文证明了在CNN训练中使用近似乘子可以在速度、功率和面积方面显著提高性能，而代价是对获得的精度产生较小的负面影响。此外，本文还提出了一种混合训练方法，以减少这种方法对精度的负面影响。利用所提出的混合方法，训练可以先使用近似乘子，然后在最后几个阶段转换为精确乘子。使用这种方法，在速度、功率和面积方面近似乘法器的性能优势可以在训练阶段的大部分时间内得到。另一方面，在训练的最后阶段使用精确的乘数可以减少对准确性的负面影响。

原文题目：Deep Learning Training with Simulated Approximate Multipliers

原文：This paper presents by simulation how approximate multipliers can be utilized to enhance the training performance of convolutional neural networks (CNNs). Approximate multipliers have significantly better performance in terms of speed, power, and area compared to exact multipliers. However, approximate multipliers have an inaccuracy which is defined in terms of the Mean Relative Error (MRE). To assess the applicability of approximate multipliers in enhancing CNN training performance, a simulation for the impact of approximate multipliers error on CNN training is presented. The paper demonstrates that using approximate multipliers for CNN training can significantly enhance the performance in terms of speed, power, and area at the cost of a small negative impact on the achieved accuracy. Additionally, the paper proposes a hybrid training method which mitigates this negative impact on the accuracy. Using the proposed hybrid method, the training can start using approximate multipliers then switches to exact multipliers for the last few epochs. Using this method, the performance benefits of approximate multipliers in terms of speed, power, and area can be attained for a large portion of the training stage. On the other hand, the negative impact on the accuracy is diminished by using the exact multipliers for the last epochs of training.

原文作者：Issam Hammad, Kamal El-Sankary, Jason Gu

原文地址：https://arxiv.org/abs/2001.00060