keras中的損失函數
- 2020 年 3 月 31 日
- 筆記
損失函數是模型優化的目標，所以又叫目標函數、優化評分函數，在keras中，模型編譯的參數loss指定了損失函數的類別，有兩種指定方法：
model.compile(loss='mean_squared_error', optimizer='sgd')
或者
from keras import losses model.compile(loss=losses.mean_squared_error, optimizer='sgd')
你可以傳遞一個現有的損失函數名，或者一個TensorFlow/Theano符號函數。該符號函數為每個數據點返回一個標量，有以下兩個參數:
- y_true: 真實標籤. TensorFlow/Theano張量
- y_pred: 預測值. TensorFlow/Theano張量，其shape與y_true相同
實際的優化目標是所有數據點的輸出數組的平均值。
mean_squared_error：均方誤差
mean_squared_error(y_true, y_pred)
源碼：
def mean_squared_error(y_true, y_pred): return K.mean(K.square(y_pred - y_true), axis=-1)
說明：
MSE:
mean_absolute_error
mean_absolute_error(y_true, y_pred)
源碼：
def mean_absolute_error(y_true, y_pred): return K.mean(K.abs(y_pred - y_true), axis=-1)
說明：
MAE：
mean_absolute_percentage_error
mean_absolute_percentage_error(y_true, y_pred)
源碼：
def mean_absolute_percentage_error(y_true, y_pred): diff = K.abs((y_true - y_pred) / K.clip(K.abs(y_true), K.epsilon(), None)) return 100. * K.mean(diff, axis=-1)
說明：
MAPE：
mean_squared_logarithmic_error
mean_squared_logarithmic_error(y_true, y_pred)
源碼：
def mean_squared_logarithmic_error(y_true, y_pred): first_log = K.log(K.clip(y_pred, K.epsilon(), None) + 1.) second_log = K.log(K.clip(y_true, K.epsilon(), None) + 1.) return K.mean(K.square(first_log - second_log), axis=-1)
說明：
MAPE：
squared_hinge
squared_hinge(y_true, y_pred)
源碼：
def squared_hinge(y_true, y_pred): return K.mean(K.square(K.maximum(1. - y_true * y_pred, 0.)), axis=-1)
說明：
hinge
hinge(y_true, y_pred)
源碼：
def hinge(y_true, y_pred): return K.mean(K.maximum(1. - y_true * y_pred, 0.), axis=-1)
說明：
categorical_hinge
categorical_hinge(y_true, y_pred)
源碼：
def categorical_hinge(y_true, y_pred): pos = K.sum(y_true * y_pred, axis=-1) neg = K.max((1. - y_true) * y_pred, axis=-1) return K.maximum(0., neg - pos + 1.)
logcosh
logcosh(y_true, y_pred)
源碼：
def logcosh(y_true, y_pred): """Logarithm of the hyperbolic cosine of the prediction error. `log(cosh(x))` is approximately equal to `(x ** 2) / 2` for small `x` and to `abs(x) - log(2)` for large `x`. This means that 'logcosh' works mostly like the mean squared error, but will not be so strongly affected by the occasional wildly incorrect prediction. # Arguments y_true: tensor of true targets. y_pred: tensor of predicted targets. # Returns Tensor with one scalar loss entry per sample. """ def _logcosh(x): return x + K.softplus(-2. * x) - K.log(2.) return K.mean(_logcosh(y_pred - y_true), axis=-1)
categorical_crossentropy
categorical_crossentropy(y_true, y_pred)
源碼：
def categorical_crossentropy(y_true, y_pred): return K.categorical_crossentropy(y_true, y_pred)
注意: 當使用categorical_crossentropy損失時，你的目標值應該是分類格式 (即，如果你有10個類，每個樣本的目標值應該是一個10維的向量，這個向量除了表示類別的那個索引為1，其他均為0)。為了將 整數目標值 轉換為 分類目標值，你可以使用Keras實用函數to_categorical：
from keras.utils.np_utils import to_categorical categorical_labels = to_categorical(int_labels, num_classes=None)
sparse_categorical_crossentropy
sparse_categorical_crossentropy(y_true, y_pred)
源碼：
def sparse_categorical_crossentropy(y_true, y_pred): return K.sparse_categorical_crossentropy(y_true, y_pred) def sparse_categorical_crossentropy(target, output, from_logits=False): """Categorical crossentropy with integer targets. # Arguments target: An integer tensor. output: A tensor resulting from a softmax (unless `from_logits` is True, in which case `output` is expected to be the logits). from_logits: Boolean, whether `output` is the result of a softmax, or is a tensor of logits. # Returns Output tensor. """ # Note: tf.nn.sparse_softmax_cross_entropy_with_logits # expects logits, Keras expects probabilities. if not from_logits: _epsilon = _to_tensor(epsilon(), output.dtype.base_dtype) output = tf.clip_by_value(output, _epsilon, 1 - _epsilon) output = tf.log(output) output_shape = output.get_shape() targets = cast(flatten(target), 'int64') logits = tf.reshape(output, [-1, int(output_shape[-1])]) res = tf.nn.sparse_softmax_cross_entropy_with_logits( labels=targets, logits=logits) if len(output_shape) >= 3: # if our output includes timestep dimension # or spatial dimensions we need to reshape return tf.reshape(res, tf.shape(output)[:-1]) else: return res
binary_crossentropy
binary_crossentropy(y_true, y_pred)
源碼：
def binary_crossentropy(y_true, y_pred): return K.mean(K.binary_crossentropy(y_true, y_pred), axis=-1) def binary_crossentropy(target, output, from_logits=False): """Binary crossentropy between an output tensor and a target tensor. # Arguments target: A tensor with the same shape as `output`. output: A tensor. from_logits: Whether `output` is expected to be a logits tensor. By default, we consider that `output` encodes a probability distribution. # Returns A tensor. """ # Note: tf.nn.sigmoid_cross_entropy_with_logits # expects logits, Keras expects probabilities. if not from_logits: # transform back to logits _epsilon = _to_tensor(epsilon(), output.dtype.base_dtype) output = tf.clip_by_value(output, _epsilon, 1 - _epsilon) output = tf.log(output / (1 - output)) return tf.nn.sigmoid_cross_entropy_with_logits(labels=target, logits=output)
kullback_leibler_divergence
kullback_leibler_divergence(y_true, y_pred)
源碼：
def kullback_leibler_divergence(y_true, y_pred): y_true = K.clip(y_true, K.epsilon(), 1) y_pred = K.clip(y_pred, K.epsilon(), 1) return K.sum(y_true * K.log(y_true / y_pred), axis=-1)
poisson
poisson(y_true, y_pred)
源碼：
def poisson(y_true, y_pred): return K.mean(y_pred - y_true * K.log(y_pred + K.epsilon()), axis=-1)
說明：
cosine_proximity
cosine_proximity(y_true, y_pred)
源碼：
def cosine_proximity(y_true, y_pred): y_true = K.l2_normalize(y_true, axis=-1) y_pred = K.l2_normalize(y_pred, axis=-1) return -K.sum(y_true * y_pred, axis=-1)
說明：
簡寫：
mse = MSE = mean_squared_error mae = MAE = mean_absolute_error mape = MAPE = mean_absolute_percentage_error msle = MSLE = mean_squared_logarithmic_error kld = KLD = kullback_leibler_divergence cosine = cosine_proximity
參考：
Keras中文文檔
Loss Functions in Artificial Neural Networks
