How to properly define custom loss function with keras

Question

I am trying to define a custom rmse loss function for Keras. I wrote the function below

import keras.backend as K

def custom_rmse(y_true, y_pred):
  loss = K.square(y_pred - y_true)
  for i in range(len(y_true)):
    for j in range(y_true.shape[1]):
      tmp = float(y_true[i][j])
      if (tmp < 0.15):
        loss[i][j] *= 0.2
      else:
        loss[i][j] *=0.8
  loss = K.sqrt(K.sum(loss, axis=1)) 
  return loss

But when I ran the model and attempted to fix it, I kept getting this error

 /usr/local/lib/python3.7/dist-packages/keras/engine/training.py:853 train_function  *
        return step_function(self, iterator)
    <ipython-input-95-efab27dd2563>:8 custom_rmse  *
        if (tmp < 0.15):
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1172 if_stmt
        _tf_if_stmt(cond, body, orelse, get_state, set_state, symbol_names, nouts)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1219 _tf_if_stmt
        cond, aug_body, aug_orelse, strict=True)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/dispatch.py:206 wrapper
        return target(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/util/deprecation.py:549 new_func
        return func(*args, **kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/control_flow_ops.py:1254 cond
        return cond_v2.cond_v2(pred, true_fn, false_fn, name)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/ops/cond_v2.py:88 cond_v2
        op_return_value=pred)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/framework/func_graph.py:1007 func_graph_from_py_func
        func_outputs = python_func(*func_args, **func_kwargs)
    /usr/local/lib/python3.7/dist-packages/tensorflow/python/autograph/operators/control_flow.py:1197 aug_body
        set_state(init_vars)
    /tmp/tmp_3e6lmrw.py:35 set_state
        (loss[i][j],) = vars_

    TypeError: 'Tensor' object does not support item assignment

I will appreciate suggestions on how to fix this. Thanks.

Asked By: Yusuf Falola

||

Source

Answer 1

If-Else statements are usually not the way to go for loss functions. Most of the time, it is better to do a "soft" way of what you are trying to achieve. This can be done by (for example) using a steep logistic function on your loss values in the following way:

def custom_rmse(y_true, y_pred):
    loss = K.square(y_pred - y_true)

    logistic_values = tf.sigmoid(1000 * (y_true - 0.15))
    loss = logistic_values * loss * 0.8 + (1-logistic_values * loss * 0.2)

    loss = K.sqrt(K.sum(loss, axis=1)) 
    return loss

This code will do the following:

We subtract 0.15 (your threshold) from your y_true so that the threshold for the new values now is at 0.
We multiply the result by a high number (I selected 1000 here, the higher the number the steeper will the "soft threshold" be. This means, that all values higher than your threshold are now very high positive values and all values lower than your threshold will now be high negative values.
We apply the sigmoid function to the resulting values. This function will be 1 for all high positive values and -0 for all high negative values (with a soft transition in between).
Now, we can just multiply our loss by logistic_values or 1-logistic_values, which basically acts as a mask that masks out all values that are 0 or 1 respectively. All the values that are not masked out can now be multiplied by their respective factor of 0.8 or 0.2.

Answered By: Marc Felix

How to properly define custom loss function with keras

Question:

Answers: