tensorflow MDA custom loss and ValueError: No gradients provided for any variable

Question

I would like to use the MDA (mean direction accuracy) as a custom loss function for a tensorflow neural network.

I am trying to implement this as described in here:
Custom Mean Directional Accuracy loss function in Keras

def mda(y_true, y_pred):
    s = K.equal(K.sign(y_true[1:] - y_true[:-1]),
                 K.sign(y_pred[1:] - y_pred[:-1]))
    return K.mean(K.cast(s, K.floatx()))

The network works fine but when I try to fit my data I am getting this error:

 ValueError: No gradients provided for any variable

I think that this is because I am loosing the gradient info from my pred tensor but I don’t know how can implement this…. or if this makes any sense at all…. Finally I want to predict is if some numeric series is going up or down, that is why this function made sense to me.

Asked By: Lord_Rafa

||

Source

Answer 1

It looks like the error you’re seeing is because the mda() function you’ve defined doesn’t have any differentiable operations. Because of this, TensorFlow doesn’t know how to compute the gradients of the function, and it’s unable to optimize the weights of your neural network using backpropagation.

To fix this, you’ll need to make sure that your mda() function uses only differentiable operations. This will allow TensorFlow to compute the gradients of the function and use them to optimize the weights of your network.

One way to do this would be to use the tf.math.sign() function instead of K.sign(), and the tf.math.reduce_mean() function instead of K.mean() in your mda() function. Both of these functions are differentiable, so TensorFlow will be able to compute the gradients of your mda() function and use them to optimize the weights of your network.

Here’s an example of how you could modify your mda() function to use differentiable operations:

import tensorflow as tf

def mda(y_true, y_pred):
    s = tf.equal(tf.math.sign(y_true[1:] - y_true[:-1]),
                 tf.math.sign(y_pred[1:] - y_pred[:-1]))
    return tf.math.reduce_mean(tf.cast(s, tf.float32))

This should allow you to use the mda() function as a custom loss function for your TensorFlow neural network.

Answered By: Reda Bourial

Answer 2

The problem is that with K.equal and K.cast, you change numbers into bools. As a result, no gradient can be calculated.

You could replace them with a calculation; using the fact that when two numbers are equal, their difference is zero, and that since sign returns only [-1, 0, 1], the absolute difference can only be 0, 1 or 2:

def mda(y_true, y_pred):
    d = K.abs(K.sign(y_true[1:] - y_true[:-1]) - (K.sign(y_pred[1:] - y_pred[:-1])))
    s = (1. - d) * (d - 1.) * (d - 2.) / 2.
return K.mean(s)

s is equal 1 when your K.equal is true, and 0 otherwise

Answered By: AndrzejO

Answer 3

Thanks Reda and AndrzeO for your answers my question. As AndrzejO mention, equals transform the data to boolean so no gradient there.

I implemented this other solution as an alternative to AndrzejO solution:

def mda_custom_loss(y_true, y_pred):
    res = tf.math.sign(y_true[1:] - y_true[:-1]) - tf.math.sign(y_pred[1:] - y_pred[:-1])
    s = tf.math.abs(tf.math.sign(res))
    return 1 - tf.math.reduce_mean(tf.math.sign(s))

Answered By: Lord_Rafa

tensorflow MDA custom loss and ValueError: No gradients provided for any variable

Question:

Answers: