How to correctly define loss function in keras network that takes an image and outputs an image?

Question

I’m trying to build a model that takes grayscale image and generates another image as output by residual CNN using keras. And the key idea here is that the amount of non-white pixels is way smaller than the amount of white ones. So the error in each non-white pixel should have more weight in the loss function than in the white one. And the darker the pixel the bigger error should be.
That’s how I’ve done it presently.
Here, total_pred_score means all the correctly guessed pixels with their weights defined by some int coefficient and total_true_score means the total expected score for all images and pixels in the train set.
But I’m not sure it’s right.Can you help me please?

def custom_loss(y_true, y_pred):
    y_true1, y_pred1 = (255 - y_true) / 255, (255 - y_pred) / 255
    dif = (1 - K.abs(y_true1 - y_pred1))
    weight = (coeff - 1) * y_true1 + 1 
    total_true_score = K.sum(weight, axis = [0,1,2,3])
    total_pred_score = K.sum(multiply([dif,weight]), axis = [0,1,2,3])
    return K.abs(total_true_score - total_pred_score) / total_true_score

Asked By: limonchellka

||

Source

Answer 1

The loss function that you are trying to build should have two properties:

Error in each non-white pixel should have more weight than in the white ones
Darker the pixel the bigger error should be

So to make it simpler you want your error to be more if you classify a dark pixel as a white one, but really not too much worried about the cases where you classify the white pixel as dark ones but you still want them to contribute to your loss function. Also, you want to make sure that your loss is proportional to the intensity of the pixel (darker the pixel more the error).

I have come up with the following loss function which obeys these two properties:

def custom_loss(y_true, y_pred, coeff):
    y_true1, y_pred1 = (255 - y_true) / 255, (255 - y_pred) / 255
    dif = y_true1 - y_pred1
    temp1 = K.l2_normalize(K.cast(K.greater(dif, 0),"float32")*y_true1,axis=-1) * coeff
    temp2 = K.cast(K.less(dif, 0),"float32")
    weight = temp1 + temp2  
    loss = K.abs(weight*dif)
    average_loss = K.mean(loss)   ##you need to return this when you use in your code
    return K.eval(loss)

So basically we are finding the difference between true value and the predicted value after doing the normalization and multiplying weights proportional only to positive values in dif (which corresponds to darker pixels) –temp1. And then we just add a weight of 1 to pixels corresponding to negative values (which corresponds to white).

SAMPLE OUTPUT

coeff = 5
y_t = np.array([0,250,30,0,255,160,210,0,2,4])
y_p = np.array([50,0,80,10,255,160,210,250,2,80])
custom_loss(y_t,y_p,coeff ) 

array([0.449957, 0.98039216, 0.39702088, 0.08999141, 0., 0., 0., 2.249785, 0., 0.67320627],dtype=float32)

Here if you look at pixels at bit 2 and 8 carefully, pixel 8 corresponds to the case where we predict a dark pixel as a while pixel which is terribly bad in our case so the loss value is really high 2.249785, likewise pixel 2 corresponds to the case where we predict a white pixel as a dark pixel which is kind of okay for us, so the loss value is 0.98039216 which is not as high as the previous case. This corresponds to property 1.

Likewise, if you look as pixels 1 and 3 the magnitude of error in both the cases are the same (they are off by 50) but pixel 1 is much darker than pixel 3 so we want the error on pixel 1 to be higher than pixel 3, which is what we observe in the error vector.

And in cases where our true and predicted values match, we have 0 error.

Answered By: Parthasarathy Subburaj

How to correctly define loss function in keras network that takes an image and outputs an image?

Question:

Answers: