Can't add random noise to weights inside a Keras Layer

Question

I am trying to add a random noise during forward pass to a convolutional layer in Keras. I wrote a wrapper class where it would add noise to the weights before computing convolution. Any addition or modifications to self.weights has no effect to the final value. There is no error either. Can someone help ?

class Conv2D_New(tf.keras.layers.Conv2D):       
    def __init__(self, *args, **kwargs):
        super().__init__(*args, **kwargs)
        
    def call(self, inputs):
        random_noise = tf.random.normal(stddev=0.01, shape=self.weights[0].shape)
        self.weights[0] = self.weights[0] + random_noise
        tf.print(self.weights[0]) ########### NO CHANGE ???? ##################
        return super().call(inputs)

Asked By: hmedu

||

Source

Answer 1

If you look at the example in the docs you’ll notice it uses a Tensor method, assign_add. Try using the same method in your case:

self.weights[0].assign_add(random_noise)

Answered By: David

Can't add random noise to weights inside a Keras Layer

Question:

Answers: