# How can I create a learnable parameter or weight vector whose values are either 1 or -1

## Question:

I need to build a neural network which is known as learnable parameter or a weight vector. In that way, only one vector will be generated and multiplied with the data. I have created that as following:

``````from tensorflow.keras.layers import Layer
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model

class LearnableMultiplier(Layer):
def __init__(self, **kwargs):
super(LearnableMultiplier, self).__init__(**kwargs)

def build(self, input_shape):
shape=(input_shape[-1],),
initializer='glorot_uniform',
trainable=True)

super(LearnableMultiplier, self).build(input_shape)

def call(self, inputs):
return inputs * self.kernel

inputs = Input(shape=(64,))
multiplier = LearnableMultiplier()(inputs)
model = Model(inputs=inputs, outputs=multiplier )
``````

I need the vector learnable parameter defined above to be selected from values of either 1 or -1. I mean each value of that vector which is multiplied with my data can only be 1 or -1. Is that feasible? How can I do it?

Yes, it is feasible to constrain the learnable weight vector to be either 1 or -1. One way to achieve this is by using the sign function to convert the weights to either 1 or -1. You can modify your call function to apply the sign function to the weights, like this:

``````import tensorflow as tf
from tensorflow.keras.layers import Layer
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model

class LearnableMultiplier(Layer):
def __init__(self, **kwargs):
super(LearnableMultiplier, self).__init__(**kwargs)

def build(self, input_shape):
shape=(input_shape[-1],),
initializer='glorot_uniform',
trainable=True)

super(LearnableMultiplier, self).build(input_shape)

def call(self, inputs):
return inputs * tf.math.sign(self.kernel)
``````

The `tf.math.sign` function will convert each element of the weight vector to either 1 or -1, depending on whether it is positive or negative. This will ensure that only 1 or -1 are used as the values for the weight vector during the multiplication.

Update: How about if I want to extend it to make every value selected from a list of different values, for example [1, -1, 0.5, -0.5], is that still feasible to be done?

To do this, you can replace the `tf.math.sign` function in the call method with a custom logic that selects the appropriate value from the list based on the sign of the `kernel` weight. Here is an example implementation:

``````import tensorflow as tf
from tensorflow.keras.layers import Layer

class LearnableMultiplier(Layer):
def __init__(self, values, **kwargs):
super(LearnableMultiplier, self).__init__(**kwargs)
self.values = values

def build(self, input_shape):
shape=(input_shape[-1],),
initializer='glorot_uniform',
trainable=True)

super(LearnableMultiplier, self).build(input_shape)

def call(self, inputs):
signs = tf.math.sign(self.kernel)
values = tf.gather(self.values, tf.cast((signs + 1) / 2, tf.int32))
return inputs * values

``````

The modified `LearnableMultiplier` layer takes an additional argument `values`, which is a list of values from which to select the learnable scalar factor. In the `call` method, the `tf.math.sign` function is replaced with a call to `tf.gather` that selects the appropriate value from the `values` list based on the sign of the `kernel` weight. Specifically, we first map the sign values from the range [-1, 1] to the range [0, 1] by adding 1 and dividing by 2, and then use the resulting integer indices to select the corresponding values from the list.

To use the modified layer in a Keras model, you can create an instance of the `LearnableMultiplier` class and pass it the values argument. For example:

``````import tensorflow as tf
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model

values = [1, -1, 0.5, -0.5]
inputs = Input(shape=(10,))
x = LearnableMultiplier(values=values)(inputs)
outputs = Dense(1)(x)
model = Model(inputs=inputs, outputs=outputs)
``````
Categories: questions
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.