# How can I create a learnable parameter or weight vector whose values are either 1 or -1

## Question:

I need to build a neural network which is known as learnable parameter or a weight vector. In that way, only one vector will be generated and multiplied with the data. I have created that as following:

```
from tensorflow.keras.layers import Layer
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
class LearnableMultiplier(Layer):
def __init__(self, **kwargs):
super(LearnableMultiplier, self).__init__(**kwargs)
def build(self, input_shape):
self.kernel = self.add_weight(name='kernel',
shape=(input_shape[-1],),
initializer='glorot_uniform',
trainable=True)
super(LearnableMultiplier, self).build(input_shape)
def call(self, inputs):
return inputs * self.kernel
inputs = Input(shape=(64,))
multiplier = LearnableMultiplier()(inputs)
model = Model(inputs=inputs, outputs=multiplier )
```

I need the vector learnable parameter defined above to be selected from values of either 1 or -1. I mean each value of that vector which is multiplied with my data can only be 1 or -1. Is that feasible? How can I do it?

## Answers:

Yes, it is feasible to constrain the learnable weight vector to be either 1 or -1. One way to achieve this is by using the sign function to convert the weights to either 1 or -1. You can modify your call function to apply the sign function to the weights, like this:

```
import tensorflow as tf
from tensorflow.keras.layers import Layer
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
class LearnableMultiplier(Layer):
def __init__(self, **kwargs):
super(LearnableMultiplier, self).__init__(**kwargs)
def build(self, input_shape):
self.kernel = self.add_weight(name='kernel',
shape=(input_shape[-1],),
initializer='glorot_uniform',
trainable=True)
super(LearnableMultiplier, self).build(input_shape)
def call(self, inputs):
return inputs * tf.math.sign(self.kernel)
```

The `tf.math.sign`

function will convert each element of the weight vector to either 1 or -1, depending on whether it is positive or negative. This will ensure that only 1 or -1 are used as the values for the weight vector during the multiplication.

**Update: How about if I want to extend it to make every value selected from a list of different values, for example [1, -1, 0.5, -0.5], is that still feasible to be done?**

To do this, you can replace the `tf.math.sign`

function in the call method with a custom logic that selects the appropriate value from the list based on the sign of the `kernel`

weight. Here is an example implementation:

```
import tensorflow as tf
from tensorflow.keras.layers import Layer
class LearnableMultiplier(Layer):
def __init__(self, values, **kwargs):
super(LearnableMultiplier, self).__init__(**kwargs)
self.values = values
def build(self, input_shape):
self.kernel = self.add_weight(name='kernel',
shape=(input_shape[-1],),
initializer='glorot_uniform',
trainable=True)
super(LearnableMultiplier, self).build(input_shape)
def call(self, inputs):
signs = tf.math.sign(self.kernel)
values = tf.gather(self.values, tf.cast((signs + 1) / 2, tf.int32))
return inputs * values
```

The modified `LearnableMultiplier`

layer takes an additional argument `values`

, which is a list of values from which to select the learnable scalar factor. In the `call`

method, the `tf.math.sign`

function is replaced with a call to `tf.gather`

that selects the appropriate value from the `values`

list based on the sign of the `kernel`

weight. Specifically, we first map the sign values from the range [-1, 1] to the range [0, 1] by adding 1 and dividing by 2, and then use the resulting integer indices to select the corresponding values from the list.

To use the modified layer in a Keras model, you can create an instance of the `LearnableMultiplier`

class and pass it the values argument. For example:

```
import tensorflow as tf
from tensorflow.keras.layers import Input, Dense
from tensorflow.keras.models import Model
values = [1, -1, 0.5, -0.5]
inputs = Input(shape=(10,))
x = LearnableMultiplier(values=values)(inputs)
outputs = Dense(1)(x)
model = Model(inputs=inputs, outputs=outputs)
```