How to assign a value to a TensorFlow variable?
Question:
I am trying to assign a new value to a tensorflow variable in python.
import tensorflow as tf
import numpy as np
x = tf.Variable(0)
init = tf.initialize_all_variables()
sess = tf.InteractiveSession()
sess.run(init)
print(x.eval())
x.assign(1)
print(x.eval())
But the output I get is
0
0
So the value has not changed. What am I missing?
Answers:
In TF1, the statement x.assign(1) does not actually assign the value 1 to x, but rather creates a tf.Operation that you have to explicitly run to update the variable.* A call to Operation.run() or Session.run() can be used to run the operation:
assign_op = x.assign(1)
sess.run(assign_op) # or `assign_op.op.run()`
print(x.eval())
# ==> 1
(* In fact, it returns a tf.Tensor, corresponding to the updated value of the variable, to make it easier to chain assignments.)
However, in TF2 x.assign(1) will now assign the value eagerly:
x.assign(1)
print(x.numpy())
# ==> 1
There is an easier approach:
x = tf.Variable(0)
x = x + 1
print x.eval()
First of all you can assign values to variables/constants just by feeding values into them the same way you do it with placeholders. So this is perfectly legal to do:
import tensorflow as tf
x = tf.Variable(0)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(x, feed_dict={x: 3})
Regarding your confusion with the tf.assign() operator. In TF nothing is executed before you run it inside of the session. So you always have to do something like this: op_name = tf.some_function_that_create_op(params) and then inside of the session you run sess.run(op_name). Using assign as an example you will do something like this:
import tensorflow as tf
x = tf.Variable(0)
y = tf.assign(x, 1)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(x)
print sess.run(y)
print sess.run(x)
Also, it has to be noted that if you’re using your_tensor.assign(), then the tf.global_variables_initializer need not be called explicitly since the assign operation does it for you in the background.
Example:
In [212]: w = tf.Variable(12)
In [213]: w_new = w.assign(34)
In [214]: with tf.Session() as sess:
...: sess.run(w_new)
...: print(w_new.eval())
# output
34
However, this will not initialize all variables, but it will only initialize the variable on which assign was executed on.
You can also assign a new value to a tf.Variable without adding an operation to the graph: tf.Variable.load(value, session). This function can also save you adding placeholders when assigning a value from outside the graph and it is useful in case the graph is finalized.
import tensorflow as tf
x = tf.Variable(0)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
print(sess.run(x)) # Prints 0.
x.load(1, sess)
print(sess.run(x)) # Prints 1.
Update: This is depricated in TF2 as eager execution is default and graphs are no longer exposed in the user-facing API.
Here is the complete working example:
import numpy as np
import tensorflow as tf
w= tf.Variable(0, dtype=tf.float32) #good practice to set the type of the variable
cost = 10 + 5*w + w*w
train = tf.train.GradientDescentOptimizer(0.01).minimize(cost)
init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)
print(session.run(w))
session.run(train)
print(session.run(w)) # runs one step of gradient descent
for i in range(10000):
session.run(train)
print(session.run(w))
Note the output will be:
0.0
-0.049999997
-2.499994
This means at the very start the Variable was 0, as defined, then after just one step of gradient decent the variable was -0.049999997, and after 10.000 more steps we are reaching -2.499994 (based on our cost function).
Note: You originally used the Interactive session. Interactive session is useful when multiple different sessions needed to be run in the same script. However, I used the non interactive session for simplicity.
Use Tensorflow eager execution mode which is latest.
import tensorflow as tf
tf.enable_eager_execution()
my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3])
print(my_int_variable)
I answered a similar question here. I looked in a lot of places that always created the same problem. Basically, I did not want to assign a value to the weights, but simply change the weights. The short version of the above answer is:
tf.keras.backend.set_value(tf_var, numpy_weights)
I am trying to assign a new value to a tensorflow variable in python.
import tensorflow as tf
import numpy as np
x = tf.Variable(0)
init = tf.initialize_all_variables()
sess = tf.InteractiveSession()
sess.run(init)
print(x.eval())
x.assign(1)
print(x.eval())
But the output I get is
0
0
So the value has not changed. What am I missing?
In TF1, the statement x.assign(1) does not actually assign the value 1 to x, but rather creates a tf.Operation that you have to explicitly run to update the variable.* A call to Operation.run() or Session.run() can be used to run the operation:
assign_op = x.assign(1)
sess.run(assign_op) # or `assign_op.op.run()`
print(x.eval())
# ==> 1
(* In fact, it returns a tf.Tensor, corresponding to the updated value of the variable, to make it easier to chain assignments.)
However, in TF2 x.assign(1) will now assign the value eagerly:
x.assign(1)
print(x.numpy())
# ==> 1
There is an easier approach:
x = tf.Variable(0)
x = x + 1
print x.eval()
First of all you can assign values to variables/constants just by feeding values into them the same way you do it with placeholders. So this is perfectly legal to do:
import tensorflow as tf
x = tf.Variable(0)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(x, feed_dict={x: 3})
Regarding your confusion with the tf.assign() operator. In TF nothing is executed before you run it inside of the session. So you always have to do something like this: op_name = tf.some_function_that_create_op(params) and then inside of the session you run sess.run(op_name). Using assign as an example you will do something like this:
import tensorflow as tf
x = tf.Variable(0)
y = tf.assign(x, 1)
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
print sess.run(x)
print sess.run(y)
print sess.run(x)
Also, it has to be noted that if you’re using your_tensor.assign(), then the tf.global_variables_initializer need not be called explicitly since the assign operation does it for you in the background.
Example:
In [212]: w = tf.Variable(12)
In [213]: w_new = w.assign(34)
In [214]: with tf.Session() as sess:
...: sess.run(w_new)
...: print(w_new.eval())
# output
34
However, this will not initialize all variables, but it will only initialize the variable on which assign was executed on.
You can also assign a new value to a tf.Variable without adding an operation to the graph: tf.Variable.load(value, session). This function can also save you adding placeholders when assigning a value from outside the graph and it is useful in case the graph is finalized.
import tensorflow as tf
x = tf.Variable(0)
sess = tf.Session()
sess.run(tf.global_variables_initializer())
print(sess.run(x)) # Prints 0.
x.load(1, sess)
print(sess.run(x)) # Prints 1.
Update: This is depricated in TF2 as eager execution is default and graphs are no longer exposed in the user-facing API.
Here is the complete working example:
import numpy as np
import tensorflow as tf
w= tf.Variable(0, dtype=tf.float32) #good practice to set the type of the variable
cost = 10 + 5*w + w*w
train = tf.train.GradientDescentOptimizer(0.01).minimize(cost)
init = tf.global_variables_initializer()
session = tf.Session()
session.run(init)
print(session.run(w))
session.run(train)
print(session.run(w)) # runs one step of gradient descent
for i in range(10000):
session.run(train)
print(session.run(w))
Note the output will be:
0.0
-0.049999997
-2.499994
This means at the very start the Variable was 0, as defined, then after just one step of gradient decent the variable was -0.049999997, and after 10.000 more steps we are reaching -2.499994 (based on our cost function).
Note: You originally used the Interactive session. Interactive session is useful when multiple different sessions needed to be run in the same script. However, I used the non interactive session for simplicity.
Use Tensorflow eager execution mode which is latest.
import tensorflow as tf
tf.enable_eager_execution()
my_int_variable = tf.get_variable("my_int_variable", [1, 2, 3])
print(my_int_variable)
I answered a similar question here. I looked in a lot of places that always created the same problem. Basically, I did not want to assign a value to the weights, but simply change the weights. The short version of the above answer is:
tf.keras.backend.set_value(tf_var, numpy_weights)