TensorFlow operator overloading
Question:
What is the difference between
tf.add(x, y)
and
x + y
in TensorFlow? What would be different in your computation graph when you construct your graph with + instead of tf.add()?
More generally, are + or other operations overloaded for tensors?
Answers:
If at least one of x or y is a tf.Tensor object, the expressions tf.add(x, y) and x + y are equivalent. The main reason you might use tf.add() is to specify an explicit name keyword argument for the created op, which is not possible with the overloaded operator version.
Note that if neither x nor y is a tf.Tensor—for example if they are NumPy arrays—then x + y will not create a TensorFlow op. tf.add() always creates a TensorFlow op and converts its arguments to tf.Tensor objects. Therefore, if you are writing a library function that might accept both tensors and NumPy arrays, you might prefer to use tf.add().
The following operators are overloaded in the TensorFlow Python API:
__neg__ (unary -)__abs__ (abs())__invert__ (unary ~)__add__ (binary +)__sub__ (binary -)__mul__ (binary elementwise *)__div__ (binary / in Python 2)__floordiv__ (binary // in Python 3)__truediv__ (binary / in Python 3)__mod__ (binary %)__pow__ (binary **)__and__ (binary &)__or__ (binary |)__xor__ (binary ^)__lt__ (binary <)__le__ (binary <=)__gt__ (binary >)__ge__ (binary >=)
Please note, __eq__ ( binary == ) is not overloaded. x == y will simply return a Python boolean whether x and y refer to the same tensor. You need to use tf.equal() explicitly to check for element-wise equality. Same goes for not equal, __ne__ ( binary != ).
Mrry nicely explained that there is no real difference. I will just add when using tf.add is beneficial.
tf.add has one important parameter which is name. It allows you to name the operation in a graph which will be visible in tensorboard. So my rule of thumb, if it will be beneficial to name an operation in tensorboard, I use tf. equivalent, otherwise I go for brevity and use overloaded version.
What is the difference between
tf.add(x, y)
and
x + y
in TensorFlow? What would be different in your computation graph when you construct your graph with + instead of tf.add()?
More generally, are + or other operations overloaded for tensors?
If at least one of x or y is a tf.Tensor object, the expressions tf.add(x, y) and x + y are equivalent. The main reason you might use tf.add() is to specify an explicit name keyword argument for the created op, which is not possible with the overloaded operator version.
Note that if neither x nor y is a tf.Tensor—for example if they are NumPy arrays—then x + y will not create a TensorFlow op. tf.add() always creates a TensorFlow op and converts its arguments to tf.Tensor objects. Therefore, if you are writing a library function that might accept both tensors and NumPy arrays, you might prefer to use tf.add().
The following operators are overloaded in the TensorFlow Python API:
__neg__(unary-)__abs__(abs())__invert__(unary~)__add__(binary+)__sub__(binary-)__mul__(binary elementwise*)__div__(binary/in Python 2)__floordiv__(binary//in Python 3)__truediv__(binary/in Python 3)__mod__(binary%)__pow__(binary**)__and__(binary&)__or__(binary|)__xor__(binary^)__lt__(binary<)__le__(binary<=)__gt__(binary>)__ge__(binary>=)
Please note, __eq__ ( binary == ) is not overloaded. x == y will simply return a Python boolean whether x and y refer to the same tensor. You need to use tf.equal() explicitly to check for element-wise equality. Same goes for not equal, __ne__ ( binary != ).
Mrry nicely explained that there is no real difference. I will just add when using tf.add is beneficial.
tf.add has one important parameter which is name. It allows you to name the operation in a graph which will be visible in tensorboard. So my rule of thumb, if it will be beneficial to name an operation in tensorboard, I use tf. equivalent, otherwise I go for brevity and use overloaded version.