# Tensor with unspecified dimension in tensorflow

## Question:

I’m playing around with tensorflow and ran into a problem with the following code:

```
def _init_parameters(self, input_data, labels):
# the input shape is (batch_size, input_size)
input_size = tf.shape(input_data)[1]
# labels in one-hot format have shape (batch_size, num_classes)
num_classes = tf.shape(labels)[1]
stddev = 1.0 / tf.cast(input_size, tf.float32)
w_shape = tf.pack([input_size, num_classes], 'w-shape')
normal_dist = tf.truncated_normal(w_shape, stddev=stddev, name='normaldist')
self.w = tf.Variable(normal_dist, name='weights')
```

(I’m using `tf.pack`

as suggested in this question, since I was getting the same error)

When I run it (from a larger script that invokes this one), I get this error:

```
ValueError: initial_value must have a shape specified: Tensor("normaldist:0", shape=TensorShape([Dimension(None), Dimension(None)]), dtype=float32)
```

I tried to replicate the process in the interactive shell. Indeed, the dimensions of `normal_dist`

are unspecified, although the supplied values do exist:

```
In [70]: input_size.eval()
Out[70]: 4
In [71]: num_classes.eval()
Out[71]: 3
In [72]: w_shape.eval()
Out[72]: array([4, 3], dtype=int32)
In [73]: normal_dist.eval()
Out[73]:
array([[-0.27035281, -0.223277 , 0.14694688],
[-0.16527176, 0.02180306, 0.00807841],
[ 0.22624688, 0.36425814, -0.03099642],
[ 0.25575709, -0.02765726, -0.26169327]], dtype=float32)
In [78]: normal_dist.get_shape()
Out[78]: TensorShape([Dimension(None), Dimension(None)])
```

This is weird. Tensorflow generates the vector but can’t say its shape. Am I doing something wrong?

## Answers:

The variable can have a dynamic shape. `get_shape()`

returns the static shape.

In your case you have a tensor that has a dynamic shape, and currently happens to hold value that is 4×3 (but at some other time it can hold a value with a different shape — because the shape is dynamic). To set the static shape, use `set_shape(w_shape)`

. After that the shape you set will be enforced, and the tensor will be a valid `initial_value`

.

As Ishamael says, all tensors have a static shape, which is known at graph construction time and accessible using `Tensor.get_shape()`

; and a dynamic shape, which is only known at runtime and is accessible by fetching the value of the tensor, or passing it to an operator like `tf.shape`

. In many cases, the static and dynamic shapes are the same, but they can be different – the static shape can be *partially defined* – in order allow the dynamic shape to vary from one step to the next.

In your code `normal_dist`

has a partially-defined static shape, because `w_shape`

is a computed value. (TensorFlow sometimes attempts to evaluate

these computed values at graph construction time, but it gets stuck at `tf.pack`

.) It infers the shape `TensorShape([Dimension(None), Dimension(None)])`

, which means “a matrix with an unknown number of rows and columns,” because it knowns that `w_shape`

is a vector of length 2, so the resulting `normal_dist`

must be 2-dimensional.

You have two options to deal with this. You can set the static shape as Ishamael suggests, but this requires you to know the shape at graph construction time. For example, the following may work:

```
normal_dist.set_shape([input_data.get_shape()[1], labels.get_shape()[1]])
```

Alternatively, you can pass `validate_shape=False`

to the `tf.Variable`

constructor. This allows you to create a variable with a partially-defined shape, but it limits the amount of static shape information that can be inferred later on in the graph.

Similar question is nicely explained in TF FAQ:

In TensorFlow, a tensor has both a static (inferred) shape and a

dynamic (true) shape. The static shape can be read using the

`tf.Tensor.get_shape`

method: this shape is inferred from the operations

that were used to create the tensor, and may be partially complete. If

the static shape is not fully defined, the dynamic shape of a Tensor t

can be determined by evaluating`tf.shape(t)`

.

So `tf.shape()`

returns you a tensor, will always have a size of `shape=(N,)`

, and can be calculated in a session:

```
a = tf.Variable(tf.zeros(shape=(2, 3, 4)))
with tf.Session() as sess:
print sess.run(tf.shape(a))
```

On the other hand you can extract the static shape by using `x.get_shape().as_list()`

and this can be calculated anywhere.