Python 2 –> 3: object of type 'zip' has no len()
Question:
I’m following a tutorial on neural nets1
It’s in Python 2.7. I’m using 3.4. This is the line that troubles me:
if test_data: n_test = len(test_data)
I get: TypeError: object of type 'zip' has no len()
.
Is there a way to rewrite it so that it works in 3.4?
Answers:
Force the zip()
to evaluate.
foo = list(zip(...))
If you know that the iterator is finite:
#NOTE: `sum()` consumes the iterator
n_test = sum(1 for _ in test_data) # find len(iterator)
Or if you know that test_data
is always small and a profiler says that the code is the bottleneck in your application then here’s code that might be more efficient for small n_test
:
test_data = list(test_data)
n_test = len(test_data)
Unfortunately, operator.length_hint()
(Python 3.4+) returns zero for a zip()
object. See PEP 0424 — A method for exposing a length hint.
A bit late now to answer, but in case anyone else stumbles on it: for that same neural net example tutorial, it turned out I had to wrap the 3 zip calls in the mnist_loader with a list(zip(…)) construct:
training_data = list(zip(training_inputs, training_results))
(...)
validation_data = list(zip(validation_inputs, va_d[1]))
(...)
test_data = list(zip(test_inputs, te_d[1]))
And then it worked.
Some Info
This is because in the Python 3.x, zip
returns a generator object. This object is not a list (it’s better) but it behaves like one. You can try iterating through it like this:
for i in zip([1,2,3,4], ['a','b','c','d']):
print i
The Quick Answer
Please show us the code surrounding where the error happened.
But I think I can still give you a quick (and not necessarily good) solution.
turn this
for i in reversed(range(1, len(x))):
into this:
for i in reversed(range(1, len(list(x))):
In mnist_loader wrap your zip results in list()
constructs like below
def load_data_wrapper():
tr_d, va_d, te_d = load_data()
training_inputs = [np.reshape(x, (784,1)) for x in tr_d[0]]
training_results = [vectorized_result(y) for y in tr_d[1]]
training_data = list(zip(training_inputs, training_results))
validation_inputs = [np.reshape(x,(784, 1))for x in va_d[0]]
validation_data = list(zip(validation_inputs, va_d[1]))
test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]
test_data = list(zip(test_inputs, te_d[1]))
return(training_data, validation_data, test_data)
I’m following a tutorial on neural nets1
It’s in Python 2.7. I’m using 3.4. This is the line that troubles me:
if test_data: n_test = len(test_data)
I get: TypeError: object of type 'zip' has no len()
.
Is there a way to rewrite it so that it works in 3.4?
Force the zip()
to evaluate.
foo = list(zip(...))
If you know that the iterator is finite:
#NOTE: `sum()` consumes the iterator
n_test = sum(1 for _ in test_data) # find len(iterator)
Or if you know that test_data
is always small and a profiler says that the code is the bottleneck in your application then here’s code that might be more efficient for small n_test
:
test_data = list(test_data)
n_test = len(test_data)
Unfortunately, operator.length_hint()
(Python 3.4+) returns zero for a zip()
object. See PEP 0424 — A method for exposing a length hint.
A bit late now to answer, but in case anyone else stumbles on it: for that same neural net example tutorial, it turned out I had to wrap the 3 zip calls in the mnist_loader with a list(zip(…)) construct:
training_data = list(zip(training_inputs, training_results))
(...)
validation_data = list(zip(validation_inputs, va_d[1]))
(...)
test_data = list(zip(test_inputs, te_d[1]))
And then it worked.
Some Info
This is because in the Python 3.x, zip
returns a generator object. This object is not a list (it’s better) but it behaves like one. You can try iterating through it like this:
for i in zip([1,2,3,4], ['a','b','c','d']):
print i
The Quick Answer
Please show us the code surrounding where the error happened.
But I think I can still give you a quick (and not necessarily good) solution.
turn this
for i in reversed(range(1, len(x))):
into this:
for i in reversed(range(1, len(list(x))):
In mnist_loader wrap your zip results in list()
constructs like below
def load_data_wrapper():
tr_d, va_d, te_d = load_data()
training_inputs = [np.reshape(x, (784,1)) for x in tr_d[0]]
training_results = [vectorized_result(y) for y in tr_d[1]]
training_data = list(zip(training_inputs, training_results))
validation_inputs = [np.reshape(x,(784, 1))for x in va_d[0]]
validation_data = list(zip(validation_inputs, va_d[1]))
test_inputs = [np.reshape(x, (784, 1)) for x in te_d[0]]
test_data = list(zip(test_inputs, te_d[1]))
return(training_data, validation_data, test_data)