Load compressed data (.npz) from file using numpy.load

Question

I have an array:

>>> data = np.ones((1,3,128))

I save it to file using savez_compressed:

>>> with open('afile','w') as f:
        np.savez_compressed(f,data=data)

When I try to load it I don’t seem to be able to access the data:

>>> with open('afile','r') as f:
        b=np.load(f)
>>> b.files
['data']
>>> b['data']

Traceback (most recent call last):
  File "<pyshell#196>", line 1, in <module>
    b['data']
  File "C:Python27libsite-packagesnumpylibnpyio.py", line 238, in __getitem__
    bytes = self.zip.read(key)
  File "C:Python27libzipfile.py", line 828, in read
    return self.open(name, "r", pwd).read()
  File "C:Python27libzipfile.py", line 853, in open
    zef_file.seek(zinfo.header_offset, 0)
ValueError: I/O operation on closed file

Am I doing something obviously wrong?

EDIT

Following @Saullo Castro’s answer I tried this:

>>> np.savez_compressed('afile.npz',data=data)
>>> b=np.load('afile.npz')
>>> b.files
['data']
>>> b['data']

and got the following error:

Traceback (most recent call last):
  File "<pyshell#253>", line 1, in <module>
    b['data']
  File "C:Python27libsite-packagesnumpylibnpyio.py", line 241, in __getitem__
    return format.read_array(value)
  File "C:Python27libsite-packagesnumpylibformat.py", line 440, in read_array
    shape, fortran_order, dtype = read_array_header_1_0(fp)
  File "C:Python27libsite-packagesnumpylibformat.py", line 336, in read_array_header_1_0
    d = safe_eval(header)
  File "C:Python27libsite-packagesnumpylibutils.py", line 1156, in safe_eval
    ast = compiler.parse(source, mode="eval")
  File "C:Python27libcompilertransformer.py", line 53, in parse
    return Transformer().parseexpr(buf)
  File "C:Python27libcompilertransformer.py", line 132, in parseexpr
    return self.transform(parser.expr(text))
  File "C:Python27libcompilertransformer.py", line 124, in transform
    return self.compile_node(tree)
  File "C:Python27libcompilertransformer.py", line 159, in compile_node
    return self.eval_input(node[1:])
  File "C:Python27libcompilertransformer.py", line 194, in eval_input
    return Expression(self.com_node(nodelist[0]))
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 578, in testlist
    return self.com_binary(Tuple, nodelist)
  File "C:Python27libcompilertransformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:Python27libcompilertransformer.py", line 596, in test
    then = self.com_node(nodelist[0])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 610, in or_test
    return self.com_binary(Or, nodelist)
  File "C:Python27libcompilertransformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:Python27libcompilertransformer.py", line 615, in and_test
    return self.com_binary(And, nodelist)
  File "C:Python27libcompilertransformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:Python27libcompilertransformer.py", line 619, in not_test
    result = self.com_node(nodelist[-1])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 626, in comparison
    node = self.com_node(nodelist[0])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 659, in expr
    return self.com_binary(Bitor, nodelist)
  File "C:Python27libcompilertransformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:Python27libcompilertransformer.py", line 663, in xor_expr
    return self.com_binary(Bitxor, nodelist)
  File "C:Python27libcompilertransformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:Python27libcompilertransformer.py", line 667, in and_expr
    return self.com_binary(Bitand, nodelist)
  File "C:Python27libcompilertransformer.py", line 1082, in com_binary
    return self.lookup_node(n)(n[1:])
  File "C:Python27libcompilertransformer.py", line 671, in shift_expr
    node = self.com_node(nodelist[0])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 683, in arith_expr
    node = self.com_node(nodelist[0])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 695, in term
    node = self.com_node(nodelist[0])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 715, in factor
    node = self.lookup_node(nodelist[-1])(nodelist[-1][1:])
  File "C:Python27libcompilertransformer.py", line 727, in power
    node = self.com_node(nodelist[0])
  File "C:Python27libcompilertransformer.py", line 805, in com_node
    return self._dispatch[node[0]](node[1:])
  File "C:Python27libcompilertransformer.py", line 739, in atom
    return self._atom_dispatch[nodelist[0][0]](nodelist)
  File "C:Python27libcompilertransformer.py", line 754, in atom_lbrace
    return self.com_dictorsetmaker(nodelist[1])
  File "C:Python27libcompilertransformer.py", line 1214, in com_dictorsetmaker
    assert nodelist[0] == symbol.dictorsetmaker
AssertionError

EDIT 2

The above error was in IDLE. It worked using Ipython.

Asked By: atomh33ls

||

Source

Answer 1

Try opening the file as binary:

with open('afile','rb') as f:

Answered By: Paul Evans

Answer 2

When using numpy.load you can pass the file name, and if the extension is .npz, it will first decompress the file:

np.savez_compressed('filename.npz', array1=array1, array2=array2)
b = np.load('filename.npz')

and do b['array1'] and so forth to retrieve the data from each array…

Answered By: Saullo G. P. Castro

Answer 3

I do have the same problem(AssertionError) when using numpy 1.7.1/1.8.0 with python 2.7.6 both on MAC OS and Windows. But the problem was automatically fixed after I switch to linux with python 2.7.5. Then I reinstall python 2.7.5 on MACOS and Windows and all the problem was gone.
Basically the problem is with python instead of numpy as the compiler is sending alert.So mightly the version matters.

But though npy is the serializable type of numpy, I don’t think the file is small enough even with savez_compressed for large matrix.

Hopefully your problem is the same with mine

Answered By: NineSPRing

Answer 4

You can also use the f attribute, which leaves you with a
np.ndarray:

images = np.load('images.npz')
images = images.f.arr_0

The name/key of the array inside the .npz-file (e.g. arr_0) can be found through

images.keys()

Note: The f attribute is not documented in the docstring of load. When load reads an npz file, it returns an instance of the class NpzFile. This class is available as numpy.lib.npyio.NpzFile. The docstring of the NpzFile class describes the f attribute. (As of this writing, the source code of the class can be found here.

Answered By: mrk

Load compressed data (.npz) from file using numpy.load

Question:

Answers: