Pickling array.array in 2.4 using cPickle

Question

I am working on a project built on python 2.4 (It is an embedded python project, so I don’t have a choice on the version of python used). Throughout the application, we use array.array to store data.

Support for pickling array.array objects was added to pickle (and cPickle) in 2.5. We have a viable workaround in 2.4 when using the pure python pickle class (we subclass Pickler/Unpickler to handle arrays) but this does not work with cPickle (we need this due to performance problems).

Any suggestions?

Asked By: Paul Osborne

||

Source

Answer 1

I’m not sure if the array type can be augmented with a __reduce__ method (perhaps with a subclass), but you could always try converting your arrays to sequences & back again… if the built-in extension mechanism won’t work for you. (hack)

I haven’t tried this before, but you could try adding support via copy_reg… essentially the same result as implementing __reduce__ on your own class or subclass, but a tad cleaner.

Answered By: pestilence669

Answer 2

You can use the standard library module copy_reg to register functions to deal with pickling instances of types that don’t natively support pickling; cPickle will use your registered functions where needed. I’d apply exactly this “hook” approach to your requirement of pickling instances of array.array.

Answered By: Alex Martelli

Answer 3

Looks like you can pickle them, but you can’t unpickle the result

Python 2.4.5 (#2, Jan 21 2010, 20:05:55) 
[GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import cPickle as pickle
>>> import array
>>> a=array.array('i','12345678')
>>> pickle.dumps(a,2)
'x80x02carraynarraynqx01)x81qx02.'
>>> b=pickle.loads(_)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: array() takes at least 1 argument (0 given)

Looks the the dumps doesn’t even include information about the typecode..or even the data 🙁

>>> a=array.array('c','abcdefghijkl') 
>>> pickle.dumps(a,2)                                                                                                                                                                             
'x80x02carraynarraynqx01)x81qx02.'
>>>

Answered By: John La Rooy

Answer 4

This is the final code that seems to be working (thanks for the suggestions):

# Add serialization for array objects
def array_unpickler(data):
    return array.array(data[0], data[1:])
def array_pickler(arr):
    return array_unpickler, ("%s%s" % (arr.typecode, arr.tostring()),)
copy_reg.pickle(array.ArrayType, array_pickler, array_unpickler)

_{This answer was posted as an edit to the question Pickling array.array in 2.4 using cPickle by the OP Paul Osborne under CC BY-SA 2.5.}

Answered By: vvvvv

Pickling array.array in 2.4 using cPickle

Question:

Answers: