StringIO replacement that works with bytes instead of strings?

Question

Is there any replacement for python StringIO class, one that will work with bytes instead of strings?

It may not be obvious but if you used StringIO for processing binary data you are out of luck with Python 2.7 or newer.

Asked By: sorin

||

Answer 1

Try io.BytesIO.

As others have pointed out, you can indeed use StringIO in 2.7, but BytesIO is a good choice for forward-compatibility.

Answered By: senderle

Answer 2

In Python 2.6/2.7, the io module is intended to be used for compatibility with Python 3.X. From the docs:

New in version 2.6.

The io module provides the Python
interfaces to stream handling. Under
Python 2.x, this is proposed as an
alternative to the built-in file
object, but in Python 3.x it is the
default interface to access files and
streams.

Note Since this module has been
designed primarily for Python 3.x, you
have to be aware that all uses of
“bytes” in this document refer to the
str type (of which bytes is an alias),
and all uses of “text” refer to the
unicode type. Furthermore, those two
types are not interchangeable in the
io APIs.

In Python versions earlier than 3.X the StringIO module contains the legacy version of StringIO, which unlike io.StringIO can be used in pre-2.6 versions of Python:

>>> import StringIO
>>> s=StringIO.StringIO()
>>> s.write('hello')
>>> s.getvalue()
'hello'
>>> import io
>>> s=io.StringIO()
>>> s.write('hello')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: string argument expected, got 'str'
>>> s.write(u'hello')
5L
>>> s.getvalue()
u'hello'

Answered By: Mark Tolonen

Answer 3

You say: “It may not be obvious but if you used StringIO for processing binary data you are out of luck with Python 2.7 or newer“.

It is not obvious because it is not true.

If you have code that works on 2.6 or earlier, it continues to work on 2.7. Unedited screen dump (Windows Command prompt window wrapping at col 80 and all):

C:UsersJohn>python26python -c"import sys,StringIO;s=StringIO.StringIO();s.wr
ite('hellon');print repr(s.getvalue()), sys.version"
'hellon' 2.6.6 (r266:84297, Aug 24 2010, 18:46:32) [MSC v.1500 32 bit (Intel)]

C:UsersJohn>python27python -c"import sys,StringIO;s=StringIO.StringIO();s.wr
ite('hellon');print repr(s.getvalue()), sys.version"
'hellon' 2.7.1 (r271:86832, Nov 27 2010, 18:30:46) [MSC v.1500 32 bit (Intel)]

If you need to write code that runs on 2.7 and 3.x, use the BytesIO class in the io module.

If you need/want a single codebase that supports 2.7, 2.6, … and 3.x, you will need to work a bit harder. Using the six module should help a lot.

Answered By: John Machin

StringIO replacement that works with bytes instead of strings?

Question:

Answers: