How do you set a string of bytes from an environment variable in Python?

Question

Say that you have a string of bytes generated via os.urandom(24),

b'x1bxbax94(xaexd0xb2xa6xf2fxf6x1fIxedxbao$xc6Dx08xbax81x96v'

and you’d like to store that in an environment variable,

export FOO='x1bxbax94(xaexd0xb2xa6xf2fxf6x1fIxedxbao$xc6Dx08xbax81x96v'

and retrieve the value from within a Python program using os.environ.

foo = os.environ['FOO']

The problem is that, here, foo has the string literal value '\x1b\xba\x94... instead of the byte sequence b'x1bxbax94....

What is the proper export value to use, or means of using os.environ to treat FOO as a string of bytes?

Asked By: ybakos

||

Source

Answer 1

You can ‘unescape’ your bytes in Python with:

import os
import sys

if sys.version_info[0] < 3:  # sadly, it's done differently in Python 2.x vs 3.x
    foo = os.environ["FOO"].decode('string_escape')  # since already in bytes...
else:
    foo = bytes(os.environ["FOO"], "utf-8").decode('unicode_escape')

Answered By: zwer

Answer 2

With zwer’s answer I tried the following

first from bash (this is the same binary literal given by ybakos)

export FOO='x1bxbax94(xaexd0xb2xa6xf2fxf6x1fIxedxbao$xc6Dx08xbax81x96v'

then I launched the python shell (I have python 3.5.2)

>>> import os
>>> # ybakos's original binary literal
>>> foo =  b'x1bxbax94(xaexd0xb2xa6xf2fxf6x1fIxedxbao$xc6Dx08xbax81x96v'
>>> # ewer's python 3.x solution
>>> FOO = bytes(os.environ["FOO"], "utf-8").decode('unicode_escape')
>>> foo == FOO
False
>>> ^D

The last line of foo == FOO should return true, so the solution does not appear to work correctly.

I noticed that there is an os.envirnb dictionary, but I couldn’t figure out to set an Environment Variable to a binary literal, so I tried the following alternative which uses base64 encoding to get an ASCII version of the binary literal.

First launch python shell

>>> import os
>>> import base64
>>> foo = os.urandom(24)
>>> foo
b'{xd9qx90x8bxbaxecvxb3xcbx1e<xd7xbaxf1xb4x99xf056x90Ux16xae'
>>> foo_base64 = base64.b64encode(foo)
>>> foo_base64
b'e9lxkIu67Hazyx4817rxtJnwNTaQVRau'
>>> ^D

Then in the bash shell

export FOO_BASE64='e9lxkIu67Hazyx4817rxtJnwNTaQVRau'

Then back in the python shell

>>> import os
>>> import base64
>>> # the original binary value from the first python shell session
>>> foo = b'{xd9qx90x8bxbaxecvxb3xcbx1e<xd7xbaxf1xb4x99xf056x90Ux16xae'
>>> dec_foo = base64.b64decode(bytes(os.environ.get('FOO_BASE64'), "utf-8"))
>>> # the values match!
>>> foo == dec_foo
True
>>> ^D

The last line shows that the 2 results are the same!!

What we are doing, is first getting a binary value from os.urandom() and Base64 encoding it. We then use the Base64 encoded value to set the environment variable. Note: base64.b64encode() returns a binary value, but it will only contain printable ASCII characters.

Then in our program we read in the Base64 encode string value from the environment variable, convert the string into it’s binary form, and finally Base64 decode it back to its original value.

Answered By: gskluzacek

Answer 3

The easiest option is to simply set it as binary data in Bash. This uses ANSI string quoting and avoids the need for any sort of conversion on the Python side.

export FOO=$'x1bxbax94(xaexd0xb2xa6xf2fxf6x1fIxedxbao$xc6Dx08xbax81x96v'

Answered By: miken32

Answer 4

This is easier than what you can ever imagine. Just store it as a plane string and encode the string. It will return a binary data. E.g:

foo = '{xd9qx90x8bxbaxecvxb3xcbx1e<xd7xbaxf1xb4x99xf056x90Ux16xae'
B_FOO = str.encode(foo)

Answered By: Emeka Augustine

How do you set a string of bytes from an environment variable in Python?

Question:

Answers: