Read Bash variables into a Python script

Question

I am running a bash script (test.sh) and it loads in environment variables (from env.sh). That works fine, but I am trying to see python can just load in the variables already in the bash script.

Yes I know it would probably be easier to just pass in the specific variables I need as arguments, but I was curious if it was possible to get the bash variables.

test.sh

#!/bin/bash
source env.sh

echo $test1

python pythontest.py

env.sh

#!/bin/bash

test1="hello"

pythontest.py

?
print test1 (that is what I want)

Asked By: Tall Paul

||

Source

Answer 1

You need to export the variables in bash, or they will be local to bash:

export test1

Then, in python

import os
print os.environ["test1"]

Answered By: AMADANON Inc.

Answer 2

Assuming the environment variables that get set are permanent, which I think they are not. You can use os.environ.

os.environ["something"]

Answered By: John

Answer 3

There’s another way using subprocess that does not depend on setting the environment. With a little more code, though.

For a shell script that looks like follows:

#!/bin/sh
myvar="here is my variable in the shell script"

function print_myvar() {
    echo $myvar
}

You can retrieve the value of the variable or even call a function in the shell script like in the following Python code:

import subprocess

def get_var(varname):
    CMD = 'echo $(source myscript.sh; echo $%s)' % varname
    p = subprocess.Popen(CMD, stdout=subprocess.PIPE, shell=True, executable='/bin/bash')
    return p.stdout.readlines()[0].strip()

def call_func(funcname):
    CMD = 'echo $(source myscript.sh; echo $(%s))' % funcname
    p = subprocess.Popen(CMD, stdout=subprocess.PIPE, shell=True, executable='/bin/bash')
    return p.stdout.readlines()[0].strip()

print get_var('myvar')
print call_func('print_myvar')

Note that both shell=True shall be set in order to process the shell command in CMD to be processed as it is, and set executable='bin/bash' to use process substitution, which is not supported by the default /bin/sh.

Answered By: Taejoon Byun

Answer 4

If you are trying to source a file, the thing you are missing is set -a. For example, to source env.sh, you would run:

set -a; source env.sh; set +a.

The reason you need this is that bash’s variables are local to bash unless they are exported. The -a option instructs bash to export all new variables (until you turn it off with +a). By using set -a before source, all of the variables imported by source will also be exported, and thus available to python.

Credit: this command comes from a comment posted by @chepner as a comment on this answer.

Answered By: Zags

Answer 5

Two additional options, which may or may not help in your specific situation:

Option 1:

if env.sh only contains regular NAME=value, get python to read it (as a text file) instead.

This only applies if you control the format of env.sh, and env.sh doesn’t contain any real shell commands, and you control the containing shell script.

Option 2:

In the shell script, once all the necessary variables are set, either save these to a file, or pipe them as stdin to your python script:

#!/bin/bash
source env.sh

echo $test1

set | python pythontest.py

or

#!/bin/bash
source env.sh

echo $test1

set > /tmp/$$_env
python pythontest.py --environment=/tmp/$$_env

You can then read the file (or stdin), and parse it into e.g. a dictionary.

Answered By: AMADANON Inc.

Read Bash variables into a Python script

Question:

Answers: