Running Python script with temporary environment variables

Question

I have a Python application and it utilizes environment variables in Kubernetes configuration such as:

apiVersion: v1
kind: ConfigMap
metadata:
  name: my-config
  namespace: default
data:
  var1: foo
  var2: bar

---

apiVersion: v1
kind: Pod
metadata:
  name: my-pod
  namespace: default
spec:
  containers:
    - envFrom:
      - configMapRef:
          name: my-config

So, it is fine when the app is Dockerized and running on a Kubernetes farm.

However, when running the application on a local machine without Docker and Kubernetes but a humble command:

python app.py

I have to make the Python modules find the environment variables with os.getenv('var1') even if there is no ConfigMap or Pod on.

Is it possible to without a need to add extra code in Python modules or adding environment variables to the local machine system?

Asked By: vahdet

||

Source

Answer 1

you can set the environment variable intended by add the key-value pair in os module’s environ dictionary.

import os 

os.environ['intended var'] = 'intended value'

Sample run

>>> os.getenv('intended var')
'intended value'

Answered By: Swadhikar

Answer 2

You will need to have a “loader” small program that will invoke the second program with the appropriate variables.
Of course it is more convenient to have it in Python as well, as everything is already setup – but it could be something along

#!/usr/bin/env python3
import subprocess
import sys

import yaml

config = yaml.load(open(sys.argv[1]))["data"]  
# Instead of "data" here, use the yaml path to the place in the configuration your variables are

cmdline = sys.argv[2:]
if cmdline[0].endswith(".py"):
    cmdline.insert(0, sys.executable)

result = subprocess.run(cmdline, env=config)
exit(result.returncode)

(Of course you need to have installed some 3rd party lib to read the yaml config files – I’ve used PyYAML to test this)

If you mark the above script as exectuable, you can use it directly in your command line, without prefixing “python3” – otherwise, if you name this file “runner.py” your command line can be

python3 runner.py myconfig.yaml myscript.py parameter1 parameter2 parameter3

If you need to pass the current enviroments and just update then with the variables in the script, then do that before the .run call:

import os
...
config = yaml.load(...)[...]
config = {**os.environ, **config}

Also, see that this will use the same Python interpreter it is running on to execute the script, in a more or less naive way (just checks the literal “.py” file extension) – if you need this to be more robust, it is recommended to perform an stat on the target script and check if it is executable – then call it directly – and only use the current Python interpreter for other files.

or modify your code

Since you have a single script as entry points, and want to simplify running it for 3rdy paties, you could change it instead of having a generic loader –
in that case, I’d recomment adding an extra config variable just to indicate the variables are already properly set – so when the script run in its container, it does nothing – otherwise, it loads the data as above, and uptates its os.envrion.

In other words, at the entry point for your code, do something along:

if not os.environ.get("EVERYTHING_SETUP"):
   import yaml
   config = yaml.load(open("myconfig.yaml"))["data"] 
   os.environ.update(config)

Answered By: jsbueno

Answer 3

In a shell, you could also simply temporarily assign the value(s) for the environment variable(s) right before calling the script. No need to change your script at all.

Consider the following app.py which just prints the environment variables ENV_PARAM and ENV_PARAM2:

#!/usr/bin/env python3
import os

print(os.environ['ENV_PARAM'])
print(os.environ['ENV_PARAM2'])

When the vars are not set and you call it like this

python app.py

you will get a KeyError.

KeyError: ‘ENV_PARAM’

When you instead specify the values in the same line and call it like this

ENV_PARAM='foo' ENV_PARAM2='bar' python app.py

it works fine. Output:

foo
bar

This will not set the environment variable beyond that, so if you do

echo "$ENV_PARAM"

afterwards, it will return nothing. The environment variable was only set temporary, like you required.

Answered By: buddemat

Running Python script with temporary environment variables

Question:

Answers:

or modify your code