Why doesn't os.environ.get() return a variable created by sourcing a .env file, but echo can find it?
Question:
Here’s one of those rare questions that gets asked a lot on SO, but there’s no answer. I’ll ask it again a little differently.
I’m trying to set an environment variable using a file and then read the variable in python. That should be pretty basic.
However, check this out:
$ cat .env.test
NAME=Bob
$ cat x.py
import os
print(os.environ.get("NAME"))
$ (source .env.test; echo $NAME)
Bob
$ (source .env.test; python3 x.py)
None
Isn’t that strange? Why doesn’t it work?
Is it because I didn’t write export in the .env file? If I add export, it does work. But why? Isn’t this what source is supposed to do?
$ cat .env.test
export NAME=Bob
$ cat x.py
import os
print(os.environ.get("NAME"))
$ (source .env.test; echo $NAME)
Bob
$ (source .env.test; python3 x.py)
Bob
I think the community could use an explanation about how sub-shells work, how source works and where python is looking for the environment variables, to answer this once and for all. Can you do that?
Related Questions
Answers:
var=value does not define an environment variable unless the option set -a is defined. By default, it only defines a shell variable.
Non-exported shell variables do not survive an exec() boundary — that is to say, when you start a new executable, only environment variables survive.
To export all variables sourced in from a file, enable the set -a shell option before sourcing it, as described in How to set environment variables from .env file:
$ (set -a; source .env.test; exec python3 x.py)
In the above case we don’t need to turn off set -a because it’s scoped to a subshell that exits when the ) is hit — or even before, because we can use exec to tell the subshell to replace itself with Python without forking again.
For a more general case, when this is used inside a script that’s going to continue to run for a while, it would instead look like:
set -a # enable auto-export
source .env.test # source file content
set +a # disable auto-export
Here’s one of those rare questions that gets asked a lot on SO, but there’s no answer. I’ll ask it again a little differently.
I’m trying to set an environment variable using a file and then read the variable in python. That should be pretty basic.
However, check this out:
$ cat .env.test
NAME=Bob
$ cat x.py
import os
print(os.environ.get("NAME"))
$ (source .env.test; echo $NAME)
Bob
$ (source .env.test; python3 x.py)
None
Isn’t that strange? Why doesn’t it work?
Is it because I didn’t write export in the .env file? If I add export, it does work. But why? Isn’t this what source is supposed to do?
$ cat .env.test
export NAME=Bob
$ cat x.py
import os
print(os.environ.get("NAME"))
$ (source .env.test; echo $NAME)
Bob
$ (source .env.test; python3 x.py)
Bob
I think the community could use an explanation about how sub-shells work, how source works and where python is looking for the environment variables, to answer this once and for all. Can you do that?
Related Questions
var=value does not define an environment variable unless the option set -a is defined. By default, it only defines a shell variable.
Non-exported shell variables do not survive an exec() boundary — that is to say, when you start a new executable, only environment variables survive.
To export all variables sourced in from a file, enable the set -a shell option before sourcing it, as described in How to set environment variables from .env file:
$ (set -a; source .env.test; exec python3 x.py)
In the above case we don’t need to turn off set -a because it’s scoped to a subshell that exits when the ) is hit — or even before, because we can use exec to tell the subshell to replace itself with Python without forking again.
For a more general case, when this is used inside a script that’s going to continue to run for a while, it would instead look like:
set -a # enable auto-export
source .env.test # source file content
set +a # disable auto-export