How to clear the conda environment variables?
Question:
While I was setting an environment variable on a conda base env, I made an error in the path that was supposed to be assigned to the variable. I was trying to set the $PYSPARK_PYTHON env variable on the conda env. The set command conda env config vars set $PYSPARK_PYTHON=errorpath executed successfully even though the path has an error, and asked me to reactivate the environment. And I am unable to activate the env.
When I check the env var list by doing the following:
conda env config vars list -n base
It shows me the incorrect path which I have set but without the variable name as follows:
= C:\ProgramData\Anaconda3\envs\some-env\python3.7
And because of this above incorrect env variable, I am unable to activate the base env. It gives me an error as follows:
Invoke-Expression : At line:6 char:1
+ $Env: = "C:\ProgramData\Anaconda3\envs\some-env\python3.7"
+ ~~~~~
Variable reference is not valid. ':' was not followed by a valid variable name character. Consider using ${} to
delimit the name.
At C:ProgramDataAnaconda3shellcondabinConda.psm1:101 char:9
+ Invoke-Expression -Command $activateCommand;
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ParserError: (:) [Invoke-Expression], ParseException
+ FullyQualifiedErrorId : InvalidVariableReferenceWithDrive,Microsoft.PowerShell.Commands.InvokeExpressionCommand
I am not sure how to fix this error, but I want to just remove the environment variable from the base env.
I tried unsetting it using the command conda env config vars unset $PYSPARK_PYTHON -n base. But it doesn’t work. I think as the variable declaration is missing in the list, I am unable to access the variable. I did try it without the $PYSPARK_PYTHON hoping it removes all the orphaned env variables but it doesn’t.
Could anyone help me with this? Is there any way to reset the base environment without affecting the other envs, or reset the env variables list on the given env?
Thanks
Answers:
Try looking for a JSON file called state that resides in the conda-meta directory of your environment.
Depending on your OS and install directory the conda-meta will be installed in different locations. The default install path for each OS is
- Windows:
C:Users<your-username>Anaconda3conda-metastate
- Mac:
/Users/<your-username>/anaconda3/conda-meta/state
or ~/opt//anaconda3/conda-meta/state for GUI install
- Linux:
/home/<your-username>/anaconda3/conda-meta/state
By editing that file you can manually change your environment variables.
Further Explanation
I recently messed up a conda environment too and only found this answer through inspecting the code for conda.
In the code you can see that the environment variables are saved and loaded from a file
def _get_environment_state_file(self):
env_vars_file = join(self.prefix_path, PREFIX_STATE_FILE)
if lexists(env_vars_file):
with open(env_vars_file, 'r') as f:
prefix_state = json.loads(f.read(), object_pairs_hook=OrderedDict)
else:
prefix_state = {}
return prefix_state
def get_environment_env_vars(self):
prefix_state = self._get_environment_state_file()
env_vars_all = OrderedDict(prefix_state.get('env_vars', {}))
env_vars = {
k: v for k, v in env_vars_all.items()
if v != CONDA_ENV_VARS_UNSET_VAR
}
return env_vars
If you print env_vars_file or look where PREFIX_STATE_FILE is defined you will find the file the environment variables are stored in for the environment.
For MacOS users who find themselves here env vars are stored in a JSON file called state located at /Users/<you>/opt/anaconda3/envs/<env>/conda-meta. It appears you can just empty the values leaving you with:
{"env_vars": {}}
There might be other state key/value pairs in there so be careful not to obliterate those.
Try conda env config vars unset PYSPARK_PYTHON
Every time I started a new shell in conda env, it returned an error complaining about a badly assigned variable. In fact, there was a parsing error, as I left space before and after "=" sign. I tried to unset using
conda env config vars unset
It didn’t work. So, I thought to clear the content of my state (/Users//opt/anaconda3/envs//conda-meta)and everything broke down since it has to have a valid json format. So, if you don’t want to have any variable assigned, just copy and past this into state:
{"env_vars": {}}
While I was setting an environment variable on a conda base env, I made an error in the path that was supposed to be assigned to the variable. I was trying to set the $PYSPARK_PYTHON env variable on the conda env. The set command conda env config vars set $PYSPARK_PYTHON=errorpath executed successfully even though the path has an error, and asked me to reactivate the environment. And I am unable to activate the env.
When I check the env var list by doing the following:
conda env config vars list -n base
It shows me the incorrect path which I have set but without the variable name as follows:
= C:\ProgramData\Anaconda3\envs\some-env\python3.7
And because of this above incorrect env variable, I am unable to activate the base env. It gives me an error as follows:
Invoke-Expression : At line:6 char:1
+ $Env: = "C:\ProgramData\Anaconda3\envs\some-env\python3.7"
+ ~~~~~
Variable reference is not valid. ':' was not followed by a valid variable name character. Consider using ${} to
delimit the name.
At C:ProgramDataAnaconda3shellcondabinConda.psm1:101 char:9
+ Invoke-Expression -Command $activateCommand;
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : ParserError: (:) [Invoke-Expression], ParseException
+ FullyQualifiedErrorId : InvalidVariableReferenceWithDrive,Microsoft.PowerShell.Commands.InvokeExpressionCommand
I am not sure how to fix this error, but I want to just remove the environment variable from the base env.
I tried unsetting it using the command conda env config vars unset $PYSPARK_PYTHON -n base. But it doesn’t work. I think as the variable declaration is missing in the list, I am unable to access the variable. I did try it without the $PYSPARK_PYTHON hoping it removes all the orphaned env variables but it doesn’t.
Could anyone help me with this? Is there any way to reset the base environment without affecting the other envs, or reset the env variables list on the given env?
Thanks
Try looking for a JSON file called state that resides in the conda-meta directory of your environment.
Depending on your OS and install directory the conda-meta will be installed in different locations. The default install path for each OS is
- Windows:
C:Users<your-username>Anaconda3conda-metastate - Mac:
/Users/<your-username>/anaconda3/conda-meta/state
or~/opt//anaconda3/conda-meta/statefor GUI install - Linux:
/home/<your-username>/anaconda3/conda-meta/state
By editing that file you can manually change your environment variables.
Further Explanation
I recently messed up a conda environment too and only found this answer through inspecting the code for conda.
In the code you can see that the environment variables are saved and loaded from a file
def _get_environment_state_file(self):
env_vars_file = join(self.prefix_path, PREFIX_STATE_FILE)
if lexists(env_vars_file):
with open(env_vars_file, 'r') as f:
prefix_state = json.loads(f.read(), object_pairs_hook=OrderedDict)
else:
prefix_state = {}
return prefix_state
def get_environment_env_vars(self):
prefix_state = self._get_environment_state_file()
env_vars_all = OrderedDict(prefix_state.get('env_vars', {}))
env_vars = {
k: v for k, v in env_vars_all.items()
if v != CONDA_ENV_VARS_UNSET_VAR
}
return env_vars
If you print env_vars_file or look where PREFIX_STATE_FILE is defined you will find the file the environment variables are stored in for the environment.
For MacOS users who find themselves here env vars are stored in a JSON file called state located at /Users/<you>/opt/anaconda3/envs/<env>/conda-meta. It appears you can just empty the values leaving you with:
{"env_vars": {}}
There might be other state key/value pairs in there so be careful not to obliterate those.
Try conda env config vars unset PYSPARK_PYTHON
Every time I started a new shell in conda env, it returned an error complaining about a badly assigned variable. In fact, there was a parsing error, as I left space before and after "=" sign. I tried to unset using
conda env config vars unset
It didn’t work. So, I thought to clear the content of my state (/Users//opt/anaconda3/envs//conda-meta)and everything broke down since it has to have a valid json format. So, if you don’t want to have any variable assigned, just copy and past this into state:
{"env_vars": {}}