Conda environment using incorrect module because of order os sys.path

Question

I am using conda version 4.14.0. When I activate a conda environment I can see that the current numpy module is 1.22.3

conda list | grep -i numpy
numpy                     1.22.3           py39hc58783e_2    conda-forge

When I run python in the conda environment and load numpy it shows version 1.19.1

Python 3.9.12 | packaged by conda-forge | (main, Mar 24 2022, 23:25:59)
[GCC 10.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import numpy as np
>>> np.__version__
'1.19.5'

Looking at the sys.path locations I see that it is searching my local account site packages before the environment site-packages, how can this order be updated?

>>> print(sys.path)
['', '/opt/tljh/user/lib/python39.zip', 
'/opt/main/user/lib/python3.9', 
'/opt/main/user/lib/python3.9/lib-dynload', 
'/home/user/.local/lib/python3.9/site-packages', 
'/opt/main/user/lib/python3.9/site-packages']

Thank you for your help.

I cannot find where these variables are added through conda.

Asked By: tgstone

||

Source

Answer 1

The user site packages (at ~/.local) remaining available is the default behaviour of conda envs, and even issues requesting adding an option to disable that have been repeatedly closed by conda maintainers (example).

It is an unfortunate decision from Conda that the user site is included by default. You can exclude it from sys.path by setting an environment variable:

export PYTHONNOUSERSITE=1

That sounds hacky but it is actually the recommended workaround from conda themselves (ref).

Or, you can do what I do and just keep the user site totally empty, to avoid such issues without needing to mess around with env vars. Uninstall everything from ~/.local and use separate venvs for everything.

~/.local/bin/python3.9 -m pip uninstall -r <(~/.local/bin/python3.9 -m pip freeze)

You can use a project like pipx as a replacement to install any utilities which should be globally available, this avoids polluting the user site (and conda envs) with their dependencies.

Watch #7707 (make python user site-packages isolation in .local directory configurable) for possible future developments from conda here.

Answered By: wim

Conda environment using incorrect module because of order os sys.path

Question:

Answers: