How to detect from within Python whether packages are managed with conda
Question:
I would like to detect in a general way from within a Python session whether it is managed by conda.
A few ideas that are not general enough to be useful:
1: Use environment variables
As mentioned in How do I find the name of the conda environment in which my code is running?
import os
is_conda = 'CONDA_PREFIX' in os.system or 'CONDA_DEFAULT_ENV' in os.system
This seems not to work in the root conda environment, where these variables are not always defined. It also has potential false positives if conda happens to be activated while you are using another Python installation.
2: Check the executable path
import sys
is_conda = ('anaconda' in sys.executable) or ('miniconda' in sys.executable)
This will work in the case that users install anaconda/miniconda in the default path. It may fail otherwise. It’s also easy to imagine false-positives.
3. Check the version info
As noted in the answers to any way to tell if user's python environment is anaconda, you can check the Python version string in some cases:
import sys
is_conda = ('Continuum Analytics' in sys.version) or ('Anaconda' in sys.version)
This works currently for Python installed from the default channel, but this is quite brittle and may break in the future, particularly with Continuum’s company name change. It also probably fails if Python is installed from a third-party source like conda-forge.
4. Check the conda import
try:
import conda
except:
is_conda = False
else:
is_conda = True
This works as long as you’re in the root conda environment, but generally fails if you’re in another conda env where the conda package is not installed by default.
5: Try conda to see if it works
Suggestion from Atto Allas below:
import subprocess
try:
retcode = subprocess.call(['conda', 'install', '-y', 'pip'])
except:
is_conda = False
else:
is_conda = (retcode == 0)
This works in the simplest cases, but fails in the common case of using multiple kernels in Jupyter, where the conda executable may or may not be connected to the current Python kernel.
Is there any entirely general way to detect from Python whether that Python installation is managed by conda?
Answers:
Maybe this?
from subprocess import check_output, CalledProcessError
import sys
def is_conda_managed():
try:
out = check_output(['conda', 'env', 'list']).decode('utf-8')
lines = (line for line in out.splitlines() if line[:1] != '#')
roots = set(line.split()[-1] for line in lines if line.strip())
except CalledProcessError:
roots = set()
return sys.prefix in roots
Using compile-time flags seems like it’d be reasonably accurate, and hopefully stable going forward.
def is_conda():
import sysconfig
return 'conda-bld' in sysconfig.get_config_var("abs_srcdir")
I’m not sure about that specific key “abs_srcdir”, but sysconfig.get_config_values() has many items that may work. With my system python it’s:
>>> sysconfig.get_config_var("abs_srcdir")
'/Library/Caches/com.apple.xbs/Binaries/python/python-97.50.7~5/TempContent/Objects/2.7/python'
And with my conda python it’s:
In [17]: sysconfig.get_config_var("abs_srcdir")
Out[17]: '/Users/ilan/minonda/conda-bld/python-3.6_1482520641897/work/Python-3.6.0'
My hope is that compile-time flags are more robust that run-time checks. Runtime things like “is conda in sys.prefix” can be confused by having a Python.org install in a “conda” subdirectory. But the compile-time flags should be OK.
import sys, os
is_conda = os.path.exists(os.path.join(sys.prefix, 'conda-meta'))
as i read thru the the ideas and their weaknesses, i am not entirely clear by what you mean by ‘managed’.
└─╼ which -a python
/Users/me/anaconda/bin/python
/usr/bin/python
┌─[ ~/myPython]
└─╼ /Users/me/anaconda/bin/python
Python 2.7.13 |Anaconda 2.2.0 (x86_64)| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> quit()
┌─[ ~/myPython]
└─╼ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
┌─[ ~/myPython]
└─╼ source activate py36
(py36) ┌─[ ~/myPython]
└─╼ which -a python
/Users/me/anaconda/envs/py36/bin/python
/Users/me/anaconda/bin/python
/usr/bin/python
└─╼ python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
(py36) ┌─[ ~/myPython]
└─╼ which -a python
/Users/me/anaconda/envs/py36/bin/python
/Users/me/anaconda/bin/python
/usr/bin/python
are the lines:
Python 2.7.10 (default
Python 2.7.13 |Anaconda 2.2.0 (x86_64)
Python 3.6.2 |Continuum Analytics, Inc.
more or less what you’re trying to capture? granted these do not always appear and would be brittle at best. examining the image using the platform might be a step in a useful direction.
In [46]: platform.python_build()
Out[46]: ('default', 'Jul 20 2017 13:14:59')
In [47]: platform.python_compiler()
Out[47]: 'GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)'
In [48]: platform.python_implementation()
Out[48]: 'CPython'
the are alternatives to conda, i expect, so the q would seem to be broader than conda.
(this an answer, only because this is my very first SO post, and i only have a single reputaion)
I think the best approach is a variation on Ross Hynten’s answer: in the build process, conda creates a file called {sys.prefix}/conda-meta/history, so checking for its presence should tell you if you’re using conda in a reasonably robust manner:
import sys, os
is_conda = os.path.exists(os.path.join(sys.prefix, 'conda-meta', 'history'))
It remains possible to have false-positives with this approach, but seems to me very unlikely unless done deliberately.
I would like to detect in a general way from within a Python session whether it is managed by conda.
A few ideas that are not general enough to be useful:
1: Use environment variables
As mentioned in How do I find the name of the conda environment in which my code is running?
import os
is_conda = 'CONDA_PREFIX' in os.system or 'CONDA_DEFAULT_ENV' in os.system
This seems not to work in the root conda environment, where these variables are not always defined. It also has potential false positives if conda happens to be activated while you are using another Python installation.
2: Check the executable path
import sys
is_conda = ('anaconda' in sys.executable) or ('miniconda' in sys.executable)
This will work in the case that users install anaconda/miniconda in the default path. It may fail otherwise. It’s also easy to imagine false-positives.
3. Check the version info
As noted in the answers to any way to tell if user's python environment is anaconda, you can check the Python version string in some cases:
import sys
is_conda = ('Continuum Analytics' in sys.version) or ('Anaconda' in sys.version)
This works currently for Python installed from the default channel, but this is quite brittle and may break in the future, particularly with Continuum’s company name change. It also probably fails if Python is installed from a third-party source like conda-forge.
4. Check the conda import
try:
import conda
except:
is_conda = False
else:
is_conda = True
This works as long as you’re in the root conda environment, but generally fails if you’re in another conda env where the conda package is not installed by default.
5: Try conda to see if it works
Suggestion from Atto Allas below:
import subprocess
try:
retcode = subprocess.call(['conda', 'install', '-y', 'pip'])
except:
is_conda = False
else:
is_conda = (retcode == 0)
This works in the simplest cases, but fails in the common case of using multiple kernels in Jupyter, where the conda executable may or may not be connected to the current Python kernel.
Is there any entirely general way to detect from Python whether that Python installation is managed by conda?
Maybe this?
from subprocess import check_output, CalledProcessError
import sys
def is_conda_managed():
try:
out = check_output(['conda', 'env', 'list']).decode('utf-8')
lines = (line for line in out.splitlines() if line[:1] != '#')
roots = set(line.split()[-1] for line in lines if line.strip())
except CalledProcessError:
roots = set()
return sys.prefix in roots
Using compile-time flags seems like it’d be reasonably accurate, and hopefully stable going forward.
def is_conda():
import sysconfig
return 'conda-bld' in sysconfig.get_config_var("abs_srcdir")
I’m not sure about that specific key “abs_srcdir”, but sysconfig.get_config_values() has many items that may work. With my system python it’s:
>>> sysconfig.get_config_var("abs_srcdir")
'/Library/Caches/com.apple.xbs/Binaries/python/python-97.50.7~5/TempContent/Objects/2.7/python'
And with my conda python it’s:
In [17]: sysconfig.get_config_var("abs_srcdir")
Out[17]: '/Users/ilan/minonda/conda-bld/python-3.6_1482520641897/work/Python-3.6.0'
My hope is that compile-time flags are more robust that run-time checks. Runtime things like “is conda in sys.prefix” can be confused by having a Python.org install in a “conda” subdirectory. But the compile-time flags should be OK.
import sys, os
is_conda = os.path.exists(os.path.join(sys.prefix, 'conda-meta'))
as i read thru the the ideas and their weaknesses, i am not entirely clear by what you mean by ‘managed’.
└─╼ which -a python
/Users/me/anaconda/bin/python
/usr/bin/python
┌─[ ~/myPython]
└─╼ /Users/me/anaconda/bin/python
Python 2.7.13 |Anaconda 2.2.0 (x86_64)| (default, Dec 20 2016, 23:05:08)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> quit()
┌─[ ~/myPython]
└─╼ /usr/bin/python
Python 2.7.10 (default, Feb 7 2017, 00:08:15)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
┌─[ ~/myPython]
└─╼ source activate py36
(py36) ┌─[ ~/myPython]
└─╼ which -a python
/Users/me/anaconda/envs/py36/bin/python
/Users/me/anaconda/bin/python
/usr/bin/python
└─╼ python
Python 3.6.2 |Continuum Analytics, Inc.| (default, Jul 20 2017, 13:14:59)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
(py36) ┌─[ ~/myPython]
└─╼ which -a python
/Users/me/anaconda/envs/py36/bin/python
/Users/me/anaconda/bin/python
/usr/bin/python
are the lines:
Python 2.7.10 (default
Python 2.7.13 |Anaconda 2.2.0 (x86_64)
Python 3.6.2 |Continuum Analytics, Inc.
more or less what you’re trying to capture? granted these do not always appear and would be brittle at best. examining the image using the platform might be a step in a useful direction.
In [46]: platform.python_build()
Out[46]: ('default', 'Jul 20 2017 13:14:59')
In [47]: platform.python_compiler()
Out[47]: 'GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)'
In [48]: platform.python_implementation()
Out[48]: 'CPython'
the are alternatives to conda, i expect, so the q would seem to be broader than conda.
(this an answer, only because this is my very first SO post, and i only have a single reputaion)
I think the best approach is a variation on Ross Hynten’s answer: in the build process, conda creates a file called {sys.prefix}/conda-meta/history, so checking for its presence should tell you if you’re using conda in a reasonably robust manner:
import sys, os
is_conda = os.path.exists(os.path.join(sys.prefix, 'conda-meta', 'history'))
It remains possible to have false-positives with this approach, but seems to me very unlikely unless done deliberately.