Is it possible to use tox with conda-based Python installations?

I don’t know how developed it is, but you can look at https://github.com/hayd/ctox.

Yes, you need the conda version of virtualenv installed for this to work.

Try to execute:

conda install virtualenv

virtualenv                15.1.0                   py36_

Change to project directory containing tox.ini and execute:

tox

I made tox and conda work together in Windows by:

• Installing virtualenv with conda in the environment which I use tox:

  conda install virtualenv

• Creating “Directory Junction” symlinks from C:PythonXY to my actual environment path. This gets around the InterpreterNotFound-error:

  mklink /J C:PythonXY C:realpathtomyPythonXYenv

I have installed Anaconda in E:Anaconda3, and all my environments in E:Anaconda3envs, e.g. E:Anaconda3envspy27

(See below for a script to make this quick and easy.)

Step 1 – Create environments with conda:

E:dev> conda create -n py27 python=2.7 --yes
E:dev> conda create -n py33 python=3.3 --yes
...
E:dev> conda create -n py36 python=3.6 --yes

Step 2 – Create all the symlinks:

E:dev> mklink /J C:Python27 E:Anaconda3envspy27
E:dev> mklink /J C:Python33 E:Anaconda3envspy33
...
E:dev> mklink /J C:Python36 E:Anaconda3envspy36

Note: I call conda create from a directory on the E-drive, so the --prefix/-p option is not required in order to install new environments in E:Anaconda3envs.

An Easier Way:

Instead of going through the cumbersome process of setting this up for each environment/python version, one can use the ToxEnvMatcher-class added further down in this manner:

my_envs = os.path.join('E:\', 'Anaconda3', 'envs')
tem = ToxEnvMatcher(my_envs)
for version in '27,34,35,36'.split(','):
    tem.make(version)

Edit: To make the script easier to use, I’ve added a new section to the file, (here assumed to be tox_with_conda.py,) so it can be called from cmd.exe:

C:dev> python tox_with_conda.py E:Anaconda3envs 27 34 35 36 37

Edit 2: Can also be installed with pip: pip install tox_with_conda and used as:

C:dev> python -m tox_with_conda E:Anaconda3envs 27 34 35 36 37

I’m using Python 3.6.3 and tox 2.9.1, but I don’t know when/if earlier versions work too.

Defence:
I assume that to some, this seems like a too cumbersome process (it’s really not, though), or to much of a hack. But keep in mind that being able to use Anaconda/conda also reduces waste of time spent trying to install libraries, packages, ++++.

Please note:

• I use tox with pytest, and have not noticed any impacts on my tests.
• My tests are simple, and there is a chance I just haven’t been exposed to issues yet.
• Assumable, there are things I haven’t thought of that might be relevant for others.

---

The class (also available here):

from subprocess import run
from os.path import join

DEFAULT_BASE = join('C:\', 'Python')


class ToxEnvMatcher:
    """
    Utility to make conda environments work with tox.

    Conda envs might be in other locations than where `tox <https://tox.readthedocs.io>`_ expects them to be.

    A symbolic link 'Directory Junction' is created from expected location to the actual location.
    Intended for Windows to get around the ``InterpreterNotFound``-error.

    E.g.: tox expects to find Python 2.7 in ``C:Python27``,
    but may actually be installed in another drive and location.

    Examples of use:

    .. code-block:: python

        my_envs = join('E:\', 'Anaconda3', 'envs')
        tem = ToxEnvMatcher(my_envs)
        for version in '27,34,35,36'.split(','):
            tem.make(version)

    The class is utilized through ``argsparse`` so it can also be used from cmd.exe.

    Examples of use of th of using ``ToxEnvMatcher`` from cmd.exe:

    .. code-block:: none

        E:dev> tox_with_conda.py E:Anaconda3envs 27 34 35 36 37

    It's possible to use the ``-b``/``--base`` option to override the default base location (``C:Python``):

    .. code-block:: none

        E:dev> tox_with_conda.py E:Anaconda3envs 27 34 35 36 37 --base D:Python

    :param str envs_dir: The path to where new conda environments will be created
    :param str default_base: The base of the 'default' location. Usually it's ``C:Python``
    """
    def __init__(self, envs_dir, default_base=DEFAULT_BASE):
        self.envs_dir = envs_dir
        self.default_base = default_base

    def __repr__(self):
        return '{}({})'.format(self.__class__.__name__, self.envs_dir)

    def make(self, version):
        """
        Take version and create conda environment with symlink from 'default tox location'.

        E.g.: given version='27' and environment folder ``{self.envs_dir}``:

         - ``conda create -p {self.envs_dir}py27 python=2.7``
         - ``mklink /J C:Python27 {self.envs_dir}py27``

        :param str version: A string on the form 'XY', e.g. '27' or '36'
        :return: None
        :rtype: NoneType
        """
        if len(version) != 2 or not int(version):
            raise ValueError("Parameter 'version' must be on the form 'XY', and not '{}'".format(version))
        conda_cmd = self._create_cmd_args(version)
        symlink_cmd = self._create_symlink_args(version)
        run(conda_cmd, shell=True)
        run(symlink_cmd, shell=True)

    def _get_env_folder(self, version):
        return join(self.envs_dir, 'py{}'.format(version))

    def _create_cmd_args(self, version):
        env_dir = self._get_env_folder(version)
        python_version = '.'.join(version)
        conda_create = 'conda create -p {} python={} --yes'.format(env_dir, python_version)
        return conda_create.split(' ')

    def _create_symlink_args(self, version):
        env_dir = self._get_env_folder(version)
        return 'mklink /J {}{} {}'.format(self.default_base, version, env_dir).split(' ')

The added code for making it work from cmd is:

if __name__ == '__main__':
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("env_dir",
                        help="The folder where conda environments should be installed.")
    parser.add_argument("versions", nargs='*',
                        help="The list of versions, formatted 'XY' where X is major and Y minor. E.g. '27 35 36'")
    parser.add_argument("-b", "--base", default=DEFAULT_BASE,
                        help="Base of the path which tox expects to find Python installed. "
                             "Default: {}.".format(DEFAULT_BASE))
    args = parser.parse_args()

    print('env_dir: ', args.env_dir)
    print('versions: ', args.versions)
    print('--base: ', args.base)

    tem = ToxEnvMatcher(args.env_dir, default_base=args.base)
    for version in args.versions:
        tem.make(version)

The tox-conda plugin should close that gap nowadays, but needs contributors who actively use conda to test and improve it.

from the README:

tox-conda is a plugin that provides integration with the conda package and environment manager for the tox automation tool. It’s like having your cake and eating it, too!

By default, tox creates isolated environments using [virtualenv](https://virtualenv.pypa.io] and installs dependencies from pip.

In contrast, when using the tox-conda plugin tox will use conda to create environments, and will install specified dependencies from conda. This is useful for developers who rely on conda for environment management and package distribution but want to take advantage of the features provided by tox for test automation.

To install that plugin it needs to be installed alongside tox in the same virutal environment. To create a virtual environment containing tox and tox-conda this should suffice:

$ python3 -m venv toxbase
$ toxbase/bin/pip install tox tox-conda
[...]
Successfully installed tox-3.13.2 tox-conda-0.2.0
$ toxbase/bin/tox --version
3.13.1 imported from /home/ob/tmp/toxbase/lib/python3.6/site-packages/tox/__init__.py
registered plugins:
    tox-conda-0.2.0 at /home/ob/tmp/toxbase/lib/python3.6/site-packages/tox_conda/plugin.py

from then on tox can be used as a command line tool and kept current by upgrading it in the toxbase virtualenv. Another, more automated way would be to use pipx