pip installing environment.yml as if it's a requirements.txt

Question

I have an environment.yml file, but don’t want to use Conda:

name: foo
channels:
  - defaults
dependencies:
  - matplotlib=2.2.2

Is it possible to have pip install the dependencies inside an environment.yml file as if it’s a requirements.txt file?

I tried pip install -r environment.yml and it doesn’t work with pip==22.1.2.

Asked By: Intrastellar Explorer

||

Source

Answer 1

No, pip does not support this format. The format it expects for a requirements file is documented here. You’ll have to convert the environment.yml file to a requirements.txt format either manually or via a script that automates this process. However, keep in mind that not all packages on Conda will be available on PyPI.

Answered By: bgfvdu3w

Answer 2

I’ve implemented what Brian suggests in his comment.

This converts the environment.yaml to requirements.txt:

import yaml

with open("environment.yaml") as file_handle:
    environment_data = yaml.load(file_handle)

with open("requirements.txt", "w") as file_handle:
    for dependency in environment_data["dependencies"]:
        package_name, package_version = dependency.split("=")
        file_handle.write("{} == {}".format(package_name, package_version))

And this installs the dependencies directly with pip:

import os
import yaml

with open("environment.yaml") as file_handle:
    environment_data = yaml.load(file_handle)

for dependency in environment_data["dependencies"]:
    package_name, package_version = dependency.split("=")
    os.system("pip install {}=={}".format(package_name, package_version))

NOTE: I’ve omitted error handling and any other variations of package definitions (e.g., specification of a package version greater than or equal to a certain version) to keep it simple.

Answered By: Beni Trainor

Answer 3

Based on Beni implementation, I just wanted to adjust the code since it has lots of errors;

import os
import yaml

with open("environment.yaml") as file_handle:
    environment_data = yaml.safe_load(file_handle)

for dependency in environment_data["dependencies"]:
    if isinstance(dependency, dict):
      for lib in dependency['pip']:
        os.system(f"pip install {lib}")

Answered By: Ahmed

Answer 4

The first answer makes important points: there is not direct conversion because Conda is a general package manager and so includes additional packages. Furthermore, Conda packages can often go by different names. None of the proposed parsing solutions cover this situation.

Personally, I think the most efficacious complete approach is to recreate the environment with Mamba, then use pip in the environment to dump out a legitimate requirements.txt.

# use mamba, not conda
mamba env create -n foo -f environment.yaml
mamba install -yn foo pip
mamba run -n foo pip list --format freeze > requirements.txt
mamba env remove -n foo

That is, don’t overthink it and use the reliable tools at hand.

Answered By: merv

pip installing environment.yml as if it's a requirements.txt

Question:

Answers: