How to maintain pip install options in requirements file made by pip freeze?

Question:

I use a pip requirements file to maintain a list of dependencies for my projects.

I now find myself having to install a library using pip install --install-option='some-setup.py.option' but pip freeze doesn’t record these options in its output which I save in my project’s requirements.txt. This causes problems because a simple pip install -r requirements.txt on a new machine installs all the dependencies without supplying the required arguments for this one library, and I’ve lost the simple round-trip operation.

So, my 2 part question is:

  • Is there a way to maintain pip install options in the pip freeze output somehow?
  • If not, is there a way to manually hack the requirements file to add the install option? I’m ok with losing the round trip nature of pip freeze if I have to, and switching to manual maintenance of the requirements file. I’ve checked the documentation but couldn’t see anything to help.

Unnecessary but possibly interesting details follow

I want to install pymongo but without building the C extension so I can use it asynchronously in an eventlet based app.

Install as desired and build requirements.txt:

(test)day@office:~/test$ pip install pymongo --install-option='--no_ext'
Downloading/unpacking pymongo
  Downloading pymongo-2.1.1.tar.gz (199Kb): 199Kb downloaded
  Running setup.py egg_info for package pymongo
Installing collected packages: pymongo
  Running setup.py install for pymongo
Successfully installed pymongo
Cleaning up...
(test)day@office:~/test$ pip freeze >  requirements.txt
(test)day@office:~/test$ cat requirements.txt
bottle==0.10.7
distribute==0.6.10
eventlet==0.9.16
greenlet==0.3.3
lxml==2.3.3
pymongo==2.1.1
simplejson==2.3.2
wsgiref==0.1.2

In new virtualenv, try to install same project from requirements.txt. pip builds the C extension for pymongo :(

(test2)day@office:~/test2$ pip install -r requirements.txt 
...
Downloading/unpacking pymongo==2.1.1 (from -r requirements.txt (line 6))
  Downloading pymongo-2.1.1.tar.gz (199Kb): 199Kb downloaded
  Running setup.py egg_info for package pymongo
Installing collected packages: pymongo
  Running setup.py install for pymongo
    building 'bson._cbson' extension
    gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -Ibson -I/usr/include/python2.6 -c bson/_cbsonmodule.c -o build/temp.linux-i686-2.6/bson/_cbsonmodule.o
...
Successfully installed pymongo
Cleaning up...

Update
Issue 271 was opened against pip in April 2011 asking for the ability to specify per-line --install-option in requirements.txt. Please vote for the issue if you have the same problem.

Asked By: Day

||

Answers:

This might probably be too naïve, but if you’re fine with managing the requirements and corresponding options manually — why not maintain them as a small shell script that includes the entire pip incantation? Just a work-around until the feature request is listened to 🙂

Answered By: fgeller

Since version 7.0 (released 2015-05-21), pip has the ability to parse --install-option and --global-option from requirement files. It should now be possible to have the following line in your requirements.txt:

pymongo==2.1.1 --install-option='--no_ext'

More information can be found here and here.

Answered By: gvalkov
Categories: questions Tags: ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.