Installing multiple versions of a package with pip
Question:
In my application I would like to use:
packageA, which requires packageX==1.3packageB, which requires packageX==1.4packageX==1.5
How can I install multiple versions of packageX with pip to handle this situation?
Answers:
pip won’t help you with this.
You can tell it to install a specific version, but it will override the other one. On the other hand, using two virtualenvs will let you install both versions on the same machine, but not use them at the same time.
You best bet is to install both version manually, by putting them in your Python path with a different name.
But if your two libs expect them to have the same name (and they should), you will have to modify them so they pick up the version they need with some import alias such as:
import dependencyname_version as dependencyname
There is currently no clean way to do this. The best you can hope is for this hack to work.
I’d rather ditch one of the two libs and replace it with an equivalent, or patch it to accept the new version of the dependency and give the patch back to the community.
Download the source for ea. package. Install each on its own separate folder. For example. I had version 1.10 package, but wanted to switch to the dev version for some work. I downloaded the source for the dev module:
git clone https://github.com/networkx/networkx.git
cd netwokrx
I created a folder for this version:
mkdir /home/username/opt/python, then I set the PYTHONPATH env var to: export PYTHONPATH=/home/username/opt/python/lib/python2.7/site-packages/. Next, I installed it using: python setup.py install --prefix=/home/username/opt/python
Now, since my PYTHONPATH is now pointing to this other site-packages folder, when I run python on the command line, and import the new module, it works. To switch switch back, remove the new folder from PYTHONPATH.
>>> import networkx as nx
>>> nx.__version__
'2.0.dev_20151209221101'
an ugly workaround I use with python in blender is I’ll install (and keep off path) a like version of python and use subprocess to have the other version do the needed work. Blenders python tends to get a little temperamental if you do much more than install pandas and scipy. I’ve tried this using virtualenvs with blender but that tends to break things.
Also on the off chance you are using blender for data visualization, you are going to want to add a config folder to your version number folder, this will keep all of your addons in that folder and it makes it far more portable and far less likely to mess up other installs of blender. Many people who make addons for blender are not ‘programmers’, so often those savvy people will do some very hackish things and this has been the best workaround I’ve been able to use.
Another workaround (and this has so many flags on the play that it should disqualify me from touching a keyboard) is to manually locate the init file and manually add it to globals with importlib … this comes with risks. Some modules will play alright when you do this, other modules will toss crapfits that can lead to extra special troubleshooting sessions. Keep it to like versions and it does cut down on the issues and I’ve had ‘alright’ luck with using this to import modulus from behind virtual envs, but there is a reason why I use subprocess calls when working with blender’s python.
def importfromfilelocation(x,y,z):
#"""x = 'tk',y = "tkinter", z =r'C:pyth"""
mod_alis = x
spec = importlib.util.spec_from_file_location(y, z)
print(spec)
mod_alis = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod_alis)
globals()[str(x)]= mod_alis
Another "workaround" is to use IPC/RPC and run isolated packages in services. If the dependencies are on the different libraries, maybe can separate by the usage of the packages.
In my application I would like to use:
packageA, which requirespackageX==1.3packageB, which requirespackageX==1.4packageX==1.5
How can I install multiple versions of packageX with pip to handle this situation?
pip won’t help you with this.
You can tell it to install a specific version, but it will override the other one. On the other hand, using two virtualenvs will let you install both versions on the same machine, but not use them at the same time.
You best bet is to install both version manually, by putting them in your Python path with a different name.
But if your two libs expect them to have the same name (and they should), you will have to modify them so they pick up the version they need with some import alias such as:
import dependencyname_version as dependencyname
There is currently no clean way to do this. The best you can hope is for this hack to work.
I’d rather ditch one of the two libs and replace it with an equivalent, or patch it to accept the new version of the dependency and give the patch back to the community.
Download the source for ea. package. Install each on its own separate folder. For example. I had version 1.10 package, but wanted to switch to the dev version for some work. I downloaded the source for the dev module:
git clone https://github.com/networkx/networkx.git
cd netwokrx
I created a folder for this version:
mkdir /home/username/opt/python, then I set the PYTHONPATH env var to: export PYTHONPATH=/home/username/opt/python/lib/python2.7/site-packages/. Next, I installed it using: python setup.py install --prefix=/home/username/opt/python
Now, since my PYTHONPATH is now pointing to this other site-packages folder, when I run python on the command line, and import the new module, it works. To switch switch back, remove the new folder from PYTHONPATH.
>>> import networkx as nx
>>> nx.__version__
'2.0.dev_20151209221101'
an ugly workaround I use with python in blender is I’ll install (and keep off path) a like version of python and use subprocess to have the other version do the needed work. Blenders python tends to get a little temperamental if you do much more than install pandas and scipy. I’ve tried this using virtualenvs with blender but that tends to break things.
Also on the off chance you are using blender for data visualization, you are going to want to add a config folder to your version number folder, this will keep all of your addons in that folder and it makes it far more portable and far less likely to mess up other installs of blender. Many people who make addons for blender are not ‘programmers’, so often those savvy people will do some very hackish things and this has been the best workaround I’ve been able to use.
Another workaround (and this has so many flags on the play that it should disqualify me from touching a keyboard) is to manually locate the init file and manually add it to globals with importlib … this comes with risks. Some modules will play alright when you do this, other modules will toss crapfits that can lead to extra special troubleshooting sessions. Keep it to like versions and it does cut down on the issues and I’ve had ‘alright’ luck with using this to import modulus from behind virtual envs, but there is a reason why I use subprocess calls when working with blender’s python.
def importfromfilelocation(x,y,z):
#"""x = 'tk',y = "tkinter", z =r'C:pyth"""
mod_alis = x
spec = importlib.util.spec_from_file_location(y, z)
print(spec)
mod_alis = importlib.util.module_from_spec(spec)
spec.loader.exec_module(mod_alis)
globals()[str(x)]= mod_alis
Another "workaround" is to use IPC/RPC and run isolated packages in services. If the dependencies are on the different libraries, maybe can separate by the usage of the packages.