Can I install Python windows packages into virtualenvs?
Question:
Virtualenv is great: it lets me keep a number of distinct Python installations so that different projects’ dependencies aren’t all thrown together into a common pile.
But if I want to install a package on Windows that’s packaged as a .exe installer, how can I direct it to install into the virtualenv? For example, I have pycuda-0.94rc.win32-py2.6.exe. When I run it, it examines the registry, and finds only one Python26 to install into, the common one that my virtualenv is based off of.
How can I direct it to install into the virtualenv?
Answers:
You can use environment’s easy_install to install PyCUDA.
dev-env-path/bin/easy_install pycuda
it will give you the same version 0.94rc.
On Windows easy_install.exe will be in Scripts directory.
If it’s a .msi, you might be able to specify command line options using msiexec. The Python installer itself allows TARGETDIR, but I’m not sure if distutils bakes this into distribution installers.
If you’re using a .exe, I don’t think there’s a clean way. One option is to use a program like 7Zip (or winzip, etc) to directly extract the contents of the exe, then copy the relevent folders into your virtual site-packages folder. For example, if I extract “processing-0.5.2.win32-py2.5.exe”, I find a folder “PLATLIBprocessing” which I copy to a virtualenv path and use without any runtime problems. (I’m not sure it’s always that simple though.)
I ended up adapting a script (http://effbot.org/zone/python-register.htm) to register a Python installation in the registry. I can pick the Python to be the Python in the registry, run the Windows installer, then set the registry back:
# -*- encoding: utf-8 -*-
#
# script to register Python 2.0 or later for use with win32all
# and other extensions that require Python registry settings
#
# Adapted by Ned Batchelder from a script
# written by Joakim Löw for Secret Labs AB / PythonWare
#
# source:
# http://www.pythonware.com/products/works/articles/regpy20.htm
import sys
from _winreg import *
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
regpath = "SOFTWARE\Python\Pythoncore\%s\" % (version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "%s;%s\Lib\;%s\DLLs\" % (
installpath, installpath, installpath
)
def RegisterPy():
try:
reg = OpenKey(HKEY_LOCAL_MACHINE, regpath)
except EnvironmentError:
try:
reg = CreateKey(HKEY_LOCAL_MACHINE, regpath)
except Exception, e:
print "*** Unable to register: %s" % e
return
SetValue(reg, installkey, REG_SZ, installpath)
SetValue(reg, pythonkey, REG_SZ, pythonpath)
CloseKey(reg)
print "--- Python %s at %s is now registered!" % (version, installpath)
if __name__ == "__main__":
RegisterPy()
Run this script with the Python you want to be registered, and it will be entered into the registry. Note that on Windows 7 and Vista, you’ll need Administrator privileges.
easy_install is able to install .exe packages as long as they were built using distutils’ bdist_wininst target, which covers many popular packages. However, there are many others that aren’t (wxPython is one that I’ve struggled with)
Yes, you can. All you need is
easy_install
binary_installer_built_with_distutils.exe
Surprised? It looks like binary installers for Windows made with distutils combine .exe with .zip into one .exe file. Change extension to .zip to see it’s a valid zip file. I discovered this after reading answers to my question Where can I download binary eggs with psycopg2 for Windows?
UPDATE
As noted by Tritium21 in his answer nowadays you should use pip instead of easy_install. Pip can’t install binary packages created by distutils but it can install binary packages in the new wheel format. You can convert from old format to the new one using wheel package, which you have to install first.
I know this is quite an old question, and predates the tools I am about to talk about, but for the sake of Google, I think it is a good idea to mention it. easy_install is the black sheep of python packaging. No one wants to admit using it with the new hotness of pip around. Also, while playing registry tricks will work best for non-standard EXE installers (someone built the installer themselves instead of using distutils, and is checking the registry for the installation path), there is now a Better Way(c) for standard EXE installers.
pip install wheel
wheel convert INSTALLER.EXE
pip install NEW_FILE_CREATED_IN_LAST_STEP.whl
The wheel format, introduced recently as of this post, is the replacement for the egg format, filling much the same role. This format is also supported by pip (a tool already installed in your virtualenv).
if for some reason pip install WHEELFILE does not work, try wheel install WHEELFILE
You should type path of your file and write ‘python ‘ before it.
Than it will run your python script without any virtual environment.
Thanks.
Virtualenv is great: it lets me keep a number of distinct Python installations so that different projects’ dependencies aren’t all thrown together into a common pile.
But if I want to install a package on Windows that’s packaged as a .exe installer, how can I direct it to install into the virtualenv? For example, I have pycuda-0.94rc.win32-py2.6.exe. When I run it, it examines the registry, and finds only one Python26 to install into, the common one that my virtualenv is based off of.
How can I direct it to install into the virtualenv?
You can use environment’s easy_install to install PyCUDA.
dev-env-path/bin/easy_install pycuda
it will give you the same version 0.94rc.
On Windows easy_install.exe will be in Scripts directory.
If it’s a .msi, you might be able to specify command line options using msiexec. The Python installer itself allows TARGETDIR, but I’m not sure if distutils bakes this into distribution installers.
If you’re using a .exe, I don’t think there’s a clean way. One option is to use a program like 7Zip (or winzip, etc) to directly extract the contents of the exe, then copy the relevent folders into your virtual site-packages folder. For example, if I extract “processing-0.5.2.win32-py2.5.exe”, I find a folder “PLATLIBprocessing” which I copy to a virtualenv path and use without any runtime problems. (I’m not sure it’s always that simple though.)
I ended up adapting a script (http://effbot.org/zone/python-register.htm) to register a Python installation in the registry. I can pick the Python to be the Python in the registry, run the Windows installer, then set the registry back:
# -*- encoding: utf-8 -*-
#
# script to register Python 2.0 or later for use with win32all
# and other extensions that require Python registry settings
#
# Adapted by Ned Batchelder from a script
# written by Joakim Löw for Secret Labs AB / PythonWare
#
# source:
# http://www.pythonware.com/products/works/articles/regpy20.htm
import sys
from _winreg import *
# tweak as necessary
version = sys.version[:3]
installpath = sys.prefix
regpath = "SOFTWARE\Python\Pythoncore\%s\" % (version)
installkey = "InstallPath"
pythonkey = "PythonPath"
pythonpath = "%s;%s\Lib\;%s\DLLs\" % (
installpath, installpath, installpath
)
def RegisterPy():
try:
reg = OpenKey(HKEY_LOCAL_MACHINE, regpath)
except EnvironmentError:
try:
reg = CreateKey(HKEY_LOCAL_MACHINE, regpath)
except Exception, e:
print "*** Unable to register: %s" % e
return
SetValue(reg, installkey, REG_SZ, installpath)
SetValue(reg, pythonkey, REG_SZ, pythonpath)
CloseKey(reg)
print "--- Python %s at %s is now registered!" % (version, installpath)
if __name__ == "__main__":
RegisterPy()
Run this script with the Python you want to be registered, and it will be entered into the registry. Note that on Windows 7 and Vista, you’ll need Administrator privileges.
easy_install is able to install .exe packages as long as they were built using distutils’ bdist_wininst target, which covers many popular packages. However, there are many others that aren’t (wxPython is one that I’ve struggled with)
Yes, you can. All you need is
easy_install
binary_installer_built_with_distutils.exe
Surprised? It looks like binary installers for Windows made with distutils combine .exe with .zip into one .exe file. Change extension to .zip to see it’s a valid zip file. I discovered this after reading answers to my question Where can I download binary eggs with psycopg2 for Windows?
UPDATE
As noted by Tritium21 in his answer nowadays you should use pip instead of easy_install. Pip can’t install binary packages created by distutils but it can install binary packages in the new wheel format. You can convert from old format to the new one using wheel package, which you have to install first.
I know this is quite an old question, and predates the tools I am about to talk about, but for the sake of Google, I think it is a good idea to mention it. easy_install is the black sheep of python packaging. No one wants to admit using it with the new hotness of pip around. Also, while playing registry tricks will work best for non-standard EXE installers (someone built the installer themselves instead of using distutils, and is checking the registry for the installation path), there is now a Better Way(c) for standard EXE installers.
pip install wheel
wheel convert INSTALLER.EXE
pip install NEW_FILE_CREATED_IN_LAST_STEP.whl
The wheel format, introduced recently as of this post, is the replacement for the egg format, filling much the same role. This format is also supported by pip (a tool already installed in your virtualenv).
if for some reason pip install WHEELFILE does not work, try wheel install WHEELFILE
You should type path of your file and write ‘python ‘ before it.
Than it will run your python script without any virtual environment.
Thanks.