pip with embedded python
Question:
I installed embedded python from here, titled “Windows x86-64 embeddable zip file”, but it does not have pip installed, it does not have site-packages either, when I try to do python get-pip.py it failed to run because this file has import pip in it. So how can I install pip within a embedded python environment.
Traceback (most recent call last):
File “.getpip.py”, line 20061, in
main()
File “.getpip.py”, line 194, in main
bootstrap(tmpdir=tmpdir)
File “.getpip.py”, line 82, in bootstrap
import pip
File “”, line 961, in _find_and_load
File “”, line 950, in _find_and_load_unlocked
File “”, line 646, in _load_unlocked
File “”, line 616, in _load_backward_compatible
The directory structure is:
Directory: C:Downloadspython-3.6.1rc1-embed-win32
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 3/4/2017 7:26 PM 157344 pyexpat.pyd
-a---- 3/4/2017 7:26 PM 97952 python.exe
-a---- 3/4/2017 7:26 PM 58016 python3.dll
-a---- 3/4/2017 7:26 PM 3263648 python36.dll
-a---- 3/4/2017 7:26 PM 2209284 python36.zip
-a---- 3/4/2017 7:26 PM 79 python36._pth
-a---- 3/4/2017 7:26 PM 96416 pythonw.exe
-a---- 3/4/2017 7:26 PM 23200 select.pyd
-a---- 3/4/2017 7:26 PM 866464 sqlite3.dll
-a---- 3/4/2017 7:26 PM 895648 unicodedata.pyd
-a---- 3/4/2017 7:26 PM 83784 vcruntime140.dll
-a---- 3/4/2017 7:26 PM 24224 winsound.pyd
-a---- 3/4/2017 7:26 PM 45216 _asyncio.pyd
-a---- 3/4/2017 7:26 PM 77984 _bz2.pyd
-a---- 3/4/2017 7:26 PM 101536 _ctypes.pyd
-a---- 3/4/2017 7:26 PM 215712 _decimal.pyd
-a---- 3/4/2017 7:26 PM 156832 _elementtree.pyd
-a---- 3/4/2017 7:26 PM 1042592 _hashlib.pyd
-a---- 3/4/2017 7:26 PM 183456 _lzma.pyd
-a---- 3/4/2017 7:26 PM 32416 _msi.pyd
-a---- 3/4/2017 7:26 PM 25760 _multiprocessing.pyd
-a---- 3/4/2017 7:26 PM 33952 _overlapped.pyd
-a---- 3/4/2017 7:26 PM 61600 _socket.pyd
-a---- 3/4/2017 7:26 PM 64160 _sqlite3.pyd
-a---- 3/4/2017 7:26 PM 1458848 _ssl.pyd
Answers:
I recently ran into the same issue. I checked the documentation for pip and they seem to say that this use case isn’t supported etc. But anyhow, here is my hack for getting the modules to work.
I installed, and by that I mean unzipped embedded python into a directory called d:python. I assumed that the modules are going to live in that same directory.
First, to install the pip module, I needed to save the extraceted files. I changed the get-pip.py using a text editor that supported unix line terminators by removing the rmtree lines that remove the temporary and unpacked tree from the blob containted in the get-pip.py file.
I changed both locations, but only the last one was needed. The line that I changed in two locations read
shutil.rmtree(tmpdir, ignore_errors=True)
and I modified it thus (I didn’t want to bother with the python indentation blocks):
print('shutil.rmtree(tmpdir, ignore_errors=True)')
I now ran my python D:pythonpython.exe on the modified get-pip.py and found the temporary directory where the files were unzipped to.
I copied this directory, (check that it contains a main.py) file into the python install D:pythonpip (this is where I wanted my modules to live), ensuring that the D:pythonpip directory contained the main.py file.
The pip module is now installed in the python directory, but you need to hack pip further to remove the exception above. I did this by changing the locations.py file (in my case located in D:pythonpiplocations.py) to return the bin_py and bin_user locations of D:python.
ie:
86 if WINDOWS:
....
bin_py = 'd:/python'
bin_user = 'd:/python'
I had to change the user_dir for pip to somewhere that would persist on this drive that was shared across multiple VMs.
The pip module now runs fine, eg
d:pythonpython.exe -m pip …
Even if explicitly stated that the embeddable version of Python does not
support Pip, it is possible with care. You need to:
-
Download and unzip Python embeddable zip file.
-
In the file python39._pth or similar, uncomment the import command.
Result should look similar to this:
python39.zip
.
import site
-
Download get-pip.py to the Python install folder
-
Run get-pip.py. this installs Pip into the Scripts directory:
python get-pip.py
-
Run Pip directly from command line as Pip is a executable program (this
example is to install Pandas):
.Scriptspip install pandas
You could find more information about this in the Pip issue 4207
Pip can also be accessed programatically within python as a command (this example installs pandas):
import pip
pip.main(['install'], 'pandas')
(Source: Installing python module within code)
how I installed pip into an embeddable python distribution:
- unpack the python embeddable distro archive and cd there
- run
python ./get-pip.py (got here)
- EITHER just uncomment string
import site inside file pythonXX._pth OR
copy folders (at least pip) from the newly-created folder Lib/site-packages/ into pythonXX.zip
now it’s possible to python -m pip ...
Solution for me was deleting this file:
python39._pth
This allows Pip to work, and also allow import from same directory.
Alternatively you can get this:
https://nuget.org/packages/python
Click "Download package", and you can extract just like a Zip file.
download https://www.python.org/ftp/python/3.10.1/python-3.10.1-amd64.exe
and install it to a Virtual box
download https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip
and unpack it to python-3.10.1-embed-amd64
copy "include" & "libs" dirs from corresponding python installation "python-3.10.1-amd64.exe" from Virtual Box to python-3.10.1-embed-amd64
goto python-3.10.1-embed-amd64 folder and:
- Download get-pip.py:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
- Execute get-pip.py:
python get-pip.py
- rename _pth:
rename python310._pth python310._pth.renamed
set environments:
set PY_PIP=python-3.10.1-embed-amd64Scripts
set PY_LIBS=python-3.10.1-embed-amd64Lib;python-3.10.1-embed-amd64Libsite-packages
now "python -m pip install PACKETNAME" should work.
PACKETNAME is tested on:
- pywin32==302
- psutil==5.8.0
- pprofile==2.1.0
- mypy==0.910
- Pillow==8.4.0
- cfractions==1.4.0
- pyyaml==6.0
- humanize==3.13.1
Note: packets installs that require to compile C/C++ are also work.
Note: ‘python-3.10.1-embed-amd64’ means something like c:mypythonspython-3.10.1-embed-amd64
I installed embedded python from here, titled “Windows x86-64 embeddable zip file”, but it does not have pip installed, it does not have site-packages either, when I try to do python get-pip.py it failed to run because this file has import pip in it. So how can I install pip within a embedded python environment.
Traceback (most recent call last):
File “.getpip.py”, line 20061, in
main()
File “.getpip.py”, line 194, in main
bootstrap(tmpdir=tmpdir)
File “.getpip.py”, line 82, in bootstrap
import pip
File “”, line 961, in _find_and_load
File “”, line 950, in _find_and_load_unlocked
File “”, line 646, in _load_unlocked
File “”, line 616, in _load_backward_compatible
The directory structure is:
Directory: C:Downloadspython-3.6.1rc1-embed-win32
Mode LastWriteTime Length Name
---- ------------- ------ ----
-a---- 3/4/2017 7:26 PM 157344 pyexpat.pyd
-a---- 3/4/2017 7:26 PM 97952 python.exe
-a---- 3/4/2017 7:26 PM 58016 python3.dll
-a---- 3/4/2017 7:26 PM 3263648 python36.dll
-a---- 3/4/2017 7:26 PM 2209284 python36.zip
-a---- 3/4/2017 7:26 PM 79 python36._pth
-a---- 3/4/2017 7:26 PM 96416 pythonw.exe
-a---- 3/4/2017 7:26 PM 23200 select.pyd
-a---- 3/4/2017 7:26 PM 866464 sqlite3.dll
-a---- 3/4/2017 7:26 PM 895648 unicodedata.pyd
-a---- 3/4/2017 7:26 PM 83784 vcruntime140.dll
-a---- 3/4/2017 7:26 PM 24224 winsound.pyd
-a---- 3/4/2017 7:26 PM 45216 _asyncio.pyd
-a---- 3/4/2017 7:26 PM 77984 _bz2.pyd
-a---- 3/4/2017 7:26 PM 101536 _ctypes.pyd
-a---- 3/4/2017 7:26 PM 215712 _decimal.pyd
-a---- 3/4/2017 7:26 PM 156832 _elementtree.pyd
-a---- 3/4/2017 7:26 PM 1042592 _hashlib.pyd
-a---- 3/4/2017 7:26 PM 183456 _lzma.pyd
-a---- 3/4/2017 7:26 PM 32416 _msi.pyd
-a---- 3/4/2017 7:26 PM 25760 _multiprocessing.pyd
-a---- 3/4/2017 7:26 PM 33952 _overlapped.pyd
-a---- 3/4/2017 7:26 PM 61600 _socket.pyd
-a---- 3/4/2017 7:26 PM 64160 _sqlite3.pyd
-a---- 3/4/2017 7:26 PM 1458848 _ssl.pyd
I recently ran into the same issue. I checked the documentation for pip and they seem to say that this use case isn’t supported etc. But anyhow, here is my hack for getting the modules to work.
I installed, and by that I mean unzipped embedded python into a directory called d:python. I assumed that the modules are going to live in that same directory.
First, to install the pip module, I needed to save the extraceted files. I changed the get-pip.py using a text editor that supported unix line terminators by removing the rmtree lines that remove the temporary and unpacked tree from the blob containted in the get-pip.py file.
I changed both locations, but only the last one was needed. The line that I changed in two locations read
shutil.rmtree(tmpdir, ignore_errors=True)
and I modified it thus (I didn’t want to bother with the python indentation blocks):
print('shutil.rmtree(tmpdir, ignore_errors=True)')
I now ran my python D:pythonpython.exe on the modified get-pip.py and found the temporary directory where the files were unzipped to.
I copied this directory, (check that it contains a main.py) file into the python install D:pythonpip (this is where I wanted my modules to live), ensuring that the D:pythonpip directory contained the main.py file.
The pip module is now installed in the python directory, but you need to hack pip further to remove the exception above. I did this by changing the locations.py file (in my case located in D:pythonpiplocations.py) to return the bin_py and bin_user locations of D:python.
ie:
86 if WINDOWS:
....
bin_py = 'd:/python'
bin_user = 'd:/python'
I had to change the user_dir for pip to somewhere that would persist on this drive that was shared across multiple VMs.
The pip module now runs fine, eg
d:pythonpython.exe -m pip …
Even if explicitly stated that the embeddable version of Python does not
support Pip, it is possible with care. You need to:
-
Download and unzip Python embeddable zip file.
-
In the file
python39._pthor similar, uncomment theimportcommand.
Result should look similar to this:python39.zip . import site -
Download get-pip.py to the Python install folder
-
Run
get-pip.py. this installs Pip into theScriptsdirectory:python get-pip.py -
Run Pip directly from command line as Pip is a executable program (this
example is to install Pandas):.Scriptspip install pandas
You could find more information about this in the Pip issue 4207
Pip can also be accessed programatically within python as a command (this example installs pandas):
import pip
pip.main(['install'], 'pandas')
(Source: Installing python module within code)
how I installed pip into an embeddable python distribution:
- unpack the python embeddable distro archive and cd there
- run
python ./get-pip.py(got here) - EITHER just uncomment string
import siteinside file pythonXX._pth OR
copy folders (at least pip) from the newly-created folder Lib/site-packages/ into pythonXX.zip
now it’s possible to python -m pip ...
Solution for me was deleting this file:
python39._pth
This allows Pip to work, and also allow import from same directory.
Alternatively you can get this:
https://nuget.org/packages/python
Click "Download package", and you can extract just like a Zip file.
download https://www.python.org/ftp/python/3.10.1/python-3.10.1-amd64.exe
and install it to a Virtual box
download https://www.python.org/ftp/python/3.10.1/python-3.10.1-embed-amd64.zip
and unpack it to python-3.10.1-embed-amd64
copy "include" & "libs" dirs from corresponding python installation "python-3.10.1-amd64.exe" from Virtual Box to python-3.10.1-embed-amd64
goto python-3.10.1-embed-amd64 folder and:
- Download get-pip.py:
curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
- Execute get-pip.py:
python get-pip.py
- rename _pth:
rename python310._pth python310._pth.renamed
set environments:
set PY_PIP=python-3.10.1-embed-amd64Scripts
set PY_LIBS=python-3.10.1-embed-amd64Lib;python-3.10.1-embed-amd64Libsite-packages
now "python -m pip install PACKETNAME" should work.
PACKETNAME is tested on:
- pywin32==302
- psutil==5.8.0
- pprofile==2.1.0
- mypy==0.910
- Pillow==8.4.0
- cfractions==1.4.0
- pyyaml==6.0
- humanize==3.13.1
Note: packets installs that require to compile C/C++ are also work.
Note: ‘python-3.10.1-embed-amd64’ means something like c:mypythonspython-3.10.1-embed-amd64