How to use Python's pip to download and keep the zipped files for a package?
Question:
If I want to use the pip command to download a package (and its dependencies), but keep all of the zipped files that get downloaded (say, django-socialregistration.tar.gz) – is there a way to do that?
I’ve tried various command-line options, but it always seems to unpack and delete the zipfile – or it gets the zipfile, but only for the original package, not the dependencies.
Answers:
The --download-cache option should do what you want:
pip install --download-cache="/pth/to/downloaded/files" package
However, when I tested this, the main package downloaded, saved and installed ok, but the the dependencies were saved with their full url path as the name – a bit annoying, but all the tar.gz files were there.
The --download option downloads the main package and its dependencies and does not install any of them. (Note that prior to version 1.1 the --download option did not download dependencies.)
pip install package --download="/pth/to/downloaded/files"
The pip documentation outlines using --download for fast & local installs.
I always do this to download the packages:
pip install --download /path/to/download/to_packagename
OR
pip install --download=/path/to/packages/downloaded -r requirements.txt
And when I want to install all of those libraries I just downloaded, I do this:
pip install --no-index --find-links="/path/to/downloaded/dependencies" packagename
OR
pip install --no-index --find-links="/path/to/downloaded/packages" -r requirements.txt
Update
Also, to get all the packages installed on one system, you can export them all to requirement.txt that will be used to intall them on another system, we do this:
pip freeze > requirement.txt
Then, the requirement.txt can be used as above for download, or do this to install them from requirement.txt:
pip install -r requirement.txt
REFERENCE: pip installer
In version 7.1.2 pip downloads the wheel of a package (if available) with the following:
pip install package -d /path/to/downloaded/file
The following downloads a source distribution:
pip install package -d /path/to/downloaded/file --no-binary :all:
These download the dependencies as well, if pip is aware of them (e.g., if pip show package lists them).
Update
As noted by Anton Khodak, pip download command is preferred since version 8. In the above examples this means that /path/to/downloaded/file needs to be given with option -d, so replacing install with download works.
pip install --download is deprecated. Starting from version 8.0.0 you should use pip download command:
pip download <package-name>
Use pip download <package1 package2 package n> to download all the packages including dependencies
Use pip install --no-index --find-links . <package1 package2 package n> to install all the packages including dependencies.
It gets all the files from CWD.
It will not download anything
pip wheel is another option you should consider:
pip wheel mypackage -w .outputdir
It will download packages and their dependencies to a directory (current working directory by default), but it performs the additional step of converting any source packages to wheels.
It conveniently supports requirements files:
pip wheel -r requirements.txt -w .outputdir
Add the --no-deps argument if you only want the specifically requested packages:
pip wheel mypackage -w .outputdir --no-deps
installing python packages offline
For windows users:
To download into a file
open your cmd and folow this:
cd <*the file-path where you want to save it*>
pip download <*package name*>
the package and the dependencies will be downloaded in the current working directory.
To install from the current working directory:
set your folder where you downloaded as the cwd then follow these:
pip install <*the package name which is downloded as .whl*> --no-index --find-links <*the file locaation where the files are downloaded*>
this will search for dependencies in that location.
I would prefer (RHEL) – pip download package==version --no-deps --no-binary=:all:
All the answers mentioned in this thread assume that the packages will be downloaded on the same OS configuration as the target OS where it has to be installed.
In my personal experience i was using windows as my work machine and had to download packages for linux environment and have seen people doing vice versa as well. I had done some extensive googling, and found sodim.dev.
All i had to do was upload requirements.txt file and select the environment configuration like OS and python version and it gives out a csv with download url, source code url etc
I guess in the backend this app spins up the OS VM as requested and installs that particular python version and then generates the report, because it does take about 15-20 minutes for 30-50 packages.
P.S.: I work in an offline environment, where security is of very high concern, and downloading packages are not that frequent. We whitelist source code and download urls for each individual requests and then after running some appsec tools, we approve/reject the source code to be downloaded.
If I want to use the pip command to download a package (and its dependencies), but keep all of the zipped files that get downloaded (say, django-socialregistration.tar.gz) – is there a way to do that?
I’ve tried various command-line options, but it always seems to unpack and delete the zipfile – or it gets the zipfile, but only for the original package, not the dependencies.
The --download-cache option should do what you want:
pip install --download-cache="/pth/to/downloaded/files" package
However, when I tested this, the main package downloaded, saved and installed ok, but the the dependencies were saved with their full url path as the name – a bit annoying, but all the tar.gz files were there.
The --download option downloads the main package and its dependencies and does not install any of them. (Note that prior to version 1.1 the --download option did not download dependencies.)
pip install package --download="/pth/to/downloaded/files"
The pip documentation outlines using --download for fast & local installs.
I always do this to download the packages:
pip install --download /path/to/download/to_packagename
OR
pip install --download=/path/to/packages/downloaded -r requirements.txt
And when I want to install all of those libraries I just downloaded, I do this:
pip install --no-index --find-links="/path/to/downloaded/dependencies" packagename
OR
pip install --no-index --find-links="/path/to/downloaded/packages" -r requirements.txt
Update
Also, to get all the packages installed on one system, you can export them all to requirement.txt that will be used to intall them on another system, we do this:
pip freeze > requirement.txt
Then, the requirement.txt can be used as above for download, or do this to install them from requirement.txt:
pip install -r requirement.txt
REFERENCE: pip installer
In version 7.1.2 pip downloads the wheel of a package (if available) with the following:
pip install package -d /path/to/downloaded/file
The following downloads a source distribution:
pip install package -d /path/to/downloaded/file --no-binary :all:
These download the dependencies as well, if pip is aware of them (e.g., if pip show package lists them).
Update
As noted by Anton Khodak, pip download command is preferred since version 8. In the above examples this means that /path/to/downloaded/file needs to be given with option -d, so replacing install with download works.
pip install --download is deprecated. Starting from version 8.0.0 you should use pip download command:
pip download <package-name>
Use pip download <package1 package2 package n> to download all the packages including dependencies
Use pip install --no-index --find-links . <package1 package2 package n> to install all the packages including dependencies.
It gets all the files from CWD.
It will not download anything
pip wheel is another option you should consider:
pip wheel mypackage -w .outputdir
It will download packages and their dependencies to a directory (current working directory by default), but it performs the additional step of converting any source packages to wheels.
It conveniently supports requirements files:
pip wheel -r requirements.txt -w .outputdir
Add the --no-deps argument if you only want the specifically requested packages:
pip wheel mypackage -w .outputdir --no-deps
installing python packages offline
For windows users:
To download into a file
open your cmd and folow this:
cd <*the file-path where you want to save it*>
pip download <*package name*>
the package and the dependencies will be downloaded in the current working directory.
To install from the current working directory:
set your folder where you downloaded as the cwd then follow these:
pip install <*the package name which is downloded as .whl*> --no-index --find-links <*the file locaation where the files are downloaded*>
this will search for dependencies in that location.
I would prefer (RHEL) – pip download package==version --no-deps --no-binary=:all:
All the answers mentioned in this thread assume that the packages will be downloaded on the same OS configuration as the target OS where it has to be installed.
In my personal experience i was using windows as my work machine and had to download packages for linux environment and have seen people doing vice versa as well. I had done some extensive googling, and found sodim.dev.
All i had to do was upload requirements.txt file and select the environment configuration like OS and python version and it gives out a csv with download url, source code url etc
I guess in the backend this app spins up the OS VM as requested and installs that particular python version and then generates the report, because it does take about 15-20 minutes for 30-50 packages.
P.S.: I work in an offline environment, where security is of very high concern, and downloading packages are not that frequent. We whitelist source code and download urls for each individual requests and then after running some appsec tools, we approve/reject the source code to be downloaded.