How can I Install a Python module within code?
Question:
I need to install a package from PyPI straight within my script.
Is there maybe some module or distutils
(distribute
, pip
, etc.) feature which allows me to just execute something like pypi.install('requests')
and requests will be installed into my virtualenv?
Answers:
You define the dependent module inside the setup.py of your own package with the “install_requires” option.
If your package needs to have some console script generated then you can use the “console_scripts” entry point in order to generate a wrapper script that will be placed
within the ‘bin’ folder (e.g. of your virtualenv environment).
This should work:
import subprocess
def install(name):
subprocess.call(['pip', 'install', name])
You can also use something like:
import pip
def install(package):
if hasattr(pip, 'main'):
pip.main(['install', package])
else:
pip._internal.main(['install', package])
# Example
if __name__ == '__main__':
install('argh')
If you want to use pip
to install required package and import it after installation, you can use this code:
def install_and_import(package):
import importlib
try:
importlib.import_module(package)
except ImportError:
import pip
pip.main(['install', package])
finally:
globals()[package] = importlib.import_module(package)
install_and_import('transliterate')
If you installed a package as a user you can encounter the problem that you cannot just import the package. See How to refresh sys.path? for additional information.
The officially recommended way to install packages from a script is by calling pip’s command-line interface via a subprocess. Most other answers presented here are not supported by pip. Furthermore since pip v10, all code has been moved to pip._internal
precisely in order to make it clear to users that programmatic use of pip is not allowed.
Use sys.executable
to ensure that you will call the same pip
associated with the current runtime.
import subprocess
import sys
def install(package):
subprocess.check_call([sys.executable, "-m", "pip", "install", package])
I added some exception handling to Aaron’s answer.
import subprocess
import sys
try:
import pandas as pd
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", 'pandas'])
finally:
import pandas as pd
For installing multiple packages, I am using a setup.py
file with the following code:
import sys
import subprocess
import pkg_resources
required = {'numpy', 'pandas', '<etc>'}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing = required - installed
if missing:
# implement pip as a subprocess:
subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])
Use:
import os
os.system('pip install requests')
I tried the above for a temporary solution instead of changing a Docker file.
Try the below. So far, it was the best that worked for me.
Install the 4 ones first and then mention the new ones in the "REQUIRED" list:
import pkg_resources
import subprocess
import sys
import os
REQUIRED = {
'spacy', 'scikit-learn', 'numpy', 'pandas', 'torch',
'pyfunctional', 'textblob', 'seaborn', 'matplotlib'
}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing = REQUIRED - installed
if missing:
python = sys.executable
subprocess.check_call([python, '-m', 'pip', 'install', *missing], stdout=subprocess.DEVNULL)
If you want a more efficient answer that expands on subprocess.check_call
. You can first check if the requirement has already been met using pkg_resources
.
This works for different requirement specifiers which is nice. E.g., >=
, ==
:
import sys
import subprocess
import pkg_resources
from pkg_resources import DistributionNotFound, VersionConflict
def should_install_requirement(requirement):
should_install = False
try:
pkg_resources.require(requirement)
except (DistributionNotFound, VersionConflict):
should_install = True
return should_install
def install_packages(requirement_list):
try:
requirements = [
requirement
for requirement in requirement_list
if should_install_requirement(requirement)
]
if len(requirements) > 0:
subprocess.check_call([sys.executable, "-m", "pip", "install", *requirements])
else:
print("Requirements already satisfied.")
except Exception as e:
print(e)
Example usage:
requirement_list = ['requests', 'httpx==0.18.2']
install_packages(requirement_list)
Relevant information: Stack Overflow question: 58612272
To conditionally install multiple packages with exact version, I’ve been using this pattern basing on @Tanmay Shrivastava’s answer:
import sys
from subprocess import run, PIPE, STDOUT
import pkg_resources
def run_cmd(cmd):
ps = run(cmd, stdout=PIPE, stderr=STDOUT, shell=True, text=True)
print(ps.stdout)
# packages to be conditionally installed with exact version
required = {"click==8.0.1", "semver==3.0.0.dev2"}
installed = {f"{pkg.key}=={pkg.version}" for pkg in pkg_resources.working_set}
missing = required - installed
if missing:
run_cmd(f'pip install --ignore-installed {" ".join([*missing])}')
import pip
try:
import imaplib
import email
import pandas as pd
# for hiding password
from pathlib import Path
from dotenv import load_dotenv
import os
import requests
#
from collections import defaultdict
from itertools import permutations,combinations
except Exception as e:
print(e)
e = str(e).split(' ')[-1].replace("'","")
pip.main(['install', e])
I need to install a package from PyPI straight within my script.
Is there maybe some module or distutils
(distribute
, pip
, etc.) feature which allows me to just execute something like pypi.install('requests')
and requests will be installed into my virtualenv?
You define the dependent module inside the setup.py of your own package with the “install_requires” option.
If your package needs to have some console script generated then you can use the “console_scripts” entry point in order to generate a wrapper script that will be placed
within the ‘bin’ folder (e.g. of your virtualenv environment).
This should work:
import subprocess
def install(name):
subprocess.call(['pip', 'install', name])
You can also use something like:
import pip
def install(package):
if hasattr(pip, 'main'):
pip.main(['install', package])
else:
pip._internal.main(['install', package])
# Example
if __name__ == '__main__':
install('argh')
If you want to use pip
to install required package and import it after installation, you can use this code:
def install_and_import(package):
import importlib
try:
importlib.import_module(package)
except ImportError:
import pip
pip.main(['install', package])
finally:
globals()[package] = importlib.import_module(package)
install_and_import('transliterate')
If you installed a package as a user you can encounter the problem that you cannot just import the package. See How to refresh sys.path? for additional information.
The officially recommended way to install packages from a script is by calling pip’s command-line interface via a subprocess. Most other answers presented here are not supported by pip. Furthermore since pip v10, all code has been moved to pip._internal
precisely in order to make it clear to users that programmatic use of pip is not allowed.
Use sys.executable
to ensure that you will call the same pip
associated with the current runtime.
import subprocess
import sys
def install(package):
subprocess.check_call([sys.executable, "-m", "pip", "install", package])
I added some exception handling to Aaron’s answer.
import subprocess
import sys
try:
import pandas as pd
except ImportError:
subprocess.check_call([sys.executable, "-m", "pip", "install", 'pandas'])
finally:
import pandas as pd
For installing multiple packages, I am using a setup.py
file with the following code:
import sys
import subprocess
import pkg_resources
required = {'numpy', 'pandas', '<etc>'}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing = required - installed
if missing:
# implement pip as a subprocess:
subprocess.check_call([sys.executable, '-m', 'pip', 'install', *missing])
Use:
import os
os.system('pip install requests')
I tried the above for a temporary solution instead of changing a Docker file.
Try the below. So far, it was the best that worked for me.
Install the 4 ones first and then mention the new ones in the "REQUIRED" list:
import pkg_resources
import subprocess
import sys
import os
REQUIRED = {
'spacy', 'scikit-learn', 'numpy', 'pandas', 'torch',
'pyfunctional', 'textblob', 'seaborn', 'matplotlib'
}
installed = {pkg.key for pkg in pkg_resources.working_set}
missing = REQUIRED - installed
if missing:
python = sys.executable
subprocess.check_call([python, '-m', 'pip', 'install', *missing], stdout=subprocess.DEVNULL)
If you want a more efficient answer that expands on subprocess.check_call
. You can first check if the requirement has already been met using pkg_resources
.
This works for different requirement specifiers which is nice. E.g., >=
, ==
:
import sys
import subprocess
import pkg_resources
from pkg_resources import DistributionNotFound, VersionConflict
def should_install_requirement(requirement):
should_install = False
try:
pkg_resources.require(requirement)
except (DistributionNotFound, VersionConflict):
should_install = True
return should_install
def install_packages(requirement_list):
try:
requirements = [
requirement
for requirement in requirement_list
if should_install_requirement(requirement)
]
if len(requirements) > 0:
subprocess.check_call([sys.executable, "-m", "pip", "install", *requirements])
else:
print("Requirements already satisfied.")
except Exception as e:
print(e)
Example usage:
requirement_list = ['requests', 'httpx==0.18.2']
install_packages(requirement_list)
Relevant information: Stack Overflow question: 58612272
To conditionally install multiple packages with exact version, I’ve been using this pattern basing on @Tanmay Shrivastava’s answer:
import sys
from subprocess import run, PIPE, STDOUT
import pkg_resources
def run_cmd(cmd):
ps = run(cmd, stdout=PIPE, stderr=STDOUT, shell=True, text=True)
print(ps.stdout)
# packages to be conditionally installed with exact version
required = {"click==8.0.1", "semver==3.0.0.dev2"}
installed = {f"{pkg.key}=={pkg.version}" for pkg in pkg_resources.working_set}
missing = required - installed
if missing:
run_cmd(f'pip install --ignore-installed {" ".join([*missing])}')
import pip
try:
import imaplib
import email
import pandas as pd
# for hiding password
from pathlib import Path
from dotenv import load_dotenv
import os
import requests
#
from collections import defaultdict
from itertools import permutations,combinations
except Exception as e:
print(e)
e = str(e).split(' ')[-1].replace("'","")
pip.main(['install', e])