How to install a missing python package from inside the script that needs it?

Question

Assuming that you already have pip or easy_install installed on your python distribution, I would like to know how can I installed a required package in the user directory from within the script itself.

From what I know pip is also a python module so the solution should look like:

try:
    import zumba
except ImportError:
    import pip
    # ... do "pip install --user zumba" or throw exception   <-- how?
    import zumba

What I am missing is doing “pip install –user zumba” from inside python, I don’t want to do it using os.system() as this may create other problems.

I assume it is possible…

Asked By: sorin

||

Source

Answer 1

Updated for newer pip version (>= 10.0):

try:
    import zumba
except ImportError:
    from pip._internal import main as pip
    pip(['install', '--user', 'zumba'])
    import zumba

Thanks to @Joop I was able to come-up with the proper answer.

try:
    import zumba
except ImportError:
    import pip
    pip.main(['install', '--user', 'zumba'])
    import zumba

One important remark is that this will work without requiring root access as it will install the module in user directory.

Not sure if it will work for binary modules or ones that would require compilation, but it clearly works well for pure-python modules.

Now you can write self contained scripts and not worry about dependencies.

Answered By: sorin

Answer 2

I wanted to note that the current accepted answer could result in a possible app name collision. Importing from the app namespace doesn’t give you the full picture of what’s installed on the system.

A better way would be:

import pip

packages = [package.project_name for package in pip.get_installed_distributions()]

if 'package' not in packages:
    pip.main(['install', 'package'])

Answered By: Casey

Answer 3

As of pip version >= 10.0.0, the above solutions will not work because of internal package restructuring. The new way to use pip inside a script is now as follows:

try: import abc
except ImportError:
    from pip._internal import main as pip
    pip(['install', '--user', 'abc'])
    import abc

Answered By: Phoenix

Answer 4

A simple function based on the above that takes the module name as a string and returns the imported module.

def pipInstallOnError(module_name):
    '''Import a module.
    Attempt to pip install the module on ImportError.

    :Parameters:
        module_name (str) = The name of the module.

    :Return:
        (obj) The imported module.
    '''
    try:
        return __import__(module_name)
    except ImportError as error:
        from pip._internal import main as pip
        pip(['install', '--user', module_name])
        return __import__(module_name)

Answered By: m3trik

Answer 5

Do not use `pip.main` or `pip._internal.main`.

Quoting directly from the official documentation (boldface emphasis and editing comments mine, italics theirs):

As noted previously, pip is a command line program. While it is… available from your Python code via import pip, you must not use pip’s internal APIs in this way. There are a number of reasons for this:

The pip code assumes that [it] is in sole control of the global state of the program. pip manages things like… without considering the possibility that user code might be affected.

pip’s code is not thread safe. If you were to run pip in a thread, there is no guarantee that either your code or pip’s would work as you expect.

pip assumes that once it has finished its work, the process will terminate… calling pip twice in the same process is likely to have issues.

This does not mean that the pip developers are opposed in principle to the idea that pip could be used as a library – it’s just that this isn’t how it was written, and it would be a lot of work to redesign the internals for use as a library [with a] stable API… And we simply don’t currently have the resources….

…[E]verything inside of pip is considered an implementation detail. Even the fact that the import name is pip is subject to change without notice. While we do try not to break things as much as possible, all the internal APIs can change at any time, for any reason….

…[I]nstalling packages into sys.path in a running Python process is something that should only be done with care. The import system caches certain data, and installing new packages while a program is running may not always behave as expected….

Having said all of the above[:] The most reliable approach, and the one that is fully supported, is to run pip in a subprocess. This is easily done using the standard subprocess module:
subprocess.check_call([sys.executable, '-m', 'pip', 'install', 'my_package'])

It goes on to describe other more appropriate tools and approaches for related problems.

Answered By: Karl Knechtel

How to install a missing python package from inside the script that needs it?

Question:

Answers:

Do not use `pip.main` or `pip._internal.main`.

How to install a missing python package from inside the script that needs it?

Question:

Answers:

Do not use pip.main or pip._internal.main.

Do not use `pip.main` or `pip._internal.main`.