Dynamically importing Python module

Question

I have a trusted remote server that stores many custom Python modules. I can fetch them via HTTP (e.g. using urllib2.urlopen) as text/plain, but I cannot save the fetched module code to the local hard disk. How can I import the code as a fully operable Python module, including its global variables and imports?
I suppose I have to use some combination of exec and imp module’s functions, but I’ve been unable to make it work yet.

Asked By: dpq

||

Source

Answer 1

It looks like this should do the trick: importing a dynamically generated module

>>> import imp
>>> foo = imp.new_module("foo")
>>> foo_code = """
... class Foo:
...     pass
... """
>>> exec foo_code in foo.__dict__
>>> foo.Foo.__module__
'foo'
>>>

Also, as suggested in the ActiveState article, you might want to add your new module to sys.modules:

>>> import sys
>>> sys.modules["foo"] = foo
>>> from foo import Foo
<class 'Foo' …>
>>>

Answered By: David Wolever

Answer 2

Here’s something I bookmarked a while back that covers something similar:

Customizing the Python Import System

It’s a bit beyond what you want, but the basic idea is there.

Answered By: ars

Answer 3

Python3 version
(attempted to edit other answer but the edit que is full)

import imp

my_dynamic_module = imp.new_module("my_dynamic_module")
exec("""
class Foo:
    pass
""", my_dynamic_module.__dict__)

Foo = my_dynamic_module.Foo
foo_object = Foo()

# register it on sys
import sys
sys.modules[my_dynamic_module.__name__] = my_dynamic_module

Answered By: Jeff Hykin

Answer 4

I recently encountered trying to do this while trying to write unit tests for source code examples I put into a project’s readme (I wanted to avoid just linking to small files or duplicating the text in a way that could get out of sync).

I came up with the following

import sys
import types
from importlib import import_module


def compile_and_install_module(module_name: str, source_code: str) -> types.ModuleType:
    """Compile source code and install it as a module.

    End result is that `import <module_name>` and `from <module_name> import ...` should work.
    """
    module = types.ModuleType(module_name, "Module created from source code")

    # Execute source in context of empty/fake module
    exec(source_code, module.__dict__)

    # Insert fake module into sys.modules. It's now a real module
    sys.modules[module_name] = module

    # Imports should work now
    return import_module(module_name)

And a quick example of how you can use it

$ cat hello.py 
def foo():
    print("Hello world")


bar = 42

$ python
Python 3.9.5 (tags/v3.9.5:0a7dcbd, May  3 2021, 17:27:52) [MSC v.1928 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> from compile import compile_and_install_module
>>> compile_and_install_module("hello", open("hello.py").read())
<module 'hello'>
>>> import hello
>>> hello.foo()
Hello world
>>> from hello import bar
>>> bar
42

You can remove the return value and import_lib import if you

Answered By: Jordan Cottle

Dynamically importing Python module

Question:

Answers: