Best practice for lazy loading Python modules

Question

Occasionally I want lazy module loading in Python. Usually because I want to keep runtime requirements or start-up times low and splitting the code into sub-modules would be cumbersome. A typical use case and my currently preferred implementation is this:

jinja2 = None

class Handler(...):
    ...
    def render_with_jinja2(self, values, template_name):
        global jinja2
        if not jinja2:
            import jinja2
        env = jinja2.Environment(...)
        ...

I wonder: is there a canonical/better way to implement lazy module loading?

Asked By: max

||

Source

Answer 1

class Handler(...):
    ...
    def render_with_jinja2(self, values, template_name):
        import jinja2
        env = jinja2.Environment(...)
        ...

There’s no need to cache the imported module; Python does that already.

Answered By: Glenn Maynard

Answer 2

There’s no reason for you to keep track of imports manually — the VM maintains a list of modules that have already been imported, and any subsequent attempts to import that module result in a quick dict lookup in sys.modules and nothing else.

The difference between your code and

def render_with_jinja2(self, values, template_name):
    import jinja2
    env = jinja2.Environment(...)

is zero — when we hit that code, if jinja2 hasn’t been imported, it is imported then. If it already has been, execution continues on.

Answered By: bgporter

Answer 3

The other answers have covered the actual details but if you are interested in a lazy loading library, check out apipkg which is part of the py package (py.test fame).

Answered By: Noufal Ibrahim

Answer 4

Nice pattern from sqlalchemy: dependency injection:

@util.dependencies("sqlalchemy.orm.query")
def merge_result(query, *args):
    #...
    query.Query(...)

Instead of declaring all “import” statements at the top of the module, it will only import a module when it’s actually needed by a function.
This can resolve circular dependency problems.

Answered By: kolypto

Best practice for lazy loading Python modules

Question:

Answers: