Python dynamic import methods from file

Question

I have multiple files with a structure like a file example.py:

    def initialize(context):
        pass


    def daj_omacku_teplu(context, data):
        pass


    def hmataj_pomaly(context, data):
        pass


    def chvatni_paku(context, data):
        pass


    def mikaj_laktom(context, data):
        pass

and I need to be able to dynamically import methods from “example.py” in a different python file like:

    for fn in os.listdir('.'):
       if os.path.isfile(fn):
           from fn import mikaj_laktom
           mikaj_laktom(example_context, sample_data)

For multiple reasons, I can not change the structure of example.py so I need to make a mechanism to load methods and evaluate them. I tried to use importlib but it can only import a class, not file with only methods defined.
Thanks for the help.

Asked By: ememem

||

Source

Answer 1

Python import does not support importing using paths, so you will need to have the files accessible as modules, see (sys.path). Assuming for now that your sources are located in the same folder as the main script, I would use the following (or similar):

import sys

def load_module(module):

    # module_path = "mypackage.%s" % module
    module_path = module

    if module_path in sys.modules:
        return sys.modules[module_path]

    return __import__(module_path, fromlist=[module])

# Main script here... Could be your for loop or anything else
# `m` is a reference to the imported module that contains the functions
m = load_module("example")
m.mikaj_laktom(None, [])

The source files can also be part of another package, in which case you will need an __init__.py in the same folder with the .py files (see packages) and you import with "mypackage.module" notation. (Note that the top level folder should be in your path, in the above example this is the folder containing "mypackage")

UDPATE:

As pointed out by @skyking there are lib that can help you do the same thing. See this post
My comment on __init__.py is outdate since things have changed in py3. See this post for some more detailed explanation

Answered By: urban

Answer 2

You were on the right track with importlib. It can be used to load modules by name, however I do not think you can load them into the global namespace in this way (as in from module import function). So you need to load them as module objects and call your required method:

import glob, importlib, os, pathlib, sys

# The directory containing your modules needs to be on the search path.
MODULE_DIR = '/path/to/modules'
sys.path.append(MODULE_DIR)

# Get the stem names (file name, without directory and '.py') of any 
# python files in your directory, load each module by name and run
# the required function.
py_files = glob.glob(os.path.join(MODULE_DIR, '*.py'))

for py_file in py_files:
    module_name = pathlib.Path(py_file).stem
    module = importlib.import_module(module_name)
    module.mikaj_laktom()

Also, be careful using '.' as your MODULE_DIR, as this will presumably try to load the current python file as well, which might cause some unexpected behaviour.

Edit: if using Python2, you won’t have pathlib in the standard library, so use

module_name = os.path.splitext(os.path.split(py_file)[1])[0]

to get the equivalent of Path.stem.

Answered By: Simon Bowly

Python dynamic import methods from file

Question:

Answers: