the pythonic way of optional imports

Question

I have a package that allows the user to use any one of 4 packages they want to connect to a database. It works great but I’m unhappy with the way I’m importing things.

I could simply import all the packages, but I don’t want to do that in case the specific user doesn’t ever need to use turbodbc for example:

import pyodbc
import pymssql
import turbodbc
from ibmdbpy.base import IdaDataBase

Currently, I have the following situation. I try to import all of them, but the ones that don’t import, no problem, My program simply assumes they will not be called and if they are it errors:

# some envs may not have all these packages installed so we try each:

try:
    import pyodbc
except:
    pass

try:
    import pymssql
except:
    pass

try:
    import turbodbc
except:
    pass

try:
    from ibmdbpy.base import IdaDataBase
except:
    pass

This doesn’t feel pythonic. So I know there are packages such as holoviews or tensorflow that allow you to specify a backend. They are of course orders of magnitude more complicated than mine, but they have to handle the same pattern.

How can I make this code right? it is technically buggy because if they intend to use pyodbc but don’t have it installed, my program will not warn them, it will error at runtime. So really this goes beyond esthetics or philosophy; this is technically error-prone code.

How would you handle this situation?

Fyi, here is an example of how the code is called:

connect('Playground', package='pymssql')

Asked By: MetaStack

||

Source

Answer 1

You can put imports in places other than the beginning of the file. “Re-importing” something doesn’t actually do anything, so it’s not computationally expensive to import x frequently:

def switch(x):
    if x == 'a':
        import json
        json.load(file)
    elif x == 'b':
        import pandas as pd
        pd.read_csv(file)

You can also use importlib to dynamically import modules. This is especially useful if you have multiple implementations of the same API that you want to choose between

class Connection:
    def __init__(self, driver_module, driver_name):
        # or driver_module, driver_name = full_path.rsplit('.', 1)
        self.driver = get_attr(importlib.load_module(driver_module), driver_name)()
    def method(self):
        return self.driver.do()

Answered By: Patrick Haugh

Answer 2

try: 
    import pyodbc
except ImportError: 
    pyodbc = None

then later:

if pyodbc is None and user_wants_to_use_pyodbc:
    print_warning()
    raise SomeConfigurationErrorOrSuch()

This approach works well for a small number of options. If you have enough options that you need to abstract out this approach, then you can use the importlib module to import modules under the control of your program.

Answered By: Sam Hartman

Answer 3

I would use import_module from importlib:

from importlib import import_module

modules_to_import = ['pyodbc', 'pymssql', 'turbodbc', 'ibmdbpy.base.IdaDataBase']
for m in modules_to_import:
    try:
        globals()[m.split('.')[-1]] = import_module(m)
    except ModuleNotFoundError:
        print('Module {} not found'.format(m))

Answered By: Osman Mamun

Answer 4

I’ve used something similar to the answers above, but sometimes you might need to mock an object to fool lint.

try:
    from neuralprophet import NeuralProphet
    using_neuralprophet = True
except ImportError:
    class NeuralMock:
       whatever=False
    using_neuralprophet = False
    NeuralProphet = NeuralMock()

Source: timemachines

Answered By: Peter Cotton

the pythonic way of optional imports

Question:

Answers: