Alembic. ModuleNotFoundError in env.py
Question:
I have the following project structure
--some db:
--some db:
--alchemy:
-- __init__.py
--alembic:
-- versions
-- env.py
-- README.py
-- script.py
--migrations:
-- __init__.py
--models:
-- model_1
-- model_2
-- __init__.py
I try to autogenerate migrations by alembic.
I have Base in __init__.py in models folder
import sqlalchemy as sa
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base, declared_attr
metadata = sa.MetaData()
Base = declarative_base(metadata=metadata)
And import this is env.py
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from models import Base
config = context.config
fileConfig(config.config_file_name)
target_metadata = Base.metadata
So, when I import Base from models in env.py in alembic directory and try to generate automigration I have some error like
ModuleNotFoundError: No module named ‘models’
How can I fix this error?
Answers:
The problem is that when env.py is executed, models is not in your PYTHONPATH, so it can’t be imported.
With the project structure you’ve outlined, it might be a little hard to address; the easiest solution might be to modify your PYTHONPATH inside env.py like this:
import sys
sys.path = ['', '..'] + sys.path[1:]
from models import Base
This will add the parent directory of alembic/ to your PYTHONPATH so that it can find the models module.
Alternatively, you add the directory containing your modules to your PYTHONPATH environment variable in your shell:
$ export PYTHONPATH='/path/to/some db/some db':$PYTHONPATH
This solution is a little more brittle because you have to remember to do this each session and it will be different for each machine you intend to run Alembic on.
When I ran into this issue, the SQLAlchemy developer suggested that if I was using pip and virtual environments, I could install my project in editable mode so that it’s in the system PYTHONPATH and Alembic would be able to find it from anywhere. More details are available in the Python packaging guide. For this, you’d need a setup.py for your project and you may want to change your project structure so that there’s a top-level module containing things like models and alchemy. For example:
myproj/
setup.py
alembic/
env.py
migrations/
myapp/
__init__.py
alchemy/
__init__.py
models/
__init__.py
If this is set up correctly, you can
$ pip install -e .
from inside the myproj/ directory. Then in env.py you would then import Base like this:
from myapp.models import Base
When a module is loaded from a file in Python, __file__ is set to its path. You can then use that with other functions to find the directory that the file is located in.
Add the following line in alembic/env.py file to resolve the ModuleNotFoundError
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASE_DIR)
Now there is a beautiful way: https://github.com/sqlalchemy/alembic/commit/d6b0c1af3df98b50c6ec52781aa411592c4e0c32
"…the default "alembic.ini" file includes a directive prepend_sys_path = . so that the local path is also in sys.path.
I found the following solution for Linux, which I find very easy.
Run the cmd as:
PYTHONPATH={your python path} {your alembic cmd}
I run alembic as a python module, for example:
python3 -m alembic.config upgrade head
I second @Abhishek Dhotre ‘s anwer since this approach is more universal to all sorts of project structures a developer might have, in my case when I run uvicorn main:app --reload I got No module named app.
I solved it by including sys.path.append(path_to_app) in env.py, where path_to_app is relative to the location of env.py as follows:
env.py
import sys
import os
BASE_DIR = os.path.abspath(os.path.dirname(__file__))
APP_DIR = os.path.join(BASE_DIR, '../app')
sys.path.append(APP_DIR)
I have the following project structure
--some db:
--some db:
--alchemy:
-- __init__.py
--alembic:
-- versions
-- env.py
-- README.py
-- script.py
--migrations:
-- __init__.py
--models:
-- model_1
-- model_2
-- __init__.py
I try to autogenerate migrations by alembic.
I have Base in __init__.py in models folder
import sqlalchemy as sa
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base, declared_attr
metadata = sa.MetaData()
Base = declarative_base(metadata=metadata)
And import this is env.py
from logging.config import fileConfig
from alembic import context
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from models import Base
config = context.config
fileConfig(config.config_file_name)
target_metadata = Base.metadata
So, when I import Base from models in env.py in alembic directory and try to generate automigration I have some error like
ModuleNotFoundError: No module named ‘models’
How can I fix this error?
The problem is that when env.py is executed, models is not in your PYTHONPATH, so it can’t be imported.
With the project structure you’ve outlined, it might be a little hard to address; the easiest solution might be to modify your PYTHONPATH inside env.py like this:
import sys
sys.path = ['', '..'] + sys.path[1:]
from models import Base
This will add the parent directory of alembic/ to your PYTHONPATH so that it can find the models module.
Alternatively, you add the directory containing your modules to your PYTHONPATH environment variable in your shell:
$ export PYTHONPATH='/path/to/some db/some db':$PYTHONPATH
This solution is a little more brittle because you have to remember to do this each session and it will be different for each machine you intend to run Alembic on.
When I ran into this issue, the SQLAlchemy developer suggested that if I was using pip and virtual environments, I could install my project in editable mode so that it’s in the system PYTHONPATH and Alembic would be able to find it from anywhere. More details are available in the Python packaging guide. For this, you’d need a setup.py for your project and you may want to change your project structure so that there’s a top-level module containing things like models and alchemy. For example:
myproj/
setup.py
alembic/
env.py
migrations/
myapp/
__init__.py
alchemy/
__init__.py
models/
__init__.py
If this is set up correctly, you can
$ pip install -e .
from inside the myproj/ directory. Then in env.py you would then import Base like this:
from myapp.models import Base
When a module is loaded from a file in Python, __file__ is set to its path. You can then use that with other functions to find the directory that the file is located in.
Add the following line in alembic/env.py file to resolve the ModuleNotFoundError
BASE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
sys.path.append(BASE_DIR)
Now there is a beautiful way: https://github.com/sqlalchemy/alembic/commit/d6b0c1af3df98b50c6ec52781aa411592c4e0c32
"…the default "alembic.ini" file includes a directive prepend_sys_path = . so that the local path is also in sys.path.
I found the following solution for Linux, which I find very easy.
Run the cmd as:
PYTHONPATH={your python path} {your alembic cmd}
I run alembic as a python module, for example:
python3 -m alembic.config upgrade head
I second @Abhishek Dhotre ‘s anwer since this approach is more universal to all sorts of project structures a developer might have, in my case when I run uvicorn main:app --reload I got No module named app.
I solved it by including sys.path.append(path_to_app) in env.py, where path_to_app is relative to the location of env.py as follows:
env.py
import sys
import os
BASE_DIR = os.path.abspath(os.path.dirname(__file__))
APP_DIR = os.path.join(BASE_DIR, '../app')
sys.path.append(APP_DIR)