Alembic –autogenerate producing empty migration
Question:
I am trying to use Alembic
for the first time and want to use --autogenerate
feature described here
My project structure looks like
project/
configuration/
__init__.py
dev.py
test.py
core/
app/
models/
__init__.py
user.py
db/
alembic/
versions/
env.py
alembic.ini
I am using Flask
and SQLAlchemy
and their Flask-SQLAlchemy
extension. my model User
looks like
class User(UserMixin, db.Model):
__tablename__ = 'users'
# noinspection PyShadowingBuiltins
uuid = Column('uuid', GUID(), default=uuid.uuid4, primary_key=True,
unique=True)
email = Column('email', String, nullable=False, unique=True)
_password = Column('password', String, nullable=False)
created_on = Column('created_on', sa.types.DateTime(timezone=True),
default=datetime.utcnow())
last_login = Column('last_login', sa.types.DateTime(timezone=True),
onupdate=datetime.utcnow())
As described here, I modified env.py
to look like
from configuration import app
alembic_config = config.get_section(config.config_ini_section)
alembic_config['sqlalchemy.url'] = app.config['SQLALCHEMY_DATABASE_URI']
engine = engine_from_config(
alembic_config,
prefix='sqlalchemy.',
poolclass=pool.NullPool)
and
from configuration import db
target_metadata = db.metadata
where configuration.__init__py
looks like
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
import dev
app = Flask(__name__)
app.config.from_envvar('SETTINGS_PT')
db = SQLAlchemy(app)
Now when I run migration
$alembic revision --autogenerate -m "Added user table"
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
Generating /Users/me/IdeaProjects/project/db/alembic/versions/55a9d5
35d8ae_added_user_table.py...done
but file alembic/versions/55a9d5
has empty upgrade()
and downgrade()
methods
"""Added user table
Revision ID: 1b62a62eef0d
Revises: None
Create Date: 2013-03-27 06:37:08.314177
"""
# revision identifiers, used by Alembic.
revision = '1b62a62eef0d'
down_revision = None
from alembic import op
import sqlalchemy as sa
def upgrade():
### commands auto generated by Alembic - please adjust! ###
pass
### end Alembic commands ###
def downgrade():
### commands auto generated by Alembic - please adjust! ###
pass
### end Alembic commands ###
How come it is not able to understand that there is a new User
model?
Please help
Answers:
As per @zzzeek, after I included the following in my env.py
, I was able to work with --autogenerate
option
in env.py
under run_migrations_online()
from configuration import app
from core.expense.models import user # added my model here
alembic_config = config.get_section(config.config_ini_section)
alembic_config['sqlalchemy.url'] = app.config['SQLALCHEMY_DATABASE_URI']
engine = engine_from_config(
alembic_config,
prefix='sqlalchemy.',
poolclass=pool.NullPool)
then I ran alembic revision --autogenerate -m "Added initial table"
and got
def upgrade():
### commands auto generated by Alembic - please adjust! ###
op.create_table('users',
sa.Column('uuid', sa.GUID(), nullable=False),
sa.Column('email', sa.String(), nullable=False),
sa.Column('password', sa.String(), nullable=False),
sa.Column('created_on', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_login', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('uuid'),
sa.UniqueConstraint('email'),
sa.UniqueConstraint('uuid')
)
### end Alembic commands ###
Thank you Michael for all your help!
I think it’s worth pointing out here that I had the same problem in the current version (0.8.4), but the method of setting metadata appears to have become more explicit: In addition to importing a model, you need to also set target_metadata
(which is present in env.py
but defaults to None
).
The documentation suggests importing something they called Base
, but it’s not clear what exactly that is; importing the DeclarativeBase instance my models inherit from did nothing for me (same result as OP).
The actual comments in the code, though, suggest setting target_metadata
using an actual model (ModelNameHere.metadata
), which did work for me (using one model’s metadata resulted in all of them being detected).
If you don’t want your flake8 throwing unused import errors, then you can also add the Record to __all__
in your models file.
At the end of users.py
__all__ = Users
More information at – https://docs.python.org/3/tutorial/modules.html#importing-from-a-package
Everything was much simplier, just from starting read the doc i solved my issue with auto generating the migrations.
Don’t forget to change
target_metadata
from None to Base.metadata
in your env.py file.
target_metadata = Base.metadata
along with the import of the all models you want to track in your __init__.py
of your module that consist all db models that inherits from Base model. Then you just need to import these Base in your env.py
from app.database.models import Base
that’s it!
I encountered the same problem and I solved it by importing class from file (note file from directory) as following:
from db.db_setup import Base
from db.models.user import User
from db.models.course import Course
And then droped all tables and autogenerated file then run alembic revision --autogenerate
again.
In my case problem was i already had target table created in database.
After i dropped this table, revision command produced filled upgrade and downgrade functions.
If anyone still has this problem for fastAPI on Postgresql database, do not forget to have your server running that is your uvicorn servers should be running. Then just change this line on alembic ini
sqlalchemy.url = driver://user:pass@localhost/dbname
to this
sqlalchemy.url =
Then you set the sqlalchemy url in the env.py
inside the alembic folder using this just below the config = context.config
:
db_string = f"postgresql+psycopg2://postgresusername:12345678@localhost:5432/mydatabase"
config.set_main_option("sqlalchemy.url", db_string)
Then import your models for example from db.base_class import Base
just before target_metadata = None
, and change the target to this:
target_metadata = Base.metadata
So my “`env.py“ looks like this:
from logging.config import fileConfig
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from core.config import settings
from alembic import context
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
db_string = f"postgresql+psycopg2://postgresusername:12345678@localhost:5432/mydatabase"
config.set_main_option("sqlalchemy.url", db_string)
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
# target_metadata = None
from db.base_class import Base
target_metadata = Base.metadata
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()
I am trying to use Alembic
for the first time and want to use --autogenerate
feature described here
My project structure looks like
project/
configuration/
__init__.py
dev.py
test.py
core/
app/
models/
__init__.py
user.py
db/
alembic/
versions/
env.py
alembic.ini
I am using Flask
and SQLAlchemy
and their Flask-SQLAlchemy
extension. my model User
looks like
class User(UserMixin, db.Model):
__tablename__ = 'users'
# noinspection PyShadowingBuiltins
uuid = Column('uuid', GUID(), default=uuid.uuid4, primary_key=True,
unique=True)
email = Column('email', String, nullable=False, unique=True)
_password = Column('password', String, nullable=False)
created_on = Column('created_on', sa.types.DateTime(timezone=True),
default=datetime.utcnow())
last_login = Column('last_login', sa.types.DateTime(timezone=True),
onupdate=datetime.utcnow())
As described here, I modified env.py
to look like
from configuration import app
alembic_config = config.get_section(config.config_ini_section)
alembic_config['sqlalchemy.url'] = app.config['SQLALCHEMY_DATABASE_URI']
engine = engine_from_config(
alembic_config,
prefix='sqlalchemy.',
poolclass=pool.NullPool)
and
from configuration import db
target_metadata = db.metadata
where configuration.__init__py
looks like
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
import dev
app = Flask(__name__)
app.config.from_envvar('SETTINGS_PT')
db = SQLAlchemy(app)
Now when I run migration
$alembic revision --autogenerate -m "Added user table"
INFO [alembic.migration] Context impl PostgresqlImpl.
INFO [alembic.migration] Will assume transactional DDL.
Generating /Users/me/IdeaProjects/project/db/alembic/versions/55a9d5
35d8ae_added_user_table.py...done
but file alembic/versions/55a9d5
has empty upgrade()
and downgrade()
methods
"""Added user table
Revision ID: 1b62a62eef0d
Revises: None
Create Date: 2013-03-27 06:37:08.314177
"""
# revision identifiers, used by Alembic.
revision = '1b62a62eef0d'
down_revision = None
from alembic import op
import sqlalchemy as sa
def upgrade():
### commands auto generated by Alembic - please adjust! ###
pass
### end Alembic commands ###
def downgrade():
### commands auto generated by Alembic - please adjust! ###
pass
### end Alembic commands ###
How come it is not able to understand that there is a new User
model?
Please help
As per @zzzeek, after I included the following in my env.py
, I was able to work with --autogenerate
option
in env.py
under run_migrations_online()
from configuration import app
from core.expense.models import user # added my model here
alembic_config = config.get_section(config.config_ini_section)
alembic_config['sqlalchemy.url'] = app.config['SQLALCHEMY_DATABASE_URI']
engine = engine_from_config(
alembic_config,
prefix='sqlalchemy.',
poolclass=pool.NullPool)
then I ran alembic revision --autogenerate -m "Added initial table"
and got
def upgrade():
### commands auto generated by Alembic - please adjust! ###
op.create_table('users',
sa.Column('uuid', sa.GUID(), nullable=False),
sa.Column('email', sa.String(), nullable=False),
sa.Column('password', sa.String(), nullable=False),
sa.Column('created_on', sa.DateTime(timezone=True), nullable=True),
sa.Column('last_login', sa.DateTime(timezone=True), nullable=True),
sa.PrimaryKeyConstraint('uuid'),
sa.UniqueConstraint('email'),
sa.UniqueConstraint('uuid')
)
### end Alembic commands ###
Thank you Michael for all your help!
I think it’s worth pointing out here that I had the same problem in the current version (0.8.4), but the method of setting metadata appears to have become more explicit: In addition to importing a model, you need to also set target_metadata
(which is present in env.py
but defaults to None
).
The documentation suggests importing something they called Base
, but it’s not clear what exactly that is; importing the DeclarativeBase instance my models inherit from did nothing for me (same result as OP).
The actual comments in the code, though, suggest setting target_metadata
using an actual model (ModelNameHere.metadata
), which did work for me (using one model’s metadata resulted in all of them being detected).
If you don’t want your flake8 throwing unused import errors, then you can also add the Record to __all__
in your models file.
At the end of users.py
__all__ = Users
More information at – https://docs.python.org/3/tutorial/modules.html#importing-from-a-package
Everything was much simplier, just from starting read the doc i solved my issue with auto generating the migrations.
Don’t forget to change
target_metadata
from None to Base.metadata
in your env.py file.
target_metadata = Base.metadata
along with the import of the all models you want to track in your __init__.py
of your module that consist all db models that inherits from Base model. Then you just need to import these Base in your env.py
from app.database.models import Base
that’s it!
I encountered the same problem and I solved it by importing class from file (note file from directory) as following:
from db.db_setup import Base
from db.models.user import User
from db.models.course import Course
And then droped all tables and autogenerated file then run alembic revision --autogenerate
again.
In my case problem was i already had target table created in database.
After i dropped this table, revision command produced filled upgrade and downgrade functions.
If anyone still has this problem for fastAPI on Postgresql database, do not forget to have your server running that is your uvicorn servers should be running. Then just change this line on alembic ini
sqlalchemy.url = driver://user:pass@localhost/dbname
to this
sqlalchemy.url =
Then you set the sqlalchemy url in the env.py
inside the alembic folder using this just below the config = context.config
:
db_string = f"postgresql+psycopg2://postgresusername:12345678@localhost:5432/mydatabase"
config.set_main_option("sqlalchemy.url", db_string)
Then import your models for example from db.base_class import Base
just before target_metadata = None
, and change the target to this:
target_metadata = Base.metadata
So my “`env.py“ looks like this:
from logging.config import fileConfig
from sqlalchemy import engine_from_config
from sqlalchemy import pool
from core.config import settings
from alembic import context
# this is the Alembic Config object, which provides
# access to the values within the .ini file in use.
config = context.config
db_string = f"postgresql+psycopg2://postgresusername:12345678@localhost:5432/mydatabase"
config.set_main_option("sqlalchemy.url", db_string)
# Interpret the config file for Python logging.
# This line sets up loggers basically.
if config.config_file_name is not None:
fileConfig(config.config_file_name)
# add your model's MetaData object here
# for 'autogenerate' support
# from myapp import mymodel
# target_metadata = mymodel.Base.metadata
# target_metadata = None
from db.base_class import Base
target_metadata = Base.metadata
# other values from the config, defined by the needs of env.py,
# can be acquired:
# my_important_option = config.get_main_option("my_important_option")
# ... etc.
def run_migrations_offline() -> None:
"""Run migrations in 'offline' mode.
This configures the context with just a URL
and not an Engine, though an Engine is acceptable
here as well. By skipping the Engine creation
we don't even need a DBAPI to be available.
Calls to context.execute() here emit the given string to the
script output.
"""
url = config.get_main_option("sqlalchemy.url")
context.configure(
url=url,
target_metadata=target_metadata,
literal_binds=True,
dialect_opts={"paramstyle": "named"},
)
with context.begin_transaction():
context.run_migrations()
def run_migrations_online() -> None:
"""Run migrations in 'online' mode.
In this scenario we need to create an Engine
and associate a connection with the context.
"""
connectable = engine_from_config(
config.get_section(config.config_ini_section),
prefix="sqlalchemy.",
poolclass=pool.NullPool,
)
with connectable.connect() as connection:
context.configure(connection=connection, target_metadata=target_metadata)
with context.begin_transaction():
context.run_migrations()
if context.is_offline_mode():
run_migrations_offline()
else:
run_migrations_online()