SQLAlchemy classes across files
Question:
I’m trying to figure out how to have SQLAlchemy classes spread across several files, and I can for my life not figure out how to do it. I am pretty new to SQLAlchemy so forgive me if this question is trivial..
Consider these 3 classes in each their own file:
A.py:
from sqlalchemy import *
from main import Base
class A(Base):
__tablename__ = "A"
id = Column(Integer, primary_key=True)
Bs = relationship("B", backref="A.id")
Cs = relationship("C", backref="A.id")
B.py:
from sqlalchemy import *
from main import Base
class B(Base):
__tablename__ = "B"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
C.py:
from sqlalchemy import *
from main import Base
class C(Base):
__tablename__ = "C"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
And then say we have a main.py something like this:
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, backref, sessionmaker
Base = declarative_base()
import A
import B
import C
engine = create_engine("sqlite:///test.db")
Base.metadata.create_all(engine, checkfirst=True)
Session = sessionmaker(bind=engine)
session = Session()
a = A.A()
b1 = B.B()
b2 = B.B()
c1 = C.C()
c2 = C.C()
a.Bs.append(b1)
a.Bs.append(b2)
a.Cs.append(c1)
a.Cs.append(c2)
session.add(a)
session.commit()
The above gives the error:
sqlalchemy.exc.NoReferencedTableError: Foreign key assocated with column 'C.A_id' could not find table 'A' with which to generate a foreign key to target column 'id'
How do I share the declarative base across these files?
What is the “the right” way to accomplish this, considering that I might throw something like Pylons or Turbogears on top of this?
edit 10-03-2011
I found this description from the Pyramids framework which describes the problem and more importantly verifies that this is an actual issue and not (only) just my confused self that’s the problem. Hope it can help others who dares down this dangerous road 🙂
Answers:
The simplest solution to your problem will be to take Base out of the module that imports A, B and C; Break the cyclic import.
base.py
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
a.py
from sqlalchemy import *
from base import Base
from sqlalchemy.orm import relationship
class A(Base):
__tablename__ = "A"
id = Column(Integer, primary_key=True)
Bs = relationship("B", backref="A.id")
Cs = relationship("C", backref="A.id")
b.py
from sqlalchemy import *
from base import Base
class B(Base):
__tablename__ = "B"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
c.py
from sqlalchemy import *
from base import Base
class C(Base):
__tablename__ = "C"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
main.py
from sqlalchemy import create_engine
from sqlalchemy.orm import relationship, backref, sessionmaker
import base
import a
import b
import c
engine = create_engine("sqlite:///:memory:")
base.Base.metadata.create_all(engine, checkfirst=True)
Session = sessionmaker(bind=engine)
session = Session()
a1 = a.A()
b1 = b.B()
b2 = b.B()
c1 = c.C()
c2 = c.C()
a1.Bs.append(b1)
a1.Bs.append(b2)
a1.Cs.append(c1)
a1.Cs.append(c2)
session.add(a1)
session.commit()
Works on my machine:
$ python main.py ; echo $?
0
I’m using Python 2.7 + Flask 0.10 + SQLAlchemy 1.0.8 + Postgres 9.4.4.1
This boilerplate comes configured with a User and UserDetail models stored in the same file “models.py” in the “user” module. These classes both inherit from an SQLAlchemy base class.
All of the additional classes I’ve added to my project also derived from this base class, and as the models.py file grew larger, I decided to split the models.py file into one file per class, and ran into the problem described here.
The solution I found, along the same lines as @computermacgyver’s Oct 23 2013 post, was to include all my classes to the init.py file of the new module I created to hold all the newly created class files. Looks like this:
/project/models/
__init__.py contains
from project.models.a import A
from project.models.b import B
etc...
If I may add my bit of sense as well since I had the same problem. You need to import the classes in the file where you create the Base = declarative_base() AFTER you created the Base and the Tables. Short example how my project is set up:
model/user.py
from sqlalchemy import *
from sqlalchemy.orm import relationship
from model import Base
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
budgets = relationship('Budget')
model/budget.py
from sqlalchemy import *
from model import Base
class Budget(Base):
__tablename__ = 'budget'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('user.id'))
model/__init__.py
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
_DB_URI = 'sqlite:///:memory:'
engine = create_engine(_DB_URI)
Base = declarative_base()
Base.metadata.create_all(engine)
DBSession = sessionmaker(bind=engine)
session = DBSession()
from .user import User
from .budget import Budget
For me, adding import app.tool.tool_entity inside app.py and from app.tool.tool_entity import Tool inside tool/__init__.py was enough to get the table to be created. I haven’t tried adding relationship yet, though.
Folder structure:
app/
app.py
tool/
__init__.py
tool_entity.py
tool_routes.py
# app/tool/tool_entity.py
from app.base import Base
from sqlalchemy import Column, Integer, String
class Tool(Base):
__tablename__ = 'tool'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False)
fullname = Column(String)
fullname2 = Column(String)
nickname = Column(String)
def __repr__(self):
return "<User(name='%s', fullname='%s', nickname='%s')>" % (
self.name, self.fullname, self.nickname)
# app/tool/__init__.py
from app.tool.tool_entity import Tool
# app/app.py
from flask import Flask
from sqlalchemy import create_engine
from app.tool.tool_routes import tool_blueprint
from app.base import Base
db_dialect = 'postgresql'
db_user = 'postgres'
db_pwd = 'postgrespwd'
db_host = 'db'
db_name = 'db_name'
engine = create_engine(f'{db_dialect}://{db_user}:{db_pwd}@{db_host}/{db_name}', echo=True)
Base.metadata.create_all(engine)
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'hello world'
app.register_blueprint(tool_blueprint, url_prefix='/tool')
if __name__ == '__main__':
# you can add this import here, or anywhere else in the file, as debug (watch mode) is on,
# the table should be created as soon as you save this file.
import app.tool.tool_entity
app.run(host='0.0.0.0', port=5000, debug=True)
I’m trying to figure out how to have SQLAlchemy classes spread across several files, and I can for my life not figure out how to do it. I am pretty new to SQLAlchemy so forgive me if this question is trivial..
Consider these 3 classes in each their own file:
A.py:
from sqlalchemy import *
from main import Base
class A(Base):
__tablename__ = "A"
id = Column(Integer, primary_key=True)
Bs = relationship("B", backref="A.id")
Cs = relationship("C", backref="A.id")
B.py:
from sqlalchemy import *
from main import Base
class B(Base):
__tablename__ = "B"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
C.py:
from sqlalchemy import *
from main import Base
class C(Base):
__tablename__ = "C"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
And then say we have a main.py something like this:
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship, backref, sessionmaker
Base = declarative_base()
import A
import B
import C
engine = create_engine("sqlite:///test.db")
Base.metadata.create_all(engine, checkfirst=True)
Session = sessionmaker(bind=engine)
session = Session()
a = A.A()
b1 = B.B()
b2 = B.B()
c1 = C.C()
c2 = C.C()
a.Bs.append(b1)
a.Bs.append(b2)
a.Cs.append(c1)
a.Cs.append(c2)
session.add(a)
session.commit()
The above gives the error:
sqlalchemy.exc.NoReferencedTableError: Foreign key assocated with column 'C.A_id' could not find table 'A' with which to generate a foreign key to target column 'id'
How do I share the declarative base across these files?
What is the “the right” way to accomplish this, considering that I might throw something like Pylons or Turbogears on top of this?
edit 10-03-2011
I found this description from the Pyramids framework which describes the problem and more importantly verifies that this is an actual issue and not (only) just my confused self that’s the problem. Hope it can help others who dares down this dangerous road 🙂
The simplest solution to your problem will be to take Base out of the module that imports A, B and C; Break the cyclic import.
base.py
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
a.py
from sqlalchemy import *
from base import Base
from sqlalchemy.orm import relationship
class A(Base):
__tablename__ = "A"
id = Column(Integer, primary_key=True)
Bs = relationship("B", backref="A.id")
Cs = relationship("C", backref="A.id")
b.py
from sqlalchemy import *
from base import Base
class B(Base):
__tablename__ = "B"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
c.py
from sqlalchemy import *
from base import Base
class C(Base):
__tablename__ = "C"
id = Column(Integer, primary_key=True)
A_id = Column(Integer, ForeignKey("A.id"))
main.py
from sqlalchemy import create_engine
from sqlalchemy.orm import relationship, backref, sessionmaker
import base
import a
import b
import c
engine = create_engine("sqlite:///:memory:")
base.Base.metadata.create_all(engine, checkfirst=True)
Session = sessionmaker(bind=engine)
session = Session()
a1 = a.A()
b1 = b.B()
b2 = b.B()
c1 = c.C()
c2 = c.C()
a1.Bs.append(b1)
a1.Bs.append(b2)
a1.Cs.append(c1)
a1.Cs.append(c2)
session.add(a1)
session.commit()
Works on my machine:
$ python main.py ; echo $?
0
I’m using Python 2.7 + Flask 0.10 + SQLAlchemy 1.0.8 + Postgres 9.4.4.1
This boilerplate comes configured with a User and UserDetail models stored in the same file “models.py” in the “user” module. These classes both inherit from an SQLAlchemy base class.
All of the additional classes I’ve added to my project also derived from this base class, and as the models.py file grew larger, I decided to split the models.py file into one file per class, and ran into the problem described here.
The solution I found, along the same lines as @computermacgyver’s Oct 23 2013 post, was to include all my classes to the init.py file of the new module I created to hold all the newly created class files. Looks like this:
/project/models/
__init__.py contains
from project.models.a import A
from project.models.b import B
etc...
If I may add my bit of sense as well since I had the same problem. You need to import the classes in the file where you create the Base = declarative_base() AFTER you created the Base and the Tables. Short example how my project is set up:
model/user.py
from sqlalchemy import *
from sqlalchemy.orm import relationship
from model import Base
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
budgets = relationship('Budget')
model/budget.py
from sqlalchemy import *
from model import Base
class Budget(Base):
__tablename__ = 'budget'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('user.id'))
model/__init__.py
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
_DB_URI = 'sqlite:///:memory:'
engine = create_engine(_DB_URI)
Base = declarative_base()
Base.metadata.create_all(engine)
DBSession = sessionmaker(bind=engine)
session = DBSession()
from .user import User
from .budget import Budget
For me, adding import app.tool.tool_entity inside app.py and from app.tool.tool_entity import Tool inside tool/__init__.py was enough to get the table to be created. I haven’t tried adding relationship yet, though.
Folder structure:
app/
app.py
tool/
__init__.py
tool_entity.py
tool_routes.py
# app/tool/tool_entity.py
from app.base import Base
from sqlalchemy import Column, Integer, String
class Tool(Base):
__tablename__ = 'tool'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False)
fullname = Column(String)
fullname2 = Column(String)
nickname = Column(String)
def __repr__(self):
return "<User(name='%s', fullname='%s', nickname='%s')>" % (
self.name, self.fullname, self.nickname)
# app/tool/__init__.py
from app.tool.tool_entity import Tool
# app/app.py
from flask import Flask
from sqlalchemy import create_engine
from app.tool.tool_routes import tool_blueprint
from app.base import Base
db_dialect = 'postgresql'
db_user = 'postgres'
db_pwd = 'postgrespwd'
db_host = 'db'
db_name = 'db_name'
engine = create_engine(f'{db_dialect}://{db_user}:{db_pwd}@{db_host}/{db_name}', echo=True)
Base.metadata.create_all(engine)
app = Flask(__name__)
@app.route('/')
def hello_world():
return 'hello world'
app.register_blueprint(tool_blueprint, url_prefix='/tool')
if __name__ == '__main__':
# you can add this import here, or anywhere else in the file, as debug (watch mode) is on,
# the table should be created as soon as you save this file.
import app.tool.tool_entity
app.run(host='0.0.0.0', port=5000, debug=True)