Case Insensitive Flask-SQLAlchemy Query
Question:
I’m using Flask-SQLAlchemy to query from a database of users; however, while
user = models.User.query.filter_by(username="ganye").first()
will return
<User u'ganye'>
doing
user = models.User.query.filter_by(username="GANYE").first()
returns
None
I’m wondering if there’s a way to query the database in a case insensitive way, so that the second example will still return
<User u'ganye'>
Answers:
You can do it by using either the lower
or upper
functions in your filter:
from sqlalchemy import func
user = models.User.query.filter(func.lower(User.username) == func.lower("GaNyE")).first()
Another option is to do searching using ilike
instead of like
:
.query.filter(Model.column.ilike("ganye"))
Improving on @plaes’s answer, this one will make the query shorter if you specify just the column(s) you need:
user = models.User.query.with_entities(models.User.username).
filter(models.User.username.ilike("%ganye%")).all()
The above example is very useful in case one needs to use Flask’s jsonify for AJAX purposes and then in your javascript access it using data.result:
from flask import jsonify
jsonify(result=user)
you can do
user = db.session.query(User).filter_by(func.lower(User.username)==func.lower("GANYE")).first()
Or you can use ilike function
user = db.session.query(User).filter_by(User.username.ilike("%ganye%")).first()
If it fits your use case, you may consider setting a custom collation on the column, such that the column automatically handles comparisons in a case-insensitive manner.
It’s worth noting:
- the collation will apply to all queries on the column
- the collation will apply to
ORDER BY
clauses as well
- a collation may be specified directly in queries, rather than being defined on the column
- this may potentially incur performance penalties
- collation definitions tend to be RDBMS / locale / language-specific – consult the relevant documentation
- collation names may not be portable between different RDBMS
- available collation attributes may vary by RDBMS
in other words, consult your RDBMS’s docs before using this feature.
This example script shows how you might use collations for MySQL, Postgresql (note the special-casing) and Sqlite; each RDBMS returns all three possible results for the query value.
import sqlalchemy as sa
from sqlalchemy import orm
data = {
'mysql': ('mysql:///test', 'utf8mb4_general_ci'),
'postgresql': ('postgresql:///test', 'coll'),
'sqlite': ('sqlite://', 'NOCASE'),
}
for dialect, (uri, collation) in data.items():
Base = orm.declarative_base()
class Test(Base):
__tablename__ = 't16573095'
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(32, collation=collation))
engine = sa.create_engine(uri, echo=False, future=True)
Base.metadata.drop_all(engine)
if dialect == 'postgresql':
# Postgres collations are more complicated
# - read the docs!
with engine.begin() as conn:
conn.execute(sa.text('DROP COLLATION IF EXISTS coll'))
stmt = """CREATE COLLATION coll (provider='icu', locale='und-u-ks-level2', deterministic=false)"""
conn.execute(sa.text(stmt))
Base.metadata.create_all(engine)
Session = orm.sessionmaker(engine, future=True)
with Session.begin() as s:
instances = [Test(name=name) for name in ['GANYE', 'ganye', 'gAnYe']]
s.add_all(instances)
with Session() as s:
results = s.execute(sa.select(Test.name).where(Test.name == 'GaNyE')).scalars()
print(f'{dialect:-<12}')
for name in results:
print(name)
print('-' * 12)
To specify a collation in a query, use an attribute’s collate method:
with Session() as s:
query = sa.select(Test).where(Test.name.collate('coll') == 'GaNyE')
results = s.execute(query)
I think the best solution would be
func.lower(User.username).contains(username.lower())
It will produce sql like
WHERE (lower(user.username) LIKE '%%' || %s || '%%')
This means your db username
will be converted to the lower case and your python value username
also will be converted to lower case. So now you do not have to worry about the case sesitivity
If your username
is Sara Smith
It will work for ara
, sar
, Sara
or event a S
- By one field
uname = "ganye"
user = models.User.query.filter(User.username.ilikef(f"%{uname}%").first()
- By several fields
return db.session.query(Post).filter(
Post.title.ilike(f"%{search_query}%")
| Post.body.ilike(f"%{search_query}%")
)
For declarative users, the equivalent without using query
is the ColumnOperators.ilike method:
class User(DeclarativeBase):
username: str
req = select(User).where(User.username.ilike("GANYE"))
result = session.execute(req)
will render as lower(User.username) LIKE lower("GANYE")
This is still the case in SQLAlchemy 2.0, and is compatible with AsyncSession
, for those using it.
I’m using Flask-SQLAlchemy to query from a database of users; however, while
user = models.User.query.filter_by(username="ganye").first()
will return
<User u'ganye'>
doing
user = models.User.query.filter_by(username="GANYE").first()
returns
None
I’m wondering if there’s a way to query the database in a case insensitive way, so that the second example will still return
<User u'ganye'>
You can do it by using either the lower
or upper
functions in your filter:
from sqlalchemy import func
user = models.User.query.filter(func.lower(User.username) == func.lower("GaNyE")).first()
Another option is to do searching using ilike
instead of like
:
.query.filter(Model.column.ilike("ganye"))
Improving on @plaes’s answer, this one will make the query shorter if you specify just the column(s) you need:
user = models.User.query.with_entities(models.User.username).
filter(models.User.username.ilike("%ganye%")).all()
The above example is very useful in case one needs to use Flask’s jsonify for AJAX purposes and then in your javascript access it using data.result:
from flask import jsonify
jsonify(result=user)
you can do
user = db.session.query(User).filter_by(func.lower(User.username)==func.lower("GANYE")).first()
Or you can use ilike function
user = db.session.query(User).filter_by(User.username.ilike("%ganye%")).first()
If it fits your use case, you may consider setting a custom collation on the column, such that the column automatically handles comparisons in a case-insensitive manner.
It’s worth noting:
- the collation will apply to all queries on the column
- the collation will apply to
ORDER BY
clauses as well - a collation may be specified directly in queries, rather than being defined on the column
- this may potentially incur performance penalties
- collation definitions tend to be RDBMS / locale / language-specific – consult the relevant documentation
- collation names may not be portable between different RDBMS
- available collation attributes may vary by RDBMS
in other words, consult your RDBMS’s docs before using this feature.
This example script shows how you might use collations for MySQL, Postgresql (note the special-casing) and Sqlite; each RDBMS returns all three possible results for the query value.
import sqlalchemy as sa
from sqlalchemy import orm
data = {
'mysql': ('mysql:///test', 'utf8mb4_general_ci'),
'postgresql': ('postgresql:///test', 'coll'),
'sqlite': ('sqlite://', 'NOCASE'),
}
for dialect, (uri, collation) in data.items():
Base = orm.declarative_base()
class Test(Base):
__tablename__ = 't16573095'
id = sa.Column(sa.Integer, primary_key=True)
name = sa.Column(sa.String(32, collation=collation))
engine = sa.create_engine(uri, echo=False, future=True)
Base.metadata.drop_all(engine)
if dialect == 'postgresql':
# Postgres collations are more complicated
# - read the docs!
with engine.begin() as conn:
conn.execute(sa.text('DROP COLLATION IF EXISTS coll'))
stmt = """CREATE COLLATION coll (provider='icu', locale='und-u-ks-level2', deterministic=false)"""
conn.execute(sa.text(stmt))
Base.metadata.create_all(engine)
Session = orm.sessionmaker(engine, future=True)
with Session.begin() as s:
instances = [Test(name=name) for name in ['GANYE', 'ganye', 'gAnYe']]
s.add_all(instances)
with Session() as s:
results = s.execute(sa.select(Test.name).where(Test.name == 'GaNyE')).scalars()
print(f'{dialect:-<12}')
for name in results:
print(name)
print('-' * 12)
To specify a collation in a query, use an attribute’s collate method:
with Session() as s:
query = sa.select(Test).where(Test.name.collate('coll') == 'GaNyE')
results = s.execute(query)
I think the best solution would be
func.lower(User.username).contains(username.lower())
It will produce sql like
WHERE (lower(user.username) LIKE '%%' || %s || '%%')
This means your db username
will be converted to the lower case and your python value username
also will be converted to lower case. So now you do not have to worry about the case sesitivity
If your username
is Sara Smith
It will work for ara
, sar
, Sara
or event a S
- By one field
uname = "ganye" user = models.User.query.filter(User.username.ilikef(f"%{uname}%").first()
- By several fields
return db.session.query(Post).filter( Post.title.ilike(f"%{search_query}%") | Post.body.ilike(f"%{search_query}%") )
For declarative users, the equivalent without using query
is the ColumnOperators.ilike method:
class User(DeclarativeBase):
username: str
req = select(User).where(User.username.ilike("GANYE"))
result = session.execute(req)
will render as lower(User.username) LIKE lower("GANYE")
This is still the case in SQLAlchemy 2.0, and is compatible with AsyncSession
, for those using it.