SQLAlchemy INSERT IGNORE
Question:
How can I insert multiple data records into table ignoring duplicates. I am using SQLAlchemy.
Thank you!
Answers:
prefix_with("TEXT") adds arbitrary text between INSERT and the rest of the SQL. execute() accepts a list of dictionaries with the records you would like to insert or a single dictionary if you only want to insert a single record.
The SQLite syntax for the behavior you’re looking for:
inserter = table_object.insert().prefix_with("OR REPLACE")
inserter.execute([{'column1':'value1'}, {'column1':'value2'}])
To always replace INSERT by INSERT OR IGNORE, you can use a compiler extension:
from sqlalchemy.ext.compiler import compiles
from sqlalchemy.sql.expression import Insert
@compiles(Insert)
def _prefix_insert_with_ignore(insert, compiler, **kw):
return compiler.visit_insert(insert.prefix_with('OR IGNORE'), **kw)
Or to do this only temporarily, call the compiles decorator manually and use deregister once you’re done:
from sqlalchemy.ext.compiler import compiles, deregister
from sqlalchemy.sql.expression import Insert
def _prefix_insert_with_ignore(insert, compiler, **kw):
return compiler.visit_insert(insert.prefix_with('OR IGNORE'), **kw)
compiles(Insert)(_prefix_insert_with_replace)
try:
# do some inserts...
finally:
deregister(Insert)
This does feel hacky because it’s still a global change, but as long as you don’t use threads and make sure everything is properly committed before the deregister call, it’s probably okay.
How can I insert multiple data records into table ignoring duplicates. I am using SQLAlchemy.
Thank you!
prefix_with("TEXT") adds arbitrary text between INSERT and the rest of the SQL. execute() accepts a list of dictionaries with the records you would like to insert or a single dictionary if you only want to insert a single record.
The SQLite syntax for the behavior you’re looking for:
inserter = table_object.insert().prefix_with("OR REPLACE")
inserter.execute([{'column1':'value1'}, {'column1':'value2'}])
To always replace INSERT by INSERT OR IGNORE, you can use a compiler extension:
from sqlalchemy.ext.compiler import compiles
from sqlalchemy.sql.expression import Insert
@compiles(Insert)
def _prefix_insert_with_ignore(insert, compiler, **kw):
return compiler.visit_insert(insert.prefix_with('OR IGNORE'), **kw)
Or to do this only temporarily, call the compiles decorator manually and use deregister once you’re done:
from sqlalchemy.ext.compiler import compiles, deregister
from sqlalchemy.sql.expression import Insert
def _prefix_insert_with_ignore(insert, compiler, **kw):
return compiler.visit_insert(insert.prefix_with('OR IGNORE'), **kw)
compiles(Insert)(_prefix_insert_with_replace)
try:
# do some inserts...
finally:
deregister(Insert)
This does feel hacky because it’s still a global change, but as long as you don’t use threads and make sure everything is properly committed before the deregister call, it’s probably okay.