Use backticks (`) or double quotes (") with Python and SQLite
Question:
I saw a similar question on Stack Overflow pertaining to Android, but I was wondering whether I should use backticks (`) or double quotes (") – using Python – to select table names or rowid or what have you.
I tried single quotes – like this select 'rowid', * from 'tbl' order by 'rowid'. The single quotes worked in some cases but not all. I learned to use double quotes or backticks, and I was looking at the SQLite database browser and I noticed that it used backticks.
I really like to put double quotes around my strings in Python, because I’m coming from Java, so it is natural to do cursor.execute("select 'rowid',* from 'table';"), and it would be just as easy to do backticks (the double quotes would require a backslash and make the query look a little confusing).
However, I just wanted to make sure that the backticks are portable (all versions of Windows, Linux, OS X, etc.).
Answers:
Prefer double quotes for quoting identifiers, such as column or table names. It’s the SQL standard.
Backticks also work, but they’re only supported for MySQL syntax compatibility.
Single quotes are for string literals, not identifiers. That’s why you’ll get the literal value when using them.
Further reading: SQLite Keywords
The SQL standard says that strings must use 'single quotes', and identifiers (such as table and column names), when quoted, must use "double quotes".
For compatibility with MySQL, SQLite also allows to use single quotes for identifiers and double quotes for strings, but only when the context makes the meaning unambiguous. (In SELECT 'rowid' ..., a string is allowed, so a string is what you get.) If possible, always use the standard SQL quotes.
For compatibility with MySQL, SQLite also allows `backticks` for identifiers.
For compatibility with Microsoft databases, SQLite also allows [brackets] for identifiers.
(This works in all SQLite versions.)
SQLite treats "mycol" as a string if mycol does not exist, unlike backticks
Therefore, if you have a typo on your query and the column name is wrong, it will be treated as a string instead of giving an error which is what any sane person wants. The backtick however gives the error.
This better error checking provides a pro of using backticks as opposed to double quotes. The huge con being of course not being standard SQL.
Example:
tmp.sql
CREATE TABLE "IntegerNames" ( value INTEGER NOT NULL, name TEXT NOT NULL );
INSERT INTO "IntegerNames" VALUES
(2, 'two'),
(3, 'three'),
(5, 'five')
;
SELECT '"value"';
SELECT * FROM "IntegerNames" WHERE "value" = 2;
SELECT '"valuee"';
SELECT * FROM "IntegerNames" WHERE "valuee" = 2;
SELECT '`value`';
SELECT * FROM "IntegerNames" WHERE `value` = 2;
SELECT '`valuee`';
SELECT * FROM "IntegerNames" WHERE `valuee` = 2;
SELECT 'value';
SELECT * FROM "IntegerNames" WHERE value = 2;
SELECT 'valuee';
SELECT * FROM "IntegerNames" WHERE valuee = 2;
run:
rm -f tmp.sqlite && sqlite3 tmp.sqlite <tmp.sql
Outcome:
"value"
2|two
"valuee"
`value`
2|two
`valuee`
Error: near line 14: in prepare, no such column: valuee (1)
value
2|two
valuee
Error: near line 18: in prepare, no such column: valuee (1)
So we see that when using "valuee" there is no error: it just gets treated as a string 'value'. And then SQLite typecasts everything implicitly, and 'valuee' is not equal to any of the integer values, so the return is empty.
Backticks in this query has the same behavior as the unquoted value and valuee, since this is not a keyword. Note however that for SQL compliance, you have to tick rows that contain upper case characters I think (or at least you do in PostgreSQL), so they are not always equivalent, even not considering keywords.
Tested on SQLite 3.37.2, Ubuntu 22.04.
This terrible behavior is documented at: https://www.sqlite.org/quirks.html#double_quoted_string_literals_are_accepted and the devs themselves acknowledge that it was a bad idea, initially motivated by a desire for MySQL compatibility. They have since learnt to copy PostgreSQL more often nowadays 😉
8. Double-quoted String Literals Are Accepted
The SQL standard requires double-quotes around identifiers and single-quotes around string literals. For example:
- "this is a legal SQL column name"
- ‘this is an SQL string literal’
SQLite accepts both of the above. But, in an effort to be compatible with MySQL 3.x (which was one of the most widely used RDBMSes when SQLite was first being designed) SQLite will also interpret a double-quotes string as string literal if it does not match any valid identifier.
This misfeature means that a misspelled double-quoted identifier will be interpreted as a string literal, rather than generating an error. It also lures developers who are new to the SQL language into the bad habit of using double-quoted string literals when they really need to learn to use the correct single-quoted string literal form.
In hindsight, we should not have tried to make SQLite accept MySQL 3.x syntax, and should have never allowed double-quoted string literals. However, there are countless applications that make use of double-quoted string literals and so we continue to support that capability to avoid breaking legacy.
As of SQLite 3.27.0 (2019-02-07) the use of a double-quoted string literal causes a warning message to be sent to the error log.
As of SQLite 3.29.0 (2019-07-10) the use of double-quoted string literals can be disabled at run-time using the SQLITE_DBCONFIG_DQS_DDL and SQLITE_DBCONFIG_DQS_DML actions to sqlite3_db_config(). The default settings can be altered at compile-time using the -DSQLITE_DQS=N compile-time option. Application developers are encouraged to compile using -DSQLITE_DQS=0 in order to disable the double-quoted string literal misfeature by default. If that is not possible, then disable double-quoted string literals for individual database connections using C-code like this:
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0);
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);
Or, if double-quoted string literals are disabled by default, but need to be selectively enabled for some historical database connections, that can be done using the same C-code as shown above except with the third parameter changed from 0 to 1.
I saw a similar question on Stack Overflow pertaining to Android, but I was wondering whether I should use backticks (`) or double quotes (") – using Python – to select table names or rowid or what have you.
I tried single quotes – like this select 'rowid', * from 'tbl' order by 'rowid'. The single quotes worked in some cases but not all. I learned to use double quotes or backticks, and I was looking at the SQLite database browser and I noticed that it used backticks.
I really like to put double quotes around my strings in Python, because I’m coming from Java, so it is natural to do cursor.execute("select 'rowid',* from 'table';"), and it would be just as easy to do backticks (the double quotes would require a backslash and make the query look a little confusing).
However, I just wanted to make sure that the backticks are portable (all versions of Windows, Linux, OS X, etc.).
Prefer double quotes for quoting identifiers, such as column or table names. It’s the SQL standard.
Backticks also work, but they’re only supported for MySQL syntax compatibility.
Single quotes are for string literals, not identifiers. That’s why you’ll get the literal value when using them.
Further reading: SQLite Keywords
The SQL standard says that strings must use 'single quotes', and identifiers (such as table and column names), when quoted, must use "double quotes".
For compatibility with MySQL, SQLite also allows to use single quotes for identifiers and double quotes for strings, but only when the context makes the meaning unambiguous. (In SELECT 'rowid' ..., a string is allowed, so a string is what you get.) If possible, always use the standard SQL quotes.
For compatibility with MySQL, SQLite also allows `backticks` for identifiers.
For compatibility with Microsoft databases, SQLite also allows [brackets] for identifiers.
(This works in all SQLite versions.)
SQLite treats "mycol" as a string if mycol does not exist, unlike backticks
Therefore, if you have a typo on your query and the column name is wrong, it will be treated as a string instead of giving an error which is what any sane person wants. The backtick however gives the error.
This better error checking provides a pro of using backticks as opposed to double quotes. The huge con being of course not being standard SQL.
Example:
tmp.sql
CREATE TABLE "IntegerNames" ( value INTEGER NOT NULL, name TEXT NOT NULL );
INSERT INTO "IntegerNames" VALUES
(2, 'two'),
(3, 'three'),
(5, 'five')
;
SELECT '"value"';
SELECT * FROM "IntegerNames" WHERE "value" = 2;
SELECT '"valuee"';
SELECT * FROM "IntegerNames" WHERE "valuee" = 2;
SELECT '`value`';
SELECT * FROM "IntegerNames" WHERE `value` = 2;
SELECT '`valuee`';
SELECT * FROM "IntegerNames" WHERE `valuee` = 2;
SELECT 'value';
SELECT * FROM "IntegerNames" WHERE value = 2;
SELECT 'valuee';
SELECT * FROM "IntegerNames" WHERE valuee = 2;
run:
rm -f tmp.sqlite && sqlite3 tmp.sqlite <tmp.sql
Outcome:
"value"
2|two
"valuee"
`value`
2|two
`valuee`
Error: near line 14: in prepare, no such column: valuee (1)
value
2|two
valuee
Error: near line 18: in prepare, no such column: valuee (1)
So we see that when using "valuee" there is no error: it just gets treated as a string 'value'. And then SQLite typecasts everything implicitly, and 'valuee' is not equal to any of the integer values, so the return is empty.
Backticks in this query has the same behavior as the unquoted value and valuee, since this is not a keyword. Note however that for SQL compliance, you have to tick rows that contain upper case characters I think (or at least you do in PostgreSQL), so they are not always equivalent, even not considering keywords.
Tested on SQLite 3.37.2, Ubuntu 22.04.
This terrible behavior is documented at: https://www.sqlite.org/quirks.html#double_quoted_string_literals_are_accepted and the devs themselves acknowledge that it was a bad idea, initially motivated by a desire for MySQL compatibility. They have since learnt to copy PostgreSQL more often nowadays 😉
8. Double-quoted String Literals Are Accepted
The SQL standard requires double-quotes around identifiers and single-quotes around string literals. For example:
- "this is a legal SQL column name"
- ‘this is an SQL string literal’
SQLite accepts both of the above. But, in an effort to be compatible with MySQL 3.x (which was one of the most widely used RDBMSes when SQLite was first being designed) SQLite will also interpret a double-quotes string as string literal if it does not match any valid identifier.This misfeature means that a misspelled double-quoted identifier will be interpreted as a string literal, rather than generating an error. It also lures developers who are new to the SQL language into the bad habit of using double-quoted string literals when they really need to learn to use the correct single-quoted string literal form.
In hindsight, we should not have tried to make SQLite accept MySQL 3.x syntax, and should have never allowed double-quoted string literals. However, there are countless applications that make use of double-quoted string literals and so we continue to support that capability to avoid breaking legacy.
As of SQLite 3.27.0 (2019-02-07) the use of a double-quoted string literal causes a warning message to be sent to the error log.
As of SQLite 3.29.0 (2019-07-10) the use of double-quoted string literals can be disabled at run-time using the SQLITE_DBCONFIG_DQS_DDL and SQLITE_DBCONFIG_DQS_DML actions to sqlite3_db_config(). The default settings can be altered at compile-time using the -DSQLITE_DQS=N compile-time option. Application developers are encouraged to compile using -DSQLITE_DQS=0 in order to disable the double-quoted string literal misfeature by default. If that is not possible, then disable double-quoted string literals for individual database connections using C-code like this:
sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DDL, 0, (void*)0); sqlite3_db_config(db, SQLITE_DBCONFIG_DQS_DML, 0, (void*)0);Or, if double-quoted string literals are disabled by default, but need to be selectively enabled for some historical database connections, that can be done using the same C-code as shown above except with the third parameter changed from 0 to 1.