Python => ValueError: unsupported format character 'Y' (0x59)
Question:
I don’t understand the ValueError with Y. I escape with %…
table = town+"_history"
db.execute("SELECT DATE_FORMAT(snapdate,'%%Y-%%m-%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM %s WHERE blockid =%%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid))
Answers:
You escape the %% but then use the string as a formatter first:
"...." % table,
that returns a new string with the %% escaped percentages replaced by single % characters. The MySQL database adapter (ab)uses string formatting with % too, so it’ll take that output and expect to be able to fill %s slots with escaped SQL literals. It is there that your '%Y-%m-%d' part of the SQL statement is being interpreted again as a string format and the error is thrown.
The solution is to either double the doubling:
db.execute("SELECT DATE_FORMAT(snapdate,'%%%%Y-%%%%m-%%%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM %s WHERE blockid = %%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid,))
or use str.format() instead and avoid having to doubly-escape:
db.execute("SELECT DATE_FORMAT(snapdate,'%%Y-%%m-%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM {0} WHERE blockid = %s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7".format(table), (blockid,))
Here {0} is replaced by the table name and the %% escapes are left untouched; the database adapter will use the %s slot to fill in the blockid parameter and return a SQL statement with the %% escapes turned into single % characters.
Finally, @Martijn Pieters you are completely right. And thank you for your useful answer. The other error comes from SUM and COUNT. Python sometimes runs in a crazy way when dealing with JSON. So the complete answer is:
db.execute("SELECT DATE_FORMAT(snapdate,'%%%%Y-%%%%m-%%%%d') AS date, CAST(SUM( population ) AS CHAR ) AS accountpopulation, CAST(count( blockid ) AS CHAR) AS number_block FROM %s WHERE blockid = %%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid,))
I found this answer more elegant and less messy. I’ve used it without any issues in my code.
I don’t understand the ValueError with Y. I escape with %…
table = town+"_history"
db.execute("SELECT DATE_FORMAT(snapdate,'%%Y-%%m-%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM %s WHERE blockid =%%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid))
You escape the %% but then use the string as a formatter first:
"...." % table,
that returns a new string with the %% escaped percentages replaced by single % characters. The MySQL database adapter (ab)uses string formatting with % too, so it’ll take that output and expect to be able to fill %s slots with escaped SQL literals. It is there that your '%Y-%m-%d' part of the SQL statement is being interpreted again as a string format and the error is thrown.
The solution is to either double the doubling:
db.execute("SELECT DATE_FORMAT(snapdate,'%%%%Y-%%%%m-%%%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM %s WHERE blockid = %%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid,))
or use str.format() instead and avoid having to doubly-escape:
db.execute("SELECT DATE_FORMAT(snapdate,'%%Y-%%m-%%d') AS date, SUM( population ) AS accountpopulation, count( blockid ) AS number_block FROM {0} WHERE blockid = %s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7".format(table), (blockid,))
Here {0} is replaced by the table name and the %% escapes are left untouched; the database adapter will use the %s slot to fill in the blockid parameter and return a SQL statement with the %% escapes turned into single % characters.
Finally, @Martijn Pieters you are completely right. And thank you for your useful answer. The other error comes from SUM and COUNT. Python sometimes runs in a crazy way when dealing with JSON. So the complete answer is:
db.execute("SELECT DATE_FORMAT(snapdate,'%%%%Y-%%%%m-%%%%d') AS date, CAST(SUM( population ) AS CHAR ) AS accountpopulation, CAST(count( blockid ) AS CHAR) AS number_block FROM %s WHERE blockid = %%s GROUP BY snapdate ORDER BY snapdate DESC LIMIT 7" % table, (blockid,))
I found this answer more elegant and less messy. I’ve used it without any issues in my code.