Psycopg2 is not only accepts None (Nonetype) as a cell entry that can be Null when using %s but not with f-strings

Question:

I am inserting data into a Postgres DB using Psycopg2 using:

params = "ts, price, volume"
entry = (1234567, None, 3)
cur.execute(f"INSERT INTO stock_data ({params}) VALUES {entry};")

I have used this pattern of f-string with params and a tuple entry countless times with no issue.

For additional information, the price column does not have any constraints and is a numeric(5).

For this particular instance, it is giving me the error:

psycopg2.ProgrammingError: column "none" does not exist

Everything I tried resulted in this error. What did end up working was not using an f-string. The following entry worked:

cur.execute(f"INSERT INTO stock_data ({params}) VALUES (%s, %s, %s);", entry)

My understanding is that these are identical. Why would the latter work, but including the entry directly into the INSERT statement not work?

Asked By: python_crypto_questions

||
Source

Answers:

Format just inserts the value into a string:

>>> x = None
>>> f"What should I expect here? {x}"
'What should I expect here? None'

You pass the already formatted string to cur.execute(). When you pass a query text with placeholders and arguments, the function knows how to convert them.

>>> cur.mogrify(f"INSERT INTO stock_data ({params}) VALUES {entry};")
b'INSERT INTO stock_data (ts, price, volume) VALUES (1234567, None, 3);'

>>> cur.mogrify(f"INSERT INTO stock_data ({params}) VALUES (%s, %s, %s);", entry)
b'INSERT INTO stock_data (ts, price, volume) VALUES (1234567, NULL, 3);'

It would be best if you always used placeholders and arguments in cursor.execute(), also for your own safety. Read Passing parameters to SQL queries, where you can find this

Warning

Never, never, NEVER use Python string concatenation (+) or string parameters interpolation (%) to pass variables to a SQL query string. Not even at gunpoint.

Answered By: klin