SQL identifier substitution – Using a list of column names

Question

I am trying to run a query of the form:

SELECT {} from TABLE where foo = VALUE

But I want to be able to provide a list to replace the {}

According to the psycopg docs, in order to do this safely, you need to use the sql.Identifier function, in order to properly escape the parameter, and then do something like this:

SQL = sql.SQL(
    "SELECT {} FROM foo WHERE bar = %s"
).format(identifier)

cursor.execute(SQL, [VALUE])

This works, when identifier is a single element, but I need it to be an arbitrary number. For example if:

identifier = ["abc", "def"]

and

VALUE = 4

SQL = SELECT abc def FROM foo WHERE bar = 4

I’ve tried running sql.Identifier(x) for every member of identifier, but that gave "abc""def", which is clearly not correct.

Asked By: alex_halford

||

Source

Answer 1

You need to use sql.join(), to make it work like a list comprehension:

from psycopg2 import sql

cols = ["abc", "def"]
query = sql.SQL(
    "select {0} from {1} where abc = %s").format(
        sql.SQL(', ').join([sql.Identifier(c) for c in cols]),
        sql.Identifier('foo')
)
cur = conn.cursor()
print(query)
print(cur.mogrify(query, ('1', )))
cur.execute(query, ('1', ))
print (cur.rowcount, cur.fetchall())

Output:

Composed([SQL('select '), Composed([Identifier('abc'), SQL(', '), Identifier('def')]), SQL(' from '), Identifier('foo'), SQL(' where abc = %s')])
select "abc", "def" from "foo" where abc = '1'
(1, [('1', '2')])

Answered By: Maurice Meyer

Answer 2

In case the accepted answer isn’t working for someone, which it should, try these two things:

1 – Make sure there’s no typo in the column name, and that it exists in the {0} table. Any of these failing would print the same error message, but the problem wouldn’t be from the double quotes!

2 – This is a work around which will probably tell you the issue was in fact "1-". psycopg2 is expecting double-quoted identifiers so Maurice’s answer should work. However, if you really need to remove them for some reason you can do:

query = query.as_string(cur).replace(""",r"")

and execute that.

Answered By: Fosh

SQL identifier substitution – Using a list of column names

Question:

Answers: