A good way to escape quotes in a database query string?
Question:
I’ve tried all manner of Python modules and they either escape too much or in the wrong way.
What’s the best way you’ve found to escape quotes (“, ‘) in Python?
Answers:
If it’s part of a Database query you should be able to use a Parameterized SQL Statement.
As well as escaping your quotes, this will deal with all special characters and will protect you from SQL injection attacks.
For a solution to a more generic problem, I have a program where I needed to store any set of characters in a flat file, tab delimited. Obviously, having tabs in the ‘set’ was causing problems.
Instead of output_f.write(str), I used output_f.write(repr(str)), which solved my problem.
It is slower to read, as I need to eval() the input when I read it, but overall, it makes the code cleaner because I don’t need to check for fringe cases anymore.
If you’re using psycopg2 that has a method for escaping strings: psycopg2.extensions.adapt() See How to quote a string value explicitly (Python DB API/Psycopg2) for the full answer
Triple-double quotes are best for escaping:
string = """This will span across 'single quotes', "double quotes",
and literal EOLs all in the same string."""
The easy and standard way to escape strings, and convert other objects to programmatic form, is to use the built in repr() function. It converts an object into the representation you would need to enter it with manual code.
E.g.:
s = "I'm happy I am "here" now"
print repr(s)
>> 'I'm happy I am "here" now'
No weird hacks, it’s built in and it just works for most purposes.
Use json.dumps.
>>> import json
>>> print json.dumps('a"bc')
"a"bc"
If using psycopg2, its execute() method has built-in escaping:
cursor.execute("SELECT column FROM table WHERE column=%s AND column2=%s", (value1, value2))
Note, that you are giving two arguments to execute method (string and tuple), instead of using Python’s % operator to modify string.
Answer stolen from here: psycopg2 equivalent of mysqldb.escape_string?
Triple single quotes will conveniently encapsulate the single quotes often used in SQL queries:
c.execute('''SELECT sval FROM sdat WHERE instime > NOW() - INTERVAL '1 days' ORDER BY instime ASC''')
For my use case, I was saving a paragraph against the database and somewhere in the paragraph there might have been some text with a single quote (example: Charlie's apple sauce was soggy)
I found this to work best:
database_cursor.execute('''INSERT INTO books.collection (book_name, book_quoted_text) VALUES ('%s', "%s")''' % (book_name, page_text.strip()))
You’ll notice that I use "" after wrapping the INSERT statement in '''
I’ve tried all manner of Python modules and they either escape too much or in the wrong way.
What’s the best way you’ve found to escape quotes (“, ‘) in Python?
If it’s part of a Database query you should be able to use a Parameterized SQL Statement.
As well as escaping your quotes, this will deal with all special characters and will protect you from SQL injection attacks.
For a solution to a more generic problem, I have a program where I needed to store any set of characters in a flat file, tab delimited. Obviously, having tabs in the ‘set’ was causing problems.
Instead of output_f.write(str), I used output_f.write(repr(str)), which solved my problem.
It is slower to read, as I need to eval() the input when I read it, but overall, it makes the code cleaner because I don’t need to check for fringe cases anymore.
If you’re using psycopg2 that has a method for escaping strings: psycopg2.extensions.adapt() See How to quote a string value explicitly (Python DB API/Psycopg2) for the full answer
Triple-double quotes are best for escaping:
string = """This will span across 'single quotes', "double quotes", and literal EOLs all in the same string."""
The easy and standard way to escape strings, and convert other objects to programmatic form, is to use the built in repr() function. It converts an object into the representation you would need to enter it with manual code.
E.g.:
s = "I'm happy I am "here" now"
print repr(s)
>> 'I'm happy I am "here" now'
No weird hacks, it’s built in and it just works for most purposes.
Use json.dumps.
>>> import json
>>> print json.dumps('a"bc')
"a"bc"
If using psycopg2, its execute() method has built-in escaping:
cursor.execute("SELECT column FROM table WHERE column=%s AND column2=%s", (value1, value2))
Note, that you are giving two arguments to execute method (string and tuple), instead of using Python’s % operator to modify string.
Answer stolen from here: psycopg2 equivalent of mysqldb.escape_string?
Triple single quotes will conveniently encapsulate the single quotes often used in SQL queries:
c.execute('''SELECT sval FROM sdat WHERE instime > NOW() - INTERVAL '1 days' ORDER BY instime ASC''')
For my use case, I was saving a paragraph against the database and somewhere in the paragraph there might have been some text with a single quote (example: Charlie's apple sauce was soggy)
I found this to work best:
database_cursor.execute('''INSERT INTO books.collection (book_name, book_quoted_text) VALUES ('%s', "%s")''' % (book_name, page_text.strip()))
You’ll notice that I use "" after wrapping the INSERT statement in '''