python and sql: issue reading a sql file

Question

I’m trying to read a sql file but it keeps giving me the error:

UnicodeError: UTF-16 stream does not start with BOM

I’ve created a fxn to read sql files specifically:

import pandas as pd
import pyodbc as db
import os
import codecs

def sql_reader_single(qry_file, server_name, database, encoding='utf16'):
    server = db.connect(str('DRIVER={SQL Server};SERVER='+server_name+';DATABASE='+database+';'))
    with codecs.open(qry_file, encoding=encoding) as qf:
        data = pd.read_sql(qf.read(), server)
    return data

then I called it to read data:

Data = sp.sql_reader_single(qry_file=QryFile, server_name='my_server', database='my_db')

what am i doing wrong?

I’ve looked into:

utf-16 file seeking in python. how?

and tried both utf-16-le or utf-16-be, but I would get an error with a bunch of japanese/chinese characters like this:

pandas.io.sql.DatabaseError: Execution failed on sql '䕓䕌呃ഠ 楤瑳湩瑣਍††⨠਍†剆䵏䔠坄䔮坄䘮捡剴捥楥楶杮潇摯⁳牦൧': ('42000', "[42000] [Microsoft][ODBC SQL Server Driver][SQL Server]Incorrect syntax near '0x0a0d'. (102) (SQLExecDirectW)")

the sql file contains a very simple query, like this:

SELECT distinct *
  FROM FactReceiving

Asked By: alwaysaskingquestions

||

Source

Answer 1

Try to read the file as UTF-8.

Answered By: clinomaniac

Answer 2

I used errors=’ignore’ with the utf-8 encoding to prevent missing hex codes from preventing processing.

def get_text(file_name):
    with open(file_name, 'r', encoding='utf-8', errors='ignore') as f:
        text = f.read()
    return text

Answered By: Golden Lion

python and sql: issue reading a sql file

Question:

Answers: