Error while using "pd_writer" to write data to snowflake – 'not all arguments converted during string formatting'

Question

Getting this error while calling the "pd_writer" method:

DatabaseError("Execution failed on sql ‘SELECT name FROM sqlite_master WHERE type=’table’ AND name=?;’: not all arguments converted during string formatting",)

def sf_To_df(self):
    sf_data = self.salesForceAuth.query_all(self.queryColumns)
    sf_data=dict(sf_data)
    try:            
        cursor=snowflake.connector.connect(user=snowflake_user_path,      
            paccount=snowflake_account,
            warehouse=snowflake_warehouse,
            database=snowflake_database,
            schema=snowflake_schema);

   for dictKey1, dictValue1 in sf_data.items():
       if dictKey1 == "records":
           sf_Df = pandas.DataFrame.from_records(dictValue1)
           **sf_Df.to_sql(dictKey1, cursor, index=False, method=pd_writer)**
   
except Exception as e:
   typ, obj, tb = sys.exc_info()
   print("exception while reading the key value: ", e.__repr__(),tb.tb_lineno)

Asked By: ADom

||

Source

Answer 1

Have a look at the answer that I gave here. The reason is because you are passing your variable ‘cursor’ – which you have incorrectly named ‘cursor’ but is actually a connection – to the to_sql method. You’re supposed to pass an SQLAlchemy engine into to_sql, not a Snowflake connection.

Example from the other link:

from sqlalchemy import create_engine
import os
import pandas as pd

snowflake_username = os.environ['SNOWFLAKE_USERNAME']
snowflake_password = os.environ['SNOWFLAKE_PASSWORD']
snowflake_account = os.environ['SNOWFLAKE_ACCOUNT']
snowflake_warehouse = os.environ['SNOWFLAKE_WAREHOUSE']
snowflake_database = 'test_db'
snowflake_schema = 'public'


if __name__ == '__main__':
    engine = create_engine(
        'snowflake://{user}:{password}@{account}/{db}/{schema}?warehouse={warehouse}'.format(
            user=snowflake_username,
            password=snowflake_password,
            account=snowflake_account,
            db=snowflake_database,
            schema=snowflake_schema,
            warehouse=snowflake_warehouse,
        )
    )
    df = pd.DataFrame([('Mark', 10), ('Luke', 20)], columns=['name', 'balance'])
    df.to_sql('TEST_TABLE', con=engine, schema='public', index=False, if_exists='append')

Answered By: Simon D

Error while using "pd_writer" to write data to snowflake – 'not all arguments converted during string formatting'

Question:

Answers: