How can I have a row-wise rank in a pyspark dataframe Question: I have a dataset for which I am going to find the rank per row. This is a toy example in pandas. import pandas as pd df = pd.DataFrame({"ID":[1,2,3,4], "a":[2,7,9,10], "b":[6,7,4,2], "c":[3,4,8,5]}) print(df) # ID a b c # 0 1 2 6 …
SQLAlchemy Mapper Events don't get triggered Question: I can’t get Mapper Events to trigger in SqlAlchemy 2.0 tl;dr: I am able to perform operations but events never trigger. Below is the example they have in their docs: def my_before_insert_listener(mapper, connection, target): # execute a stored procedure upon INSERT, # apply the value to the row …
How to remove all duplicates and similiar queries from Django admin Question: I have a lot different models in my project and in every of them there’s a duplicated SQL queries I want to know how to remove them but the main problem is that in one model when I’m trying to take a look …
copy temp from stdin with skip id Question: I have a huge file with some data and need to insert it into the crm database. I have tested it with pandas.to_sql, but I also need to check for duplications and update data in duplications case, so I decided to use this: SQL_STATEMENT = """ CREATE …
How to fetch data in Python from SQL as a list of strings instead of tuples? Question: I’m trying to get the columns names of an SQLite table, but I’m getting back a list of tuples with the names. cur.execute(”’SELECT name FROM PRAGMA_TABLE_INFO(‘Emails’)”’) .fetchall()) If I print this the output is: [(’email’,), (‘count’,)] How can …
UPDATE row with ID in Azure SQL table with pyodbc (passing variables) Question: I am attempting to update an entire row with a specific ID in my azure SQL database using pyodbc. I currently have the below query but know that the syntax is not correct. def update_invoice(fullName, invoiceNo, date, address, description, total, invoiceID): cursor.execute("UPDATE …
how to make loop in pyspark Question: i have a code: list_files = glob.glob("/t/main_folder/*/file_*[0-9].csv") test = sorted(list_files, key = lambda x:x[-5:]) so this code has helped me to find files that i need to work with. I found 5 csv files in different folders. next step-im using a code down below , to work with …
Maximum number of values we can have in IN clause Question: Is there a limit to the maximum number of values we can have in the in clause sql_query = """SELECT * FROM table1 WHERE column1 IN %(list_of_values)s ORDER BY CASE WHEN column2 LIKE ‘a%’ THEN 1 WHEN column2 LIKE ‘b%’ THEN 2 WHEN column2 …
Can we do a switch case with an in clause Question: Can we do use an in clause with case sql_query=f"""SELECT * FROM table1 where column1 in (‘{list_of_values}’) order by CASE WHEN column2 like’a%’ THEN 1 WHEN column2 like’b%’ THEN 2 WHEN column2 like’c%’ THEN 3 ELSE 99 END; """ I am not getting any …
Sqlalchemy is slow when doing query the first time Question: I’m using Sqlalchemy(2.0.3) with python3.10 and after fresh container boot it takes ~2.2s to execute specific query, all consecutive calls of the same query take ~70ms to execute. I’m using PostgreSQL and it takes 40-70ms to execute raw query in DataGrip. Here is the code: …
