copy temp from stdin with skip id


I have a huge file with some data and need to insert it into the crm database.

I have tested it with pandas.to_sql, but I also need to check for duplications and update data in duplications case, so I decided to use this:

                            CREATE TEMP TABLE temp
                                LIKE metal
                            ON COMMIT DROP;

                            COPY temp FROM STDIN WITH
                                DELIMITER AS ',';

                            INSERT INTO metal
                            SELECT *
                            FROM temp
                            ON CONFLICT (title) DO UPDATE SET main_category = EXCLUDED.main_category

My issue is, that crm db has autogenerated id’s, and I can’t it add to my file

So can I just write in the statement to skip the first column (the id’s column)?

Asked By: Nero



I found the solution to my problem, all I need is to find the latest id and make it +1 in my csv file

max_id = pd.read_sql_table('metal', engine)['id'].max()

And then, using my statement above, if it is new data, it will add a new uniq id, if it is old, it will update the selected column (main_category) in my case

Answered By: Nero
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.