How to use python mysqldb to insert many rows at once

Question

I have a list of lists, e.g [['a','b'],['c','d']].

I have a table called T and two fields F1, F2. The first item in the field list maps to F1, second to F2.

How can I insert rows for each inner list in a single command or call, rather than using a for loop like this?

for i in [['a','b'],['c','d']]:
    c.execute("insert into T (F1,F2) values (%s, %s)", (i[0], i[1]))

Asked By: Tampa

||

Source

Answer 1

From MySQLdb User’s Guide:

c.executemany(
      """INSERT INTO breakfast (name, spam, eggs, sausage, price)
      VALUES (%s, %s, %s, %s, %s)""",
      [
      ("Spam and Sausage Lover's Plate", 5, 1, 8, 7.95 ),
      ("Not So Much Spam Plate", 3, 2, 0, 3.95 ),
      ("Don't Wany ANY SPAM! Plate", 0, 4, 3, 5.95 )
      ] )

so in your case:

c.executemany("insert into T (F1,F2) values (%s, %s)",
    [('a','b'),('c','d')])

Answered By: zenpoy

Answer 2

It’s possible to insert all rows in one single statement like @adamhajari, and avoid sql injections like @zenpoy, at the same time. You just need to create a big insert statement and let mysqldb’s execute do the formatting.

values_to_insert = [('a','b'),('c','d')]
query = "INSERT INTO T (F1, F2) VALUES " + ",".join("(%s, %s)" for _ in values_to_insert)
flattened_values = [item for sublist in values_to_insert for item in sublist]
c.execute(query, flattened_values)

Not super readable, but can be slightly faster than executemany (I tried inserting batches of 50000 rows in a local DB, executemany was 20% slower).

Answered By: Rems

Answer 3

def multiple_insert(cursor, table, cols, rows):
    sql_insert = 'INSERT INTO %s(%s) values %s' % (
        table,
        ','.join(cols),
        ','.join('(%s , %s)' for _ in rows)
    )
    values = [_ for r in rows for _ in r]
    cursor.execute(sql_insert, values)


# eg:
rows = [(a1 , b1),(a2 , b2),(a3 , b3)]
multiple_insert(cursor, 'your_table',('col1', 'col2'), rows)
conn.commit()

Answered By: PHPJungle

How to use python mysqldb to insert many rows at once

Question:

Answers: