Inserting yfinance API data from Python into MYSQL Table
Question:
just after some help (I’m a python beginner so please excuse me). I have installed the yfinance api in python and extracted historic stock data using the following commands:
import yfinance as yf
fltdata = yf.download(tickers="FLT", start="2021-12-06", end="2022-12-06", period="1d")
print (fltdata)
Which gives me an output looking like this:
[*********************100%***********************] 1 of 1 completed
Open High Low Close Adj Close Volume
Date
2021-12-06 210.050003 219.720001 208.690002 217.259995 217.259995 660600
2021-12-07 219.250000 224.389999 219.250000 220.149994 220.149994 901000
2021-12-08 220.940002 224.020004 219.889999 223.050003 223.050003 609100
2021-12-09 222.039993 224.539993 219.649994 221.949997 221.949997 611700
2021-12-10 223.130005 224.220001 218.869995 222.809998 222.809998 460600
... ... ... ... ... ... ...
2022-11-29 187.380005 191.289993 187.380005 190.199997 190.199997 406400
2022-11-30 190.270004 196.440002 188.800003 196.199997 196.199997 591400
2022-12-01 197.139999 199.179993 194.729996 196.279999 196.279999 396300
2022-12-02 192.289993 194.320007 191.059998 193.600006 193.600006 425200
2022-12-05 192.130005 193.110001 189.610001 191.250000 191.250000 471100
[252 rows x 6 columns]
I then installed the mysql python connector and created a mysql database which works all well and good:
import mysql.connector
mybd = mysql.connector.connect(
host="XXXXX",
user="root",
password="XXXX",
database="XXXXX"
)
mycursor=mybd.cursor()
mycursor.execute("SHOW DATABASES")
for db in mycursor:
print(db)
I have created a table in the database with three columns as i only need to present these variables- Date, Opening Price and CLosing Price:
mycursor.execute("CREATE TABLE fltstocks (date VARCHAR(10), openingprice decimal(10,6), closingprice decimal(10,6))")
mycursor.execute("SHOW TABLES")
for tb in mycursor:
print(tb)
What’s the best way to insert the data from the API output in Python into the corresponding columns in the MYSQL database? Or is it best to create a table in MYSQL that has all the columns from the output and then drop columns? Obviously the output is 252 rows and i want to insert all with avoiding writing them all out and I can’t seem to get that working.
Much appreciate any help as im clueless.
Thanks 🙂
I orignally tried creating a table with all variables in the output and inserting all the values with the following:
mycursor.execute("CREATE TABLE fltstocks (date VARCHAR(10), openingprice decimal(10,6), closingprice decimal(10,6), adjclose decimal(10,6), volume int(6))")
mycursor.execute("SHOW TABLES")
for tb in mycursor:
print(tb)
"Insert Data into fltstock table - fix below this line"
mycursor.execute("INSERT INTO fltstocks")
for i, row in fltdata.iterrows():
sql = 'INSERT INTO fltstocks VALUES(%s, %s, %s, %s, %s, %s, %s)'
mycursor.execute(sql, tuple(row))
print("Record Inserted")
mybd.commit()
mybd.close()
but get this error:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ” at line 1
Any help in inserting this data to my MYSQL database would be amazing,
Thanks everyone!
Answers:
What’s the best way to insert the data from the API output in Python into the corresponding columns in the MYSQL database?
You have a dataframe in hand.
So go the simplest route: just call .to_sql()
df.to_sql("fltstocks", con=mycursor)
That will create a new table.
Feel free to adjust the name, if a fltstocks already exists.
Also, the if_exists parameter can be very helpful.
You might prefer to subset your dataframe prior to writing those rows.
For example, you might want just these three columns:
df = df[['Date', 'Open', 'Close']]
Sqlalchemy has a lot going for it,
and it’s a natural fit for dataframes.
But it’s big, so maybe wait for your next project
to try it out.
just after some help (I’m a python beginner so please excuse me). I have installed the yfinance api in python and extracted historic stock data using the following commands:
import yfinance as yf
fltdata = yf.download(tickers="FLT", start="2021-12-06", end="2022-12-06", period="1d")
print (fltdata)
Which gives me an output looking like this:
[*********************100%***********************] 1 of 1 completed
Open High Low Close Adj Close Volume
Date
2021-12-06 210.050003 219.720001 208.690002 217.259995 217.259995 660600
2021-12-07 219.250000 224.389999 219.250000 220.149994 220.149994 901000
2021-12-08 220.940002 224.020004 219.889999 223.050003 223.050003 609100
2021-12-09 222.039993 224.539993 219.649994 221.949997 221.949997 611700
2021-12-10 223.130005 224.220001 218.869995 222.809998 222.809998 460600
... ... ... ... ... ... ...
2022-11-29 187.380005 191.289993 187.380005 190.199997 190.199997 406400
2022-11-30 190.270004 196.440002 188.800003 196.199997 196.199997 591400
2022-12-01 197.139999 199.179993 194.729996 196.279999 196.279999 396300
2022-12-02 192.289993 194.320007 191.059998 193.600006 193.600006 425200
2022-12-05 192.130005 193.110001 189.610001 191.250000 191.250000 471100
[252 rows x 6 columns]
I then installed the mysql python connector and created a mysql database which works all well and good:
import mysql.connector
mybd = mysql.connector.connect(
host="XXXXX",
user="root",
password="XXXX",
database="XXXXX"
)
mycursor=mybd.cursor()
mycursor.execute("SHOW DATABASES")
for db in mycursor:
print(db)
I have created a table in the database with three columns as i only need to present these variables- Date, Opening Price and CLosing Price:
mycursor.execute("CREATE TABLE fltstocks (date VARCHAR(10), openingprice decimal(10,6), closingprice decimal(10,6))")
mycursor.execute("SHOW TABLES")
for tb in mycursor:
print(tb)
What’s the best way to insert the data from the API output in Python into the corresponding columns in the MYSQL database? Or is it best to create a table in MYSQL that has all the columns from the output and then drop columns? Obviously the output is 252 rows and i want to insert all with avoiding writing them all out and I can’t seem to get that working.
Much appreciate any help as im clueless.
Thanks 🙂
I orignally tried creating a table with all variables in the output and inserting all the values with the following:
mycursor.execute("CREATE TABLE fltstocks (date VARCHAR(10), openingprice decimal(10,6), closingprice decimal(10,6), adjclose decimal(10,6), volume int(6))")
mycursor.execute("SHOW TABLES")
for tb in mycursor:
print(tb)
"Insert Data into fltstock table - fix below this line"
mycursor.execute("INSERT INTO fltstocks")
for i, row in fltdata.iterrows():
sql = 'INSERT INTO fltstocks VALUES(%s, %s, %s, %s, %s, %s, %s)'
mycursor.execute(sql, tuple(row))
print("Record Inserted")
mybd.commit()
mybd.close()
but get this error:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ” at line 1
Any help in inserting this data to my MYSQL database would be amazing,
Thanks everyone!
What’s the best way to insert the data from the API output in Python into the corresponding columns in the MYSQL database?
You have a dataframe in hand.
So go the simplest route: just call .to_sql()
df.to_sql("fltstocks", con=mycursor)
That will create a new table.
Feel free to adjust the name, if a fltstocks already exists.
Also, the if_exists parameter can be very helpful.
You might prefer to subset your dataframe prior to writing those rows.
For example, you might want just these three columns:
df = df[['Date', 'Open', 'Close']]
Sqlalchemy has a lot going for it,
and it’s a natural fit for dataframes.
But it’s big, so maybe wait for your next project
to try it out.