How to scrape data from an interactive chart using python?
Question:
I would like to scrape the date and total value data from the "Value History" chart on the following link when the chart is set to 1Y:
https://csgoskins.gg/markets/csgofloat
I’ve tried to look and see if the data on the site is stored in a json file, but it does not seem like it.
I found this looking at the javascript code of the site, which makes me think the data is stored within a variable called t, but I haven’t been able to find it:
series: [{
name: "Total Value",
type: "line",
showInNavigator: !0,
color: "#60a5fa",
marker: {
lineColor: "#60a5fa"
},
data: t.map((function(t) {
return [Date.parse(t.day), t.sales_value]
}
Which is part of a function called: window.createMarketValueHistoryChart = function(t)
Any help would be much appreciated!
Answers:
Check the history
variable in the console. Something like:
history.forEach(v => {console.log(v.day, v.sales_value)})
will print the data you need.
Getting it to pandas
If you use Chrome (but I’d assume that should work for most browsers), in the console write history
, hit enter, right-click on the output, "Copy object". Paste it into data.json
file and read with:
df = pd.read_json('data.json')
I would like to scrape the date and total value data from the "Value History" chart on the following link when the chart is set to 1Y:
https://csgoskins.gg/markets/csgofloat
I’ve tried to look and see if the data on the site is stored in a json file, but it does not seem like it.
I found this looking at the javascript code of the site, which makes me think the data is stored within a variable called t, but I haven’t been able to find it:
series: [{
name: "Total Value",
type: "line",
showInNavigator: !0,
color: "#60a5fa",
marker: {
lineColor: "#60a5fa"
},
data: t.map((function(t) {
return [Date.parse(t.day), t.sales_value]
}
Which is part of a function called: window.createMarketValueHistoryChart = function(t)
Any help would be much appreciated!
Check the history
variable in the console. Something like:
history.forEach(v => {console.log(v.day, v.sales_value)})
will print the data you need.
Getting it to pandas
If you use Chrome (but I’d assume that should work for most browsers), in the console write history
, hit enter, right-click on the output, "Copy object". Paste it into data.json
file and read with:
df = pd.read_json('data.json')