How to Load the Earnings Calendar data from TradingView link and into Dataframe
Question:
Answers:
I noticed that there are few hidden columns characterized by the class i-hidden
. So as first thing we select only the visible columns with the css selector :not([class*=i-hidden])
. Then we get the attribute data-field
of these columns, so that we can select the corresponding values in the rows. Examples of values for this attribute are
name
market_cap_basic
earnings_per_share_forecast_next_fq
eps_surprise_fq
Next we get the header of the table and the rows. Then we loop over the data-field to get all the cell values in each column. Finally we create a dataframe from a dictionary having the header as keys and the columns as values.
visible_columns = driver.find_elements(By.CSS_SELECTOR, 'div.tv-screener__content-pane thead th:not([class*=i-hidden])')
data_field = [c.get_attribute('data-field') for c in visible_columns]
header = [c.text.split('n')[0] for c in visible_columns]
rows = driver.find_elements(By.XPATH, "//div[@class='tv-screener__content-pane']//tbody/tr")
columns = []
for field in data_field:
column = driver.find_elements(By.XPATH, f"//div[@class='tv-screener__content-pane']//tbody/tr/td[@data-field-key='{field}']")
columns.append([col.text.replace('n',' - ') for col in column])
pd.DataFrame(dict(zip(header, columns)))
Output
I noticed that there are few hidden columns characterized by the class i-hidden
. So as first thing we select only the visible columns with the css selector :not([class*=i-hidden])
. Then we get the attribute data-field
of these columns, so that we can select the corresponding values in the rows. Examples of values for this attribute are
name
market_cap_basic
earnings_per_share_forecast_next_fq
eps_surprise_fq
Next we get the header of the table and the rows. Then we loop over the data-field to get all the cell values in each column. Finally we create a dataframe from a dictionary having the header as keys and the columns as values.
visible_columns = driver.find_elements(By.CSS_SELECTOR, 'div.tv-screener__content-pane thead th:not([class*=i-hidden])')
data_field = [c.get_attribute('data-field') for c in visible_columns]
header = [c.text.split('n')[0] for c in visible_columns]
rows = driver.find_elements(By.XPATH, "//div[@class='tv-screener__content-pane']//tbody/tr")
columns = []
for field in data_field:
column = driver.find_elements(By.XPATH, f"//div[@class='tv-screener__content-pane']//tbody/tr/td[@data-field-key='{field}']")
columns.append([col.text.replace('n',' - ') for col in column])
pd.DataFrame(dict(zip(header, columns)))
Output