Using requests I am creating an object which is in .csv format. How can I then write that object to a DataFrame with pandas?
To get the requests object in text format:
import requests import pandas as pd url = r'http://test.url' r = requests.get(url) r.text #this will return the data as text in csv format
I tried (doesn’t work):
I think you can use
filepath_or_buffer : str, pathlib.Path, py._path.local.LocalPath or any object with a read() method (such as a file handle or StringIO)
The string could be a URL. Valid URL schemes include http, ftp, s3, and file. For file URLs, a host is expected. For instance, a local file could be file ://localhost/path/to/table.csv
import pandas as pd import io import requests url = r'http://...' r = requests.get(url) df = pd.read_csv(io.StringIO(r))
If it doesnt work, try update last line:
import pandas as pd import io import requests url = r'http://...' r = requests.get(url) df = pd.read_csv(io.StringIO(r.text))
if the url has no authentication then you can directly use read_csv(url)
if you have authentication you can use request to get it un-pickel and print the csv and make sure the result is CSV and use panda.
You can directly use importing
import requests import pandas as pd import io urlData = requests.get(url).content rawData = pd.read_csv(io.StringIO(urlData.decode('utf-8')))
Using “read_csv with url” worked:
import requests, csv import pandas as pd url = 'https://arte.folha.uol.com.br/ciencia/2020/coronavirus/csv/mundo/dados-bra.csv' corona_bra = pd.read_csv(url) print(corona_bra.head())