JSON API to Dataframe

Question:

I created the function below to read from an API, it happens that I need the result to return in a datatrame, each key being a value, does anyone know how I can do this? When I transform into dataframe it doesn’t come columnar, it just brings the values ​​in the dataframe as json.

Function:

    def get_ip_list(self):
        response_all = list()

        while True:
            url = "www.apitest.com/testing/"
            response = requests.request("GET", url)

            try:
                if response.status_code != 200:
                    self.logger.log_info(f"An error has occured. Status code of API Page: {response.status_code}")
                else:
                    response = response.json()
                if response.count('no blacklist monitors found in account') > 0:
                    break
            except Exception as e:
                break
            response_all += response

            self.counter += 1
        df = pd.DataFrame.from_records(response_all)
        
        print(response_all)

Sample:

[[{'ID': '5a6sd465sdf465asd498a798a', 'Type': 'IPv4', 'Target': '300.400.500.600', 'Add_Date': 1519929239, 'Last_Check': 1658187303, 'Status': 'Active', 'Label': '', 'Contact_List_ID': 'f56sdf465sdf465sd4f', 'Blacklisted_Count': '0', 'Blacklisted_On': None, 'Links': {'Report_Link': 'https://hetrixtools.com/report/blacklist/456465465/', 'Whitelabel_Report_Link': 'https://rbl.allin.com.br/report/blacklist/545665/'}}, {'ID': 'mdkljkljsdflkjsd123232', 'Type': 'IPv4', 'Target': '200.109.500.900', 'Add_Date': 1519929239, 'Last_Check': 1658188249, 'Status': 'Active', 'Label': '', 'Contact_List_ID': 'jfkldjflksjflkjsdf5456465', 'Blacklisted_Count': '0', 'Blacklisted_On': None, 'Links': {'Report_Link': 'https://hetrixtools.com/report/blacklist/23565456465/', 'Whitelabel_Report_Link': 'https://rbl.allin.com.br/report/blacklist/2156465465/'}}, {'ID': '165sdf465s4df654d56', 'Type': 'IPv4', 'Target': '155.111.255.300', 'Add_Date': 1519929239, 'Last_Check': 1658187644, 'Status': 'Active', 'Label': '', 'Contact_List_ID': 'ds54f65sd4f65d4sf654df65', 'Blacklisted_Count': '0', 'Blacklisted_On': None, 'Links': {'Report_Link': 'https://hetrixtools.com/report/blacklist/45465465/', 'Whitelabel_Report_Link': 'https://rbl.allin.com.br/report/blacklist/5465465/'}}], {'Meta': {'Total_Records': '24576'}, 'Links': {'Pages': {'Prev': 'https://api.hetrixtools.com/v2/a8b6c925e6d01613deaf5f5c48581f8f/blacklist/monitors/22/1024/'}}}]
Asked By: Caroline Leite

||

Answers:

I’m assuming the sample you’ve provided is the contents of response_all. To create DataFrame from it you can do:

df = pd.DataFrame(response_all[0])
print(df.to_markdown(index=False))

Prints:

ID Type Target Add_Date Last_Check Status Label Contact_List_ID Blacklisted_Count Blacklisted_On Links
5a6sd465sdf465asd498a798a IPv4 300.400.500.600 1519929239 1658187303 Active f56sdf465sdf465sd4f 0 {‘Report_Link’: ‘https://hetrixtools.com/report/blacklist/456465465/’, ‘Whitelabel_Report_Link’: ‘https://rbl.allin.com.br/report/blacklist/545665/’}
mdkljkljsdflkjsd123232 IPv4 200.109.500.900 1519929239 1658188249 Active jfkldjflksjflkjsdf5456465 0 {‘Report_Link’: ‘https://hetrixtools.com/report/blacklist/23565456465/’, ‘Whitelabel_Report_Link’: ‘https://rbl.allin.com.br/report/blacklist/2156465465/’}
165sdf465s4df654d56 IPv4 155.111.255.300 1519929239 1658187644 Active ds54f65sd4f65d4sf654df65 0 {‘Report_Link’: ‘https://hetrixtools.com/report/blacklist/45465465/’, ‘Whitelabel_Report_Link’: ‘https://rbl.allin.com.br/report/blacklist/5465465/’}
Answered By: Andrej Kesely
Categories: questions Tags: , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.