Parse JSON response with nested values (sometimes nonevalues)

Question:

I’m trying to parse multiple JSON responses from an API request using Python (one example below, but I have a list containing hundreds of these JSON responses). I’m looking to create a dictonary with the values from the "dealID", "companyName", and "dealSize": "amount" keys.

I’m able to successfully create a dictionary containing values from the first 3 keys above, but not the nested "amount". The issue I’m running into is that "amount" sometimes has a none value. To get around this, I created a function that replaces the none value with a default value. But now when I try to create a dictionary, I’m getting the error message "TypeError: string indices must be integers". Grateful for any advice. I’ve seemingly tried everything.

[{
    "dealId": "330843-24T",
    "dealNumber": 1,
    "companyId": "62224",
    "companyName": "Company A",
    "dealDate": "2014-04-16",
    "dealSize": {
        "amount": 1500000.0,
        "currency": "USD",
        "nativeAmount": 1500000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "dealSizeStatus": "Actual",
    "dealStatus": {
        "code": "COMP",
        "description": "Completed"
    },
    "percentAcquired": "None",
    "raisedToDate": {
        "amount": 1500000.0,
        "currency": "USD",
        "nativeAmount": 1500000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "vcRound": "1st Round",
    "vcRoundUpDownFlat": "None",
    "totalInvestedCapital": {
        "amount": 1500000.0,
        "currency": "USD",
        "nativeAmount": 1500000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "investorOwnership": "None",
    "stockSplit": "None",
    "dealType1": {
        "code": "Seed",
        "description": "Seed Round"
    },
    "dealType2": "None",
    "dealType3": "None",
    "dealClass": {
        "code": "VC",
        "description": "Venture Capital"
    },
    "dealSynopsis": "The company raised $1.5 million of seed funding",
    "totalInvestedEquity": {
        "amount": 300000.0,
        "currency": "USD",
        "nativeAmount": 300000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "debtType1": "None",
    "debtType2": "None",
    "debtType3": "None",
    "debtAmount1": {
        "amount": 1200000.0,
        "currency": "USD",
        "nativeAmount": 1200000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "debtAmount2": "None",
    "debtAmount3": "None",
    "debtRaisedInRound": {
        "amount": 1200000.0,
        "currency": "USD",
        "nativeAmount": 1200000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "contingentPayout": "None",
    "valuationAvailable": true,
    "capTableAvailable": false,
    "trancheInfoAvailable": false,
    "debtLenderInfoAvailable": true
}, {
    "dealId": "4053346-57T",
    "dealNumber": 3,
    "companyId": "624245-02",
    "companyName": "Company A",
    "dealDate": "2017-02-27",
    "dealSize": {
        "amount": 2800000.0,
        "currency": "USD",
        "nativeAmount": 2800000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "dealSizeStatus": "Actual",
    "dealStatus": {
        "code": "COMP",
        "description": "Completed"
    },
    "percentAcquired": "None",
    "raisedToDate": {
        "amount": 14298988.0,
        "currency": "USD",
        "nativeAmount": 14298988.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "vcRound": "3rd Round",
    "vcRoundUpDownFlat": "None",
    "totalInvestedCapital": {
        "amount": 2800000.0,
        "currency": "USD",
        "nativeAmount": 2800000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "investorOwnership": "None",
    "stockSplit": "None",
    "dealType1": {
        "code": "EVC",
        "description": "Early Stage VC"
    },
    "dealType2": {
        "code": "A1",
        "description": "Series A1"
    },
    "dealType3": "None",
    "dealClass": {
        "code": "VC",
        "description": "Venture Capital"
    },
    "dealSynopsis": "The company raised $2.8 million",
    "totalInvestedEquity": {
        "amount": 2800000.0,
        "currency": "USD",
        "nativeAmount": 2800000.0,
        "nativeCurrency": "USD",
        "estimated": false
    },
    "debtType1": "None",
    "debtType2": "None",
    "debtType3": "None",
    "debtAmount1": "None",
    "debtAmount2": "None",
    "debtAmount3": "None",
    "debtRaisedInRound": "None",
    "contingentPayout": "None",
    "valuationAvailable": false,
    "capTableAvailable": false,
    "trancheInfoAvailable": false,
    "debtLenderInfoAvailable": false
},

This is the function I’ve used to replace the none values in the original list (called dealinfo) containing the JSON responses

def replace_none_in_ddict(dealinfo):
    replacement = "0"
    return {k: v if v is not None else replacement for k, v in dealinfo}

json_str = json.dumps(dealinfo)
dealinfo2 = json.loads(json_str, object_pairs_hook=replace_none_in_ddict)
dealinfo3 = json.dumps(dealinfo2)

And this is my attempt to create a dictionary that results in the string TypeError. dealinfo3 contains the JSON responses with the none values replaced with 0.

elements = []
for data in dealinfo3: 
    dealdata = {   
         
        "DealID" : data["dealId"],
        "Deal_Size": data["dealSize"]["amount"],
        "CompanyID" : data["companyId"],        
   }
    elements.append(dealdata)
Asked By: Pballer88

||
Source

Answers:

I have created a minimal example of your input-data like this:

input_data = [
    {
        "dealId": "330843-24T",
        "companyName": "Company A",
        "dealSize": {
            "amount": 1500000.0,
        }},
    {
        "dealId": "4053346-57T",
        "companyName": "Company A",
        "dealSize": {
            "amount": None,
        }
    }]

How do you want to replace your None values? In this solution dealSize is set to 0, if the amount is None.
Feel free to ask any question or correct me if my assumptions were wrong 🙂

output = [{"dealId": deal.get("dealId"), 
           "companyName": deal.get("companyName"), 
           "dealSize": deal.get("dealSize", {}).get("amount") or 0} for deal in input_data]

output:

[{'dealId': '330843-24T', 'companyName': 'Company A', 'dealSize': 1500000.0}, 
 {'dealId': '4053346-57T', 'companyName': 'Company A', 'dealSize': 0}]

EDIT: copy/paste

input_data = [
    {
        "dealId": "330843-24T",
        "companyName": "Company A",
        "dealSize": {
            "amount": 1500000.0,
        }},
    {
        "dealId": "4053346-57T",
        "companyName": "Company A",
        "dealSize": {
            "amount": None,
        }
    }]

output = [{"dealId": deal.get("dealId"),
           "companyName": deal.get("companyName"),
           "dealSize": deal.get("dealSize", {}).get("amount") or 0} for deal in input_data]
Answered By: bitflip

There is "[", what means this json inside the list.
You need modify every element:

for my_json in some_json:
    print(my_json['argument'])

example:

response = r.get(url)
response_text = json.loads(response.text)
for responce_json in responce_text:
    lat = responce_json['lat']
    lon = responce_json['lon']
Answered By: Satisapp

I am not sure exactly the output you want.

if the output like this:

[{‘dealID’: ‘330843-24T’, ‘companyName’: ‘Company A’, ‘dealSize’: 1500000.0}, {‘dealID’: ‘4053346-57T’, ‘companyName’: ‘Company A’, ‘dealSize’: 2800000.0}]

so the code like this:

keys_needed = ['dealID', 'companyName', 'dealSize']

grand_dict = []
for i, d in enumerate(data):
    i = {'dealID': '',
            'companyName': '',
            'dealSize': ''}
            
    i['dealID'] = d.get('dealId')
    i['companyName'] = d.get('companyName')
    i['dealSize'] = d.get('dealSize') and d.get('dealSize').get('amount')
    grand_dict.append(i)

print (grand_dict)
Answered By: Henro Sutrisno Tanjung