Grouping list of dictionary by keys (in a nested dictionary)

Question

I have a list of dictionaries like below (sample):

{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'on_demand_price': {'usd': '40.00'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'on_demand_price': {'eur': '33.33'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'capacity_storage_price': {'usd': '23.00'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'capacity_storage_price': {'eur': '19.17'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastohio', 'data': {'on_demand_price': {'usd': '40.00'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastohio', 'data': {'on_demand_price': {'eur': '33.33'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastohio', 'data': {'capacity_storage_price': {'usd': '23.00'}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastohio', 'data': {'capacity_storage_price': {'eur': '19.17'}}}
{'platform': 'microsoftazure', 'region': 'uksouthlondon', 'data': {'tier': {'enterprise': {'eur': '3.33', 'usd': '4.00', 'gbp': ''}}}}
{'platform': 'microsoftazure', 'region': 'uksouthlondon', 'data': {'tier': {'business-critical': {'eur': '4.50', 'usd': '5.40', 'gbp': ''}}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'tier': {'standard': {'eur': '1.67', 'usd': '2.00', 'gbp': ''}}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'tier': {'enterprise': {'eur': '2.50', 'usd': '3.00', 'gbp': ''}}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastnorthernvirginia', 'data': {'tier': {'business-critical': {'eur': '3.33', 'usd': '4.00', 'gbp': ''}}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastohio', 'data': {'tier': {'standard': {'eur': '1.67', 'usd': '2.00', 'gbp': ''}}}}
{'platform': 'amazonwebservicesaws', 'region': 'useastohio', 'data': {'tier': {'enterprise': {'eur': '2.50', 'usd': '3.00', 'gbp': ''}}}}

That I somehow need to merge to the following (JSON):

{
  "platform": {
    "amazonwebserviceaws": {
      "asiapacificosaka": {
        "on_demand_price": {
          "eur": "1.00",
          "usd": "3.00",
          "gbp": "4.00"
        },
        "capacity_storage_price": {
          "eur": "5.00",
          "usd": "6.00",
          "gbp": "7.00"
        },
        "tier": {
          "standard": {
            "eur": "1.00",
            "usd": "2.00",
            "gbp": "3.00"
          },
          "enterprise": {
            "eur": "4.00",
            "usd": "5.00",
            "gbp": "6.00"
          },
          "business-critical": {
            "eur": "7.00",
            "usd": "8.00",
            "gbp": "9.00"
          }
        }
      },
      "asia_test": {
        "on_demand_price": {
          "eur": "1.00",
          "usd": "1.00",
          "gbp": "1.00"
        },
        "capacity_storage_price": {
          "eur": "1.00",
          "usd": "1.00",
          "gbp": "1.00"
        },
        "tier": {
          "standard": {
            "eur": "1.00",
            "usd": "2.00",
            "gbp": "3.00"
          },
          "enterprise": {
            "eur": "4.00",
            "usd": "5.00",
            "gbp": "6.00"
          },
          "business-critical": {
            "eur": "7.00",
            "usd": "8.00",
            "gbp": "9.00"
          }
        }
      }
    },
    "googlecloudplatform": {
      "europe-west": {
        "on_demand_price": {
          "eur": "1.00",
          "usd": "1.00",
          "gbp": "1.00"
        },
        "capacity_storage_price": {
          "eur": "1.00",
          "usd": "1.00",
          "gbp": "1.00"
        },
        "tier": {
          "standard": {
            "eur": "1.00",
            "usd": "2.00",
            "gbp": "3.00"
          },
          "enterprise": {
            "eur": "4.00",
            "usd": "5.00",
            "gbp": "6.00"
          },
          "business-critical": {
            "eur": "7.00",
            "usd": "8.00",
            "gbp": "9.00"
          }
        }
      },
      "europe_east": {
        "on_demand_price": {
          "eur": "1.00",
          "usd": "1.00",
          "gbp": "1.00"
        },
        "capacity_storage_price": {
          "eur": "1.00",
          "usd": "1.00",
          "gbp": "1.00"
        },
        "tier": {
          "standard": {
            "eur": "1.00",
            "usd": "2.00",
            "gbp": "3.00"
          },
          "enterprise": {
            "eur": "4.00",
            "usd": "5.00",
            "gbp": "6.00"
          },
          "business-critical": {
            "eur": "7.00",
            "usd": "8.00",
            "gbp": "9.00"
          }
        }
      }
    }
  }
}

So I somehow have to group by both platform and region and then append to the data key using either the on_demand_price, capacity_storage_price of tier key.. So far haven’t found anything that works.

Asked By: Ferron Hooi

||

Source

Answer 1

Try (lst is your list of dicts from the question):

out = {}
for dct in lst:
    if "tier" in dct["data"]:
        out.setdefault(dct["platform"], {}).setdefault(
            dct["region"], {}
        ).setdefault("tier", {}).setdefault(
            (n := list(dct["data"]["tier"])[0]), {}
        ).update(
            dct["data"]["tier"][n]
        )
    else:
        out.setdefault(dct["platform"], {}).setdefault(
            dct["region"], {}
        ).setdefault((n := list(dct["data"])[0]), {}).update(dct["data"][n])

print(out)

Prints:

{
    "amazonwebservicesaws": {
        "useastnorthernvirginia": {
            "on_demand_price": {"usd": "40.00", "eur": "33.33"},
            "capacity_storage_price": {"usd": "23.00", "eur": "19.17"},
            "tier": {
                "standard": {"eur": "1.67", "usd": "2.00", "gbp": ""},
                "enterprise": {"eur": "2.50", "usd": "3.00", "gbp": ""},
                "business-critical": {"eur": "3.33", "usd": "4.00", "gbp": ""},
            },
        },
        "useastohio": {
            "on_demand_price": {"usd": "40.00", "eur": "33.33"},
            "capacity_storage_price": {"usd": "23.00", "eur": "19.17"},
            "tier": {
                "standard": {"eur": "1.67", "usd": "2.00", "gbp": ""},
                "enterprise": {"eur": "2.50", "usd": "3.00", "gbp": ""},
            },
        },
    },
    "microsoftazure": {
        "uksouthlondon": {
            "tier": {
                "enterprise": {"eur": "3.33", "usd": "4.00", "gbp": ""},
                "business-critical": {"eur": "4.50", "usd": "5.40", "gbp": ""},
            }
        }
    },
}

Answered By: Andrej Kesely

Grouping list of dictionary by keys (in a nested dictionary)

Question:

Answers: