Can't flatten lists and remove duplicates when it's a dict value, for each dict key?

Question

I have the following dict:

my_dict = {'a': [['a','b','c'], ['a',1,4]], 'b':[[1,2,3], [1],[8,2,2,1]]}

(The original dict is much larger)

I want to go over all values, merge lists in one and remove duplicates, for each key.

I am doing it with the following method:

merged_dicts_list = [{k:list(set(list(chain.from_iterable(v))))} for k,v in my_dict.items()]

But I keep getting the error:

TypeError: unhashable type: 'dict'

Please advise how to flatten/merge the lists and remove the duplicates + convert back to list to avoid this error.

Finally I want to get:

[{'a': ['a','b','c',1,4]}, {'b': [1,2,3,8]}]

Asked By: SteveS

||

Source

Answer 1

Slightly alternate solution using sum, then converting to set to remove duplicate and finally converting back to list:

>>> {k: list(set(sum(v, []))) for k,v in my_dict.items()}

# Output:
{'a': [1, 4, 'a', 'b', 'c'], 'b': [8, 1, 2, 3]}

But if you want to maintain the order of the values, then conversion to set will not work, you will have to follow alternate approach to remove the duplicates. You can follow How do I remove duplicates from a list, while preserving order? thread for that.

Answered By: ThePyGuy

Answer 2

this works :

for k, v in my_dict.items():
    my_dict[k] =  [item for sublist in v for item in sublist]

Edit : OOPS my bad ! that did not work, now this does :

for k, v in my_dict.items():
    my_dict[k] = list( dict.fromkeys( [item for sublist in v for item in sublist]) )

Output :

{'a': ['a', 'b', 'c', 1, 4], 'b': [1, 2, 3, 8]}

Answered By: grymlin

Answer 3

Finally I want to get:

[{‘a’: [‘a’,’b’,’c’,1,4]}, {‘b’: [1,2,3,8]}]

my_dict = {'a': [['a','b','c'], ['a',1,4]], 'b':[[1,2,3], [1],[8,2,2,1]]}
merged_dicts_list = list(dict())
for k, v in my_dict.items():
    distinct_val = []
    for v_in in v:
        for v_in_in in v_in:
            if v_in_in not in distinct_val:
                distinct_val.append(v_in_in)
    merged_dicts_list.append({k : distinct_val})

print(merged_dicts_list)

Outputs exactly what were you asking for:

[{'a': ['a', 'b', 'c', 1, 4]}, {'b': [1, 2, 3, 8]}]

Answered By: RobertG

Can't flatten lists and remove duplicates when it's a dict value, for each dict key?

Question:

Answers: