remove lists with same elements but in different order, from a list of lists

Question

I want to filter a list of lists for duplicates. I consider two lists to be a duplicate of each other when they contain the same elements but not necessarily in the same order. So for example

[['A', 'B', 'C'], ['C', 'B', 'A'], ['D', 'B', 'A']]

should become

[['A', 'B', 'C'], ['D', 'B', 'A']]

since ['C', 'B', 'A'] is a duplicate of ['A', 'B', 'C'].
It does not matter which one of the duplicates gets removed, as long as the final list of lists does not contain any duplicates anymore. And all lists need to keep the order of there elements. So using set() may not be an option.

I found this related questions:
Determine if 2 lists have the same elements, regardless of order? ,
How to efficiently compare two unordered lists (not sets)?
But they only talk about how to compare two lists, not how too efficiently remove duplicates. I’m using python.

Asked By: Akut Luna

||

Source

Answer 1

You can use frozenset

>>> x = [['A', 'B', 'C'], ['C', 'B', 'A'], ['D', 'B', 'A']]
>>> [list(s) for s in set([frozenset(item) for item in x])]
[['A', 'B', 'D'], ['A', 'B', 'C']]

Or, with map:

>>> [list(s) for s in set(map(frozenset, x))]
[['A', 'B', 'D'], ['A', 'B', 'C']]

Answered By: The Thonnu

Answer 2

using dictionary comprehension

>>> data = [['A', 'B', 'C'], ['C', 'B', 'A'], ['D', 'B', 'A']]
>>> result = {tuple(sorted(i)): i for i in data}.values()
>>> result 
dict_values([['C', 'B', 'A'], ['D', 'B', 'A']])
>>> list( result )
[['C', 'B', 'A'], ['D', 'B', 'A']]

Answered By: sahasrara62

Answer 3

Do you want to keep the order of elements?

from itertools import groupby
data = [['A', 'B', 'C'], ['C', 'B', 'A'], ['D', 'B', 'A']]

print([k for k, _ in groupby(data, key=sorted)])

Output:

[['A', 'B', 'C'], ['A', 'B', 'D']]

Answered By: funnydman

Answer 4

In python you have to remember that you can’t change existing data but you can somehow append / update data.

The simplest way is as follows:

dict = [['A', 'B', 'C'], ['C', 'B', 'A'], ['D', 'B', 'A']]

temp = []
for i in dict:
    if sorted(i) in temp:
        pass
    else:
        temp.append(i)
print(temp)

cheers, athrv

Answered By: athrvvv

remove lists with same elements but in different order, from a list of lists

Question:

Answers: