convert Python list to ordered unique values

Question

I encounter many tasks in which I need to filter python (2.7) list to keep only ordered unique values. My usual approach is by using odereddict from collections:

from collections import OrderedDict

ls = [1,2,3,4,1,23,4,12,3,41]

ls = OrderedDict(zip(ls,['']*len(ls))).keys()

print ls

the output is:

[1, 2, 3, 4, 23, 12, 41]

is there any other state of the art method to do it in Python?

Note – the input and the output should be given as list

edit – a comparison of the methods can be found here:
https://www.peterbe.com/plog/uniqifiers-benchmark

the best solution meanwhile is:

def get_unique(seq):
    seen = set()
    seen_add = seen.add
    return [x for x in seq if not (x in seen or seen_add(x))]

Asked By: Dimgold

||

Source

Answer 1

If you need to preserve the order and get rid of the duplicates, you can do it like:

ls = [1, 2, 3, 4, 1, 23, 4, 12, 3, 41]

lookup = set()  # a temporary lookup set
ls = [x for x in ls if x not in lookup and lookup.add(x) is None]
# [1, 2, 3, 4, 23, 12, 41]

This should be considerably faster than your approach.

Answered By: zwer

Answer 2

You could use a set like this:

newls = []
seen = set()

for elem in ls:
    if not elem in seen:
        newls.append(elem)
        seen.add(elem)

Answered By: Eugene Yarmash

Answer 3

Define a function to do so:

def uniques(l):
    retl = []
    for x in l:
        if x not in retl:
            retl.append(x)
    return retl
ls = [1,2,3,4,1,23,4,12,3,41]
uniques(ls)
[1, 2, 3, 4, 23, 12, 41]

Answered By: Netwave

Answer 4

Another solution would be using list comprehension like this:

[x for i, x in enumerate(ls) if x not in ls[:i]]

Output:

[1, 2, 3, 4, 23, 12, 41]

Answered By: Carles Mitjans

convert Python list to ordered unique values

Question:

Answers: