How to ignore duplicate string?

Question

I had some script below

unique=[]

thislist =['idn_type1=：111-22-3333',
'idn_type1=：111-22-3333',
'idn_type1=：222-33-4444',
'idn_type2=：333-44-5555',
'idn_type2=：222-33-4444',
'idn_type3=：222-33-4444',
'idn_type4=：222-33-4444',
'idn_type1=：',
'idn_type1=：444-55-6666',
'idn_type1=：555-66-7777',
'idn_type1=：']

for name in thislist:
      if name not in unique and name.partition('：')[-1] not in unique and name.partition('：')[-1]!='':
            unique.append(name)

for i in range(len(unique)):
    print(unique[i])

And the result is wrong

idn_type1=：111-22-3333
idn_type1=：222-33-4444
idn_type2=：333-44-5555
idn_type2=：222-33-4444
idn_type3=：222-33-4444
idn_type4=：222-33-4444
idn_type1=：444-55-6666
idn_type1=：555-66-7777

But what I want is

idn_type1=：111-22-3333
idn_type1=：222-33-4444
idn_type2=：333-44-5555
idn_type1=：444-55-6666
idn_type1=：555-66-7777

I want idn '222-33-4444' append only once when it meets first with idn_type1.
Only idn_type1=：222-33-4444 appended to the new list.

How to modify this script?

Thank Mr.Martineau to correct my description!
https://stackoverflow.com/users/355230/martineau

Asked By: Stanley1

||

Source

Answer 1

you can use set to remember the numbers you already saw

unique = []

thislist = ['idn_type1=：111-22-3333',
            'idn_type1=：111-22-3333',
            'idn_type1=：222-33-4444',
            'idn_type2=：333-44-5555',
            'idn_type2=：222-33-4444',
            'idn_type3=：222-33-4444',
            'idn_type4=：222-33-4444',
            'idn_type1=：',
            'idn_type1=：444-55-6666',
            'idn_type1=：555-66-7777',
            'idn_type1=：']

seen = set()
for name in thislist:
    numbers = name.partition('：')[-1]
    if name not in unique and numbers not in seen and numbers != '':
        unique.append(name)
        seen.add(numbers)

for i in range(len(unique)):
    print(unique[i])

output:

idn_type1=：111-22-3333
idn_type1=：222-33-4444
idn_type2=：333-44-5555
idn_type1=：444-55-6666
idn_type1=：555-66-7777

hope i could help, if you need me to clarify anything feel free to ask in the comments 🙂

Answered By: Ohad Sharet

Answer 2

use set to remove duplicate elements in your list.

thislist =['idn_type1=：111-22-3333',
'idn_type1=：111-22-3333',
'idn_type1=：222-33-4444'
,
'idn_type2=：333-44-5555',
'idn_type2=：222-33-4444',
'idn_type3=：222-33-4444',
'idn_type4=：222-33-4444',
'idn_type1=：',
'idn_type1=：444-55-6666',
'idn_type1=：555-66-7777',
'idn_type1=：']

unique = list(set(thislist))

for each in unique:
    print(each)

Answered By: Amandeep Singh

How to ignore duplicate string?

Question:

Answers: