我没有看到一个纯粹使用迭代器的解决方案，所以我们开始吧

这需要对列表进行排序，这可能是这里的缺点。

a = [1,2,3,2,1,5,6,5,5,5]
a.sort()
set(map(lambda x: x[0], filter(lambda x: x[0] == x[1], zip(a, a[1:]))))

{1, 2, 5}

你可以用这段代码轻松检查你的机器有多快，有一百万潜在的重复:

首先生成数据

import random
from itertools import chain
a = list(chain(*[[n] * random.randint(1, 2) for n in range(1000000)]))

并运行测试:

set(map(lambda x: x[0], filter(lambda x: x[0] == x[1], zip(a, a[1:]))))

不用说，这个解决方案只在列表已经排序的情况下才有效。

使用Set函数 如:-

arr=[1,4,2,5,2,3,4,1,4,5,2,3]
arr2=list(set(arr))
print(arr2)

输出:- [1,2,3,4,5]

使用array删除副本

eg:-

arr=[1,4,2,5,2,3,4,1,4,5,2,3]
arr3=[]
for i in arr:
    if(i not in arr3):
     arr3.append(i)
print(arr3)

输出: [1,4,2,5,3]

使用Lambda函数

eg:-

rem_duplicate_func=lambda arr:set(arr)
print(rem_duplicate_func(arr))

输出: {1,2,3,4,5}

从字典中删除重复值

eg:-

dict1={
    'car':["Ford","Toyota","Ford","Toyota"],
    'brand':["Mustang","Ranz","Mustang","Ranz"] } dict2={} for key,value in dict1.items():
    dict2[key]=set(value) print(dict2)

输出: {“车”:{“丰田”、“福特”},“品牌”:{“主攻”、“野马”}}

对称差异-删除重复元素

eg:-

set1={1,2,4,5}
set2={2,1,5,7}
rem_dup_ele=set1.symmetric_difference(set2)
print(rem_dup_ele)

输出: {4 7}

def removeduplicates(a):
  seen = set()

  for i in a:
    if i not in seen:
      seen.add(i)
  return seen 

print(removeduplicates([1,1,2,2]))

some_list = ['a', 'b', 'c', 'b', 'd', 'm', 'n', 'n']
some_dictionary = {}

for element in some_list:
    if element not in some_dictionary:
       some_dictionary[element] = 1
    else:
        some_dictionary[element] += 1

for key, value in some_dictionary.items():
    if value > 1:
       print(key, end = ' ')

# another way
duplicates = []

for x in some_list:
    if some_list.count(x) > 1 and x not in duplicates:
        duplicates.append(x)

print()
print(duplicates)

来源:这里

list2 = [1, 2, 3, 4, 1, 2, 3]
lset = set()
[(lset.add(item), list2.append(item))
 for item in list2 if item not in lset]
print list(lset)

我必须这样做，因为我挑战自己不使用其他方法:

def dupList(oldlist):
    if type(oldlist)==type((2,2)):
        oldlist=[x for x in oldlist]
    newList=[]
    newList=newList+oldlist
    oldlist=oldlist
    forbidden=[]
    checkPoint=0
    for i in range(len(oldlist)):
        #print 'start i', i
        if i in forbidden:
            continue
        else:
            for j in range(len(oldlist)):
                #print 'start j', j
                if j in forbidden:
                    continue
                else:
                    #print 'after Else'
                    if i!=j: 
                        #print 'i,j', i,j
                        #print oldlist
                        #print newList
                        if oldlist[j]==oldlist[i]:
                            #print 'oldlist[i],oldlist[j]', oldlist[i],oldlist[j]
                            forbidden.append(j)
                            #print 'forbidden', forbidden
                            del newList[j-checkPoint]
                            #print newList
                            checkPoint=checkPoint+1
    return newList

所以你的样本工作如下:

>>>a = [1,2,3,3,3,4,5,6,6,7]
>>>dupList(a)
[1, 2, 3, 4, 5, 6, 7]