我如何检查一个列表是否有任何重复,并返回一个没有重复的新列表?


当前回答

很晚才回答。 如果你不关心列表顺序,你可以使用*arg扩展集唯一性来删除dupes,即:

l = [*{*l}]

Python3演示

其他回答

它需要安装一个第三方模块,但包iteration_utilities包含一个unique_everseen1函数,可以删除所有重复的同时保留顺序:

>>> from iteration_utilities import unique_everseen

>>> list(unique_everseen(['a', 'b', 'c', 'd'] + ['a', 'c', 'd']))
['a', 'b', 'c', 'd']

如果你想避免列表添加操作的开销,你可以使用itertools。链:

>>> from itertools import chain
>>> list(unique_everseen(chain(['a', 'b', 'c', 'd'], ['a', 'c', 'd'])))
['a', 'b', 'c', 'd']

unique_everseen也适用于列表中有不可哈希项(例如列表)的情况:

>>> from iteration_utilities import unique_everseen
>>> list(unique_everseen([['a'], ['b'], 'c', 'd'] + ['a', 'c', 'd']))
[['a'], ['b'], 'c', 'd', 'a']

然而,这将比项目是可哈希的(多)慢。


1披露:我是iteration_utilities-library的作者。

要删除重复的,将其设置为SET,然后再次将其设置为LIST,并打印/使用它。 一个集合保证有唯一的元素。例如:

a = [1,2,3,4,5,9,11,15]
b = [4,5,6,7,8]
c=a+b
print c
print list(set(c)) #one line for getting unique elements of c

输出将如下所示(在python 2.7中检查)

[1, 2, 3, 4, 5, 9, 11, 15, 4, 5, 6, 7, 8]  #simple list addition with duplicates
[1, 2, 3, 4, 5, 6, 7, 8, 9, 11, 15] #duplicates removed!!

与回复中列出的其他解决方案相比,下面是最快的python解决方案。

使用短路计算的实现细节允许使用列表理解,这足够快。visit .add(item)总是返回None作为结果,它被赋值为False,所以or的右边总是这样的表达式的结果。

自己计时

def deduplicate(sequence):
    visited = set()
    adder = visited.add  # get rid of qualification overhead
    out = [adder(item) or item for item in sequence if item not in visited]
    return out
def remove_duplicates(A):
   [A.pop(count) for count,elem in enumerate(A) if A.count(elem)!=1]
   return A

用于删除重复项的列表推导

Write a Python program to create a list of numbers by taking input from the user and then remove  the duplicates from the list. You can take input of non-zero numbers, with an appropriate  prompt, from the user until the user enters a zero to create the list assuming that the numbers  are non-zero.  
Sample Input: [10, 34, 18, 10, 12, 34, 18, 20, 25, 20]  
Output: [10, 34, 18, 12, 20, 25] 

 lst = []
print("ENTER ZERO NUMBER FOR EXIT !!!!!!!!!!!!")
print("ENTER LIST ELEMENTS  :: ")
while True:
    n = int(input())
    if n == 0 :
       print("!!!!!!!!!!! EXIT !!!!!!!!!!!!")
       break
    else :
        lst.append(n)
print("LIST ELEMENR ARE :: ",lst)
#dup = set()
uniq = []
for x in lst:
    if x not in uniq:
        uniq.append(x)
       # dup.add(x)
print("UNIQUE ELEMENTS IN LIST ARE :: ",uniq)