我需要从一个相当大的SQL Server表(即300,000+行)中删除重复的行。

当然,由于RowID标识字段的存在,这些行不会完全重复。

MyTable

RowID int not null identity(1,1) primary key,
Col1 varchar(20) not null,
Col2 varchar(2048) not null,
Col3 tinyint not null

我该怎么做呢?


当前回答

删除重复记录

在这种情况下,大于操作符删除除第一条记录以外的所有记录

u1 FROM users u1 JOIN users u2 u1的地方。Id > u2.id 和u1.email = u2.email

<小于操作符在这种情况下删除除最后一条记录以外的所有记录

u1 FROM users u1 JOIN users u2 u1的地方。Id < u2.id 和u1.email = u2.email

其他回答

使用这个

WITH tblTemp as
(
SELECT ROW_NUMBER() Over(PARTITION BY Name,Department ORDER BY Name)
   As RowNumber,* FROM <table_name>
)
DELETE FROM tblTemp where RowNumber >1
delete t1
from table t1, table t2
where t1.columnA = t2.columnA
and t1.rowid>t2.rowid

邮政:

delete
from table t1
using table t2
where t1.columnA = t2.columnA
and t1.rowid > t2.rowid

创建另一个由原始值组成的表:

CREATE TABLE table2 AS SELECT *, COUNT(*) FROM table1 GROUP BY name HAVING COUNT (*) > 0

首先,您可以使用MIN()和Group By选择最小RowId。我们将保留这些行。

   SELECT MIN(RowId) as RowId
   FROM MyTable 
   GROUP BY Col1, Col2, Col3

删除那些不在最小RowId使用范围内的RowId

DELETE FROM MyTable WHERE RowId Not IN()

最后的查询:

DELETE FROM MyTable WHERE RowId Not IN(

    SELECT MIN(RowId) as RowId
    FROM MyTable 
    GROUP BY Col1, Col2, Col3
)

你也可以在SQL Fiddle中检查我的答案

对于表结构

MyTable

RowID int not null identity(1,1) primary key,
Col1 varchar(20) not null,
Col2 varchar(2048) not null,
Col3 tinyint not null

删除重复项的查询:

DELETE t1
FROM MyTable t1
INNER JOIN MyTable t2
WHERE t1.RowID > t2.RowID
  AND t1.Col1 = t2.Col1
  AND t1.Col2=t2.Col2
  AND t1.Col3=t2.Col3;

我假设RowID是一种自动递增,其余列有重复的值。