我有一个PostgreSQL数据库表称为“user_links”,目前允许以下重复字段:

year, user_id, sid, cid

唯一的约束目前是第一个字段称为“id”,但我现在希望添加一个约束,以确保年份,user_id, sid和cid都是唯一的,但我不能应用约束,因为重复的值已经存在,违反这一约束。

有没有办法找到所有的副本?


当前回答

为了方便起见,我假设您希望仅对列year应用唯一约束,并且主键是名为id的列。

为了找到重复的值,您应该运行,

SELECT year, COUNT(id)
FROM YOUR_TABLE
GROUP BY year
HAVING COUNT(id) > 1
ORDER BY COUNT(id);

使用上面的sql语句,您将得到一个包含表中所有重复年份的表。为了删除除最新重复条目外的所有重复条目,您应该使用上面的sql语句。

DELETE
FROM YOUR_TABLE A USING YOUR_TABLE_AGAIN B
WHERE A.year=B.year AND A.id<B.id;

其他回答

为了方便起见,我假设您希望仅对列year应用唯一约束,并且主键是名为id的列。

为了找到重复的值,您应该运行,

SELECT year, COUNT(id)
FROM YOUR_TABLE
GROUP BY year
HAVING COUNT(id) > 1
ORDER BY COUNT(id);

使用上面的sql语句,您将得到一个包含表中所有重复年份的表。为了删除除最新重复条目外的所有重复条目,您应该使用上面的sql语句。

DELETE
FROM YOUR_TABLE A USING YOUR_TABLE_AGAIN B
WHERE A.year=B.year AND A.id<B.id;

您可以在将被复制的字段上连接到同一个表,然后在id字段上反连接。从第一个表别名(tn1)中选择id字段,然后对第二个表别名的id字段使用array_agg函数。最后,为了使array_agg函数正常工作,您将根据tn1对结果进行分组。id字段。这将产生一个结果集,其中包含记录的id和符合连接条件的所有id的数组。

select tn1.id,
       array_agg(tn2.id) as duplicate_entries, 
from table_name tn1 join table_name tn2 on 
    tn1.year = tn2.year 
    and tn1.sid = tn2.sid 
    and tn1.user_id = tn2.user_id 
    and tn1.cid = tn2.cid
    and tn1.id <> tn2.id
group by tn1.id;

显然,在duplicate_entries数组中的id在结果集中也有自己的条目。你必须使用这个结果集来决定你想要哪个id成为“真相”的来源。唯一不应该被删除的记录。也许你可以这样做:

with dupe_set as (
select tn1.id,
       array_agg(tn2.id) as duplicate_entries, 
from table_name tn1 join table_name tn2 on 
    tn1.year = tn2.year 
    and tn1.sid = tn2.sid 
    and tn1.user_id = tn2.user_id 
    and tn1.cid = tn2.cid
    and tn1.id <> tn2.id
group by tn1.id
order by tn1.id asc)
select ds.id from dupe_set ds where not exists 
 (select de from unnest(ds.duplicate_entries) as de where de < ds.id)

选择具有重复的最小数字ID(假设ID在PK中递增)。这些就是你要保存的ID。

受到Sandro Wiggers的启发,我做了一些类似的事情

WITH ordered AS ( 
  SELECT id,year, user_id, sid, cid,
    rank() OVER (PARTITION BY year, user_id, sid, cid ORDER BY id) AS rnk 
  FROM user_links 
), 
to_delete AS ( 
  SELECT id
  FROM   ordered 
  WHERE  rnk > 1
) 
DELETE 
FROM user_links
USING to_delete 
WHERE user_link.id = to_delete.id;

如果你想测试它,稍微改变一下:

WITH ordered AS ( 
  SELECT id,year, user_id, sid, cid,
    rank() OVER (PARTITION BY year, user_id, sid, cid ORDER BY id) AS rnk 
  FROM user_links 
), 
to_delete AS ( 
  SELECT id,year,user_id,sid, cid
  FROM   ordered 
  WHERE  rnk > 1
) 
SELECT * FROM to_delete;

这将给出将要删除的内容的概述(在运行删除时,在to_delete查询中保留year,user_id,sid,cid是没有问题的,但随后它们就不需要了)

在您的情况下,由于限制,您需要删除重复的记录。

查找重复的行 根据created_at日期组织它们——在本例中,我保留了最老的日期 使用USING删除记录以过滤正确的行

WITH duplicated AS ( 
    SELECT id,
        count(*) 
    FROM products 
    GROUP BY id 
    HAVING count(*) > 1), 
ordered AS ( 
    SELECT p.id, 
        created_at, 
        rank() OVER (partition BY p.id ORDER BY p.created_at) AS rnk 
    FROM products o 
    JOIN     duplicated d ON d.id = p.id ), 
products_to_delete AS ( 
    SELECT id, 
        created_at 
    FROM   ordered 
    WHERE  rnk = 2
) 
DELETE 
FROM products 
USING products_to_delete 
WHERE products.id = products_to_delete.id 
    AND products.created_at = products_to_delete.created_at;

基本思想将使用一个嵌套查询计数聚合:

select * from yourTable ou
where (select count(*) from yourTable inr
where inr.sid = ou.sid) > 1

您可以调整内部查询中的where子句来缩小搜索范围。


评论中提到了另一个很好的解决方案,(但不是每个人都读它们):

select Column1, Column2, count(*)
from yourTable
group by Column1, Column2
HAVING count(*) > 1

或更短:

SELECT (yourTable.*)::text, count(*)
FROM yourTable
GROUP BY yourTable.*
HAVING count(*) > 1