我有一个varchar列的表,我想找到在这个列中有重复值的所有记录。我可以使用什么查询来查找重复项?


当前回答

一个非常晚的贡献…万一这能帮助到以后的任何人…我有一个任务是在一个银行应用程序中找到匹配的交易对(实际上是账户到账户转账的双方),以识别每个账户间转账交易的“从”和“到”,所以我们最终得到了这个:

SELECT 
    LEAST(primaryid, secondaryid) AS transactionid1,
    GREATEST(primaryid, secondaryid) AS transactionid2
FROM (
    SELECT table1.transactionid AS primaryid, 
        table2.transactionid AS secondaryid
    FROM financial_transactions table1
    INNER JOIN financial_transactions table2 
    ON table1.accountid = table2.accountid
    AND table1.transactionid <> table2.transactionid 
    AND table1.transactiondate = table2.transactiondate
    AND table1.sourceref = table2.destinationref
    AND table1.amount = (0 - table2.amount)
) AS DuplicateResultsTable
GROUP BY transactionid1
ORDER BY transactionid1;

The result is that the DuplicateResultsTable provides rows containing matching (i.e. duplicate) transactions, but it also provides the same transaction id's in reverse the second time it matches the same pair, so the outer SELECT is there to group by the first transaction ID, which is done by using LEAST and GREATEST to make sure the two transactionid's are always in the same order in the results, which makes it safe to GROUP by the first one, thus eliminating all the duplicate matches. Ran through nearly a million records and identified 12,000+ matches in just under 2 seconds. Of course the transactionid is the primary index, which really helped.

其他回答

SELECT  *
FROM    mytable mto
WHERE   EXISTS
        (
        SELECT  1
        FROM    mytable mti
        WHERE   mti.varchar_column = mto.varchar_column
        LIMIT 1, 1
        )
ORDER BY varchar_column

这个查询返回完整的记录,而不仅仅是不同的varchar_column。

这个查询不使用COUNT(*)。如果有很多重复项,COUNT(*)是昂贵的,并且不需要整个COUNT(*),只需要知道是否有两行具有相同的值。

这是通过相关查询底部的LIMIT 1,1来实现的(本质上意味着“返回第二行”)。EXISTS只在前面提到的第二行存在时才返回true(即至少有两行具有相同的varchar_column值)。

当然,在varchar_column上建立索引将大大加快此查询的速度。

下面的代码将找到所有使用了不止一次的product_id。每个product_id只能得到一条记录。

SELECT product_id FROM oc_product_reward GROUP BY product_id HAVING count( product_id ) >1

代码取自:http://chandreshrana.blogspot.in/2014/12/find-duplicate-records-based-on-any.html

感谢@novocaine的精彩回答,他的解决方案对我很有效。我稍微改变了它,以包括一个百分比的循环值,这在我的例子中是需要的。以下是修改后的版本。它将百分比减少到小数点后两位。如果你把2改成0,它就不会显示小数,改成1,它就会显示一位小数,以此类推。

SELECT GROUP_CONCAT(id), name, COUNT(*) c, 
COUNT(*) OVER() AS totalRecords, 
CONCAT(FORMAT(COUNT(*)/COUNT(*) OVER()*100,2),'%') as recurringPecentage
FROM table
GROUP BY name
HAVING c > 1

进一步采取@maxyfc的答案,我需要找到所有返回的重复值的行,这样我就可以在MySQL Workbench中编辑它们:

SELECT * FROM table
   WHERE field IN (
     SELECT field FROM table GROUP BY field HAVING count(*) > 1
   ) ORDER BY field

我看到上面的结果和查询将工作良好,如果你需要检查单列值是重复的。比如电子邮件。

但如果你需要检查更多的列,并希望检查结果的组合,那么这个查询将正常工作:

SELECT COUNT(CONCAT(name,email)) AS tot,
       name,
       email
FROM users
GROUP BY CONCAT(name,email)
HAVING tot>1 (This query will SHOW the USER list which ARE greater THAN 1
              AND also COUNT)