有一个表消息,其中包含如下所示的数据:

Id   Name   Other_Columns
-------------------------
1    A       A_data_1
2    A       A_data_2
3    A       A_data_3
4    B       B_data_1
5    B       B_data_2
6    C       C_data_1

如果我按名称从消息组中运行查询select *,我将得到如下结果:

1    A       A_data_1
4    B       B_data_1
6    C       C_data_1

哪个查询将返回以下结果?

3    A       A_data_3
5    B       B_data_2
6    C       C_data_1

也就是说,应该返回每个组中的最后一条记录。

目前,这是我使用的查询:

SELECT
  *
FROM (SELECT
  *
FROM messages
ORDER BY id DESC) AS x
GROUP BY name

但这看起来效率很低。还有其他方法可以达到同样的效果吗?


当前回答

希望以下Oracle查询能有所帮助:

WITH Temp_table AS
(
    Select id, name, othercolumns, ROW_NUMBER() over (PARTITION BY name ORDER BY ID 
    desc)as rank from messages
)
Select id, name,othercolumns from Temp_table where rank=1

其他回答

MySQL 8.0现在支持窗口函数,就像几乎所有流行的SQL实现一样。使用这个标准语法,我们可以编写每组最大n个查询:

WITH ranked_messages AS (
  SELECT m.*, ROW_NUMBER() OVER (PARTITION BY name ORDER BY id DESC) AS rn
  FROM messages AS m
)
SELECT * FROM ranked_messages WHERE rn = 1;

这种方法和其他查找分组最大行的方法在MySQL手册中有说明。

以下是我在2009年写的关于这个问题的原始答案:


我这样写解:

SELECT m1.*
FROM messages m1 LEFT JOIN messages m2
 ON (m1.name = m2.name AND m1.id < m2.id)
WHERE m2.id IS NULL;

关于性能,一种解决方案或另一种解决方案可能更好,这取决于数据的性质。因此,您应该同时测试这两个查询,并在给定数据库的情况下使用性能更好的查询。

例如,我有一个StackOverflow八月数据转储的副本。我将使用它进行基准测试。Posts表中有1,114,357行。这是在我的Macbook Pro 2.40GHz的MySQL 5.0.75上运行的。

我将编写一个查询,为给定的用户ID(我的)查找最近的帖子。

首先在子查询中使用@Eric所展示的GROUP by技术:

SELECT p1.postid
FROM Posts p1
INNER JOIN (SELECT pi.owneruserid, MAX(pi.postid) AS maxpostid
            FROM Posts pi GROUP BY pi.owneruserid) p2
  ON (p1.postid = p2.maxpostid)
WHERE p1.owneruserid = 20860;

1 row in set (1 min 17.89 sec)

即使是EXPLAIN分析也需要超过16秒:

+----+-------------+------------+--------+----------------------------+-------------+---------+--------------+---------+-------------+
| id | select_type | table      | type   | possible_keys              | key         | key_len | ref          | rows    | Extra       |
+----+-------------+------------+--------+----------------------------+-------------+---------+--------------+---------+-------------+
|  1 | PRIMARY     | <derived2> | ALL    | NULL                       | NULL        | NULL    | NULL         |   76756 |             | 
|  1 | PRIMARY     | p1         | eq_ref | PRIMARY,PostId,OwnerUserId | PRIMARY     | 8       | p2.maxpostid |       1 | Using where | 
|  2 | DERIVED     | pi         | index  | NULL                       | OwnerUserId | 8       | NULL         | 1151268 | Using index | 
+----+-------------+------------+--------+----------------------------+-------------+---------+--------------+---------+-------------+
3 rows in set (16.09 sec)

现在使用我的LEFT JOIN技术生成相同的查询结果:

SELECT p1.postid
FROM Posts p1 LEFT JOIN posts p2
  ON (p1.owneruserid = p2.owneruserid AND p1.postid < p2.postid)
WHERE p2.postid IS NULL AND p1.owneruserid = 20860;

1 row in set (0.28 sec)

EXPLAIN分析表明这两个表都可以使用它们的索引:

+----+-------------+-------+------+----------------------------+-------------+---------+-------+------+--------------------------------------+
| id | select_type | table | type | possible_keys              | key         | key_len | ref   | rows | Extra                                |
+----+-------------+-------+------+----------------------------+-------------+---------+-------+------+--------------------------------------+
|  1 | SIMPLE      | p1    | ref  | OwnerUserId                | OwnerUserId | 8       | const | 1384 | Using index                          | 
|  1 | SIMPLE      | p2    | ref  | PRIMARY,PostId,OwnerUserId | OwnerUserId | 8       | const | 1384 | Using where; Using index; Not exists | 
+----+-------------+-------+------+----------------------------+-------------+---------+-------+------+--------------------------------------+
2 rows in set (0.00 sec)

下面是我Posts表的DDL:

CREATE TABLE `posts` (
  `PostId` bigint(20) unsigned NOT NULL auto_increment,
  `PostTypeId` bigint(20) unsigned NOT NULL,
  `AcceptedAnswerId` bigint(20) unsigned default NULL,
  `ParentId` bigint(20) unsigned default NULL,
  `CreationDate` datetime NOT NULL,
  `Score` int(11) NOT NULL default '0',
  `ViewCount` int(11) NOT NULL default '0',
  `Body` text NOT NULL,
  `OwnerUserId` bigint(20) unsigned NOT NULL,
  `OwnerDisplayName` varchar(40) default NULL,
  `LastEditorUserId` bigint(20) unsigned default NULL,
  `LastEditDate` datetime default NULL,
  `LastActivityDate` datetime default NULL,
  `Title` varchar(250) NOT NULL default '',
  `Tags` varchar(150) NOT NULL default '',
  `AnswerCount` int(11) NOT NULL default '0',
  `CommentCount` int(11) NOT NULL default '0',
  `FavoriteCount` int(11) NOT NULL default '0',
  `ClosedDate` datetime default NULL,
  PRIMARY KEY  (`PostId`),
  UNIQUE KEY `PostId` (`PostId`),
  KEY `PostTypeId` (`PostTypeId`),
  KEY `AcceptedAnswerId` (`AcceptedAnswerId`),
  KEY `OwnerUserId` (`OwnerUserId`),
  KEY `LastEditorUserId` (`LastEditorUserId`),
  KEY `ParentId` (`ParentId`),
  CONSTRAINT `posts_ibfk_1` FOREIGN KEY (`PostTypeId`) REFERENCES `posttypes` (`PostTypeId`)
) ENGINE=InnoDB;

评论者注意:如果你想用不同版本的MySQL,不同的数据集,或者不同的表设计来做另一个基准测试,你可以自己做。我已经展示了上面的技术。Stack Overflow在这里向您展示如何进行软件开发工作,而不是为您做所有的工作。

MariaDB 10.3及更新版本使用GROUP_CONCAT。

这个想法是使用ORDER BY + LIMIT:

SELECT GROUP_CONCAT(id ORDER BY id DESC LIMIT 1) AS id,
       name,
       GROUP_CONCAT(Other_columns ORDER BY id DESC LIMIT 1) AS Other_columns
FROM t
GROUP BY name;

db < > fiddle演示

这里有两个建议。首先,如果mysql支持ROW_NUMBER(),这很简单:

WITH Ranked AS (
  SELECT Id, Name, OtherColumns,
    ROW_NUMBER() OVER (
      PARTITION BY Name
      ORDER BY Id DESC
    ) AS rk
  FROM messages
)
  SELECT Id, Name, OtherColumns
  FROM messages
  WHERE rk = 1;

我猜你说的"最后"是指最后一个。如果不是,则相应地更改ROW_NUMBER()窗口的ORDER BY子句。如果ROW_NUMBER()不可用,这是另一个解决方案:

其次,如果没有,这通常是一个很好的方法:

SELECT
  Id, Name, OtherColumns
FROM messages
WHERE NOT EXISTS (
  SELECT * FROM messages as M2
  WHERE M2.Name = messages.Name
  AND M2.Id > messages.Id
)

换句话说,选择没有相同名称的later-Id消息的消息。

另一种方法:

找到每个程序中最大m2_price的属性(一个程序中有n个属性):

select * from properties p
join (
    select max(m2_price) as max_price 
    from properties 
    group by program_id
) p2 on (p.program_id = p2.program_id)
having p.m2_price = max_price

UPD: 2017-03-31, MySQL 5.7.5版本默认启用了ONLY_FULL_GROUP_BY开关(因此,不确定的GROUP by查询被禁用)。此外,他们更新了GROUP BY实现,即使禁用了开关,解决方案也可能不再像预期的那样工作。我们需要检查一下。

Bill Karwin的上述解决方案在组内的项目计数相当小时工作得很好,但当组相当大时查询性能就会变差,因为该解决方案只需要n*n/2 + n/2个is NULL比较。

我在一个包含18684446行和1182个组的InnoDB表上进行了测试。该表包含功能测试的测试结果,并将(test_id, request_id)作为主键。因此,test_id是一个组,我正在为每个test_id搜索最后的request_id。

Bill的解决方案已经在我的dell e4310上运行了几个小时,我不知道它什么时候会完成,即使它在覆盖索引上运行(因此在EXPLAIN中使用索引)。

基于同样的想法,我有一些其他的解决方案:

if the underlying index is BTREE index (which is usually the case), the largest (group_id, item_value) pair is the last value within each group_id, that is the first for each group_id if we walk through the index in descending order; if we read the values which are covered by an index, the values are read in the order of the index; each index implicitly contains primary key columns appended to that (that is the primary key is in the coverage index). In solutions below I operate directly on the primary key, in you case, you will just need to add primary key columns in the result. in many cases it is much cheaper to collect the required row ids in the required order in a subquery and join the result of the subquery on the id. Since for each row in the subquery result MySQL will need a single fetch based on primary key, the subquery will be put first in the join and the rows will be output in the order of the ids in the subquery (if we omit explicit ORDER BY for the join)

MySQL使用索引的3种方式是一篇了解一些细节的好文章。

解决方案1

这个是非常快的,在我的18M+行上大约需要0.8秒:

SELECT test_id, MAX(request_id) AS request_id
FROM testresults
GROUP BY test_id DESC;

如果你想改变顺序为ASC,把它放在一个子查询中,只返回id,并使用它作为子查询连接到其他列:

SELECT test_id, request_id
FROM (
    SELECT test_id, MAX(request_id) AS request_id
    FROM testresults
    GROUP BY test_id DESC) as ids
ORDER BY test_id;

在我的数据上,这个大约需要1,2秒。

解决方案2

下面是另一个解决方案,对我的表来说大约需要19秒:

SELECT test_id, request_id
FROM testresults, (SELECT @group:=NULL) as init
WHERE IF(IFNULL(@group, -1)=@group:=test_id, 0, 1)
ORDER BY test_id DESC, request_id DESC

它还按降序返回测试。它的速度要慢得多,因为它做了一个完整的索引扫描,但它在这里给你一个想法,如何为每个组输出最多N行。

查询的缺点是查询缓存不能缓存其结果。