有一个表消息,其中包含如下所示的数据:

Id   Name   Other_Columns
-------------------------
1    A       A_data_1
2    A       A_data_2
3    A       A_data_3
4    B       B_data_1
5    B       B_data_2
6    C       C_data_1

如果我按名称从消息组中运行查询select *,我将得到如下结果:

1    A       A_data_1
4    B       B_data_1
6    C       C_data_1

哪个查询将返回以下结果?

3    A       A_data_3
5    B       B_data_2
6    C       C_data_1

也就是说,应该返回每个组中的最后一条记录。

目前,这是我使用的查询:

SELECT
  *
FROM (SELECT
  *
FROM messages
ORDER BY id DESC) AS x
GROUP BY name

但这看起来效率很低。还有其他方法可以达到同样的效果吗?


当前回答

我们将了解如何使用MySQL获取Group By记录中的最后一条记录。例如,如果你有这个帖子的结果集。

id category_id post_title
1 1 Title 1
2 1 Title 2
3 1 Title 3
4 2 Title 4
5 2 Title 5
6 3 Title 6

我想能够得到最后的职位在每个类别是标题3,标题5和标题6。要按类别获取文章,您将使用MySQL Group by键盘。

select * from posts group by category_id

但是我们从这个查询中得到的结果是。

id category_id post_title
1 1 Title 1
4 2 Title 4
6 3 Title 6

组by将始终返回结果集中该组中的第一个记录。

SELECT id, category_id, post_title
FROM posts
WHERE id IN (
    SELECT MAX(id)
    FROM posts
    GROUP BY category_id );

这将返回每个组中id最高的帖子。

id category_id post_title
3 1 Title 3
5 2 Title 5
6 3 Title 6

参考资料

其他回答

显然,有很多不同的方法得到相同的结果,你的问题似乎是什么是一个有效的方法得到最后的结果在每组MySQL。如果你正在处理大量的数据,并且假设你正在使用InnoDB,即使是最新版本的MySQL(比如5.7.21和8.0.4-rc),那么可能没有一个有效的方法来做到这一点。

有时我们需要对超过6000万行的表执行此操作。

对于这些示例,我将使用只有大约150万行的数据,其中查询需要为数据中的所有组找到结果。在我们的实际情况中,我们经常需要返回大约2000个组的数据(假设不需要检查很多数据)。

我将使用以下表格:

CREATE TABLE temperature(
  id INT UNSIGNED NOT NULL AUTO_INCREMENT, 
  groupID INT UNSIGNED NOT NULL, 
  recordedTimestamp TIMESTAMP NOT NULL, 
  recordedValue INT NOT NULL,
  INDEX groupIndex(groupID, recordedTimestamp), 
  PRIMARY KEY (id)
);

CREATE TEMPORARY TABLE selected_group(id INT UNSIGNED NOT NULL, PRIMARY KEY(id)); 

温度表由大约150万条随机记录和100个不同的组填充。 selected_group由这100个组填充(在我们的例子中,所有组通常小于20%)。

由于该数据是随机的,这意味着多行可以具有相同的recordedTimestamps。我们想要的是获得所有选中的组的列表,这些组按groupID顺序排列,每个组的最后一个recordedTimestamp,如果同一个组有多个这样的匹配行,那么这些行的最后一个匹配id。

如果假设MySQL有一个last()函数,它从一个特殊的ORDER BY子句中返回最后一行的值,那么我们可以简单地这样做:

SELECT 
  last(t1.id) AS id, 
  t1.groupID, 
  last(t1.recordedTimestamp) AS recordedTimestamp, 
  last(t1.recordedValue) AS recordedValue
FROM selected_group g
INNER JOIN temperature t1 ON t1.groupID = g.id
ORDER BY t1.recordedTimestamp, t1.id
GROUP BY t1.groupID;

which would only need to examine a few 100 rows in this case as it doesn't use any of the normal GROUP BY functions. This would execute in 0 seconds and hence be highly efficient. Note that normally in MySQL we would see an ORDER BY clause following the GROUP BY clause however this ORDER BY clause is used to determine the ORDER for the last() function, if it was after the GROUP BY then it would be ordering the GROUPS. If no GROUP BY clause is present then the last values will be the same in all of the returned rows.

但是MySQL没有这个,所以让我们看看它有什么不同的想法,并证明这些都不是有效的。

示例1

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue
FROM selected_group g
INNER JOIN temperature t1 ON t1.id = (
  SELECT t2.id
  FROM temperature t2 
  WHERE t2.groupID = g.id
  ORDER BY t2.recordedTimestamp DESC, t2.id DESC
  LIMIT 1
);

这检查了3,009,254行,在5.7.21上花费了0.859秒,在8.0.4-rc上花费了稍长的时间

示例2

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue 
FROM temperature t1
INNER JOIN ( 
  SELECT max(t2.id) AS id   
  FROM temperature t2
  INNER JOIN (
    SELECT t3.groupID, max(t3.recordedTimestamp) AS recordedTimestamp
    FROM selected_group g
    INNER JOIN temperature t3 ON t3.groupID = g.id
    GROUP BY t3.groupID
  ) t4 ON t4.groupID = t2.groupID AND t4.recordedTimestamp = t2.recordedTimestamp
  GROUP BY t2.groupID
) t5 ON t5.id = t1.id;

这检查了1,505,331行,在5.7.21上花费了约1.25秒,在8.0.4-rc上花费了稍长时间

示例3

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue 
FROM temperature t1
WHERE t1.id IN ( 
  SELECT max(t2.id) AS id   
  FROM temperature t2
  INNER JOIN (
    SELECT t3.groupID, max(t3.recordedTimestamp) AS recordedTimestamp
    FROM selected_group g
    INNER JOIN temperature t3 ON t3.groupID = g.id
    GROUP BY t3.groupID
  ) t4 ON t4.groupID = t2.groupID AND t4.recordedTimestamp = t2.recordedTimestamp
  GROUP BY t2.groupID
)
ORDER BY t1.groupID;

这检查了3,009,685行,在5.7.21上花费了约1.95秒,在8.0.4-rc上花费了稍长时间

示例4

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue
FROM selected_group g
INNER JOIN temperature t1 ON t1.id = (
  SELECT max(t2.id)
  FROM temperature t2 
  WHERE t2.groupID = g.id AND t2.recordedTimestamp = (
      SELECT max(t3.recordedTimestamp)
      FROM temperature t3 
      WHERE t3.groupID = g.id
    )
);

这检查了6,137,810行,在5.7.21上花费了约2.2秒,在8.0.4-rc上花费了稍长时间

示例5

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue
FROM (
  SELECT 
    t2.id, 
    t2.groupID, 
    t2.recordedTimestamp, 
    t2.recordedValue, 
    row_number() OVER (
      PARTITION BY t2.groupID ORDER BY t2.recordedTimestamp DESC, t2.id DESC
    ) AS rowNumber
  FROM selected_group g 
  INNER JOIN temperature t2 ON t2.groupID = g.id
) t1 WHERE t1.rowNumber = 1;

这检查了6,017,808行,在8.0.4-rc上花费了约4.2秒

例子6

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue 
FROM (
  SELECT 
    last_value(t2.id) OVER w AS id, 
    t2.groupID, 
    last_value(t2.recordedTimestamp) OVER w AS recordedTimestamp, 
    last_value(t2.recordedValue) OVER w AS recordedValue
  FROM selected_group g
  INNER JOIN temperature t2 ON t2.groupID = g.id
  WINDOW w AS (
    PARTITION BY t2.groupID 
    ORDER BY t2.recordedTimestamp, t2.id 
    RANGE BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
  )
) t1
GROUP BY t1.groupID;

这检查了6,017,908行,在8.0.4-rc上花费了约17.5秒

例7

SELECT t1.id, t1.groupID, t1.recordedTimestamp, t1.recordedValue 
FROM selected_group g
INNER JOIN temperature t1 ON t1.groupID = g.id
LEFT JOIN temperature t2 
  ON t2.groupID = g.id 
  AND (
    t2.recordedTimestamp > t1.recordedTimestamp 
    OR (t2.recordedTimestamp = t1.recordedTimestamp AND t2.id > t1.id)
  )
WHERE t2.id IS NULL
ORDER BY t1.groupID;

这只花了很长时间,所以我不得不杀了它。

SELECT 
  column1,
  column2 
FROM
  table_name 
WHERE id IN 
  (SELECT 
    MAX(id) 
  FROM
    table_name 
  GROUP BY column1) 
ORDER BY column1 ;

我还没有测试大DB,但我认为这可能比连接表更快:

SELECT *, Max(Id) FROM messages GROUP BY Name

使用子查询返回正确的分组,因为您已经完成了一半。

试试这个:

select
    a.*
from
    messages a
    inner join 
        (select name, max(id) as maxid from messages group by name) as b on
        a.id = b.maxid

如果它不是id,你想要的最大值:

select
    a.*
from
    messages a
    inner join 
        (select name, max(other_col) as other_col 
         from messages group by name) as b on
        a.name = b.name
        and a.other_col = b.other_col

通过这种方式,可以避免在子查询中进行相关子查询和/或排序,这往往非常缓慢/低效。

下面是一个更有效的版本,只需一行,只要表有时间戳列即可。

SELECT Id, Name, SUBSTRING_INDEX(MAX(CONCAT(TimeStamp, ',', Other_Columns)), ',', -1)
FROM Messages
ORDER BY id DESC GROUP BY Name 

这将返回“Other_Columns”上组的最新记录