如何在纯SQL中请求随机行(或尽可能接近真正的随机)?


当前回答

我不得不同意CD-MaN:使用“ORDER BY RAND()”将很好地用于小表或当你只做几次SELECT时。

我还使用“num_value >= RAND() *…”技术,如果我真的想获得随机结果,我在表中有一个特殊的“随机”列,我大约每天更新一次。单个UPDATE运行将花费一些时间(特别是因为必须在该列上建立索引),但它比每次运行select时为每一行创建随机数快得多。

其他回答

晚了,但通过谷歌到达这里,所以为了子孙后代,我将添加一个替代解决方案。

另一种方法是使用TOP两次,顺序交替。我不知道它是否是“纯SQL”,因为它在TOP中使用了一个变量,但它在SQL Server 2008中工作。这里有一个例子,如果我想要一个随机的单词,我使用字典单词表。

SELECT TOP 1
  word
FROM (
  SELECT TOP(@idx)
    word 
  FROM
    dbo.DictionaryAbridged WITH(NOLOCK)
  ORDER BY
    word DESC
) AS D
ORDER BY
  word ASC

当然,@idx是目标表上从1到COUNT(*)的随机生成的整数。如果您的列被索引,您也会从中受益。另一个优点是可以在函数中使用它,因为NEWID()是不允许的。

最后,在同一个表上,上述查询的执行时间大约是NEWID()类型查询的1/10。YYMV。

我不得不同意CD-MaN:使用“ORDER BY RAND()”将很好地用于小表或当你只做几次SELECT时。

我还使用“num_value >= RAND() *…”技术,如果我真的想获得随机结果,我在表中有一个特殊的“随机”列,我大约每天更新一次。单个UPDATE运行将花费一些时间(特别是因为必须在该列上建立索引),但它比每次运行select时为每一行创建随机数快得多。

SQL Server

Newid ()/order by可以工作,但对于大型结果集来说代价非常高,因为它必须为每一行生成一个id,然后对它们进行排序。

从性能的角度来看,TABLESAMPLE()很好,但是您将得到结果的聚集(将返回页面上的所有行)。

为了获得更好的真实随机样本,最好的方法是随机过滤掉行。我在SQL Server Books Online文章使用TABLESAMPLE限制结果集中找到了以下代码示例:

If you really want a random sample of individual rows, modify your query to filter out rows randomly, instead of using TABLESAMPLE. For example, the following query uses the NEWID function to return approximately one percent of the rows of the Sales.SalesOrderDetail table: SELECT * FROM Sales.SalesOrderDetail WHERE 0.01 >= CAST(CHECKSUM(NEWID(),SalesOrderID) & 0x7fffffff AS float) / CAST (0x7fffffff AS int) The SalesOrderID column is included in the CHECKSUM expression so that NEWID() evaluates once per row to achieve sampling on a per-row basis. The expression CAST(CHECKSUM(NEWID(), SalesOrderID) & 0x7fffffff AS float / CAST (0x7fffffff AS int) evaluates to a random float value between 0 and 1.

当对一个有1,000,000行的表运行时,下面是我的结果:

SET STATISTICS TIME ON
SET STATISTICS IO ON

/* newid()
   rows returned: 10000
   logical reads: 3359
   CPU time: 3312 ms
   elapsed time = 3359 ms
*/
SELECT TOP 1 PERCENT Number
FROM Numbers
ORDER BY newid()

/* TABLESAMPLE
   rows returned: 9269 (varies)
   logical reads: 32
   CPU time: 0 ms
   elapsed time: 5 ms
*/
SELECT Number
FROM Numbers
TABLESAMPLE (1 PERCENT)

/* Filter
   rows returned: 9994 (varies)
   logical reads: 3359
   CPU time: 641 ms
   elapsed time: 627 ms
*/    
SELECT Number
FROM Numbers
WHERE 0.01 >= CAST(CHECKSUM(NEWID(), Number) & 0x7fffffff AS float) 
              / CAST (0x7fffffff AS int)

SET STATISTICS IO OFF
SET STATISTICS TIME OFF

如果您可以使用TABLESAMPLE,它将为您提供最佳性能。否则使用newwid ()/filter方法。如果结果集很大,Newid ()/order by应该是最后的选择。

火鸟:

Select FIRST 1 column from table ORDER BY RAND()

对于SQL Server和需要“单个随机行”..

如果不需要真采样,生成一个随机值[0,max_rows)并使用ORDER BY..OFFSET..从SQL Server 2012+获取。

如果COUNT和ORDER BY在适当的索引上,这是非常快的——这样数据就已经沿着查询行“排序”了。如果涵盖了这些操作,那么它就是一个快速请求,并且不会受到使用ORDER BY NEWID()或类似方法的可怕可伸缩性的影响。显然,这种方法在非索引的HEAP表上不能很好地伸缩。

declare @rows int
select @rows = count(1) from t

-- Other issues if row counts in the bigint range..
-- This is also not 'true random', although such is likely not required.
declare @skip int = convert(int, @rows * rand())

select t.*
from t
order by t.id -- Make sure this is clustered PK or IX/UCL axis!
offset (@skip) rows
fetch first 1 row only

确保使用了适当的事务隔离级别和/或考虑0结果。


对于SQL Server,需要一个“一般行样本”的方法..

注意:这是一个在SQL Server上找到的关于获取行样本的特定问题的答案的改编。它是根据上下文量身定制的。

虽然这里应该谨慎使用一般抽样方法,但对于其他答案(以及关于非伸缩和/或有问题的实现的重复建议),它仍然是潜在的有用信息。如果目标是找到“单个随机行”,那么这种抽样方法的效率低于所示的第一个代码,并且容易出错。


这是一个更新和改进的对行百分比进行抽样的形式。它基于与其他一些使用CHECKSUM / BINARY_CHECKSUM和modulus的答案相同的概念。

It is relatively fast over huge data sets and can be efficiently used in/with derived queries. Millions of pre-filtered rows can be sampled in seconds with no tempdb usage and, if aligned with the rest of the query, the overhead is often minimal. Does not suffer from CHECKSUM(*) / BINARY_CHECKSUM(*) issues with runs of data. When using the CHECKSUM(*) approach, the rows can be selected in "chunks" and not "random" at all! This is because CHECKSUM prefers speed over distribution. Results in a stable/repeatable row selection and can be trivially changed to produce different rows on subsequent query executions. Approaches that use NEWID() can never be stable/repeatable. Does not use ORDER BY NEWID() of the entire input set, as ordering can become a significant bottleneck with large input sets. Avoiding unnecessary sorting also reduces memory and tempdb usage. Does not use TABLESAMPLE and thus works with a WHERE pre-filter.

这是要点。有关更多细节和注意事项,请参阅这个答案。

Naï亿一下:

declare @sample_percent decimal(7, 4)
-- Looking at this value should be an indicator of why a
-- general sampling approach can be error-prone to select 1 row.
select @sample_percent = 100.0 / count(1) from t

-- BAD!
-- When choosing appropriate sample percent of "approximately 1 row"
-- it is very reasonable to expect 0 rows, which definitely fails the ask!
-- If choosing a larger sample size the distribution is heavily skewed forward,
-- and is very much NOT 'true random'.
select top 1
    t.*
from t
where 1=1
    and ( -- sample
        @sample_percent = 100
        or abs(
            convert(bigint, hashbytes('SHA1', convert(varbinary(32), t.rowguid)))
        ) % (1000 * 100) < (1000 * @sample_percent)
    )

这可以在很大程度上通过混合抽样和ORDER by从小得多的样本集中选择的混合查询来补救。这将排序操作限制为样本大小,而不是原始表的大小。

-- Sample "approximately 1000 rows" from the table,
-- dealing with some edge-cases.
declare @rows int
select @rows = count(1) from t

declare @sample_size int = 1000
declare @sample_percent decimal(7, 4) = case
    when @rows <= 1000 then 100                              -- not enough rows
    when (100.0 * @sample_size / @rows) < 0.0001 then 0.0001 -- min sample percent
    else 100.0 * @sample_size / @rows                        -- everything else
    end

-- There is a statistical "guarantee" of having sampled a limited-yet-non-zero number of rows.
-- The limited rows are then sorted randomly before the first is selected.
select top 1
    t.*
from t
where 1=1
    and ( -- sample
        @sample_percent = 100
        or abs(
            convert(bigint, hashbytes('SHA1', convert(varbinary(32), t.rowguid)))
        ) % (1000 * 100) < (1000 * @sample_percent)
    )
-- ONLY the sampled rows are ordered, which improves scalability.
order by newid()