在回答另一个Stack Overflow问题时,我偶然发现了一个有趣的子问题。对6个整数的数组进行排序的最快方法是什么?

因为问题层次很低:

我们不能假设库是可用的(而且调用本身也有开销),只有纯C 为了避免清空指令管道(这有非常高的成本),我们可能应该最小化分支、跳转和其他类型的控制流中断(比如隐藏在&&或||序列点后面的那些)。 空间是有限的,最小化寄存器和内存的使用是一个问题,理想情况下,就地排序可能是最好的。

实际上,这个问题是一种Golf,其目标不是最小化源长度,而是最小化执行时间。我称之为“Zening”代码,就像Michael Abrash在《Zen of code optimization》一书及其续集中所使用的那样。

至于为什么它有趣,有几个层面:

示例简单,易于理解和测量,不需要太多的C技能 它显示了对问题选择好的算法的影响,也显示了编译器和底层硬件的影响。

下面是我的参考(简单的,不是优化的)实现和测试集。

#include <stdio.h>

static __inline__ int sort6(int * d){

    char j, i, imin;
    int tmp;
    for (j = 0 ; j < 5 ; j++){
        imin = j;
        for (i = j + 1; i < 6 ; i++){
            if (d[i] < d[imin]){
                imin = i;
            }
        }
        tmp = d[j];
        d[j] = d[imin];
        d[imin] = tmp;
    }
}

static __inline__ unsigned long long rdtsc(void)
{
  unsigned long long int x;
     __asm__ volatile (".byte 0x0f, 0x31" : "=A" (x));
     return x;
}

int main(int argc, char ** argv){
    int i;
    int d[6][5] = {
        {1, 2, 3, 4, 5, 6},
        {6, 5, 4, 3, 2, 1},
        {100, 2, 300, 4, 500, 6},
        {100, 2, 3, 4, 500, 6},
        {1, 200, 3, 4, 5, 600},
        {1, 1, 2, 1, 2, 1}
    };

    unsigned long long cycles = rdtsc();
    for (i = 0; i < 6 ; i++){
        sort6(d[i]);
        /*
         * printf("d%d : %d %d %d %d %d %d\n", i,
         *  d[i][0], d[i][6], d[i][7],
         *  d[i][8], d[i][9], d[i][10]);
        */
    }
    cycles = rdtsc() - cycles;
    printf("Time is %d\n", (unsigned)cycles);
}

生的结果

随着变体的数量越来越多,我将它们都收集到一个测试套件中,可以在这里找到。在Kevin Stock的帮助下,实际使用的测试没有上面展示的那么简单。您可以在自己的环境中编译和执行它。我对不同目标架构/编译器上的行为很感兴趣。(好了,伙计们,把它放在答案里,我将+1一个新结果集的每个贡献者)。

一年前,我把答案给了Daniel Stutzbach(高尔夫),因为他是当时最快的解决方案(排序网络)的来源。

Linux 64位,gcc 4.6.1 64位,Intel Core 2 Duo E8400, -O2

Direct call to qsort library function : 689.38 Naive implementation (insertion sort) : 285.70 Insertion Sort (Daniel Stutzbach) : 142.12 Insertion Sort Unrolled : 125.47 Rank Order : 102.26 Rank Order with registers : 58.03 Sorting Networks (Daniel Stutzbach) : 111.68 Sorting Networks (Paul R) : 66.36 Sorting Networks 12 with Fast Swap : 58.86 Sorting Networks 12 reordered Swap : 53.74 Sorting Networks 12 reordered Simple Swap : 31.54 Reordered Sorting Network w/ fast swap : 31.54 Reordered Sorting Network w/ fast swap V2 : 33.63 Inlined Bubble Sort (Paolo Bonzini) : 48.85 Unrolled Insertion Sort (Paolo Bonzini) : 75.30

Linux 64位,gcc 4.6.1 64位,Intel Core 2 Duo E8400, -O1

Direct call to qsort library function : 705.93 Naive implementation (insertion sort) : 135.60 Insertion Sort (Daniel Stutzbach) : 142.11 Insertion Sort Unrolled : 126.75 Rank Order : 46.42 Rank Order with registers : 43.58 Sorting Networks (Daniel Stutzbach) : 115.57 Sorting Networks (Paul R) : 64.44 Sorting Networks 12 with Fast Swap : 61.98 Sorting Networks 12 reordered Swap : 54.67 Sorting Networks 12 reordered Simple Swap : 31.54 Reordered Sorting Network w/ fast swap : 31.24 Reordered Sorting Network w/ fast swap V2 : 33.07 Inlined Bubble Sort (Paolo Bonzini) : 45.79 Unrolled Insertion Sort (Paolo Bonzini) : 80.15

我包括了-O1和-O2的结果,因为令人惊讶的是,在一些程序中,O2的效率低于O1。我想知道什么具体的优化有这种效果?

对建议解决方案的评论

插入排序(丹尼尔·斯图茨巴赫)

正如预期的那样,最小化分支确实是一个好主意。

排序网络(丹尼尔·斯图茨巴赫)

比插入排序好。我想知道主要的效果是不是避免外部循环。我试着通过展开插入排序来检查,确实我们得到了大致相同的数字(代码在这里)。

排序网络(保罗R)

迄今为止最好的。我用来测试的实际代码在这里。目前还不知道为什么它的速度几乎是其他排序网络实现的两倍。参数传递?快速max ?

排序网络12 SWAP与快速交换

根据Daniel Stutzbach的建议,我将他的12交换排序网络与无分支快速交换相结合(代码在这里)。它确实更快,到目前为止最好的,只有很小的利润率(大约5%),因为可以使用更少的交换。

同样有趣的是,无分支交换似乎比在PPC架构上使用if的简单交换效率低得多(4倍)。

调用库qsort

To give another reference point I also tried as suggested to just call library qsort (code is here). As expected it is much slower : 10 to 30 times slower... as it became obvious with the new test suite, the main problem seems to be the initial load of the library after the first call, and it compares not so poorly with other version. It is just between 3 and 20 times slower on my Linux. On some architecture used for tests by others it seems even to be faster (I'm really surprised by that one, as library qsort use a more complex API).

等级次序

Rex Kerr proposed another completely different method : for each item of the array compute directly its final position. This is efficient because computing rank order do not need branch. The drawback of this method is that it takes three times the amount of memory of the array (one copy of array and variables to store rank orders). The performance results are very surprising (and interesting). On my reference architecture with 32 bits OS and Intel Core2 Quad E8300, cycle count was slightly below 1000 (like sorting networks with branching swap). But when compiled and executed on my 64 bits box (Intel Core2 Duo) it performed much better : it became the fastest so far. I finally found out the true reason. My 32bits box use gcc 4.4.1 and my 64bits box gcc 4.4.3 and the last one seems much better at optimizing this particular code (there was very little difference for other proposals).

更新:

正如上面公布的数字所示,这种效果在gcc的后续版本中仍然得到了增强,Rank Order的速度始终是其他任何替代版本的两倍。

用重新排序的交换对网络进行排序

The amazing efficiency of the Rex Kerr proposal with gcc 4.4.3 made me wonder : how could a program with 3 times as much memory usage be faster than branchless sorting networks? My hypothesis was that it had less dependencies of the kind read after write, allowing for better use of the superscalar instruction scheduler of the x86. That gave me an idea: reorder swaps to minimize read after write dependencies. More simply put: when you do SWAP(1, 2); SWAP(0, 2); you have to wait for the first swap to be finished before performing the second one because both access to a common memory cell. When you do SWAP(1, 2); SWAP(4, 5);the processor can execute both in parallel. I tried it and it works as expected, the sorting networks is running about 10% faster.

使用简单交换对网络进行排序

One year after the original post Steinar H. Gunderson suggested, that we should not try to outsmart the compiler and keep the swap code simple. It's indeed a good idea as the resulting code is about 40% faster! He also proposed a swap optimized by hand using x86 inline assembly code that can still spare some more cycles. The most surprising (it says volumes on programmer's psychology) is that one year ago none of used tried that version of swap. Code I used to test is here. Others suggested other ways to write a C fast swap, but it yields the same performances as the simple one with a decent compiler.

“最佳”代码如下:

static inline void sort6_sorting_network_simple_swap(int * d){
#define min(x, y) (x<y?x:y)
#define max(x, y) (x<y?y:x) 
#define SWAP(x,y) { const int a = min(d[x], d[y]); \
                    const int b = max(d[x], d[y]); \
                    d[x] = a; d[y] = b; }
    SWAP(1, 2);
    SWAP(4, 5);
    SWAP(0, 2);
    SWAP(3, 5);
    SWAP(0, 1);
    SWAP(3, 4);
    SWAP(1, 4);
    SWAP(0, 3);
    SWAP(2, 5);
    SWAP(1, 3);
    SWAP(2, 4);
    SWAP(2, 3);
#undef SWAP
#undef min
#undef max
}

如果我们相信我们的测试集(是的,它很差,它的唯一好处是简短,简单,易于理解我们所测量的内容),那么一个排序的结果代码的平均循环次数低于40个循环(执行6个测试)。这使得每次交换平均为4个周期。我称之为惊人的快。还有其他可能的改进吗?


当前回答

看来我迟到了一年,但我们开始吧…

Looking at the assembly generated by gcc 4.5.2 I observed that loads and stores are being done for every swap, which really isn't needed. It would be better to load the 6 values into registers, sort those, and store them back into memory. I ordered the loads at stores to be as close as possible to there the registers are first needed and last used. I also used Steinar H. Gunderson's SWAP macro. Update: I switched to Paolo Bonzini's SWAP macro which gcc converts into something similar to Gunderson's, but gcc is able to better order the instructions since they aren't given as explicit assembly.

我使用与重新排序的交换网络相同的交换顺序,尽管可能有更好的顺序。如果我有更多的时间,我会生成并测试一堆排列。

我修改了测试代码,考虑超过4000个数组,并显示排序每个数组所需的平均周期数。在i5-650上,我得到~34.1循环/排序(使用-O3),相比之下,原始的重新排序网络得到~65.3循环/排序(使用-O1,击败-O2和-O3)。

#include <stdio.h>

static inline void sort6_fast(int * d) {
#define SWAP(x,y) { int dx = x, dy = y, tmp; tmp = x = dx < dy ? dx : dy; y ^= dx ^ tmp; }
    register int x0,x1,x2,x3,x4,x5;
    x1 = d[1];
    x2 = d[2];
    SWAP(x1, x2);
    x4 = d[4];
    x5 = d[5];
    SWAP(x4, x5);
    x0 = d[0];
    SWAP(x0, x2);
    x3 = d[3];
    SWAP(x3, x5);
    SWAP(x0, x1);
    SWAP(x3, x4);
    SWAP(x1, x4);
    SWAP(x0, x3);
    d[0] = x0;
    SWAP(x2, x5);
    d[5] = x5;
    SWAP(x1, x3);
    d[1] = x1;
    SWAP(x2, x4);
    d[4] = x4;
    SWAP(x2, x3);
    d[2] = x2;
    d[3] = x3;

#undef SWAP
#undef min
#undef max
}

static __inline__ unsigned long long rdtsc(void)
{
    unsigned long long int x;
    __asm__ volatile ("rdtsc; shlq $32, %%rdx; orq %%rdx, %0" : "=a" (x) : : "rdx");
    return x;
}

void ran_fill(int n, int *a) {
    static int seed = 76521;
    while (n--) *a++ = (seed = seed *1812433253 + 12345);
}

#define NTESTS 4096
int main() {
    int i;
    int d[6*NTESTS];
    ran_fill(6*NTESTS, d);

    unsigned long long cycles = rdtsc();
    for (i = 0; i < 6*NTESTS ; i+=6) {
        sort6_fast(d+i);
    }
    cycles = rdtsc() - cycles;
    printf("Time is %.2lf\n", (double)cycles/(double)NTESTS);

    for (i = 0; i < 6*NTESTS ; i+=6) {
        if (d[i+0] > d[i+1] || d[i+1] > d[i+2] || d[i+2] > d[i+3] || d[i+3] > d[i+4] || d[i+4] > d[i+5])
            printf("d%d : %d %d %d %d %d %d\n", i,
                    d[i+0], d[i+1], d[i+2],
                    d[i+3], d[i+4], d[i+5]);
    }
    return 0;
}

我修改了测试套件,以报告每次排序的时钟,并运行更多的测试(cmp函数也更新为处理整数溢出),下面是一些不同架构上的结果。我尝试在AMD cpu上测试,但rdtsc在我可用的X6 1100T上不可靠。

Clarkdale (i5-650)
==================
Direct call to qsort library function      635.14   575.65   581.61   577.76   521.12
Naive implementation (insertion sort)      538.30   135.36   134.89   240.62   101.23
Insertion Sort (Daniel Stutzbach)          424.48   159.85   160.76   152.01   151.92
Insertion Sort Unrolled                    339.16   125.16   125.81   129.93   123.16
Rank Order                                 184.34   106.58   54.74    93.24    94.09
Rank Order with registers                  127.45   104.65   53.79    98.05    97.95
Sorting Networks (Daniel Stutzbach)        269.77   130.56   128.15   126.70   127.30
Sorting Networks (Paul R)                  551.64   103.20   64.57    73.68    73.51
Sorting Networks 12 with Fast Swap         321.74   61.61    63.90    67.92    67.76
Sorting Networks 12 reordered Swap         318.75   60.69    65.90    70.25    70.06
Reordered Sorting Network w/ fast swap     145.91   34.17    32.66    32.22    32.18

Kentsfield (Core 2 Quad)
========================
Direct call to qsort library function      870.01   736.39   723.39   725.48   721.85
Naive implementation (insertion sort)      503.67   174.09   182.13   284.41   191.10
Insertion Sort (Daniel Stutzbach)          345.32   152.84   157.67   151.23   150.96
Insertion Sort Unrolled                    316.20   133.03   129.86   118.96   105.06
Rank Order                                 164.37   138.32   46.29    99.87    99.81
Rank Order with registers                  115.44   116.02   44.04    116.04   116.03
Sorting Networks (Daniel Stutzbach)        230.35   114.31   119.15   110.51   111.45
Sorting Networks (Paul R)                  498.94   77.24    63.98    62.17    65.67
Sorting Networks 12 with Fast Swap         315.98   59.41    58.36    60.29    55.15
Sorting Networks 12 reordered Swap         307.67   55.78    51.48    51.67    50.74
Reordered Sorting Network w/ fast swap     149.68   31.46    30.91    31.54    31.58

Sandy Bridge (i7-2600k)
=======================
Direct call to qsort library function      559.97   451.88   464.84   491.35   458.11
Naive implementation (insertion sort)      341.15   160.26   160.45   154.40   106.54
Insertion Sort (Daniel Stutzbach)          284.17   136.74   132.69   123.85   121.77
Insertion Sort Unrolled                    239.40   110.49   114.81   110.79   117.30
Rank Order                                 114.24   76.42    45.31    36.96    36.73
Rank Order with registers                  105.09   32.31    48.54    32.51    33.29
Sorting Networks (Daniel Stutzbach)        210.56   115.68   116.69   107.05   124.08
Sorting Networks (Paul R)                  364.03   66.02    61.64    45.70    44.19
Sorting Networks 12 with Fast Swap         246.97   41.36    59.03    41.66    38.98
Sorting Networks 12 reordered Swap         235.39   38.84    47.36    38.61    37.29
Reordered Sorting Network w/ fast swap     115.58   27.23    27.75    27.25    26.54

Nehalem (Xeon E5640)
====================
Direct call to qsort library function      911.62   890.88   681.80   876.03   872.89
Naive implementation (insertion sort)      457.69   236.87   127.68   388.74   175.28
Insertion Sort (Daniel Stutzbach)          317.89   279.74   147.78   247.97   245.09
Insertion Sort Unrolled                    259.63   220.60   116.55   221.66   212.93
Rank Order                                 140.62   197.04   52.10    163.66   153.63
Rank Order with registers                  84.83    96.78    50.93    109.96   54.73
Sorting Networks (Daniel Stutzbach)        214.59   220.94   118.68   120.60   116.09
Sorting Networks (Paul R)                  459.17   163.76   56.40    61.83    58.69
Sorting Networks 12 with Fast Swap         284.58   95.01    50.66    53.19    55.47
Sorting Networks 12 reordered Swap         281.20   96.72    44.15    56.38    54.57
Reordered Sorting Network w/ fast swap     128.34   50.87    26.87    27.91    28.02

其他回答

永远不要在没有基准测试和查看实际编译器生成的程序集的情况下优化min/max。如果我让GCC用条件移动指令优化最小值,我得到了33%的加速:

#define SWAP(x,y) { int dx = d[x], dy = d[y], tmp; tmp = d[x] = dx < dy ? dx : dy; d[y] ^= dx ^ tmp; }

(测试代码中的循环为280 vs. 420)。用?:做max或多或少是一样的,几乎淹没在噪音中,但上面的速度稍微快一点。这个SWAP在GCC和Clang中都更快。

编译器在寄存器分配和别名分析方面也做得很出色,有效地将d[x]提前移动到局部变量中,并且只在结束时复制回内存。事实上,它们甚至比完全使用局部变量(如d0 = d[0], d1 = d[1], d2 = d[2], d3 = d[3], d4 = d[4], d5 = d[5])更好。我写这个是因为你假设强优化,但试图在min/max上胜过编译器。:)

顺便说一下,我尝试了Clang和GCC。它们做了相同的优化,但由于调度差异,两者在结果上有一些变化,不能说哪个更快或更慢。GCC在排序网络上速度较快,Clang在二次排序网络上速度较快。

为了完整起见,展开冒泡排序和插入排序也是可能的。下面是冒泡排序:

SWAP(0,1); SWAP(1,2); SWAP(2,3); SWAP(3,4); SWAP(4,5);
SWAP(0,1); SWAP(1,2); SWAP(2,3); SWAP(3,4);
SWAP(0,1); SWAP(1,2); SWAP(2,3);
SWAP(0,1); SWAP(1,2);
SWAP(0,1);

这是插入排序:

//#define ITER(x) { if (t < d[x]) { d[x+1] = d[x]; d[x] = t; } }
//Faster on x86, probably slower on ARM or similar:
#define ITER(x) { d[x+1] ^= t < d[x] ? d[x] ^ d[x+1] : 0; d[x] = t < d[x] ? t : d[x]; }
static inline void sort6_insertion_sort_unrolled_v2(int * d){
    int t;
    t = d[1]; ITER(0);
    t = d[2]; ITER(1); ITER(0);
    t = d[3]; ITER(2); ITER(1); ITER(0);
    t = d[4]; ITER(3); ITER(2); ITER(1); ITER(0);
    t = d[5]; ITER(4); ITER(3); ITER(2); ITER(1); ITER(0);

这种插入排序比Daniel Stutzbach的更快,在GPU或有预测的计算机上特别好,因为ITER只需要3条指令就可以完成(而SWAP则需要4条指令)。例如,这里是t = d[2];ITER (1);ITER (0);ARM装配线:

    MOV    r6, r2
    CMP    r6, r1
    MOVLT  r2, r1
    MOVLT  r1, r6
    CMP    r6, r0
    MOVLT  r1, r0
    MOVLT  r0, r6

对于6个元素,插入排序与排序网络竞争(12次交换vs. 15次迭代平衡4条指令/交换vs. 3条指令/迭代);泡沫当然要慢一些。但当大小增加时就不成立了,因为插入排序是O(n²)而排序网络是O(n log n)。

这是我对这个线程的贡献:一个包含唯一值的6成员int向量(valp)的优化1,4间隙壳排序。

void shellsort (int *valp)
{      
  int c,a,*cp,*ip=valp,*ep=valp+5;

  c=*valp;    a=*(valp+4);if (c>a) {*valp=    a;*(valp+4)=c;}
  c=*(valp+1);a=*(valp+5);if (c>a) {*(valp+1)=a;*(valp+5)=c;}

  cp=ip;    
  do
  {
    c=*cp;
    a=*(cp+1);
    do
    {
      if (c<a) break;

      *cp=a;
      *(cp+1)=c;
      cp-=1;
      c=*cp;
    } while (cp>=valp);
    ip+=1;
    cp=ip;
  } while (ip<ep);
}

在我的惠普dv7-3010so笔记本电脑上,双核Athlon M300 @ 2 Ghz (DDR2内存),它在165个时钟周期内执行。这是一个平均计算从计时每个独特的序列(6!/总共720)。使用OpenWatcom 1.8编译到Win32。这个循环本质上是一个插入排序,有16条指令/37字节长。

我没有一个64位的环境来编译。

异或交换在交换函数中可能很有用。

void xorSwap (int *x, int *y) {
     if (*x != *y) {
         *x ^= *y;
         *y ^= *x;
         *x ^= *y;
     }
 }

if可能会在代码中导致太多的分歧,但如果你能保证所有int都是唯一的,这可能会很方便。

我将测试套件移植到一台我无法识别的PPC架构机器上(不需要触摸代码,只需增加测试的迭代,使用8个测试用例来避免mods污染结果,并替换x86特定的rdtsc):

直接调用qsort库函数:101

简单实现(插入排序):299

插入排序(Daniel Stutzbach): 108

插入排序展开:51

排序网络(Daniel Stutzbach): 26

排序网络(Paul R): 85

排序网络12与快速交换:117

排序网络12重排序交换:116

排名顺序:56

//Bruteforce compute unrolled count dumbsort(min to 0-index)
void bcudc_sort6(int* a)
{
    int t[6] = {0};
    int r1,r2;

    r1=0;
    r1 += (a[0] > a[1]);
    r1 += (a[0] > a[2]);
    r1 += (a[0] > a[3]);
    r1 += (a[0] > a[4]);
    r1 += (a[0] > a[5]);
    while(t[r1]){r1++;}
    t[r1] = a[0];

    r2=0;
    r2 += (a[1] > a[0]);
    r2 += (a[1] > a[2]);
    r2 += (a[1] > a[3]);
    r2 += (a[1] > a[4]);
    r2 += (a[1] > a[5]);
    while(t[r2]){r2++;} 
    t[r2] = a[1];

    r1=0;
    r1 += (a[2] > a[0]);
    r1 += (a[2] > a[1]);
    r1 += (a[2] > a[3]);
    r1 += (a[2] > a[4]);
    r1 += (a[2] > a[5]);
    while(t[r1]){r1++;}
    t[r1] = a[2];

    r2=0;
    r2 += (a[3] > a[0]);
    r2 += (a[3] > a[1]);
    r2 += (a[3] > a[2]);
    r2 += (a[3] > a[4]);
    r2 += (a[3] > a[5]);
    while(t[r2]){r2++;} 
    t[r2] = a[3];

    r1=0;
    r1 += (a[4] > a[0]);
    r1 += (a[4] > a[1]);
    r1 += (a[4] > a[2]);
    r1 += (a[4] > a[3]);
    r1 += (a[4] > a[5]);
    while(t[r1]){r1++;}
    t[r1] = a[4];

    r2=0;
    r2 += (a[5] > a[0]);
    r2 += (a[5] > a[1]);
    r2 += (a[5] > a[2]);
    r2 += (a[5] > a[3]);
    r2 += (a[5] > a[4]);
    while(t[r2]){r2++;} 
    t[r2] = a[5];

    a[0]=t[0];
    a[1]=t[1];
    a[2]=t[2];
    a[3]=t[3];
    a[4]=t[4];
    a[5]=t[5];
}

static __inline__ void sort6(int* a)
{
    #define wire(x,y); t = a[x] ^ a[y] ^ ( (a[x] ^ a[y]) & -(a[x] < a[y]) ); a[x] = a[x] ^ t; a[y] = a[y] ^ t;
    register int t;

    wire( 0, 1); wire( 2, 3); wire( 4, 5);
    wire( 3, 5); wire( 0, 2); wire( 1, 4);
    wire( 4, 5); wire( 2, 3); wire( 0, 1); 
    wire( 3, 4); wire( 1, 2); 
    wire( 2, 3);

    #undef wire
}