当你想在堆上添加一项时，会发生的事情是计算机必须搜索空间来容纳该项。这就是为什么动态分配不在内存池上执行或使用池分配程序会“减慢”速度的原因。对于一个沉重的STL应用程序，如果你正在做多线程，有囤积分配器或TBB英特尔版本。

现在，当内存碎片化时，会发生两件事:

There will have to be more searches to find a good space to stick "large" objects. That is, with many small objects scattered about finding a nice contigous chunk of memory could under certain conditions be difficult (these are extreme.) Memory is not some easily read entity. Processors are limited to how much they can hold and where. They do this by swapping pages if an item they need is one place but the current addresses are another. If you are constantly having to swap pages, processing can slow down (again, extreme scenarios where this impacts performance.) See this posting on virtual memory.

更新: 谷歌TCMalloc:线程缓存Malloc 已经发现它在处理长时间运行进程中的碎片方面相当出色。

我一直在开发一个服务器应用程序，它在HP-UX 11.23/11.31 ia64上存在内存碎片问题。

它是这样的。有一个进程进行内存分配和释放，并运行了几天。即使没有内存泄漏，进程的内存消耗也在不断增加。

About my experience. On HP-UX it is very easy to find memory fragmentation using HP-UX gdb. You set a break-point and when you hit it you run this command: info heap and see all memory allocations for the process and the total size of heap. Then your continue your program and then some time later your again hit the break-point. You do again info heap. If the total size of heap is bigger but the number and the size of separate allocations are the same then it is likely that you have memory allocation problems. If necessary do this check few fore times.

My way of improving the situation was this. After I had done some analysis with HP-UX gdb I saw that memory problems were caused by the fact that I used std::vector for storing some types of information from a database. std::vector requires that its data must be kept in one block. I had a few containers based on std::vector. These containers were regularly recreated. There were often situations when new records were added to the database and after that the containers were recreated. And since the recreated containers were bigger their did not fit into available blocks of free memory and the runtime asked for a new bigger block from the OS. As a result even though there were no memory leaks the memory consumption of the process grew. I improved the situation when I changed the containers. Instead of std::vector I started using std::deque which has a different way of allocating memory for data.

我知道在HP-UX上避免内存碎片的方法之一是使用小块分配器或使用MallocNextGen。在RedHat Linux上，默认的分配器似乎可以很好地处理大量小块的分配。在Windows上有低碎片堆，它解决了大量小分配的问题。

My understanding is that in an STL-heavy application you have first to identify problems. Memory allocators (like in libc) actually handle the problem of a lot of small allocations, which is typical for std::string (for instance in my server application there are lots of STL strings but as I see from running info heap they are not causing any problems). My impression is that you need to avoid frequent large allocations. Unfortunately there are situations when you can't avoid them and have to change your code. As I say in my case I improved the situation when switched to std::deque. If you identify your memory fragmention it might be possible to talk about it more precisely.

什么是内存碎片?

内存碎片是指当您的大部分内存被分配为大量不连续的块或块时——留下很大比例的内存未分配，但在大多数典型场景下无法使用。这将导致内存溢出异常或分配错误(即malloc返回null)。

The easiest way to think about this is to imagine you have a big empty wall that you need to put pictures of varying sizes on. Each picture takes up a certain size and you obviously can't split it into smaller pieces to make it fit. You need an empty spot on the wall, the size of the picture, or else you can't put it up. Now, if you start hanging pictures on the wall and you're not careful about how you arrange them, you will soon end up with a wall that's partially covered with pictures and even though you may have empty spots most new pictures won't fit because they're larger than the available spots. You can still hang really small pictures, but most ones won't fit. So you'll have to re-arrange (compact) the ones already on the wall to make room for more..

现在，想象墙是你的(堆)内存，图片是物体。这就是内存碎片。

如何判断内存碎片是否是我的应用程序的一个问题?什么样的程序最可能受到影响?

您可能正在处理内存碎片的一个明显迹象是，如果您得到许多分配错误，特别是当已使用内存的百分比很高时(但不是您还没有使用完所有内存)，那么从技术上讲，您应该有足够的空间用于您试图分配的对象。

当内存严重碎片化时，内存分配可能需要更长的时间，因为内存分配器必须做更多的工作来为新对象找到合适的空间。如果您有许多内存分配(您可能会这样做，因为您最终会产生内存碎片)，分配时间甚至可能会导致明显的延迟。

处理内存碎片的常用方法是什么?

使用好的算法分配内存。不是为许多小对象分配内存，而是为这些小对象的连续数组预分配内存。有时，在分配内存时稍微浪费一点可以提高性能，并且可以省去必须处理内存碎片的麻烦。

什么是内存碎片?

Memory fragmentation is the problem of memory becoming unusable even though it is theoretically available. There are two kinds of fragmentation: internal fragmentation is memory that is allocated but cannot be used (e.g. when memory is allocated in 8 byte chunks but the program repeatedly does single allocations when it needs only 4 bytes). external fragmentation is the problem of free memory becoming divided into many small chunks so that large allocation requests cannot be met although there is enough overall free memory.

如何判断内存碎片是否是我的应用程序的一个问题?什么样的程序最可能受到影响?

如果您的程序使用的系统内存远远超过其实际有效负载数据所需的内存(并且您已经排除了内存泄漏)，那么内存碎片就是一个问题。

处理内存碎片的常用方法是什么?

使用一个好的内存分配器。在IIRC中，那些使用“最佳匹配”策略的人通常在避免碎片化方面要出色得多，只是速度稍慢一些。然而，事实也表明，对于任何配置策略，都存在病态的最坏情况。幸运的是，对于分配器来说，大多数应用程序的典型分配模式实际上是相对友好的。如果你对细节感兴趣，这里有一堆文件:

Paul R. Wilson, Mark S. Johnstone, Michael Neely and David Boles. Dynamic Storage Allocation: A Survey and Critical Review. In Proceedings of the 1995 International Workshop on Memory Management, Springer Verlag LNCS, 1995 Mark S.Johnstone, Paul R. Wilson. The Memory Fragmentation Problem: Solved? In ACM SIG-PLAN Notices, volume 34 No. 3, pages 26-36, 1999 M.R. Garey, R.L. Graham and J.D. Ullman. Worst-Case analysis of memory allocation algorithms. In Fourth Annual ACM Symposium on the Theory of Computing, 1972

当分配和释放许多大小不同的对象时，最可能发生内存碎片。假设你在内存中有如下布局:

obj1 (10kb) | obj2(20kb) | obj3(5kb) | unused space (100kb)

现在，当obj2被释放时，您有120kb的未使用内存，但是您不能分配120kb的完整块，因为内存是碎片化的。

避免这种影响的常用技术包括环形缓冲区和对象池。在STL的上下文中，像std::vector::reserve()这样的方法可以提供帮助。

什么是内存碎片?

When your app uses dynamic memory, it allocates and frees chunks of memory. In the beginning, the whole memory space of your app is one contiguous block of free memory. However, when you allocate and free blocks of different size, the memory starts to get fragmented, i.e. instead of a big contiguous free block and a number of contiguous allocated blocks, there will be a allocated and free blocks mixed up. Since the free blocks have limited size, it is difficult to reuse them. E.g. you may have 1000 bytes of free memory, but can't allocate memory for a 100 byte block, because all the free blocks are at most 50 bytes long.

另一个不可避免但问题较少的碎片来源是，在大多数架构中，内存地址必须对齐到2,4,8等字节边界(即地址必须是2,4,8的倍数等)这意味着，即使你有一个包含3个char字段的结构，你的结构可能有12而不是3，因为每个字段都对齐到4字节边界。

如何判断内存碎片是否是我的应用程序的一个问题?什么样的程序最可能受到影响?

最明显的答案是内存不足异常。

显然，在c++应用程序中，没有一种好的便携式方法来检测内存碎片。更多细节请看这个答案。

处理内存碎片的常用方法是什么?

这在c++中很困难，因为你在指针中使用直接内存地址，你无法控制谁引用特定的内存地址。因此，重新安排已分配的内存块(Java垃圾收集器的方式)是不可取的。

自定义分配器可以通过在较大内存块中管理小对象的分配，并重用该块中的空闲插槽来提供帮助。