Node.js堆内存不足

今天，我运行了文件系统索引的脚本来刷新RAID文件索引，4h后它崩溃了，出现以下错误:

[md5:]  241613/241627 97.5%  
[md5:]  241614/241627 97.5%  
[md5:]  241625/241627 98.1%
Creating missing list... (79570 files missing)
Creating new files list... (241627 new files)

<--- Last few GCs --->

11629672 ms: Mark-sweep 1174.6 (1426.5) -> 1172.4 (1418.3) MB, 659.9 / 0 ms [allocation failure] [GC in old space requested].
11630371 ms: Mark-sweep 1172.4 (1418.3) -> 1172.4 (1411.3) MB, 698.9 / 0 ms [allocation failure] [GC in old space requested].
11631105 ms: Mark-sweep 1172.4 (1411.3) -> 1172.4 (1389.3) MB, 733.5 / 0 ms [last resort gc].
11631778 ms: Mark-sweep 1172.4 (1389.3) -> 1172.4 (1368.3) MB, 673.6 / 0 ms [last resort gc].


<--- JS stacktrace --->

==== JS stack trace =========================================

Security context: 0x3d1d329c9e59 <JS Object>
1: SparseJoinWithSeparatorJS(aka SparseJoinWithSeparatorJS) [native array.js:~84] [pc=0x3629ef689ad0] (this=0x3d1d32904189 <undefined>,w=0x2b690ce91071 <JS Array[241627]>,L=241627,M=0x3d1d329b4a11 <JS Function ConvertToString (SharedFunctionInfo 0x3d1d3294ef79)>,N=0x7c953bf4d49 <String[4]\: ,\n  >)
2: Join(aka Join) [native array.js:143] [pc=0x3629ef616696] (this=0x3d1d32904189 <undefin...

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory
 1: node::Abort() [/usr/bin/node]
 2: 0xe2c5fc [/usr/bin/node]
 3: v8::Utils::ReportApiFailure(char const*, char const*) [/usr/bin/node]
 4: v8::internal::V8::FatalProcessOutOfMemory(char const*, bool) [/usr/bin/node]
 5: v8::internal::Factory::NewRawTwoByteString(int, v8::internal::PretenureFlag) [/usr/bin/node]
 6: v8::internal::Runtime_SparseJoinWithSeparator(int, v8::internal::Object**, v8::internal::Isolate*) [/usr/bin/node]
 7: 0x3629ef50961b

服务器配置16gb RAM和24gb SSD交换盘。我非常怀疑我的脚本内存超过了36gb。至少不应该是这样

脚本创建文件索引存储为对象数组与文件元数据(修改日期，权限等，没有大数据)

以下是完整的脚本代码: http://pastebin.com/mjaD76c3

我已经经历了奇怪的节点问题在过去与这个脚本迫使我eg。分割索引到多个文件作为节点是故障时，工作在这样的大文件字符串。对于庞大的数据集，有什么方法可以改善nodejs的内存管理吗?

当前回答

如果你想更改节点(windows)的全局内存，请进入高级系统设置->环境变量->新用户变量

variable name = NODE_OPTIONS
variable value = --max-old-space-size=4096

2019-09-23 14:38:18

其他回答

所有这些答案都不适合我(我没有尝试更新npm)。

我的程序使用了两个数组。一个是在JSON上解析的，另一个是根据第一个上的数据生成的。在第二个循环之前，我只需要将第一个JSON解析数组设置为[]。

这样就释放了大量的内存，允许程序继续执行而不会在某个时候内存分配失败。

干杯!

2022-12-20 20:03:49

你可以通过以下方法修复Node.js中的“堆出内存”错误。

Increase the amount of memory allocated to the Node.js process by using the --max-old-space-size flag when starting the application. For example, you can increase the limit to 4GB by running node --max-old-space-size=4096 index.js. Use a memory leak detection tool, such as the Node.js heap dump module, to identify and fix memory leaks in your application. You can also use the node inspector and use chrome://inspect to check memory usage. Optimize your code to reduce the amount of memory needed. This might involve reducing the size of data structures, reusing objects instead of creating new ones, or using more efficient algorithms. Use a garbage collector (GC) algorithm to manage memory automatically. Node.js uses the V8 engine's garbage collector by default, but you can also use other GC algorithms such as the Garbage Collection in Node.js Use a containerization technology like Docker which limits the amount of memory available to the container. Use a process manager like pm2 which allows to automatically restart the node application if it goes out of memory.

2023-01-12 16:00:59

对于像我这样没有找到解决此错误的合适解决方案的初学者，请检查已安装的节点版本(x32, x64, x86)。我有一个64位CPU，我已经安装了x86节点版本，这导致CALL_AND_RETRY_LAST分配失败- JavaScript堆出内存错误。

2020-09-22 22:33:58

如果你想更改节点(windows)的全局内存，请进入高级系统设置->环境变量->新用户变量

variable name = NODE_OPTIONS
variable value = --max-old-space-size=4096

2019-09-23 14:38:18

在我的例子中，我把node.js版本升级到最新版(12.8.0版本)，它就像一个魅力。

2020-06-12 08:06:13

Node.js堆内存不足

推荐文章

最新文章

标签