Hmmm, Node-RED grinds to a halt and is restarting. Any Ideas?

General
1

Seeing this in the logs, or at least, this is the first one I caught!...

<--- Last few GCs --->
[3690:0x7f9d100000] 799596 ms: Mark-Compact 1005.1 (1073.6) -> 989.6 (1071.6) MB, pooled: 1 MB, 578.63 / 0.00 ms (average mu = 0.467, current mu = 0.434) allocation failure; scavenge might not succeed
[3690:0x7f9d100000] 800337 ms: Mark-Compact 998.8 (1071.9) -> 989.8 (1072.6) MB, pooled: 0 MB, 535.35 / 0.00 ms (average mu = 0.384, current mu = 0.278) allocation failure; scavenge might not succeed
<--- JS stacktrace --->
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
----- Native stack trace -----
1: 0xe15218 node::OOMErrorHandler(char const*, v8::OOMDetails const&) [node-red]
2: 0x11ab5cc v8::Utils::ReportOOMFailure(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node-red]
3: 0x11ab77c v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char const*, v8::OOMDetails const&) [node-red]
4: 0x13d137c [node-red]
5: 0x13e92f0 v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace, v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node-red]
6: 0x13bfc78 v8::internal::HeapAllocator::AllocateRawWithLightRetrySlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node-red]
7: 0x13c0ab0 v8::internal::HeapAllocator::AllocateRawWithRetryOrFailSlowPath(int, v8::internal::AllocationType, v8::internal::AllocationOrigin, v8::internal::AllocationAlignment) [node-red]
8: 0x139b1b0 v8::internal::Factory::AllocateRaw(int, v8::internal::AllocationType, v8::internal::AllocationAlignment) [node-red]
9: 0x13893f8 v8::internal::FactoryBase<v8::internal::Factory>::AllocateRawWithImmortalMap(int, v8::internal::AllocationType, v8::internal::Tagged<v8::internal::Map>, v8::internal::AllocationAlignment) [node-red]
10: 0x138a90c v8::internal::FactoryBase<v8::internal::Factory>::NewRawOneByteString(int, v8::internal::AllocationType) [node-red]
11: 0x138a99c v8::internal::FactoryBase<v8::internal::Factory>::NewStringFromOneByte(v8::base::Vector<unsigned char const>, v8::internal::AllocationType) [node-red]
12: 0x14fd7bc v8::internal::JsonStringifier::Stringify(v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node-red]
13: 0x14fd95c v8::internal::JsonStringify(v8::internal::Isolate*, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>, v8::internal::Handle<v8::internal::Object>) [node-red]
14: 0x12263b4 v8::internal::Builtin_JsonStringify(int, unsigned long*, v8::internal::Isolate*) [node-red]
15: 0x1cc8bf4 [node-red]
nodered.service: Main process exited, code=killed, status=6/ABRT
nodered.service: Failed with result 'signal'.
nodered.service: Consumed 15min 27.649s CPU time.
nodered.service: Scheduled restart job, restart counter is at 2.
Stopped nodered.service - Node-RED graphical event wiring tool.
nodered.service: Consumed 15min 27.649s CPU time.
Started nodered.service - Node-RED graphical event wiring tool.
26 May 18:10:48 - [info]
Welcome to Node-RED
===================
26 May 18:10:48 - [info] Node-RED version: v4.1.10
26 May 18:10:48 - [info] Node.js version: v22.22.2
26 May 18:10:48 - [info] Linux 6.12.87+rpt-rpi-v8 arm64 LE
26 May 18:10:48 - [info] Loading palette nodes
26 May 18:10:50 - [info] node-red-contrib-telegrambot version: v17.4.12
26 May 18:10:55 - [info] Dashboard version 3.6.6 started at /ui
26 May 18:10:55 - [info] Settings file : /home/nodered/.node-red/settings.js
26 May 18:10:55 - [info] Context store : 'default' [module=memory]
26 May 18:10:55 - [info] Context store : 'storeInFile' [module=localfilesystem]
(node:5811) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
26 May 18:10:55 - [info] User directory : /home/nodered/.node-red
26 May 18:10:55 - [info] Projects directory: /home/nodered/.node-red/projects
26 May 18:10:56 - [info] Server now running at ``http://127.0.0.1:1880/
26 May 18:10:56 - [info] Active project : 2401091_Tennyson
26 May 18:10:56 - [info] Flows file : /home/nodered/.node-red/projects/2401091_Tennyson/flows.json
26 May 18:10:56 - [warn] Using unencrypted credentials
26 May 18:10:56 - [info] +-----------------------------------------------------
26 May 18:10:56 - [info] | šŸŒ uibuilder v7.6.2 initialised
26 May 18:10:56 - [info] | root folder: /home/nodered/.node-red/projects/2401091_Tennyson/uibuilder
26 May 18:10:56 - [info] | Using Node-RED's webserver at:
26 May 18:10:56 - [info] | ``http://0.0.0.0:1880/
26 May 18:10:56 - [info] | Installed packages:
26 May 18:10:56 - [info] | justgage, uplot
26 May 18:10:56 - [info] +-----------------------------------------------------
26 May 18:10:56 - [info] Starting flows
26 May 18:10:56 - [warn] Unknown context store 'memoryOnly' specified. Using default store.
26 May 18:10:57 - [warn] Unknown context store 'memory' specified. Using default store.
26 May 18:10:58 - [info] Started flows
26 May 18:10:59 - [info] [serialconfig:eaecafaec2da87bc] serial port /dev/ttyUSB0 opened at 115200 baud 8N1
26 May 18:10:59 - [info] [mqtt-broker:Node-RED MQTT] Connected to broker: mqtt://172.27.123.58:1883
26 May 18:11:03 - [info] [position-config:8aac1f268a7d4357] getSunCalc, time difference since last output to low, do no calculation

I have been reviewing SWP Memory on another project so I find this rather perplexing. Really? No SWP Memory used?? (Must admit, I can't recall what it might have been before getting this error.

image
image2450Ɨ1083 410 KB

Note the 100% use. It has built up to this

I have InfluxDB and Grafana on this RPi5. I have already stopped Grafana as it is only needed for display. There are calls to load and fetch to/from InfluxDB, so I have left it running. System OS is up to date.

Any ideas anyone please?

TIA,
Colin J

2

The usual cause is an infinite loop, either explicit in your code or involving an external process such as subscribing and publishing to the same MQTT topic.

It's also possible for an overly expensive regular expression to max out CPU usage.

3

Looking at this log, Node-RED crashed because a node attempted to run JSON.stringify() on a massive object (or an object with a circular reference that caused an infinite loop, though a massive payload is more likely here). The heap memory allocation failed right around the default 1 GB limit (you can see 1005.1 -> 989.6 MB in the GC logs), which is typical for Node.js on a 32-bit system or a constrained environment like a Raspberry Pi.

Are you selecting all from DB then passing it to a node that stringifies (like a debug node?)

An infinite loop is also a common reason (and just so happens the last thing to push it over the edge was a stringify operation)

4

I would get rid of the max_old_space paramter, it shouldn't be needed. That isn't your main issue though.

5

Hmm, good point, I should add a check for that to my saferSerialize() function and the various tools that use it.

6

Thank you Peoples!

I 'suspected' the JSON.stringify() command might be part of the problem, Not changed anything. This was all working OK before I had a problem with my main WiFi Router.

I can't recall how long this installation of NR has been running for. I am wondering if perhaps some kind of corruption has taken place. I have just modified some of the MQTT node calls to make sure I only have one MQTT In and one Out.

I now have the relevant information and now need to eliminate the flow causing the problem.

Many thanks!
Colin J

Oh, the max_space parameter was done in the original install all those years ago. Probably to cover the transfer from the RPi3 to RPi5.

7

Yes, really shouldn't be needed now I don't believe and will be restricting your memory allocation on larger memory devices.

I know this doesn't help you now, but the new function node uibuilder functions should help in uibuilder's next release. These can serialize pretty much anything that JavaScript/Node.js can create without crashing. They give sensible results where they can and a hint where they can't (functions, circular references, etc).