[ADVICE] NR -100% memory use & lockup (reboot usually req'd)

General
1

After numerous encounters with having Node Red freeze (or the appearance thereof) and having not the faintest idea as to why, yet spending hour upon hours trying to resolve it, I thought I'd write this up in the hopes that I can help others avoid the long road of misery and understanding I've gone through. Full disclosure, I am not a programming genius by any stretch, I've been using NR for about 3.5 years now and have a better then basic understanding of it.

Usually when the issue occurs, if you open a terminal window and type TOP or HTOP, you'll find node is always at the top, using > 95% RAM memory. Even when you reboot you find NR either refuses repeatedly to launch or immediately "locks up" again. For reasons way above my pay grade the NR error messages didn't help at all.

When the problem occurs NR will completely ignore the command "node-red-stop" and even the CLI command "kill xxxx" (PID) has no affect. So I thought I'd provide some wisdom to others that I've learned via the school of frustration. If this does happen to you I suggest you look in the last location where you were adjusting code and look for the following issues (I'm sure there are others but these are the 2 that kept getting me).

1. LOOPS
First word of advice, be very, very careful when using loops like while or for because NR will live in them forever without any hint to you as to being the reason the RAM just got chewed up and it refusing to run normally. All it takes is to forget the line "++x" ,or put it in the wrong place, or a similar loop irregularity. I sure wish NR could detect this and tell me about my stupidity in the error messages.

2. MQTT
This one may surprise you cause it did me. And it was very hard to track down. I have a bunch of GUI switches (well, buttons actually) that can have a number of modes (Sched ON, Sched OFF, Hard ON, Hard OFF, Offline, E-Save, Hydro Fail). They control BigTimer who sends MQTT messages to the switches and MQTT responses tell the switch what it should look like. If the switch (button) is active you can click it to change between the first 4 modes. It is easier than you'd think to have a loop be created through your MQTT broker when you start doing this kind of thing.

For this I suggest you consider using a delay node set to rate limit on your primary (or every) MQTT IN node. It will prevent NR running away in the loop and provides a visual indicator that something is very wrong.
temp
In the example above you see the delay node is set to rate limit to 20 msgs per second. I recreated a fault on purpose to show you how the number appears below the node when msgs start to back up (I saw it go to +1200 once). While at times (like startup) a low number may appear, it should disappear quickly.

Using a delay node this way will prevent NR locking up and give you a visual indicator that you probably have an MQTT broker loop that just keeps sending and receiving stuff. Now you know where to look to resolve it.

(Writing this out is my bit of therapy for having to figure it out, twice).

Cheers

2 Likes
2

You did not mention the fact that one can get out of the lockup situation by starting node-red using
node-red --safe
which starts the editor without starting the flows, allowing the flows to be edited and then started by Deploying.

MQTT loops are easy to trigger and when anyone here says they have such a lockup MQTT will always be near the top of the list of things to check. it can be difficult to work out what is going on. In this situation a useful technique is to use an MQTT client such as mosquitto-sub or mqtt-explorer, subscribe to # and it usually it very quickly becomes clear what topics are causing the problem.

Edit: I forgot to say to always check in ui widgets that Pass Through is disabled unless you are sure you need it.

3

Thanks Colin, I did plan to mention about node-red --safe but forgot. Also, the MQTT IN node I show above is, as you suggest, a topic%/# listener. Good point about Pass Through as well. Thanks for helping make this clearer for others.

4

MQTT loops are a special and (for many newcomers to NR) surprising experience. Program loops are not really any greater danger in JavaScript or the node.js/NR environment than in most other languages. You can't expect a programming environment to protect you from every system crash or freeze -- just live dangerously and keep good backups.

5

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.