Crashing Every Few Minutes - How Do I Debug?

Hey All,
My install of node red is crashing every few minutes, it doesn't go more than an hour.

I'm trying to debut the problem but I can't locate the log files. I checked the settings.js file and it says logging is set to "Console Only" and says that is the only option. Sounds like there is no log file?

So I should watch the console for errors? I've got a lot of flows, it could take a while..

What's the best way to proceed?
Thanks

Rich

"Every few minutes" and "doesn't go more than an hour" are two different things. but...

Stop Node-Red from the terminal.

node-red-stop from the directory where node-red is.

Then node-red and watch what happens.

Somewhere in the stuff you see scrolling past should be something to help.
Actually - if I am understanding things: the scrolling should stop just after the error happens.
So run it and wait for NR to crash and look at the terminal where you started node-red and that should help some what.

1 Like

It depends on the platform. On a Raspberry Pi, Node-red logs to /var/log/syslog.

You can start Node-red in safe mode - the editor is available but the flows do not start.
Look through your flows for loops.
If you use MQTT check that you don't have a flow subscribing to a topic then publishing to the same topic.
You can disable a portion of your flows, deploy (which drops out of safe mode) and see if the problem still exists.

1 Like

What OS are you running on and how did you install node red?

Raspberry Pi 4.

You did not answer that.

Assuming that you used the recommended method then run
node-red-stop
node-red-start
And post the full startup log here. Copy/paste please, not screenshot. Then wait till it crashes and post what you see in the log then.

ok, here's the crash report. Something is sucking up memory. Note, I'm in the process of removing all hardware from my pi and stripping it down to just be the broker and node red. I say this because I'm making a lot of changes to my system right now.

FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
nodered.service: Main process exited, code=killed, status=6/ABRT

Thanks once again for your help Colin. Much appreciated!!

Note: I am running some charts/graphs but it's less and less as time goes by. And I loose the datapoints each time.

My newest pallet is suspect "light transition"

I just disconnected a bunch of flows using this. I suspect this this is the culprit.

nope. Crashed again, same error

You can disable individual tabs in the editor to help find what is leaking memory.

Are you doing anything with large files or videos, for example.

Also show us the startup log as I requested.
Also show us the lines in the log around the crash.

Are you viewing the editor and dashboard from a browser on the Pi?

If you have one, try accessing editor and dashboard from a device with much more memory, a PC for instance.

Make sure that your charts don't have substantially more points than the available pixel width.

1 Like

Crash details

      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/python /hom                                                                                             e/pi/scripts/temperature/currentTemps/calculateHighTemp.py
pam_unix(sudo:session): session opened for user root by (uid=0)
pam_unix(sudo:session): session closed for user root
      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/python /hom                                                                                             e/pi/scripts/temperature/currentTemps/calculateHighTemp.py
pam_unix(sudo:session): session opened for user root by (uid=0)
pam_unix(sudo:session): session closed for user root
      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/python /hom                                                                                             e/pi/scripts/temperature/temp.py
pam_unix(sudo:session): session opened for user root by (uid=0)
      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/python /hom                                                                                             e/pi/scripts/temperature/currentTemps/calculateHighTemp.py
pam_unix(sudo:session): session opened for user root by (uid=0)
pam_unix(sudo:session): session closed for user root
pam_unix(sudo:session): session closed for user root
      pi : TTY=unknown ; PWD=/home/pi ; USER=root ; COMMAND=/usr/bin/python /hom                                                                                             e/pi/scripts/temperature/currentTemps/calculateHighTemp.py
pam_unix(sudo:session): session opened for user root by (uid=0)
pam_unix(sudo:session): session closed for user root
<--- Last few GCs --->
[3599:0x52ddef0]  1949856 ms: Mark-sweep 497.8 (520.4) -> 494.2 (520.7) MB, 3704                                                                                             .6 / 0.1 ms  (average mu = 0.242, current mu = 0.079) task scavenge might not su                                                                                             cceed
[3599:0x52ddef0]  1953979 ms: Mark-sweep 497.8 (520.7) -> 494.4 (520.7) MB, 3683                                                                                             .2 / 0.1 ms  (average mu = 0.181, current mu = 0.107) task scavenge might not su                                                                                             cceed
[3599:0x52ddef0]  1958643 ms: Mark-sweep 498.0 (520.7) -> 494.5 (520.9) MB, 4351                                                                                             .6 / 0.2 ms  (average mu = 0.124, current mu = 0.067) task scavenge might not su                                                                                             cceed
<--- JS stacktrace --->
FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaS                                                                                             cript heap out of memory
nodered.service: Main process exited, code=killed, status=6/ABRT
pam_unix(sudo:session): session closed for user root
pam_unix(sudo:session): session closed for user root
nodered.service: Failed with result 'signal'.
nodered.service: Service RestartSec=20s expired, scheduling restart.
nodered.service: Scheduled restart job, restart counter is at 3.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.

Startup details

Welcome to Node-RED
===================
7 Aug 13:27:08 - [info] Node-RED version: v3.0.2
7 Aug 13:27:08 - [info] Node.js  version: v16.20.1
7 Aug 13:27:08 - [info] Linux 5.10.103-v7l+ arm LE
7 Aug 13:27:09 - [info] Loading palette nodes
7 Aug 13:27:13 - [info] Dashboard version 3.5.0 started at /ui
(node:4548) Warning: Accessing non-existent property 'padLevels' of module expor                                                                                             ts inside circular dependency
(Use `node --trace-warnings ...` to show where the warning was created)
7 Aug 13:27:13 - [info] Settings file  : /home/pi/.node-red/settings.js
7 Aug 13:27:13 - [info] Context store  : 'default' [module=memory]
7 Aug 13:27:13 - [info] User directory : /home/pi/.node-red
7 Aug 13:27:13 - [warn] Projects disabled : editorTheme.projects.enabled=false
7 Aug 13:27:13 - [info] Flows file     : /home/pi/.node-red/flows.json
7 Aug 13:27:13 - [info] Server now running at http://127.0.0.1:1880/
7 Aug 13:27:13 - [warn]
---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.
If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.
You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------
7 Aug 13:27:13 - [info] Starting flows
7 Aug 13:27:14 - [info] [thingzi-timer:blood pressure] Scheduled 'ON' (tod) for                                                                                              Mon Aug 07 2023 20:01:00 GMT-0400
7 Aug 13:27:14 - [info] [thingzi-timer:blood pressure] Scheduled 'ON' (tod) for                                                                                              Mon Aug 14 2023 09:00:00 GMT-0400
7 Aug 13:27:14 - [info] [thingzi-timer:Dusk] Scheduled 'ON' (sun) for Mon Aug 07                                                                                              2023 21:14:28 GMT-0400
7 Aug 13:27:14 - [info] [thingzi-timer:Dawn] Scheduled 'ON' (sun) for Tue Aug 08                                                                                              2023 06:02:53 GMT-0400
7 Aug 13:27:14 - [info] [thingzi-timer:Dusk] Scheduled 'ON' (sun) for Mon Aug 07                                                                                              2023 21:09:28 GMT-0400
7 Aug 13:27:14 - [info] [thingzi-timer:Dawn] Scheduled 'ON' (sun) for Tue Aug 08                                                                                              2023 06:02:53 GMT-0400
7 Aug 13:27:14 - [info] Started flows
7 Aug 13:27:14 - [info] [serialconfig:71aef95893b7db0d] serial port /dev/ttyS0 o                                                                                             pened at 115200 baud 8N1
7 Aug 13:27:14 - [info] [tasmota-mqtt-broker:6e9362666c472291] Connected to brok                                                                                             er: mqtt://192.168.1.217:1883
7 Aug 13:27:14 - [info] [mqtt-broker:1a947cf4658331dc] Connected to broker: mqtt                                                                                             ://192.168.1.217:1883
7 Aug 13:27:29 - [info] Stopping modified nodes
7 Aug 13:27:29 - [info] Stopped modified nodes
7 Aug 13:27:29 - [info] Updated flows
7 Aug 13:27:29 - [info] Starting modified nodes
7 Aug 13:27:29 - [info] Started modified nodes

2 hours later things seem to have stabilized. Not sure what happened but things stabilized after I updated an esp8266 board I've been working on. Meanwhile I'm looking to move my install to a pi4 8G running on a usb drive.

Thanks to everyone who chimed in, I now have a sense of how to debug

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.