HOSED. I don't know where to start. Node-red not working / crashing etc

General
1

I successfully managed to HOSE my platform. Initially I seen some mongodb errors flood my debug once in Node-red... then it would lose connection.

I don't know where to begin... I have the console output. I suspect it was after an OS release upgrade to Jammy Ubuntu.

Here is the dump of the console...

Starting as a systemd service.
23 Aug 09:11:55 - [warn] [node-red-contrib-oauth2] Node module cannot be loaded on this version. Requires: 4.0.2
23 Aug 09:11:59 - [warn] ------------------------------------------------------
23 Aug 09:11:59 - [warn] [node-red-contrib-v8-cpu-profiler/v8-cpu-profiler] Error: Cannot find module '/home/ubuntu/.node-red/node_modules/v8-profiler-next/build/binding/Release/node-v115-linux-x64/profiler.node'
Require stack:
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/lib/binding.js
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/dispatch.js
- /home/ubuntu/.node-red/node_modules/node-red-contrib-v8-cpu-profiler/v8_cpu_profiler.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/loader.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/index.js
- /usr/lib/node_modules/node-red/lib/red.js
- /usr/lib/node_modules/node-red/red.js
23 Aug 09:11:59 - [warn] ------------------------------------------------------
23 Aug 09:11:59 - [warn] Missing node modules:
23 Aug 09:11:59 - [warn]  - node-red-contrib-oauth2 (4.0.1): oauth2, oauth2-credentials
23 Aug 09:11:59 - [warn] Projects disabled : editorTheme.projects.enabled=false
(node:9433) [DEP0128] DeprecationWarning: Invalid 'main' field in '/home/ubuntu/.node-red/node_modules/node-blob/package.json' of 'server.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)
23 Aug 09:11:59 - [error] Uncaught Exception:
23 Aug 09:11:59 - [error] Error: listen EADDRINUSE: address already in use 0.0.0.0:18800
    at Server.setupListenHandle [as _listen2] (node:net:1872:16)
    at listenInCluster (node:net:1920:12)
    at doListen (node:net:2069:7)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Scheduled restart job, restart counter is at 18.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
23 Aug 09:12:21 - [warn] [node-red-contrib-oauth2] Node module cannot be loaded on this version. Requires: 4.0.2
23 Aug 09:12:25 - [warn] ------------------------------------------------------
23 Aug 09:12:25 - [warn] [node-red-contrib-v8-cpu-profiler/v8-cpu-profiler] Error: Cannot find module '/home/ubuntu/.node-red/node_modules/v8-profiler-next/build/binding/Release/node-v115-linux-x64/profiler.node'
Require stack:
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/lib/binding.js
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/dispatch.js
- /home/ubuntu/.node-red/node_modules/node-red-contrib-v8-cpu-profiler/v8_cpu_profiler.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/loader.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/index.js
- /usr/lib/node_modules/node-red/lib/red.js
- /usr/lib/node_modules/node-red/red.js
23 Aug 09:12:25 - [warn] ------------------------------------------------------
23 Aug 09:12:25 - [warn] Missing node modules:
23 Aug 09:12:25 - [warn]  - node-red-contrib-oauth2 (4.0.1): oauth2, oauth2-credentials
23 Aug 09:12:25 - [warn] Projects disabled : editorTheme.projects.enabled=false
(node:9513) [DEP0128] DeprecationWarning: Invalid 'main' field in '/home/ubuntu/.node-red/node_modules/node-blob/package.json' of 'server.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)
23 Aug 09:12:25 - [error] Uncaught Exception:
23 Aug 09:12:25 - [error] Error: listen EADDRINUSE: address already in use 0.0.0.0:18800
    at Server.setupListenHandle [as _listen2] (node:net:1872:16)
    at listenInCluster (node:net:1920:12)
    at doListen (node:net:2069:7)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Scheduled restart job, restart counter is at 19.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
23 Aug 09:12:47 - [warn] [node-red-contrib-oauth2] Node module cannot be loaded on this version. Requires: 4.0.2
23 Aug 09:12:51 - [warn] ------------------------------------------------------
23 Aug 09:12:51 - [warn] [node-red-contrib-v8-cpu-profiler/v8-cpu-profiler] Error: Cannot find module '/home/ubuntu/.node-red/node_modules/v8-profiler-next/build/binding/Release/node-v115-linux-x64/profiler.node'
Require stack:
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/lib/binding.js
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/dispatch.js
- /home/ubuntu/.node-red/node_modules/node-red-contrib-v8-cpu-profiler/v8_cpu_profiler.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/loader.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/index.js
- /usr/lib/node_modules/node-red/lib/red.js
- /usr/lib/node_modules/node-red/red.js
23 Aug 09:12:51 - [warn] ------------------------------------------------------
23 Aug 09:12:51 - [warn] Missing node modules:
23 Aug 09:12:51 - [warn]  - node-red-contrib-oauth2 (4.0.1): oauth2, oauth2-credentials
23 Aug 09:12:51 - [warn] Projects disabled : editorTheme.projects.enabled=false
(node:9591) [DEP0128] DeprecationWarning: Invalid 'main' field in '/home/ubuntu/.node-red/node_modules/node-blob/package.json' of 'server.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)
23 Aug 09:12:51 - [error] Uncaught Exception:
23 Aug 09:12:51 - [error] Error: listen EADDRINUSE: address already in use 0.0.0.0:18800
    at Server.setupListenHandle [as _listen2] (node:net:1872:16)
    at listenInCluster (node:net:1920:12)
    at doListen (node:net:2069:7)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Scheduled restart job, restart counter is at 20.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
23 Aug 09:13:12 - [warn] [node-red-contrib-oauth2] Node module cannot be loaded on this version. Requires: 4.0.2
23 Aug 09:13:16 - [warn] ------------------------------------------------------
23 Aug 09:13:16 - [warn] [node-red-contrib-v8-cpu-profiler/v8-cpu-profiler] Error: Cannot find module '/home/ubuntu/.node-red/node_modules/v8-profiler-next/build/binding/Release/node-v115-linux-x64/profiler.node'
Require stack:
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/lib/binding.js
- /home/ubuntu/.node-red/node_modules/v8-profiler-next/dispatch.js
- /home/ubuntu/.node-red/node_modules/node-red-contrib-v8-cpu-profiler/v8_cpu_profiler.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/loader.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/registry/lib/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/index.js
- /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/index.js
- /usr/lib/node_modules/node-red/lib/red.js
- /usr/lib/node_modules/node-red/red.js
23 Aug 09:13:16 - [warn] ------------------------------------------------------
23 Aug 09:13:16 - [warn] Missing node modules:
23 Aug 09:13:16 - [warn]  - node-red-contrib-oauth2 (4.0.1): oauth2, oauth2-credentials
23 Aug 09:13:16 - [warn] Projects disabled : editorTheme.projects.enabled=false
(node:9668) [DEP0128] DeprecationWarning: Invalid 'main' field in '/home/ubuntu/.node-red/node_modules/node-blob/package.json' of 'server.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)
23 Aug 09:13:16 - [error] Uncaught Exception:
23 Aug 09:13:16 - [error] Error: listen EADDRINUSE: address already in use 0.0.0.0:18800
    at Server.setupListenHandle [as _listen2] (node:net:1872:16)
    at listenInCluster (node:net:1920:12)
    at doListen (node:net:2069:7)
    at process.processTicksAndRejections (node:internal/process/task_queues:83:21)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.

!!

2

I might be wrong on the Jammy,
its actually

Distributor ID: Ubuntu

Description: Ubuntu 20.04.6 LTS

Release: 20.04

Codename: focal```
3

The error EADDRINUSE usually means that Node-red is already running, for instance because it is run at startup from a systemd script.

IP address 0.0.0.0 and port 18800 look distinctly odd too.

4

What happens if you try to start in safe mode by running
node-red --safe

What version of node-red are you using?
What version of nodejs (use node -v to get that).

5

hey jbudd,

Thanks for pointing that out... I am aware of this issue... My main issue is I can't get into node red at all despite it being ran on the port.

You have a keen eye on the port change, which I have for good reason, as some creeps out there like to look for node-red on it's original port. I'm explicitly using port 18800 in hopes to avoid the hackers trying to portscan for node-red on 1880.

I have had a machine in the past that wasn't secured during testing, and in short time they managed to find my node-red and put a crypto miner on my server... but I digress.

6

Did you update nodejs at some point? The v8 errors are common for when a node has compiled components that have not been recompiled.

7

Also, since you are running on ubuntu, did you use the official script to install?

8

Actually I don't think it is the upgrade, because after I did upgrate to Ubuntu 20 from 18, I had ironed out any issues and it was working to my satisfaction.

Originally I came back into NR today to find there we mongodb4 errors flooding my debug console in NR, and I couldn't connect.
Then NR would keep disconnecting.

Not sure what is going on.

The mongodb4 error I was getting looked like this:
node-red MongoNetworkError: 00180F56777F0000:error:0A000438:SSL routines:ssl3_read_bytes:tlsv1 alert internal error:../deps/openssl/openssl/ssl/record/rec_layer_s3.c:1586:SSL alert number 80

I'm not sure what I did, but now I simply can't get into NR at all.

Doing a node-red-start freezes after Starting as a system service

9

Node-red?
I always use the official script to install.
Granted though I always opt for the RPi script to install NR, but I don't do the RPi extras as asked in the installer.
I might try to reinstall NR via the script.

10

You didn't answer my questions.

However, I think you might be best to run the install script as you suggest.
Add --node18 to the end to force it to install a good version.

11

Hey Colin,

I apologize I did not see your question.

I was originally running 16.
I have ran the script installer with the --node18 and I am still experiencing the same issues.

I check the node -v and it still at version 16

Safe mode allows me to log into NR.

Here is the output from safe mode:

$ node-red --safe
(node:23998) [DEP0128] DeprecationWarning: Invalid 'main' field in '/home/ubuntu/.node-red/node_modules/node-blob/package.json' of 'server.js'. Please either fix that or report it to the module author
(Use `node --trace-deprecation ...` to show where the warning was created)
23 Aug 10:07:49 - [warn] Projects disabled : editorTheme.projects.enabled=false

I have been down this road before when dealing with the stick old version of Node not getting replaced by the NR script installer.
Not sure what is going on in that case.

12

Do these commands work without error?
sudo apt update
sudo apt full-upgrade

13

Further: the script installer had something I have not seen before RE: TLS

Using NVM to manage Node.js + please run nvm use lts before running Node-RED

$ nvm use lts
N/A: version "lts -> N/A" is not yet installed.
$ nvm install lts
Version 'lts' not found - try `nvm ls-remote` to browse available versions.

That is where I left off... I can't seem to nvm install lts@18.17.1 or anything .. I know I'm doing something wrong installing that.

14

Don't use nvm unless you need it for some reason. Uninstall it and delete ~/.nvm. Then try the script again.

15 
Reading package lists... Done
Building dependency tree       
Reading state information... Done
Calculating upgrade... Done
Get more security updates through Ubuntu Pro with 'esm-apps' enabled:
  python2.7-minimal python2.7 libgsasl7 ruby-rack libjs-jquery-ui
  libpython2.7-minimal libpython2.7-stdlib
Learn more about Ubuntu Pro on AWS at https://ubuntu.com/aws/pro
#
# You can verify the status of security fixes using the `pro fix` command.
# E.g., a recent Ruby vulnerability can be checked with: `pro fix USN-6219-1`
# For more detail see: https://ubuntu.com/security/notices/USN-6219-1
#
The following packages will be upgraded:
  docker.io sosreport
2 upgraded, 0 newly installed, 0 to remove and 0 not upgraded.
Need to get 31.1 MB of archives.
After this operation, 1202 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Abort.

sudo apt update works no error. But as above sudo apt full-upgrade it aborts after I enter y

16

You have a big problem then. Does it run if you do
sudo apt full-upgrade -y

17

Ya.. That seems to do the trick getting the sudo apt full-upgrade -y to work.

Removed the ~/.nvm directory and will try the script again.

18

Colin, thank you so much for your help.
I seem to now be back to normal with operation.
I have no clue what happened that caused this issue.

After the your tip to get the sudo apt full-upgrade -y working after the abort error allowed me to rerun the NR install script, and this time it did not say anything about the tls when using the --node18 argument.

And now NR runs with minimal complains in the console:

(node:29034) NOTE: We are formalizing our plans to enter AWS SDK for JavaScript (v2) into maintenance mode in 2023.
Please migrate your code to use AWS SDK for JavaScript (v3).```

Thanks again!
19

Somehow you started using nvm, or if you were using it before then you were lucky in that the version of nodejs in .nvm was the same as the apt installed one.

The problem was nvm, not the -y issue. I don't know what that is and you might run into it again at some point.

1 Like
20

That will be this or a related issue. Possibly related to a node you have installed.