Node Red 3, Crashing After Reboot? Seems like npm (db repo) is getting corrupted?

(error resulting)
Starting as a systemd service.
nodered.service: Main process exited, code=killed, status=4/ILL
nodered.service: Failed with result 'signal'.

Steps to recreate issue...

  1. install Pi OS 11
  2. apt update
  3. apt full-upgrade
  4. install NR via the standard script --node16, since script reports reports bad Node install otherwise. There is no Node install on a clean build.
  5. used the following settings file (see attached)
  6. loaded the following modules via npm, stopping and restarting NR each time to test
  7. reboot as test, KABOOM,

List of modules installed listed in order installed...

(already in pace from script)
node-red
node-red-contrib-play-audio
node-red-node-pi-gpio
node-red-node-ping
node-red-node-random
node-red-node-serialport
node-red-node-smooth

(installed following stop and start of NR)
npm install @tuyapi/cli i2c-bus javascript-color-gradient n os debug@4.3.1
node-red-dashboard
node-red-node-mysql
node-red-node-random
node-red-node-ui-list
node-red-node-ui-table
node-red-contrib-ui-led
node-red-node-ping
node-red-node-pi-gpiod
node-red-node-snmp
node-red-node-email

This is consistent, have used 3 different SD cards, and clean image per SD card.

What other information is needed? Cannot seem to find any solution, even steps for same error in forum do not seem to be applicable. Same Pi 3B running same modules and NR 2.2.2 rock solid. Node version 14 via ---node14.

Settings file...

module.exports = {
        flowFile: 'flows.json',
        credentialSecret: 'BOZO',
        flowFilePretty: true,
        uiPort: process.env.PORT || 1880,
        logging: {
                console: {
                        level: 'info',
                        metrics: false,
                        audit: false
                }
        },
        exportGlobalContextKeys: false,
        contextStorage: {
                default: {
                        module:'memory'
                },
                persistent: {
                        module:'localfilesystem'
                },
        },
        externalModules: {
        },
        editorTheme: {
                tours: false,
                palette: {
                },
                projects: {
                        enabled: false,
                        workflow: {
                                mode: 'manual'
                        }
                },
                codeEditor: {
                        lib: 'ace',
                        options: {
                                theme: 'vs',
                        }
                }
        },
        functionExternalModules: true,
        functionGlobalContext: {
                os:require('os'),
                i2c:require('i2c-bus'),
                gradient:require('javascript-color-gradient'),
        },
        debugMaxLength: 1000,
        mqttReconnectTime: 15000,
        serialReconnectTime: 15000,
}

Packge.json

{
  "name": "node-red-project",
  "description": "initially created for you by Node-RED 3.0.0",
  "version": "0.0.1",
  "private": true,
  "dependencies": {
    "node-red-contrib-play-audio": "^2.5.0",
    "node-red-node-pi-gpio": "^2.0.5",
    "node-red-node-ping": "^0.3.1",
    "node-red-node-random": "^0.4.0",
    "node-red-node-serialport": "^1.0.1",
    "node-red-node-smooth": "^0.1.2"
  }
}

npm list...
$ npm list
Illegal instruction


**npm list was not crashing before reboot!**

I am going reimage... Install NR, and do reboots after each module install to see if I can figure out when npm is getting trashed.  REALLY weird!  Since I have been using 3 different SD cards.

Forgot to mention the 3B is 64bit version of Pi OS 11... Not sure that is material but maybe there is hidden gotcha with Node or npm I am not aware of?

I would suspect either i2c-bus or n to start with. I2c-bus as it has a binary component that may not be happy with 64bit, and n because it is a node version manager. What do you need it at all ? It always makes things worse.

If npm is crashing then it is nothing to do with node-red.

Are you using n? Dave suggests you are but I can't see where that is mentioned.

If not then what does
apt policy nodejs
show, also (running in the .node-red folder)
node -v
npm -v
which node
which npm
and post the node red install log here. /var/log/nodered-install.log.

Right, that is why I qualified the title as I did. NR is downstream of the real issue, which I believe I isolated, the TUYA API generates a warning about issues with the javascript debug module have some regressive issues. I tried to use the recommended version of debug which is less than 3.2.7 or greater than 4.3.0. The versions in between have this 'bug' TUYA notes. Once I moved to debug module 4.3.4, the npm corruption seems to resolve.

I also explicitly tested the SD cards using 'SD card formatter' they all passed but that has happened before, that for some reason a Pi device does not like a given SD card when the formatter passes it. But I digress. I finally used an almost new SD card.

Right now the only issue I am having is NR 3.0.0 is still crashing at times when I access the dashboard, I get a connection lost, and when I check the systemctl status, nodred has failed. I am working on tracking this down. Reviewing the logs, the following seems suspect.

Jul 20 13:19:47 eccentric Node-RED[3719]: 20 Jul 13:19:47 - [info] [mqtt-broker:Broker] Connected to broker: mqtt://broker.dachshund-digital.org:1883
Jul 20 13:19:47 eccentric Node-RED[3719]: 20 Jul 13:19:47 - [info] [wifiscan:Scan] wifiscan - Input received
Jul 20 13:19:47 eccentric Node-RED[3719]: 20 Jul 13:19:47 - [red] Uncaught Exception:
Jul 20 13:19:47 eccentric Node-RED[3719]: 20 Jul 13:19:47 - [error] Error: No connection has been made to the device.
Jul 20 13:19:47 eccentric Node-RED[3719]:     at TuyaDevice._send (/home/pi/.node-red/node_modules/node-red-contrib-tuya-local/node_modules/tuyapi/index.js:235:13)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at /home/pi/.node-red/node_modules/node-red-contrib-tuya-local/node_modules/tuyapi/index.js:117:14
Jul 20 13:19:47 eccentric Node-RED[3719]:     at new Promise (<anonymous>)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at TuyaDevice.get (/home/pi/.node-red/node_modules/node-red-contrib-tuya-local/node_modules/tuyapi/index.js:114:12)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at setDevice (/home/pi/.node-red/node_modules/node-red-contrib-tuya-local/tuya-local.js:42:12)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at TuyaNode._inputCallback (/home/pi/.node-red/node_modules/node-red-contrib-tuya-local/tuya-local.js:122:4)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at /usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:210:26
Jul 20 13:19:47 eccentric Node-RED[3719]:     at Object.trigger (/usr/lib/node_modules/node-red/node_modules/@node-red/util/lib/hooks.js:166:13)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at TuyaNode.Node._emitInput (/usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:202:11)
Jul 20 13:19:47 eccentric Node-RED[3719]:     at TuyaNode.Node.emit (/usr/lib/node_modules/node-red/node_modules/@node-red/runtime/lib/nodes/Node.js:186:25)
Jul 20 13:19:47 eccentric systemd[1]: nodered.service: Main process exited, code=exited, status=1/FAILURE

Right now to me, it makes no sense that the TUYA API not being able to talk to a IoT device could chain react is such a why that it causes Node fault? Then downstream, NR to stop? This was not happening under my previous NR 2.2.2 environment, which was Node 14 based. Whereas now NR 3.0 and Node 16. The same error happened once in a while in the old environment, seems more frequent now. Maybe just their code is struggling more under Node 16/NR 3.0 based envrionment. TUYA definitely has something sloppy under the hood.

The last step, I will do later today, is build an image with out the TUYA API modules, and I suspect it will be solid.

The npm illegal instruction was a surprise, and has not occurred again, once I moved to the newer SD card. The 3 SD cards that exhibited the issue were all the same vendor, so once I had a working image, I reimaged these SD cards from the working image, rather than using the default Pi OS image and loading individual files, that worked so far, but the cards may just be suspect, only time will tell.

Still testing, but once I disabled the TUYA flow (which was using node-red-contrib-tuya-local) and uninstalled @tuyapi/cli@1.18.3, the connection issues appear to have resolved. So, moving to Node 16 and NR 3.0, the current TUYA API module freaks out. Again, NR is just downstream impacted IMHO.

Kind of a bummer, because I had to abandon SmartLife Air for TUYA compatible IoT devices, now it appears I can't use the node-red-contrib-tuya-local module?

Pretty sure its been abandoned (and your issue is reported here: Following error happens often and at times seems to be crashing Node and thus Node Red? · Issue #36 · subzero79/node-red-contrib-tuya-local · GitHub)

Additionally, it references tuyapi v5 (latest is v7.3)

You might want to try out node-red-contrib-tuya-smart-device which was published only 21 days ago and uses tuyapi v7.3 under the hood.

Yeah, started to try 'smart-device' variant but the other seemed easier to implement, guess not! The issue reported was by me in fact.

@Colin,. this specific Pi device is my data collector for the entire environment, so it tracks version information and other things, hence by 'n' used.

@dceejay, as for i2c-bus, I have been using it for some time and since I have long history with i2c, I keep an eye on its implementation code in Node. It was the first thing I ruled out in this case. This specific Pi also has a few sensors that I can't directly integrate to NR, since there are not existing modules for the specific sensors. I started writing nodes for the sensors to add to NR, but that project keeps sliding back in the queue as other projects are prioritized higher.

good to hear - we have heard other reports of issues on node18 on 64bit Raspi, so keep an eye open.

Again, under Node 14 and NR 2.2.2 everything has been fine for more than a year, starting to move to NR 3.0 has just exposed some quirks. The npm illegal instruction fault was zinger, but since I moved to a different SD card vendor, that has not happened again. I still need to create a 32 baseline, but once I removed the TUYA components everything has been stable so far.

Yeah, I once had a few issues with 16, but that was when I was pushing the edge, and tried 16 really too early in the life cycle. I have no plans to jump to 18 as yet, but good to know that there has been buzz about it. Thanks for the heads up!