Node-Red v1.2.7 Crashing

General
1

Hi,

I have been using node-red v1.X (don’t remember the exact version) for well over a year. Everything has been extremely stable – no crashes.

About 1 one month ago, I upgraded to node-red v1.2.7. Shortly after, I also started using the node-red-mysql node. I was getting the “Pool is closed” error. I searched the forums and found suggestions on how to solve this problem, i.e deleted all the mysql nodes and the configuration node and restarted node-red and my raspi 4. Overall the “Pool is closed” error went away, however once in a while it comes back and I have to restart node-red and then things run well for a while, but eventually the problem comes back.

The bigger problem I am having is node-red is crashing – sometimes on its own or just as I start to deploy. Not sure if it this is related to the mysql node, the node-red upgrade, or something I am now doing wrong? I am attaching the log file which I hope will have the necessary information to solve this problem.

25 Feb 15:30:58 - [info] Stopping flows
25 Feb 15:30:58 - [info] [MSSQL-CN:2MXPRO7] Disconnecting server : 192.168.21.236, database : CanLine, port : 1433, user : 192.168.21.236
25 Feb 15:30:58 - [red] Uncaught Exception:
25 Feb 15:30:58 - Error [ERR_INTERNAL_ASSERTION]: This is caused by either a bug in Node.js or incorrect usage of Node.js internals.
Please open an issue with this stack trace at https://github.com/nodejs/node/issues
    at assert (internal/assert.js:14:11)
    at JSStreamSocket.doWrite (internal/js_stream_socket.js:165:5)
    at JSStream.onwrite (internal/js_stream_socket.js:33:57)
    at Socket.ondata (internal/js_stream_socket.js:77:22)
    at Socket.emit (events.js:314:20)
    at addChunk (_stream_readable.js:298:12)
    at readableAddChunk (_stream_readable.js:273:9)
    at Socket.Readable.push (_stream_readable.js:214:10)
    at TCP.onStreamRead (internal/stream_base_commons.js:188:23)
nodered.service: Main process exited, code=exited, status=1/FAILURE
nodered.service: Failed with result 'exit-code'.
nodered.service: Service RestartSec=20s expired, scheduling restart.
nodered.service: Scheduled restart job, restart counter is at 1.
Stopped Node-RED graphical event wiring tool.
Started Node-RED graphical event wiring tool.
25 Feb 15:31:19 - [info]
Welcome to Node-RED
===================
25 Feb 15:31:19 - [info] Node-RED version: v1.2.7
25 Feb 15:31:19 - [info] Node.js  version: v12.19.1
25 Feb 15:31:19 - [info] Linux 5.4.72-v7l+ arm LE
25 Feb 15:31:20 - [info] Loading palette nodes
25 Feb 15:31:22 - [info] +-----------------------------------------------------
25 Feb 15:31:22 - [info] | uibuilder initialised:
25 Feb 15:31:22 - [info] |   root folder: /home/pi/.node-red/uibuilder
25 Feb 15:31:22 - [info] |   version . .: 3.1.3
25 Feb 15:31:22 - [info] |   packages . : jquery,socket.io
25 Feb 15:31:22 - [info] +-----------------------------------------------------
25 Feb 15:31:23 - [info] Dashboard version 2.27.0 started at /ui
25 Feb 15:31:23 - [info] Settings file  : /home/pi/.node-red/settings.js
25 Feb 15:31:23 - [info] Context store  : 'default' [module=memory]
25 Feb 15:31:23 - [info] User directory : /home/pi/.node-red
25 Feb 15:31:23 - [warn] Projects disabled : editorTheme.projects.enabled=false
25 Feb 15:31:23 - [info] Flows file     : /home/pi/.node-red/flows_raspberrypi.json
25 Feb 15:31:23 - [info] Server now running at http://127.0.0.1:1880/
25 Feb 15:31:24 - [info] Starting flows
25 Feb 15:31:24 - [info] [tcp in:9b3b85c9.e68da8] connecting to 192.168.21.231:10001
25 Feb 15:31:24 - [info] Started flows
25 Feb 15:31:24 - [info] [tcp in:9b3b85c9.e68da8] connected to 192.168.21.231:10001

I am far from a computer expert and do not have a github account. I am running node.js v12.19.1 on a raspi 4. I upgraded node-red using the recommended scripts I found on the node-red site:

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

I am hoping that someone can help me resolve this intermittent problem. I realize that I may need to provide more information in order to solve this problem – which occurs a couple of times a day. I have also been trying to do more backups as of today because when node-red restarted, all the flows were empty. I stopped node-red and then restarted it. Fortunately all the flows came back.

2

I know this isn't exactly helpful, but is something of concern.

Ok, thought:
Elsewhere on the computer (say Home/BACKUP)

node-red-stop

MOVE all the .json files there.

node-red-start

What happens?

3

So here is what I did:

Created new directory under home/pi called backuptest.
Copied ALL files (not just JSON) to home/pi/backuptest.
Started node red from home/pi/backuptest

Everything works like before. No issues. What were you expecting?

4

I wasn't sure the flow (code?) was causing the problem or not.

Sorry, it is way beyond my skill set.

Push coming to shove: Uninstall/remove any extra/new nodes and see what happens.

Maybe one of the nodes is causing the problem.

5

@Curious in your original description you say you are using the node-red-mysql node but the error is reporting a [MSSQL-CN:2MXPRO7] which I beleive is from MSSQl, not MySQL.

So my question s are you using MSSQL or MySQL and if you are using MSSQL why aren't you using an mssql node?

6

Actually I am using both MSSQL and MySQL. MSSQL has been installed since I first started using node-red and MySQL since the upgrade. The MSSQL and MySQL nodes seem to work correctly, although every once in a while I still get the MySQL "Pool is closed" error, eve after completely deleting these nodes and configuration.

In the app both SQL nodes are enabled but no messages flow to the MySQL or the MSSQL nodes depending on how the "switches" are set up. The "Pool is Closed" error has decreased significantly since I deleted the nodes along with the configuration node, nevertheless the problem remains but not as bad as it was. (I brought up this error as I added the MySQL nodes AFTER the node-red upgrade as I wsn't sure if it could be the cause but wanted to share as much information as possible).

I was thinking of Disabling the MySQL nodes (disable - not to uninstall them, at least for now), and then observe if Node-Red still crashes?

To be honest, node-red has been very stable prior to the node-red upgrade and the installation of MySQL. Only within the last month have I been having problems.

Are there any other known reasons as to why node-red would crash, especially during the start of deployment? Has any one else had these problems? Are there any other log files that could be looked at?

Again, my experience and knowledge in this area is very limited, so I am really grasping at straws and do not know what else I should be looking for?

7

I you can check syslog and node-red logs to find the moment it crashes, you will get a stack trace & will will be able to pin point the issue. Until then it is guess work.

Search the forum on how to / where logs are (it has been covered hundreds of times).

Then paste a copy to a reply.

8

Which mssql node have you installed? I believe that node-red-contrib-mssql is not maintained and should not be used. I think that node-red-contrib-mssql-plus is generally recommended. If you are not currently using it then I suggest uninstalling it, and when you get to the stage of requiring it again then use the new one.

I am confused by this, which nodes did you delete and are still getting the pool error?

9

Thank you for your replies. I am using the node-red-contrib-mssql-plus node v0.6.4. This has been working very well - no errors.

It is the node-red-node-mysql v0.1.1 that has been giving me errors. Is this the version I should be using?

As per your suggestion, I will check the forum regarding the logs to see what I can find.

10

Do you know it is that node that is causing the crash? There is no evidence of that in the log that I can see. Since it was immediately preceded by an MSSQL message that may be a clue that that node is involved, hence my post.
To increase the amount of information you get when it crashes I suggest editing .node-red/settings.js and setting the log level to trace. Then when it next crashes you may get more clues.

11

Just a quick update. I have increased the level of logging in the trace. Below is a partial extract:

6 Feb 07:58:32 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:33 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:33 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:33 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:34 - [error] [MySQLdatabase:7994ff75.b9322] Error: Cannot enqueue Query after fatal error.
26 Feb 07:58:34 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:34 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:34 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:36 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:36 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:40 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout
26 Feb 07:58:40 - [error] [MySQLdatabase:7994ff75.b9322] Error: Handshake inactivity timeout

Just to clarify, I am running both MSSQL and MySQL on a development computer. Node-Red will write data to one or the other SQL Server depending on the selection I have made. I use switch nodes to route the msgs to the desired SQL Server nodes. When I change the selection, I use the console.log to send a msg to indicate the SQL nodes was changed. That is why “MSSQL” would show up in the log file.

I believe that the issue happens when the development computer goes into Windows "sleep" mode while MySQL is selected. (This does not happen when MSSQL is selected). Even when I "wake" up the development computer, the errors continue. (When the "Handshake inactivity timeout" occurs, I believe that Node-Red is trying to reconnect, but is not able to). The only way to recover Node-Red and stop the errors, is to stop and to restart Node-Red. One would think that since the MySQL node is trying to reconnect, it should eventually be able to reconnect and stop throwing errors? This appears to be the problem: the inability to reconnect after throwing this error.

The good news is that I haven't experienced any severe Node-Red crashes recently, which seem only to happen when I would deploy some new code. I now believe that this was related to the MySQL Handshake errors which makes the system UNSTABLE. Even deploying an empty function block would cause Node-Red crash, but only when the system was in an error state with MySQL.

I will continue to monitor and advise if I find anything else. Has anyone else experienced this problem with MySQL and Node-Red?

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.