I am running Node-Red (v.0.19.4) with Node.js (v8.12) on a Raspberry Pi 3. After about a day the dashboard will not load or load with very minimal data, like just the logo in the top right corner. I get a lot of "failed to load resource: net::ERR_CONNECTION_TIMED_OUT" errors in the chrome console. Looking at htop, I have plenty of resources available. I tried increasing the open file limit. Not sure what else to try. The flows continue to operate normally and there is nothing unusual in the node-red log (even with verbose logging).
Does the issue happen if you use another browser?
Yes. It becomes unresponsive with any browser/computer. Weirdly, sometimes I have to restart the process twice to get it back. Nothing in the logs with the verbose option turned on.
so what do you have in your flows and on th dashboard?
Graphs?
any MQTT in the flows?
Any possibilities of loops in the flows?
All my flows are simple https endpoints that then trigger an api call to another server. No loops, mqtt or really anything weird. All the flow continue to operate even when the web interface does not. Since increasing the max number of open files in linux, it seems to have helped and the web interface has been stable all week.
I just realized I was running it from an open command-line and not with systemd. Perhaps that's why it's been stable all week. Restarted the service to see if that is the problem. So far nothing weird in the journal log. We'll see if it's still working tomorrow.
OK there is still a problem. I switched back to running as a systemd service and after about a week the dashboard became unresponsive. After another day the flows stopped working. Nothing in the log running in debug mode. The only difference is that the systemd script starts node-red with the node-red-pi script and the
--max-old-space-size=128 option. I tried changing the --max-old-space-size to 256, but the dashboard never loads.
Unless you a very early Pi1 with only 512MB of memory changing that should affect the initial loading of the dashboard at all...
The most likely cause is charts with too much data - do you have any charts ? and how many data points are they showing ? (typically a 6 unit wide graph is only 300 pixels so sending thousands of points is ill-advised)
It's a Pi3. Really nothing complex and not using UI at all, so no charts. Here is the output of the startup, anything weird? I know there is one DeprecationWarning.
===================
9 Oct 12:15:26 - [info] Node-RED version: v0.19.4
9 Oct 12:15:26 - [info] Node.js version: v8.12.0
9 Oct 12:15:26 - [info] Linux 4.14.52-v7+ arm LE
9 Oct 12:15:27 - [info] Loading palette nodes
9 Oct 12:15:27 - [debug] Module: node-red-node-random 0.1.0
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-node-random
9 Oct 12:15:27 - [debug] Module: node-red-contrib-ibm-watson-iot 0.2.8
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-contrib-ibm-watson-iot
9 Oct 12:15:27 - [debug] Module: node-red-contrib-slack 0.1.2
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-contrib-slack
9 Oct 12:15:27 - [debug] Module: node-red-contrib-slackbot 1.0.3
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-contrib-slackbot
9 Oct 12:15:27 - [debug] Module: node-red-contrib-sse 0.1.1
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-contrib-sse
9 Oct 12:15:27 - [debug] Module: node-red-contrib-voicerss-tts 0.0.1
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-contrib-voicerss-tts
9 Oct 12:15:27 - [debug] Module: node-red-node-ping 0.0.15
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-node-ping
9 Oct 12:15:27 - [debug] Module: node-red-contrib-play-audio 2.3.2
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-contrib-play-audio
9 Oct 12:15:27 - [debug] Module: node-red-node-serialport 0.6.8
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-node-serialport
9 Oct 12:15:27 - [debug] Module: node-red-node-smooth 0.1.0
9 Oct 12:15:27 - [debug] /root/.node-red/node_modules/node-red-node-smooth
9 Oct 12:15:27 - [debug] Module: node-red-node-email 0.1.29
9 Oct 12:15:27 - [debug] /usr/lib/node_modules/node-red/node_modules/node-red-node-em ail
9 Oct 12:15:27 - [debug] Module: node-red-node-feedparser 0.1.14
9 Oct 12:15:27 - [debug] /usr/lib/node_modules/node-red/node_modules/node-red-node-fe edparser
9 Oct 12:15:27 - [debug] Module: node-red-node-rbe 0.2.3
9 Oct 12:15:27 - [debug] /usr/lib/node_modules/node-red/node_modules/node-red-node-rb e
9 Oct 12:15:27 - [debug] Module: node-red-node-twitter 1.1.2
9 Oct 12:15:27 - [debug] /usr/lib/node_modules/node-red/node_modules/node-red-node-tw itter
9 Oct 12:15:31 - [info] Settings file : /root/.node-red/settings.js
9 Oct 12:15:31 - [info] Context store : 'default' [module=memory]
9 Oct 12:15:31 - [info] User directory : /root/.node-red
9 Oct 12:15:31 - [warn] Projects disabled : set editorTheme.projects.enabled=true to enable
9 Oct 12:15:31 - [info] Flows file : /root/.node-red/flows_callme.json
9 Oct 12:15:31 - [info] Server now running at https://127.0.0.1:443/
9 Oct 12:15:31 - [debug] loaded flow revision: f41a069f54cbe29e2cadb5d55d1d1039
9 Oct 12:15:31 - [debug] red/runtime/nodes/credentials.load : user provided key
9 Oct 12:15:31 - [debug] red/runtime/nodes/credentials.load : keyType=user
9 Oct 12:15:32 - [info] Starting flows
9 Oct 12:15:32 - [debug] red/nodes/flows.start : starting flow : global
9 Oct 12:15:32 - [debug] red/nodes/flows.start : not starting disabled flow : 7cd0f0bd.aa5b6
9 Oct 12:15:32 - [debug] red/nodes/flows.start : starting flow : 881e05c.06facf8
9 Oct 12:15:32 - [debug] red/nodes/flows.start : starting flow : 18bd1870.e23c98
9 Oct 12:15:32 - [debug] red/nodes/flows.start : starting flow : b2f65432.367ca8
9 Oct 12:15:32 - [debug] red/nodes/flows.start : starting flow : 7672f2de.39615c
9 Oct 12:15:32 - [debug] red/nodes/flows.start : starting flow : 1fcd5003.94b04
9 Oct 12:15:32 - [info] Started flows
(node:32668) [DEP0001] DeprecationWarning: OutgoingMessage.flush is deprecated. Use flushHead ers instead.Oh - so you mean it won't load the editor ? not dashboard ....
Are the incoming http calls high volume ? bursty ?
There will be some overhead due to the https on a PI.
Oh sorry, yes the editor. I finally found some error messages that are useful, but I don't know how to fix. Getting:
Error: EMFILE: too many open files, open '/usr/lib/node_modules/node-red/public/red/images/node-red.png'
Error: EMFILE: too many open files, open '/usr/lib/node_modules/node-red/public/red/images/spin.svg'
Ran:
lsof -i -n -P | grep node
and there are thousands of connections similar to this:
node-red 23962 root 14u IPv4 50409435 0t0 TCP 192.168.1.10:443->96.76.124.177:12555 (ESTABLISHED)
Seems like some sort of cleanup is not happening and over time overflowing max open files limits.
ulimit -n
500000
When connecting to a website, if Google Chrome browser fails to fetch the website to the browser, it throws an error saying "This site can't be reached Error" – ERR_CONNECTION_TIMED_OUT. It means the server is taking too much time to replying. The main reason behind this error is your computer can’t be able to access the internet connection or maybe something blocking your network to establishing a connection. Apart from the Network issue, there can be multiple reasons why this error shows up. Before going to any fix, please make sure the server you want to open is exist. If server exist, there are numerous solutions which can be used to solve this error.
Clear your Chrome browsing data
Check your Network Cables, Restart your Router and Reconnect
Check your Windows Host File
Remove Proxy
Flush DNS and reset TCP/IP
Run Chrome Cleanup Tool
Yes, I understand it is failing to connect. The reason why has nothing to do with my browser. Over time (couple of days) node-red stops responding. My resources don't seem exceeded and the process is still running. Nothing in the logs to indicate why.
That external IP address is a Comcast cable IP address. Is it your external address or something different?
My live system on a Pi2 has been up for months and only has 12 entries for that lsof -i -n -P | grep node command.
lsof -i -n -P | grep node for mine has 1700 entries almost immediately after being restarted. But I am running https://github.com/PointSource/node-red-contrib-sse the server sent events server with about 200 clients.
Just updated to latest node-red and updated a few modules. We'll see if that helps.
And the address?
It probably one of the clients connected, it's not the main address.
I think what I was really trying to get at is whether or not that is an Internet address and whether you know who owns that address? If you don't know who owns it, is it a valid address for your system to be connecting to? In other words, have you been infected with something?
I don't know, but whomever owns the IP also has an API key, so it's likely a legit address
1 Like
Sounds like this ip is not closing it's connections, but instead creating a new one for every request.. can you force close after each succesfull request?
