I noticed that repetitive deployments increase the CPU load. I noticed it first on 0.19.5 but it still holds for 0.20.2. Stopping and starting Node Red reduces the load to a normal level.
I did several (like 20 to 30) deployments during 2000 and 2300.
I'm running NR on a Raspberry Pi 3 running Raspian, Node v10.15.0.
Is this a known behaviour?
Hi @guardiande. No, this isn't a known behaviour.
I can't say we explicitly check this sort of thing in our tests, but we haven't had any similar reports.
It could well be a node you are using failing to free up some resource or stop some processing when the deploy is done.
Tracking down that node would require some detective work. What nodes are you using in your flows?
You also need to check some other metrics on your Pi. Such as SWAP use. Because the Pi's typically use SD cards for storage, paging things from memory to swap and back has a significant overhead.
Without understanding a lot more about what is running on the Pi, how active things are and so on, it is hard to make a judgement.
This is from my Pi3:
And this from my much busier Pi2:
As you can see, under relatively busy conditions, the loads are so variable it would be hard to pick out a trend like yours.
What I can see though is that your average CPU load is quite high. I think my Pi2 is quite busy but average overall is relatively low still.
Here's the full list of installed nodes:
Nodes Types State
node-red-contrib-alexa-local/alexa-local alexa-local enabled
node-red-contrib-better-sonos/better-sonos-config better-sonos-config enabled
node-red-contrib-better-sonos/better-sonos-control better-sonos-control enabled
node-red-contrib-better-sonos/better-sonos-get-queue better-sonos-get-queue enabled
node-red-contrib-better-sonos/better-sonos-queue better-sonos-queue enabled
node-red-contrib-better-sonos/better-sonos-status better-sonos-status enabled
node-red-contrib-bigtimer/bigtimer bigtimer enabled
node-red-contrib-cast/cast-to-client cast-to-client enabled
node-red-contrib-contextbrowser/contextbrowser contextbrowser enabled
contextbrowser-sidebar
node-red-contrib-eztimer/eztimer eztimer enabled
node-red-contrib-flow-combine/Flow-Combine Combine Start enabled
Combine End
node-red-contrib-fritz/fritzbox fritzbox-config enabled
fritzbox-in
fritzbox-calllist
fritzbox-phonebook
node-red-contrib-fritz/fritzbox-callmonitor fritzbox-callmonitor enabled
node-red-contrib-fritz/fritzbox-contact fritzbox-contact enabled
node-red-contrib-huemagic/hue-bridge hue-bridge enabled
node-red-contrib-huemagic/hue-bridge-node hue-bridge-node enabled
node-red-contrib-huemagic/hue-brightness hue-brightness enabled
node-red-contrib-huemagic/hue-group hue-group enabled
node-red-contrib-huemagic/hue-light hue-light enabled
node-red-contrib-huemagic/hue-motion hue-motion enabled
node-red-contrib-huemagic/hue-scene hue-scene enabled
node-red-contrib-huemagic/hue-switch hue-switch enabled
node-red-contrib-huemagic/hue-tap hue-tap enabled
node-red-contrib-huemagic/hue-temperature hue-temperature enabled
node-red-contrib-influxdb/influxdb influxdb enabled
influxdb out
influxdb batch
influxdb in
node-red-contrib-lgtv/lgtv-app lgtv-app enabled
node-red-contrib-lgtv/lgtv-browser lgtv-browser enabled
node-red-contrib-lgtv/lgtv-button lgtv-button enabled
node-red-contrib-lgtv/lgtv-channel lgtv-channel enabled
node-red-contrib-lgtv/lgtv-config lgtv-config enabled
node-red-contrib-lgtv/lgtv-control lgtv-control enabled
node-red-contrib-lgtv/lgtv-mouse lgtv-mouse enabled
node-red-contrib-lgtv/lgtv-mute lgtv-mute enabled
node-red-contrib-lgtv/lgtv-request lgtv-request enabled
node-red-contrib-lgtv/lgtv-toast lgtv-toast enabled
node-red-contrib-lgtv/lgtv-volume lgtv-volume enabled
node-red-contrib-lgtv/lgtv-youtube lgtv-youtube enabled
node-red-contrib-light-scheduler/light-scheduler light-scheduler enabled
node-red-contrib-light-scheduler/light-scheduler-settings light-scheduler-settings enabled
node-red-contrib-moment/humanizer humanizer enabled
node-red-contrib-moment/moment moment enabled
node-red-contrib-openhab2/openhab2 openhab2-controller enabled
openhab2-in
openhab2-get
openhab2-monitor
openhab2-out
openhab2-events
node-red-contrib-persist/persist persist-store enabled
persist in
persist out
node-red-contrib-pushover/pushover pushover-keys enabled
pushover api
glances
node-red-contrib-schedex/schedex schedex enabled
node-red-contrib-simple-gate/gate gate enabled
node-red-contrib-stoptimer/stoptimer stoptimer enabled
node-red-contrib-string/string string enabled
node-red-contrib-sun-position/moon-position moon-position enabled
node-red-contrib-sun-position/position-config position-config enabled
node-red-contrib-sun-position/sun-position sun-position enabled
node-red-contrib-sun-position/time-comp time-comp enabled
node-red-contrib-sun-position/time-inject time-inject enabled
node-red-contrib-sun-position/time-span time-span enabled
node-red-contrib-sun-position/within-time-switch within-time-switch enabled
node-red-contrib-time-range-switch/time-range-switch time-range-switch enabled
node-red-contrib-timerswitch/timerswitch timerswitch enabled
node-red-contrib-traffic/traffic traffic enabled
node-red-contrib-uibuilder/uibuilder uibuilder enabled
node-red-contrib-wait-paths/wait-paths wait-paths enabled
node-red-dashboard/ui_audio ui_audio enabled
node-red-dashboard/ui_base ui_base enabled
node-red-dashboard/ui_button ui_button enabled
node-red-dashboard/ui_chart ui_chart enabled
node-red-dashboard/ui_colour_picker ui_colour_picker enabled
node-red-dashboard/ui_date_picker ui_date_picker enabled
node-red-dashboard/ui_dropdown ui_dropdown enabled
node-red-dashboard/ui_form ui_form enabled
node-red-dashboard/ui_gauge ui_gauge enabled
node-red-dashboard/ui_group ui_group enabled
node-red-dashboard/ui_link ui_link enabled
node-red-dashboard/ui_numeric ui_numeric enabled
node-red-dashboard/ui_slider ui_slider enabled
node-red-dashboard/ui_spacer ui_spacer enabled
node-red-dashboard/ui_switch ui_switch enabled
node-red-dashboard/ui_tab ui_tab enabled
node-red-dashboard/ui_template ui_template enabled
node-red-dashboard/ui_text ui_text enabled
node-red-dashboard/ui_text_input ui_text_input enabled
node-red-dashboard/ui_toast ui_toast enabled
node-red-dashboard/ui_ui_control ui_ui_control enabled
node-red-node-email/email e-mail enabled
e-mail in
node-red-node-feedparser/feedparse feedparse enabled
node-red-node-rbe/rbe rbe enabled
node-red-node-sentiment/sentiment sentiment enabled
node-red-node-tail/tail tail enabled
node-red-node-timeswitch/timeswitch timeswitch enabled
node-red-node-twitter/twitter twitter-credentials enabled
twitter in
twitter out
node-red/batch batch enabled
node-red/catch catch enabled
node-red/change change enabled
node-red/comment comment enabled
node-red/CSV csv enabled
node-red/debug debug enabled
node-red/delay delay enabled
node-red/exec exec enabled
node-red/file file enabled
file in
node-red/function function enabled
node-red/HTML html enabled
node-red/httpin http in enabled
http response
node-red/httpproxy http proxy enabled
node-red/httprequest http request enabled
node-red/inject inject enabled
node-red/JSON json enabled
node-red/link link in enabled
link out
node-red/mqtt mqtt in enabled
mqtt out
mqtt-broker
node-red/range range enabled
node-red/rpi-gpio rpi-gpio in enabled
rpi-gpio out
rpi-mouse
rpi-keyboard
node-red/sort sort enabled
node-red/split split enabled
join
node-red/status status enabled
node-red/switch switch enabled
node-red/tcpin tcp in enabled
tcp out
tcp request
node-red/template template enabled
node-red/tls tls-config enabled
node-red/trigger trigger enabled
node-red/udp udp in enabled
udp out
node-red/unknown unknown enabled
node-red/watch watch enabled
node-red/websocket websocket in enabled
websocket out
websocket-listener
websocket-client
node-red/XML xml enabled
node-red/YAML yaml enabled
I quite extensively use the alexa-local and openhab2 functionality.
Are there any good hints on how to analyze the node.js activity in terms of event loop and synchronous activity, e.g. what is it doing. Something like a sampling or stacktrace of the running process?
Oh, sorry. The graph is misleading: The CPU load on the machine is comfortably below 20%, I'm just filling up with the idle percentage (the space between the orange and the green line).
The effect (user load increases with each deployment - the orange line) is 100% reproducible and is not caused by other workloads on the machine.
If you run something like 'top' you will see which processors are consuming the processor, which may be helpful.
As knolleary suggested, it is most likely that one node or other isn't freeing up it's resources correctly when we re-deploy (i.e. not shutting down old ports, links, listeners, etc correctly). However, given the extensive list you have provided - finding the one (or several) - is going to be hard. The brute force approach is to remove them one at a time, and re-measure - but that will no doubt break your flow as well.
Would I be right in thinking that it is likely that if @guardiande was doing partial deploys rather than full deploys and still sees the issue, then that would indicate that it is likely to be to do with nodes being re-deployed?
Well clearly moment and uibuilder can't be a problem 
I think that could be a clever way, just move a node a bit in the editor, make a partial deploy, check. Repat for all nodes one by one, hopefully the blaming one is found
I generally recommend glances which is a Python-based tool. Although it takes a fair bit of processor itself, it is great for a quick peek into what's happening.
If you need longer-term monitoring, you could do worse than use a combination of InfluxDB, Telegraf (from the same people, which can monitor lots of different things and logs the outputs to all sorts of places including InfluxDB) and Grafana to give you the charts and dashboard - that's what produced the charts I showed.
It is out of question that it is Node Red causing the increasing CPU load.
Although I only took a quick look at glances I come to the conclusion that this is at much to high abstraction level (like top) to debug the issue.
I need something like jstack for Java applications, not sure if there's such a thing for node.js apps.
Good idea, will check that out.
You should check that it is the node-red process, rather than something associated with node-red. It is not inconceivable that it is influx for example as it is could be affected by the re-deploys. Or perhaps one of the added nodes uses a server of some sort that is being re-invoked so you end up with multiple copies of it.
This is what I meant. It is definitely the NR process.
What are you using to measure and produce the graph?
This is a concert of collectd on the Pi for collecting the system statistics, Influxdb for storing the time-series data and Grafana for visualizing it.
Meanwhile I did some tests: Restarting the flows repeatedly as possible with NR 0.20 expectedly causes the load to increase. I then deployed only changes flows and did that with every single flow that I have repeatedly several times. This causes no load increase.
I suspect the configuration nodes to cause the increase.
Indeed.
To go further, you can use the --inspect flag and use a Chromium based browser - I'm using Vivaldi or use the debug features of VScode - documented on my blog and elsewhere.
Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
The OpenJS Foundation | Terms of Use | Privacy Policy | OpenJS Foundation Bylaws | Trademark Policy | Trademark List | Cookie Policy