Node-Red *CPU load* -- wierd things going on

General
1

(Yeah, I'm a gluten for this.)

Just mentioning.
If anyone knows something, I'm interested.
But more so it is a statement.

My system:
Intel NUC I5 CPU. 15 Gig or RAM. 500 G SSD.
Ubuntu 18.04 LTS.
NR 1.0.3
Firefox 72.0.1

And, yes: I have quite a few session of FireFox (FF) and Node-Red (NR) open.
On this window I have 11 tabs with NR. About half/half edit screen/dashboard.

Now and then when I deploy NR, FF just sits there "waiting waiting waiting"

I have been told of the Fire fox taskmaster - of which I didn't know until recently.

This is a nominal view of it's load.
Note the top one.

I have seen it way (WAY) bigger. Like in the thousands value.
It is only pointing to one of the IP addresses.

The pages (NR dashboards) are a bit complicated, but not exactly NASA type complicated.
(NASA - Rocket science)

What gives with these spikes?

Screenshot from 2020-01-26 09-15-02
Screenshot from 2020-01-26 09-15-021360Ă—768 119 KB

2

Are you doing a full deploy or a partial deploy?

3

My first thought would be to get rid of Firefox (it's almost as bad as Chrome for eating resources) and switch to Opera.

My second thought would be to have far, far fewer tabs open, as they'll kill performance as well.

Even with 16 GB of RAM (I guess you meant) you're still putting a lot of load on memory and the processor.

4

Paul,

I usually do a modified nodes deploy.
Other than ip 192.168.0.6, they are RasPies. They don't need the extra stress of a full deploy.

5

Yes... 15/16. I mucked up with hitting the wrong number.

I use FF because way back when, there were IE and FF.

IE made Swiss Cheese look solid with all its problems.

Though I think I rememeber Netscape.

My real computer though was a whole different O/S (etc) which is now kinda not usable.

I got thrown into the M$ world early 2000's I think. So between IE and FF..... I went FF.

Yeah, maybe I should have/get 32 G of sheep in the machine. (Sheep: Ram - bad pun)

6

I work with a lot of different O/S's in my day job, and actually Windows isn't too bad these days (although working at low level can be a nightmare). MS may get slated a lot, but Edge is actually one of the better browsers in terms of resource use.

My home automation system sounds very similar to yours, but less powerful - a Brix Mini with a dual core Celeron and 8 GB of RAM, and it runs Node-RED plus various other apps quite happily, with almost instantaneous response to the "Deploy" button.

I'd suspect that Firefox, with a lot of tabs, is eating most of the memory and processor, which is causing your problems. But check the processor and memory use for everything running (top & ps) and maybe you'll see something else is the culprit.

7

On Windows, the new Edge is excellent - though the released version still has a couple of rough edges (pun intended). I've been using the dev version for around 6 months now. It is based on Chromium as most browsers are now. It is fast, efficient and stable. Microsoft have added some nice UI features but kept things fairly simple which is good. My second goto browser is Vivaldi, also based on Chromium. On mobile, I tend to use Brave (Vivaldi mobile is still in beta) but I sometimes also use Edge as it is great for sharing bookmarks and open tabs with the PC. I have FF and FF dev installed on the PC but rarely fire them up - usually only to do a quick test of some code.

A browser with lots of tabs open shouldn't take more than a couple of GB of RAM. Occasionally, some websites continue processing when a tab is in the background which isn't good. I install a tab manager wherever possible that lets me suspend tabs - that takes care of that problem without forcing me to keep closing and opening them. Currently using "Cluster Tab Manager" on Edge/Vivaldi. Can't remember what I used on FF.

1 Like
8

Um, mine is a Raspberry Pi Model B Rev 2

It is certainly less than a Brix Mini at a guess. 8GB ram. Runs rings around what the RasPi has.

9

Ah, OK. I was going by the specs you gave in your first post :

That should be able to handle reasonable loads, although see previous comments re Firefox, tabs and other processes to try to identify resource hogs.

The RPi is a reasonably powerful beast too, and certainly amazing value for money, but again, it has its limits, and you'll need to find out what's taking up processor time and memory if it's slowing down.

10

No, that's the testing machine where I design flows, etc.

And my main use machine.

No problems.

(Looking at your avatar, you from Kiwi land?)

11

So which machine are you looking at the CPU load on?

The title implies it's the device running Node-RED. But the description suggestions it's the NUC where you use FF to access the Node-RED editor and dashboard.

12

It is more the Firefox load that I am showing which is running on the NUC. (The bigger machine - 192.168.0.6)

Which in the first screen shot is showing 86.94.

I guess in retrospect it should/could have been more "FireFox CPU load. . . . ."
But it is only tabs in FireFox with are dealing with Node-Red which are getting such spikes.

They may last for a couple of seconds or they may last . . . . . . an annoyingly long time.
Not to be confused with a "REBOOT annoyingly long time".

I didn't know of the firefox taskmanager and was told of it only recently.
I'm monitoring it and created this more to help (show?) other people who may have similar problems that they are not alone.

Granted: having 30 "sessions" of Firefox won't help.
I'm not even going to have at a guess how many tabs.

It is just it happens now and then and for no particular reason.
None of the remote machines running node-red are straining. Granted there are spikes when they do their cycles but that is only for maybe a second at most. But for the main one where it could last 8 seconds as it pings, and does other stuff.

But it is not always that machine's node-red which is maxing out this machine's Fire fox. It could even be Node-red on this machine.

I really can't see anything which could cause it.
Again: I am mentioning it so if others have the problem....... (De ja vous)

Does that help?

13

I've never been, although it's on the (big) list.

I didn't know the adventures of molesworth 1, or public schools, were big in NZ, but that's been my avatar and nym on t'intarwebs since, well, before there even was an internet. If you see the same name and av on anything to do with software, games, sailing or astronomy, it's probably me :wink:

[/off-topic]

14

[off-topic]
background info on (the fictional) Molesworth.... https://en.wikipedia.org/wiki/Nigel_Molesworth
[/off-topic]

1 Like
15

You need to dig into the utilisation of the FF machine to see what is causing the spikes. FF's task manager only shows FF tasks, you need to look at a machine level monitor to see what the machine is doing. Look at the command line tools: top, atop htop - or Glances though if you are having problems with performance, Glances can make it worse as it is Python based - it is the best though.

Check to see what is causing spikes and make sure that it is actually FF or Node-RED or something else. On an I/O limited device (e.g. one with a slow filing system) for example, the I/O itself can cause noticeable spikes.

If you do track it down to FF, use the developer tools to monitor exactly what is going on - perhaps a slow network response is slowing things down for example.

Note that in your original image, you were see "Energy Impact" as being high. This doesn't always correspond to high CPU. Network activities have a high energy requirement. Or complex, high-speed graphics.

2 Likes
16

Or, from experience, webpages utilising websockets that continuously send/receive information through it, like the Node-RED editor when outputting a lot of debug messages or deploying a flow, or dashboard pages that refresh information occasionally. I’m seeing this with my own creations too that use websockets or (god forbid) broadcast server sent events

1 Like
17

Interesting. I'd not tried this before. I'm using Microsoft Edge Dev (based on Chromium).

With a uibuilder node using the standard template with two tabs open to the same uibuilder url and Node-RED sending a small json payload msg every 0.1 seconds, the uibuilder pages rarely go above 10% CPU and seem to average maybe 7-8 - I think that is probably mainly VueJS because it updates the whole msg on-screen on each receipt. The Node-RED page, after an initial high-cpu when it starts sending, settles down to almost no CPU, occasionally going up to about 2%. Memory utilisation on both is also really stable though the uibulder page cycles up and down over perhaps 20-40 MiB.

So I do get some spikes at the start of a flow but it quickly settles down. Unfortunately, I'm not seeing any network throughput as both server and client are on the same device. 16GiB, i7 Surface Book 2.

18

Going slightly offtopic, but I noticed it in Mac’s task manager that both firefox and chrome would get energy usage spikes while working on a Vue based application that communicated with a backend python server over websockets for bi-directional communication, followed by a steady stream of SSE messages. This was visible both when running everything on that macbook, as well as when the backend application ran on heroku, and the frontend was deployed on a (different) 3rd party hosting, with the backend also functioning as CDN for most of the files except the outer HTML. It would connect/authorise to an oauth 2 service through the backend, interact with it, then parse the output, filter it on the backend as the service didn’t allow that, then send the filtered output over SSE back to the frontend every time it finds an item that matches the filter.

19

Hi Julian.

The problem started a while ago. I would be using the NUC to write NR code on a machine (a machine: Any, including this one) and I would press deploy and it would sit there, deploying.

Sometimes it would be quite a while and it was starting to annoy me, so I would - now and then if not needing to program in NR - load another browser and do what I needed and all would be sweet.

Yes, of course all would be sweet, as there are no NR tabs loaded.
To further test the water I opened a few NR Dashboards in the other browser as now and then I would need to check the status of things in the NR world.
Oh, I also don't edit any NR stuff on the remote machines as they are password locked and the password is only of FireFox. (FF)

Again there would be no big lags.

So when it became apparent that it was the NR Edit windows, I asked around about how to track what is going on in FF.

I was shown the whole new world of the task manager in FF and so I opened it up and keep it open at all times. Then if I want to see what is going on, I can look there.

To using top, atop, htop.

Yeah, I have been down that path too. Though mostly with just top.
All it showed me was something like Web content and a huge number.
To me that isn't useful in finding out WHAT exactly it is.

Ah-ha! Something to do with the interweb.
Which then lead on to the FF task manager.

That screen shot I showed you was 89.xx. While watching it some times - not often - but it is up to THOUSANDS. Only for less than a second. But wow! It really spikes.

My reflexes aren't that good to actually catch them, and I don't really think getting a screen shot of it is really going to achieve anything useful.

So what ever it is: it is tricky.
These spikes can be on any of the 4 machines - this one included.
There are 3 RasPies on 24/7 and this machine when I am surfing the web.

The other 3 machines - as said - do very little. Though the main one is a bit higher loaded as it polls the others at regular intervals. I've upped it from 20 seconds to 30 seconds.
Doesn't really change things.

I may look more at htop - I think that was recently installed on this machine. But if it is on another machine........

That is another problem I see.
It happens only now and then.
So to check all the other three machines, I would have to SSH into them and get top (or equivalent) running and be monitoring it all the time.

Yeah, if I want to find the problem I need to do the work. I am just explaining what I have done and why.

It is not quite at the threshold of being/needing to be fixed. But it is getting there.
This is just to tell others of the problem so if they get it, they know they aren't alone.

Using FF's developer tools is WAY beyond my skill set. I am having enough troubles with NR and getting things done with it.

But thanks.