Advice needed on Pi CPU usage

I think it very unlikely that it is fundamentally a problem with Node-RED itself since so many of us run it for very long periods of time with no issues. It will certainly be one of the contrib nodes I think.

Afraid not, just got to pull bits apart until you find the problem. In my experience, you will also find many ways to optimise your flows along the way.

Again, unlikely. I assume you have checked for unattached config nodes? Though NR tells you about those these days anyway. You can open your flows file in an editor, copy the whole lot and paste it into the flow inspector. You can also run a flow that will examine your flows file, the details are in the forum somewhere. But I doubt you will find anything.

Of course, as you've already found one old node, you should certainly check to make sure that you don't have any other really old nodes around. And maybe see if you can reduce the number of custom nodes. Certainly remove any node types that you aren't using.

Nope, maybe I wasn't clear... Not in NR itself, but in system setup on the pi, a permission or something that ** I ** might have cocked up...

I can tell you something for nothing... I have even tidied up the "layout" of the flows!! Lol!!

Yep, sorted them out as they have cropped up...

Hmmmm..... I must look into that, will save it till tomorrow!

I wouldn't be able to recognise an old from a new!! Lol.... But I can say that all that is installed is at the latest version according to the palette manager.... I have done updates regularly as I can of late...

The only custom ones I know of in use are the process specific ones I have written as subflows... and none of them relate to TCP as such....

Thanks for coming back to me!!
I appreciate it!!

Regds
Ed

Hmm, tricky. All I can say is have a good tidy up if you haven't already. Ditch anything you don't need and try to make things you think you need go away as well. Do you need REDIS? .....

Or even buy a second Pi and move some of the services to it :slight_smile: Anything to simplify the device you are having issues with.

Go through each of your installed node types on the flows site and find out when it was last updated. Compare those dates to how often the underlying service gets updated. For example, if you are using a MySQL node, when was it last updated compared to when you updated MySQL itself? The bigger the difference, the more likely you are to have an issue.

Run an npm audit, it will tell you all of the vulnerabilities in all installed modules which may give you further clues that a node module is not being updated. Don't try to run npm audit fix though, it will almost certainly break things spectacularly!

You can also run npm outdated to see if any of your nodes need an update. Also run npm outdated -g to see if you have any old globally installed nodes.

I don't think that can be true, didn't you have loads of nodes? Anything that starts with node-red-contrib- is a custom node.

Aha !!

I learn every day!!

Been there a few times already... As far as I can make out, the logging side, Emoncms highly recc's redis to not "wear out" the sdcard.... Early on in this creature's life, before I installed redis, on the 3b, I went through and fried a couple of flash drives after a fairly short life(running system off them) before I put in redis and things lasted a good while longer after that... Moved entirely to sdcard now but still kept redis...

Couldn't resist poking the flow inspector....

This is what it said:

Flows: 49
Subflows: 14
Nodes: 3459

 Type                    | Count | Module
-------------------------|-------|-------------------
 adv ping                |    50 | node-red-contrib-advanced-ping
 bigtimer                |    12 | node-red-contrib-bigtimer
 change                  |   482 | @node-red/nodes
 chronos-config          |     1 | node-red-contrib-chronos
 chronos-filter          |     1 | node-red-contrib-chronos
 comment                 |   270 | @node-red/nodes
 cpu                     |     3 | node-red-contrib-cpu
 debug                   |   157 | @node-red/nodes
 delay                   |    13 | @node-red/nodes
 emoncms                 |    19 | node-red-node-emoncms
 emoncms-server          |     1 | node-red-node-emoncms
 exec                    |     9 | @node-red/nodes
 function                |   356 | @node-red/nodes
 inject                  |   140 | @node-red/nodes
 is online               |    39 | node-red-contrib-isonline
 json                    |     8 | @node-red/nodes
 link in                 |   502 | @node-red/nodes
 link out                |   360 | @node-red/nodes
 mqtt in                 |    97 | @node-red/nodes
 mqtt out                |    51 | @node-red/nodes
 mqtt-broker             |     1 | node-red-contrib-vhome
 rbe                     |    42 | node-red-node-rbe
 schedex                 |     1 | node-red-contrib-schedex
 smooth                  |    22 | node-red-node-smooth
 string                  |    57 | node-red-contrib-string
 switch                  |    82 | @node-red/nodes
 telegram bot            |     3 | node-red-contrib-telegrambot
 telegram command        |     1 | node-red-contrib-telegrambot
 telegram receiver       |     4 | node-red-contrib-telegrambot
 telegram sender         |     4 | node-red-contrib-telegrambot
 toggle                  |     2 | node-red-contrib-toggle
 trigger                 |    63 | @node-red/nodes
 ui_base                 |     1 | node-red-dashboard
 ui_dropdown             |    12 | node-red-dashboard
 ui_group                |    34 | node-red-dashboard
 ui_multistate_switch    |    34 | node-red-contrib-ui-multistate-switch
 ui_spacer               |    88 | node-red-dashboard
 moment                  |     4 | node-red-contrib-moment-bug_locale, node-red-contrib-moment2
 openweathermap          |     1 | node-red-node-openweathermap, node-red-node-web-nodes
 ui_button               |    12 | node-red-dashboard
 ui_gauge                |    37 | node-red-dashboard
 ui_led                  |    85 | node-red-contrib-ui-led, node-red-contrib-ui-led-fork
 ui_numeric              |    28 | node-red-dashboard
 ui_slider               |     4 | node-red-dashboard
 ui_switch               |     2 | node-red-dashboard
 ui_tab                  |    12 | node-red-dashboard
 ui_text                 |   138 | node-red-dashboard
 ui_text_input           |    12 | node-red-dashboard
 Drives                  |     1 | node-red-contrib-os, node-red-contrib-ovia-os, @ekristoffe/node-red-contrib-os
 Memory                  |     1 | node-red-contrib-os, node-red-contrib-ovia-os, @ekristoffe/node-red-contrib-os
 ping                    |     2 | node-red-contrib-balena, node-red-contrib-resinio, node-red-node-ping
 Uptime                  |     1 | node-red-contrib-os, node-red-contrib-ovia-os, @ekristoffe/node-red-contrib-os
 subflow:2a3e5f6c.c1c978 |    32 | 
 subflow:4ea703af.383ab4 |    19 | 
 subflow:c7b6dfd.ce717a  |    10 | 
 subflow:7c47ba0c.67f06c |    15 | 
 subflow:a96c0d99.92551  |     0 | 
 subflow:c2bd8105.a4c1d  |     1 | 
 subflow:6fd4c7a8.7adcb8 |     3 | 
 subflow:7ff4ba73.77eb14 |     0 | 
 subflow:4f99a4ff.38fc5c |     0 | 
 subflow:da85ee92.d65ae  |     0 | 
 subflow:3fc7efae.1834f  |     3 | 
 subflow:d205017e.38fcb8 |     0 | 
 subflow:3ff9d316.b5cda4 |     0 | 
 subflow:975c594d.3c0d78 |    14 | 

And...... I just spotted something.... Hold thumbs!!

Ed

Edit: Bit early to tell, but 5 mins operating with no discernible increase in Skts... Hold both thumbs!!

Edit2: Daaaaamn..... This just sank in:Flows: 49 Subflows: 14 Nodes: 3459 This things getting big!!

@TotallyInformation @Colin @dceejay @Steve-Mcl

I thinks we got it!!

Still waaaay to early to tell definitively, but..... Here is a 10 min section of graph showing the gradual increase:

screenshot-192.168.0.118-2021.04.12-21_53_19

Disabled the suspect and here is a slightly longer section of graph after the disable:
screenshot-192.168.0.118-2021.04.12-21_53_35

Spot the difference?

from 21:42 or so onwards, it's virtually flat!!

Thanks for dangling the flow inspector in front of me!!

It triggered something in my rather murky grey matter - A node that I was suspect of initially, but ruled out in error...

This one seems to be the bugbear in my case:
is online | 39 | node-red-contrib-isonline

And Look:

Guess where I disabled it!! Right at 21h42 or so....

Woohoo.... It's taken months to find it!!

A super big, awesome thank you to EVERYONE EVERYONE EVERYONE who has helped me!!

:kissing_heart:

Tonight I sleep well!!

Cheerz
Ed

Can you move all of that to a new Pi?

A good quality SD-Card has wear levelling. You need that and plenty of space. Once you have those, you can hammer an SD-Card pretty hard for many years. I always use Samsung Evo or Evo Pro cards of at least 32GB.

??? What the heck is that?

You have bigtimer, chronos and schedex - are they all needed? (not really an issue here though)

Balena - do you really need that service? Not updated in 2 years and doing lots of network stuff - a prime candidate to at least temporarily remove.

The OS nodes aren't really that helpful I never think. They take resources and really don't give enough detail to be able to analyse problems. Even worse if you pass the data to a Dashboard chart and worse still if you view that chart from a desktop on the same Pi. Use Telegraf with InfluxDB and Grafana to get really efficient OS and other monitors over time. Just remember to keep the InfluxDB telegraf db trimmed or things will get hairy after a year or so. You can even run InfluxDB and Grafana on a different system if you need to save resources (though they will certainly use less than a Linux desktop so get rid of that first if you can).

Open Weathermap may be OK but again, you don't really need it since you can get the data straight from the API with a simple http request node and a bit of processsing.

Are you really using that? It hasn't been updated in over 5 years! Get rid of it and use the proper node-red-contrib-moment. If there is a problem with that node, raise an issue on GitHub and I will get it fixed for you.

Great news.

I see node-red-contrib-isonline hasn't been updated in 5 years, which isn't a good sign.

You didn't waste your time changing the MQTT node over, you will be much better using the core node which is guaranteed to be kept up to date.

Right, now go fix those other things I've raised so you don't get back into this situation or something worse.

Give the guy a break, he needs some sleep :slight_smile:

Tomorrow will do.

Haha, no rest for the wicked, you know that!

A 3rd-party version of node-red-contrib-moment from over 5 years ago!!!

Dude...... Really!!

I'm just happy its working!!

Remember, a lot of these nodes have been operating for a good couple of years now, I'm happy to stay on "crisis management" stance till the next time....

I will improve where I can, but I find a lot of the stuff in there I actually use, certainly more than 2x a week...

This IS my new pi!! Lol... Let me get over the financial effects of the lightning strike/no insurance/everything in a smoking heap first.... This is my New inverter running with my new pi, on my new network infrastructure, next to my new pabx, alongside my old and buggered dvr, with my new, new, new, new, lots of things.... Not a planned replacement either... It hurts!!

At least in the next day or 2 I will get my NEW tig welder so I can at least earn a few bux to carry on replacing the other stuff that is fried... I got hit hard!! The strike hit the building less than 10m from where I was standing in the workshop at the time... I didn't know that a 200m square IBR steel roof could light up blue... in its entirety... Luckily I was wearing my brown pants.....

Yep, yep and yep... Took a lot of finding those nodes, they do exactly what I need..... Plenty things come close, but no cigar...

Broker setup of mqtt...

Part of the Ping node.... I use the ping regularly...

part of the moment node as far as I know....

I have a small flow set I would like you to update for me and fix any outstanding issues.... Its a charity job, so you would enjoy it.... only about 3500 nodes or so....

Thanks Again!!
E

1 Like

I do what I can... I hate things left dangling... If there's a chance it might have caused the problem, I am only too happy to change it...

E

At least I know I will live forever.... Only the good die young..... That's the problem of being 16 going on 60!

Lol
E

PS:

Haha, sorry to hear about your disaster though. Hope things get back to normal for you quickly.

For those other nodes, please do yourself a favour and just check that it is the flow inspector that is finding the wrong thing and that you really don't have a bunch of odd nodes installed. When time permits :slight_smile:

Lots of things can be done without specific nodes as well and many of us have gone through (and still go through) periodic clear-outs.

Anyway, enjoy your renewed use of Node-RED.

Cron-plus?

Node-red-node-ping any day of the week.

:slight_smile:

Ah, I remember those days. I'm 30 going on 62 now - downhill fast! (which hopefully I will be doing again next year if we can at last get another skiing holiday in!).

Will look into them Thanks!

Will do indeed!

Might call on you for some pointers though!!

Every day is a holiday for me!! I'm doing what I enjoy!!

Shooting, paintball range, metal fabrication, gunsmithing, electronics, bit bashing, 3d printing, more shooting, collecting bangy things, lots of coffee, late nights, loud music(I'm deaf so I have an excuse), machining, welding, restoring diesel engines, building gensets, chem work, renewable energy r&d, building oil fracturing thingies, restoring sewing machines, you name it!! (and these are only a few of the things I get up to) .... I live in a man cave!!

Life is way too short to live slowly!!

Cya
E

Edit: Would like to see the longbow you're drawing in your profile pic...

Nope, Telegraf every day of the week (actually every 5 or 15 min, can't quite remember what I set it to now). I ping about 10 different internal and external endpoints and query all of the major global DNS services (and my routers dns).

Data goes straight into InfluxDB along with all of the system monitoring data so I can review it on and be alerted (to telegram) by Grafana.

As Telegraf is written in Go, it is extremely efficient. I can still get the data back to Node-RED of course if I need to process it - but truth is, I don't need to. Telegraf can update MQTT as well if I want it to. I generally don't bother.

Ha, no problem, you know there are sad people like us that relish the challenge - so sad that we don't have a real life ...... hang on, what! Nah, we love it, keeps the cobwebs away and I can't stand doing crosswords :rofl: :mage:

Me too, though it is just an office mainly filled with computers :grinning:

Sounds like you should be running a YouTube channel!

Too true.

Haha, well that wasn't mine I'm afraid. We live not that far from Sherwood forest and that was a Robin Hood day in the forest when I was doing archery and my son was a massive Robin Hood fan.

It was fun shooting one and I managed to "bring down" a (plastic) deer. At far too short a range to be realistic of course. Nice, fun change from the recurve bows I was used to. My upper-body strength was never up to the task of shooting a real longbow though, you are looking at 100lbs+ for those. I always looked on with envy at the longbow shooters at our club.