Raspi 3B+ running Buster + NR4.0.2, NodeJS 20.17, NPM 10.8.2
Main problem is the Dashboard gets increasingly sluggish over time including disconnects despite a running on a good network with very low latency. So I started looking further.
Using htop, I have found that there is a large amount of swap memory (50 out of 100MB) being used despite only 400MB out of 900+MB of memory being used. If I restart the Raspi this drops back to 1MB and system response time at dashboard returns to normal. but over a few days or more, up goes Swap memory used (Saw 71M out of 100M after 3 days once!). would I be right assuming this large swap usage cause the dashboard lag as the system swaps stuff in and out of the SD card swap memory (and accelerate SD card degradation?
The next thing I saw in HTOP was that Node-red, Node, npm start along with many other processes are shown in htop as running with many PIDs (counted 15 for some), albeit the PIDs use identical memory, but different CPU total run time). Is this normal or do I have something in settings wrong.
Was considering upgrading the RasPi, but as I see it, this Raspi is idling, not using more the 60% of its resource.
Not just Dashboard but probably lags just about everything. Swap on Pi's with SD-Card storage is notoriously slow.
It will certainly have an impact on card wear. How significant is debatable since, by default, the card will be hit a lot by log writes which likely dwarf the swaps. Actual card wear is also dependent on card quality and spare capacity - the use of cards with wear levelling and plenty of spare capacity is strongly recommended for Pi's.
Yes, not uncommon. Though I wouldn't expect to see npm running if using Dave's install script; that would imply you are running node-red using npm rather than the more normal systemd.
50MB of swap is not actually large by any means. But it does indicate that you are running something with greater memory requirements than your device can easily handle. Possibly excessive node-red context memory or a large flow with nodes requiring large memory use. Though it could also be caused by a different process such as a database engine.
In truth, the problem for you is caused by running a Linux desktop on the same device as Node-RED and other services such as MQTT.
Run the desktop on a different device. Preferably one designed to do so - which the Pi, with the greatest of respect, is not.
Run the Pi headless and you will almost certainly have no issues. I used to run Node-RED, Mosquitto, InfluxDB, Telegraf, Grafana and other servers on a Pi3 with no issues at all.
Thanks TI for your great response, yep desktop is installed, and enabled but I do not use. All access is remote through putty so it is about to be disabled and uninstalled.
No big databases (SBFSpot uses a small one that logs a hundred bytes every 2 minutes) but my flow is massive, running over about 24 tabs, amongst those are graphs with a lot of datapoints so plenty in memory. But most of those graphs hold no more than 24 hours of data. As a result wouldn't expect memory usage to co tinue to increase after a day.
Interesting about npm running. I usually use script to instal and sysd to start on powerup. But I think this install was one I tried to run node 22 on (flagged to many errors or warnings, plus stuffed up telegram_Bot so went back to node 20) and even tried the node version manager (till I found NR didn’t like it) so maybe something leftover from that activity is adding npm to run on start.
I will get on and finish a clean build (bookworm) on my other raspi (remove linux desktop) and do the swap (always a worry... what did I forget?)
Once
It isn't that NR doesn't like it. It means that, for every different version of Node.JS used, you have to install all global npm installs (including Node-RED) again. It is confusing and leaves lots of unnecessary "cruft" behind.
Just to note that, especially if using one of the Dashboards (1 or 2), it is worth doing a quick calculation on your charts since any number of points above the likely number of pixels in your typical display all have a significant overhead to the performance of the Dashboard.
All sounds good, just needs a bit of rationalisation and it sounds like you have that in hand.
I think that you can install Rasbian without the desktop which is the best option if possible so that you don't get unnecessary library and apps cluttering your SD-Card.
The fact that it is using a bit of swap is not an issue provided there is lots of free memory. It means that at some point there has been a shortage of memory so it used some swap (50MB is not a lot), but now there is available memory, so if it needed to access the data in swap it would fetch it. I don't like htop, I much prefer top, I find htop confusing.
The fact that you have the desktop enabled but not in use will make a bit of difference, but probably not vast. As Julian said, you can install the Lite version of the OS, which only has the terminal interface installed.
Also as Julian said, check your chart usage. Work out how many points are added to the chart for its full width and make sure it is not a lot more than the number of pixels on screen., otherwise you are just wasting memory and processor.
Performance growing more sluggish over a period matches well with chart data points gradually accumulating. I suggest making sure you don't have any charts with more data points than pixels.
Swapping on an SD card is definitely bad.
Take a look at zram, which lets you reserve some RAM for a [compressed?] swap partition, probably eliminating sd card swaps.
This on a Pi Zero 2 which runs Node-red:
/dev/zram0 is a virtual swap partition.
/var/swap is the default swap space on the sd card and it is not in use.
Note the Zero 2 only has 512MB but Node-red still works when zram grabs some of it.
There must be some impact on performance of stealing memory like this, far from clear if it would be a negative impact for a Pi which is already swapping.
MEMORY : total used free shared buff/cache available
Mem: 416Mi 172Mi 136Mi 8.0Ki 172Mi 244Mi
Swap: 721Mi 112Mi 608Mi
SWAP : Size Used Priority
/var/swap : 200M 0M -2
/dev/zram0 : 521M 113M 15
I suspect that there is not much benefit in stealing RAM to use as swap in this sort of situation. It just means you will run out of RAM earlier.
The most important thing is to find why it using lots of ram in the first place.
It's swapping. You are already running out of RAM. The important thing is to stop it killing your hard disk.
Have you done any tests?
It does mean that.
It also means swapping is much faster and card wear is reduced.
There is lots of free memory most of the time, so it is only transiently running out of RAM. With swap to disc/card enabled the total amount of memory available during that transient is the real ram plus the swap. By using up ram for swap then the total amount of memory available is reduced to that of the real ram only, so there is an increased risk of running out completely and crashing.
Also, by moving rarely used stuff out to swap there is more real ram available for caching and so on, which speeds up the system. Linux actually starts moving unused pages out to swap before it runs out of real ram, for exactly this reason. It is only when RAM gets so low that it has to keep moving pages in and out that performance and disc/SD card wear become an issue.
I've never really quite understood that. If you are getting paging activity to swap, you are already short of memory. So wouldn't reserving MORE memory make matters worse, not better?
SORRY, only just read the other responses saying similar.
Yes indeed. It isn't the use of swap per-se, it is the continual swapping (paging) that is an issue - on ANY system but magnified of course on a system using just a low-performance SD-Card for storage.
Like you I find the idea of reserving part of the RAM counterintuitive, so I've tried it.
I don't have quantitative results because I don't know of a way to measure sd wear, nor a suitable Node-red performance measure.
It causes the Pi to swap earlier but swapping is much faster.
Does it speed things up or slow them down? Like I said, the answer is not obvious.
It will certainly speed swapping up, but that is significant only if pages are regularly swapped in and out. As you say, whether it will speed the system up or slow it down will vary from situation to situation.
It will certainly reduce the total amount of virtual memory available.
