I'm no expert in tuning a Raspberry Pi but the 100% swap space usage looks really bad news.
I suspect that until you fix that, there's little point in hunting for guilty processes.
Maybe you can find something useful in https://haydenjames.io/raspberry-pi-performance-add-zram-kernel-parameters/
You can change the htop display to view as a process hierarchy which is useful when trying to make sense of more complex applications.
It is best to look at the mem % rather than virtual mem.
You seem to have a lot of InfluxDB threads running. My live system only has 1.
OK, I've spotted your problem.
You are running a Linux desktop - you need to get rid of that and run headless only.
Do I really get rid of that, does that make all the difference in the world, or just a bit better?
I just love the desktop!! I also make use of SD Card copier. With this tool I can easily make perfect backups while running Node-red etc.
It makes a MASSIVE difference 
On a Pi3 with limited memory and cores, it is devastating to overall performance.
Get another Pi3 
Why do I have so many threads (17) running? And you only one?
Not sure, I think it has something to do with sharding in the database but I'm no expert in this area I'm afraid. I think that the db just got too complex. At least as an experiment, you might want to create a new db with the config as recommended previously and see how that looks. Also check the db sizes. My combined size for all dbs is currently 378M.
This command will show you the total size:
sudo du -sh /var/lib/influxdb/data/
My size is 185M
Seem to be smaller then yours.
OK. Here is my breakdown:
Unsurprisingly, the Telegraf db is largest since it is capturing loads of system data every 10s or so. Even though I only keep the data for a week. The home db is split into 2 as mentioned previously.
How often are you writing to the db?
Have a look at the influx log and see what it is doing.
I don't see weird things, looks good to me, Colin
If you tail the log is it running continuously or are there time gaps in it? Have you got it set to show the queries being run?
I used this command:
sudo journalctl -u influxdb.service
And in the output log I see indeed the queries that I run, I don't know what you mean by time gaps?
I mean was it continually outputing log lines (as if maybe it cannot keep up) or ws it intermittent, with at least short gaps of inactivity.
Yes, indeed, Colin, it is continuously outputing data as expected. I don't see any gaps.
If it is continually outputing data then perhaps it is not keeping up with what it is trying to do.
Running sudo iotop may be interesting. That will show you (in the IO> column) how much of a processes time is spent waiting for IO (so waiting for the SD card in this case).
If iotop is not installed then
sudo apt install iotop
That's ok, if it were tens of percent then I would be worried.
@Colin @jbudd @dceejay @TotallyInformation @BartButenaers
I now think that something went wrong about 2 or maybe 3 weeks ago during minor change and consecutive incremental compile. I have experienced wrongful compiles before, but only minor faults. One time a changed node gives completely weird messages, after replacing node with new one and compile problem was over. One time a changed or placed new node and after compile this node gave 2 output messages with 1 input message. So I am worried something must have gone wrong during this minor change, that resulted in high cpu usage.
As I might never find this problem, I have decided to rebuild my project on second RPI. And I am now in the process of doing just that.
I think I can manage that, but I have just one question for now. I have a lot of settings in global data.
Is it possible to backup the data to a file in order to restore later on new system?
What do you mean by 'compile'?
Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
The OpenJS Foundation | Terms of Use | Privacy Policy | OpenJS Foundation Bylaws | Trademark Policy | Trademark List | Cookie Policy
