Memory leak ?... trouble performance now

General
1

When I first started with Node-red. I had trouble with memory leak, but after 1 or 2 months, it disappeared.
Now for as long as 2 months, I have the same issue again. But my application works fine, so far.
Every now and then the memory hit some 95%, and then resets to about 50 to 55 % and then builds up again slowly. If I restart Node-red or reboot RPI, it also drops again.
Today I did some minor chages, and now the memory will not go lower then about 85% and is very fast over 90%. Als the CPU is working like crazy and temp has gone up from about 32 dgrC to 47 dgrC.

Most of my application exist of some fast modbus connections and some slow http requests. Data saved to influxDB and some graphs and tables. Nothing special I would think. The Node-red log looks fine.

Anyone has any idea where to look for the cause ????

Here is some info,

afbeelding

afbeelding
afbeelding394Ă—677 45.6 KB

afbeelding

2

In a terminal run htop to see which applications are hogging the memory.

3

@jbudd Here is the output of htop, but so far I can only see that Node-red is the cause.

afbeelding
afbeelding1045Ă—959 91.9 KB

afbeelding
afbeelding1025Ă—947 89.4 KB

4

Check that charts and displayed tables do not hold an unnecessary number of points and that you have implemented a retention policy.

5

@ozpos and @jbudd , I have killed all access to influxDB. usage is back to 7% again.
But memory is still very high, about 78%.

6

Have you added any new nodes ? I have found that with a Pi Zero some nodes pull in huge amount of js.

7

Influx is using most CPU time, Node-Red using most memory.
Is it a Pi 3B? Only 1GB memory?

I wonder if you have accidentally created a loop in Node-Red.

Try disabling all the flows and watch memory usage as you reneable them one by one.

1 Like
8

@ozpos , No I didn't lateley, only simple gate, but this was working fine up till about 2 houres ago.
Like I said, did only minor changes.

9

@jbud Indeed influx has been killed, but restored all write actions again. No problem with performance so far.
It is an RPI 3B+ with 1 GB memory , yes.

I will try that, and see where it is coming from.

10

@jbudd , I disabled almost all flows. But memory still high,

afbeelding

afbeelding
afbeelding1612Ă—1227 233 KB

11

@jbudd , I restored all flows again. Performance is still rather good(usage and temp.) But memory still slow on the rise.Very strange where bad performance came from.......
Any other ideas where I can find where the memory build up comes from?

afbeelding
afbeelding389Ă—759 46.5 KB

12

Not really. Is influx still disabled?

I think there are discussions here about profiling and debug tools, I've not used them.

13

@jbudd No almost everything is back running again. Only a test flow I have disabled.

14

Don't worry about the memory for the moment, the main problem is the CPU usage. Possibly you have some sort of loop in the system. Or it could be influx. Does the CPU drop right down if you stop influx?

15

@Colin and @jbudd . Yes, already found a loop, just now, on feedback to multi switch.

That is not the issue anymore. Still the memory is high, and won't go down anymore like it did before.

afbeelding

16

Can you show us what top shows please?

17

@Colin

afbeelding
afbeelding1598Ă—1221 166 KB

afbeelding
afbeelding880Ă—1238 146 KB

18

I don't see any problem there. Linux tries to make best use of all the memory there is, keeping disc buffers in RAM and caching stuff where possible. To get actual amount of memory available for use you have to look at the mem available figure. Node red is only using 17% of the ram.

1 Like
19

Probably not related to your recent problems but certainly do look at InfluxDB and make certain that you have a retention policy in place for any detailed data. On your Pi, eventually you will start to get CPU spikes and SWAP use.

Try to keep detailed records only for a limited time. Typically I keep 1 minute records for 7 days but I have a continuous query that creates hourly records of everything in the 7 day store (keeping avg, max and min values over the hour for all values in the detailed store). I keep the hourly record for 5 years.

20

@Colin . Thanks. Yes, I can see also, that node-red is in fact only using 17 - 18% of the total memory it shows in TOP. Why dos it say in Node-red that it uses almost all of 924 MB available? Is there a way to increase mem available for Node-red?

afbeelding