# Memory leak ?... trouble performance now

**URL:** <https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119>\
**Category:** General\
**Created:** [13 December 2021 11:58 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119 "2021-12-13T11:58:31Z")\
**Posts on this page:** 17\
**Page:** 2

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [13 December 2021 19:25 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/21 "2021-12-13T19:25:40Z")

</div>

@TotallyInformation , thanks for your input. I try to keep good use of databases. I have a "real-time" database with standard retention of 8 hours. And at midnight I store my daily values in long term database.  
Then I have some other values I want to keep a bit longer then 7 days I have 2 databases for that perpose.  
About the continious query I don't know, I still have to figure out how that works, among other things.  
The memory problems in Node-red, I already had before I started using influx. I started using influx because I had several trendings running directly and that I backed up in files. I thought the trending was taking up much memory, and so I started to think about database to store the data.

![afbeelding](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/4/1/412b6f5c7981c2e6ba51b51c0b215b75636d5293.png)

---

<div class="post-metadata">

**Author:** ![ozpos](https://avatars.discourse-cdn.com/v4/letter/o/278dde/32.png) [@ozpos](https://discourse.nodered.org/u/ozpos)\
**Post date:** [13 December 2021 19:30 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/22 "2021-12-13T19:30:15Z")

</div>

There is this 'Environment="NODE\_OPTIONS=--max\_old\_space\_size=512"' when using systemd [https://raw.githubusercontent.com/node-red/linux-installers/master/resources/nodered.service](https://raw.githubusercontent.com/node-red/linux-installers/master/resources/nodered.service)

I would also recommend trying quite an aggressive retention policy as @TotallyInformation described. Doing so allowed me to run influxDB for a couple of years on a Pi4B with 64GB EVO Plus sdcard without any problems at all.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [13 December 2021 19:39 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/23 "2021-12-13T19:39:13Z")

</div>

> [@henkkas](#):
>
> Why dos it say in Node-red that it uses almost all of 924 MB available

Look at top again, whatever node you are using is showing the free mem and total figures, instead of of mem available.

> [@henkkas](#):
>
> Is there a way to increase mem available for Node-red?

Why? It is not short of memory.

---

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [13 December 2021 19:44 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/24 "2021-12-13T19:44:17Z")

</div>

@Colin I get this information from nodes provided by

`> node-red-contrib-cpu` and `> node-red-contrib-os`  
I also read about garbage collection and how to monitor with `> node-red-contrib-gc` that gave me graphical info about this, see pic.Time frame is 15 min.

![afbeelding](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/7/1/71234fec9c7940c960800388625c132f9ccccc77.png)

---

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [13 December 2021 19:49 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/25 "2021-12-13T19:49:18Z")

</div>

@ozpoz, I make use of the same SD card you mention. As I said, I try to keep the load on influx low.  
But Am I to understand, that all of you have the same kind of behaviour on memory, that's mounting up until almost max. and then resets again (by lack a better word)?

---

<div class="post-metadata">

**Author:** ![ozpos](https://avatars.discourse-cdn.com/v4/letter/o/278dde/32.png) [@ozpos](https://discourse.nodered.org/u/ozpos)\
**Post date:** [13 December 2021 19:50 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/26 "2021-12-13T19:50:06Z")

</div>

Is it possible that the level of monitoring is contributing to the problem. How many points are you showing ?

---

<div class="post-metadata">

**Author:** ![ozpos](https://avatars.discourse-cdn.com/v4/letter/o/278dde/32.png) [@ozpos](https://discourse.nodered.org/u/ozpos)\
**Post date:** [13 December 2021 19:51 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/27 "2021-12-13T19:51:25Z")

</div>

> [@henkkas](#):
>
> Am I to understand, that all of you have the same kind of behaviour on memory, that's mounting up

I have not experienced this.

---

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [13 December 2021 19:53 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/28 "2021-12-13T19:53:55Z")

</div>

> [@ozpos](#):
>
> Is it possible that the level of monitoring is contributing to the problem. How many points are you showing ?

Showing where?

---

<div class="post-metadata">

**Author:** ![ozpos](https://avatars.discourse-cdn.com/v4/letter/o/278dde/32.png) [@ozpos](https://discourse.nodered.org/u/ozpos)\
**Post date:** [13 December 2021 20:01 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/29 "2021-12-13T20:01:21Z")

</div>

Dashboard graphs or tables.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [13 December 2021 20:09 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/30 "2021-12-13T20:09:29Z")

</div>

> [@henkkas](#):
>
> Am I to understand, that all of you have the same kind of behaviour on memory, that's mounting up until almost max. and then resets again

It isn't, all you are seeing is Linux making good use of as much memory as it can. It tries to keep the free memory as low as possible. If you were really running out of memory then you would see swap being used.  
This may help to explain it [Why does Red Hat Linux report less free memory on the system than is actually available? - Server Fault](https://serverfault.com/questions/9442/why-does-red-hat-linux-report-less-free-memory-on-the-system-than-is-actually-av#:~:text=Linux%20)

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [13 December 2021 20:15 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/31 "2021-12-13T20:15:28Z")

</div>

> [@ozpos](#):
>
> Dashboard graphs

These in particular can be a real drag on performance.

> [@henkkas](#):
>
> And at midnight I store my daily values in long term database.

Much more efficient to let InfluxDB do that for you.

> [@henkkas](#):
>
> About the continious query I don't know, I still have to figure out how that works

Here are some examples:

```auto
		# Limit the domotica db to 24hrs data by default (.8*60*60*24=69,120 69,120*11*9=6,842,880 
		CREATE RETENTION POLICY twenty_four_hours ON domotica DURATION 24h REPLICATION 1 DEFAULT
		# Create 2nd Limit for the env_daily table for 5yrs (24*365*5=43,800 ). 5y = 5*365=1,825 d
		CREATE RETENTION POLICY one_week ON domotica DURATION 1825d REPLICATION 1
		
		# Aggregate details to 60minute data
		CREATE CONTINUOUS QUERY cq_60min_environment ON test BEGIN  
			SELECT mean(value) AS value, max(value), min(value), max(value) - min(value) AS range  
			INTO test.five_years.environment_hourly  
			FROM test.one_week.environment  
			GROUP BY location, type, time(1h) 
		END
		
		CREATE CONTINUOUS QUERY cq_60min ON domotica BEGIN SELECT mean("value") as value,max("value"),min("value"),max("value")-min("value") as range INTO domotica.one_week.env_daily FROM environment GROUP BY location,type,time(60m) END
			SELECT mean("value") as value,max("value"),min("value"),max("value")-min("value") as range INTO domotica.one_week.env_daily FROM environment WHERE time >= 0 GROUP BY location,type,time(60m)
			# SELECT mean("value") as value,max("value"),min("value"),max("value")-min("value") as range FROM environment WHERE time >= now() - 1d GROUP BY location,type,time(60m)
			# SELECT * FROM environment WHERE location = 'HOME/IN/00/HAL' AND type = 'DewPoint' AND time >= '2016-04-28 15:00:00' AND time <= '2016-04-28 16:00:00'
			DELETE FROM environment WHERE time < '2016-04-26'
			select count("value") from domotica.'default'.environment

```

I think there is a post in the forum somewhere from me where I give a much better and more up-to-date example.

* * *

Here we go, this is better:

[Need more detailed information on influxdb - General - Node-RED Forum (nodered.org)](https://discourse.nodered.org/t/need-more-detailed-information-on-influxdb/36885/5)

---

<div class="post-metadata">

**Author:** ![edje11](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/edje11/32/572_2.png) [@edje11](https://discourse.nodered.org/u/edje11)\
**Post date:** [13 December 2021 20:18 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/32 "2021-12-13T20:18:18Z")

</div>

To reduce the cpu (spike) load by influxdb significant you can disable the database tables internal used by influxdb.

Edit **/etc/influxdb/influxdb.conf** and find the section **[monitor]**  
Uncomment the line store-enabled and give it the value false.

```auto
[monitor]
  # Whether to record statistics internally.
    store-enabled = false

  # The destination database for recorded statistics
  # store-database = "_internal"

  # The interval at which to record statistics
  # store-interval = "10s"

```

Restart infuxdb **sudo service influxdb restart**

Influxdb will now stop using the tables for internal use, use at your own risk.  
Note from the [influxdb website](https://docs.influxdata.com/influxdb/v1.8/administration/config/#monitoring-settings):

> Set to `false` to disable recording statistics internally. If set to `false` it will make it substantially more difficult to diagnose issues with your installation.

---

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [13 December 2021 20:20 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/33 "2021-12-13T20:20:32Z")

</div>

@Colin , Yes, Indeed, I can see it's more complex then I Thought, although I don't understand all of it.  
I think I leave this subject to rest, because I maybe I shouldn't worry about it....  
The panic struck when at the same time the performance went down. 😀 But that was explainable by a mistake I made.

Anyway thank you all for your input, @ozpos, @TotallyInformation, @jbudd . etc.  
I think you guys are doing very well with your suppport to us!!! Keep it up !!!

---

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [15 December 2021 09:06 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/34 "2021-12-15T09:06:24Z")

</div>

@TotallyInformation , thanks for the examples. I will study this at a later time, at this moment it looks quite complex to me, I am only a novice at databases and Node-red.  
Like I said in the beginning of this post, I don't have issues with performance (yet) or influx. It works just fine for me right now. Also I am reaching the end of the functionality of my application. The only worry I have at this moment is the use of memory and the strange behaviour I notice, the saw thooth graph and build up of memory use, I do not really understand this. I just wonder if there is any way to avoid this or make it more stable. Am I doing something wrong to get this behaviour. I get this info from the nodes I use, like I mentioned before. But I found out this info is derived from command

`> free -m`

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [15 December 2021 09:29 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/35 "2021-12-15T09:29:37Z")

</div>

> [@henkkas](#):
>
> The only worry I have at this moment is the use of memory and the strange behaviour I notice, the saw thooth graph and build up of memory use,

How many times do I have to say that there is nothing unusual about this and it is nothing to do with node red. Look at the amount of memory node red is using and you will see that.  
Any well used Linux system will show that behaviour.

---

<div class="post-metadata">

**Author:** ![henkkas](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/henkkas/32/39844_2.png) [@henkkas](https://discourse.nodered.org/u/henkkas)\
**Post date:** [15 December 2021 09:35 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/36 "2021-12-15T09:35:32Z")

</div>

@Colin . Well that was the assurance I was looking for!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [13 February 2022 09:36 UTC](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119/37 "2022-02-13T09:36:07Z")

</div>

Closed on 13 February 2022 09:36 UTC

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

[Previous page](https://discourse.nodered.org/t/memory-leak-trouble-performance-now/55119.md?page=1)
