# Struggling with memoryleak

**URL:** <https://discourse.nodered.org/t/struggling-with-memoryleak/88983>\
**Category:** General\
**Created:** [26 June 2024 14:20 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983 "2024-06-26T14:20:42Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [26 June 2024 14:20 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/1 "2024-06-26T14:20:42Z")

</div>

Hello!,

I notice i have a memoryleak when listening to a mqtt broker, and putting the payload into a queue:  
[@ba47/node-red-queue - npm (npmjs.com)](https://www.npmjs.com/package/@ba47/node-red-queue)

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/a/e/ae9dc9e351b050bc24e3c4c27178a2561401e52b.png)

It seems like this node always creeps up, but never frees memory.  
I have tried several other queue nodes as well with the same issue. Also same issue if i disconnect the output as well.  
Is there some core issue i am missing here?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [26 June 2024 15:08 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/2 "2024-06-26T15:08:12Z")

</div>

Welcome to the forum @RonnyR

For what reason are you queuing the messages? There is probably a better way of doing it.

---

<div class="post-metadata">

**Author:** ![zenofmud](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/zenofmud/32/316_2.png) [@zenofmud](https://discourse.nodered.org/u/zenofmud)\
**Post date:** [26 June 2024 18:08 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/3 "2024-06-26T18:08:58Z")

</div>

Did you read the `Issues` section in the link you provided? Could that be describing your problem?

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [27 June 2024 07:37 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/4 "2024-06-27T07:37:27Z")

</div>

Hi, thanks for replying, yes i have read the issues section and its not solving the issue.  
About the reason for doing this:  
I am trying to create a timeframe of 3 seconds where i store all the messages to create a snapshot of the set of datapackages to detect deviations.

I use a mqttbroker node-\>decode with a protobuf file-\>to this queue.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [27 June 2024 08:01 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/5 "2024-06-27T08:01:35Z")

</div>

Unfortunately the node does not have a link to a repository where you can submit issues, which is often a bad sign. You may have to find an alternative solution.

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [27 June 2024 08:05 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/6 "2024-06-27T08:05:46Z")

</div>

Thank you Colin, ill try some other nodes as well, if people have any suggestions i appreciate it. 🙂

---

<div class="post-metadata">

**Author:** ![jbudd](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jbudd](https://discourse.nodered.org/u/jbudd)\
**Post date:** [27 June 2024 08:12 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/7 "2024-06-27T08:12:05Z")

</div>

> [@RonnyR](#):
>
> create a snapshot of the set of datapackages to detect deviations.

What does a "deviation" mean? Can you give an example?

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [27 June 2024 08:17 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/8 "2024-06-27T08:17:10Z")

</div>

Sure!, I want to gather 1000 of payloads, containing a address and a number, if the number deviates i want to collect them into a "bucket" to process them even more as a deviation and create a report. 🙂

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [27 June 2024 08:34 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/9 "2024-06-27T08:34:50Z")

</div>

Do you mean you want to collect all the payloads (just the payloads) over a fixed time and then release them in one message?

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [27 June 2024 08:38 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/10 "2024-06-27T08:38:20Z")

</div>

Correct, i want to collect all payloads over a fixed time and them release them collected in one message for further processing. First stage of a filtering sequence really.

---

<div class="post-metadata">

**Author:** ![jbudd](https://avatars.discourse-cdn.com/v4/letter/j/5f8ce5/32.png) [@jbudd](https://discourse.nodered.org/u/jbudd)\
**Post date:** [27 June 2024 09:57 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/11 "2024-06-27T09:57:46Z")

</div>

There is a Filter node which you can setup to only pass messages where the payload has changed.  
You could use that to only report when the number for any address changes.

Another approach I would consider is to save the data to a database and use SQL to report on deviations.  
If you accumulate thousands of records in a relatively short time this would need careful housekeeping to keep the db size reasonable.

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [27 June 2024 10:39 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/12 "2024-06-27T10:39:29Z")

</div>

Thank you for your response.  
I will look into the filter node as well.  
Also I have a database, but I would expect a standard queue handling should be easy to memory manage, not to leak. I have tried several nodes now, also q-gate, here its a lot less, so I guess I have to deep dive more into node.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [27 June 2024 13:18 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/13 "2024-06-27T13:18:45Z")

</div>

Perhaps something like this will do what you want

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/1/7/1733615678503dcda43d064c13f17a5fe290c064.png)

```auto
[{"id":"a4f83620bc9c7846","type":"function","z":"bdd7be38.d3b55","name":"Queue payloads","func":"let payloads = context.get(\"payloads\") || []\nif (msg._release) {\n // the incoming message includes msg._release true so pass on the payloads\n msg.payload = payloads\n payloads = []\n} else {\n payloads.push(msg.payload)\n msg = null // stop the return from sending a message\n}\ncontext.set(\"payloads\", payloads)\nnode.status({text: `queued: ${payloads.length}`})\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":560,"y":2920,"wires":[["0abb31cae9c20c03"]]},{"id":"aef84022f34c57bd","type":"inject","z":"bdd7be38.d3b55","name":"Simulated incoming messages 1 per second","props":[{"p":"payload"}],"repeat":"1","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"iso","payloadType":"date","x":270,"y":2880,"wires":[["a4f83620bc9c7846"]]},{"id":"fec86de385320a44","type":"inject","z":"bdd7be38.d3b55","name":"Release every 5 seconds","props":[{"p":"_release","v":"true","vt":"bool"}],"repeat":"5","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":300,"y":2980,"wires":[["a4f83620bc9c7846"]]},{"id":"0abb31cae9c20c03","type":"debug","z":"bdd7be38.d3b55","name":"Queued payloads","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":800,"y":2920,"wires":[]}]

```

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [27 June 2024 13:35 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/14 "2024-06-27T13:35:20Z")

</div>

Absolutely, i will try to test this out, as I had a similar solution.  
Thank you 🙂

---

<div class="post-metadata">

**Author:** ![RonnyR](https://avatars.discourse-cdn.com/v4/letter/r/eb9ed0/32.png) [@RonnyR](https://discourse.nodered.org/u/RonnyR)\
**Post date:** [1 July 2024 10:55 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/15 "2024-07-01T10:55:30Z")

</div>

It seems like its my gRPC node which i have no lifecycle control over thats leaking memory.  
If the gRPC node is the last node the memory slowly creeps upwards.  
If i disconnect it, the memory is stable.  
I use:  
[node-red-contrib-grpc (node) - Node-RED (nodered.org)](https://flows.nodered.org/node/node-red-contrib-grpc)

Is there anything else you should think of when i am using this node?

I trigger it quite a few times, like 5 times a second. I will try to add a delay to see if that helps as well.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [1 July 2024 11:09 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/16 "2024-07-01T11:09:24Z")

</div>

I would first ask the developer to update dependencies to see if that helps any. It is using versions of gRPC and protobuf that are ~4y old and have had lots of updates since the node was released.

NOTE: the repo has [bump updates sitting waiting](https://github.com/julien-pal/node-red-contrib-grpc/pulls) since 2022 - so dont hold your breath.

Alternatively, fork it, update the dependencies, install your fork direct from GitHub.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [29 September 2024 11:09 UTC](https://discourse.nodered.org/t/struggling-with-memoryleak/88983/17 "2024-09-29T11:09:54Z")

</div>

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.