Why pondering on a better solution, how do I increase the node.js memory?
Why pondering on a better solution, hod do I increase the node.ja heap memory ?
Firstly check how you are starting up Node-RED. If it has a memory (old-space) parameter on the startup, take that off so that node.js can fully manage its memory itself.
Other than that, trying to micro-manage the heap is generally a waste of time, node.js is very good at managing its own memory.
I see that you are using SQLite so that would be something obvious to check. Has the db exceeded what will easily fit in memory?
Other things you can look at are reviewing your flows for unnecessary things. Common issues can arise from over-use of sub-flows (each use creates a new instance of all of the nodes in the sub-flow) and having lots of nodes with 2 or more wires out of the same output port of a node (which forces node-red to CLONE the msg data multiple times).
Thanks for the reply.
How do I check my sqlite data length because to be honest, I have very lengthy data tables in my database
Assuming that they are written to the filing system, you can simply check their file size. I expect there are admin statements as well but I’m afraid that my SQL is very rusty.
Okay thank you
Hi Colin, observing memory usage by node js I noticed it start at 200mb during program startup and sums up gradually and eventually crashes when it goes above 2gb. That’s after some few hours of run.
Are you using charts in the dashboard?
Hi Baknan2.
No I don’t have any charts on my dashboard
Since you can see the amount increasing over a fairly short time, I suggest disabling sections of the flows until you find which section is causing the problem.
Thank you.
It is actually the sqlite node. The frequency at which data are treated and written into the SQLite database is too high. At a frequency of 1 second, node js takes 1 hour to hit memory limit(2G). At 5 minutes, it takes close to 7 hours. But my project’s target is to monitor Gensets’ operations and bearing in mind what current can do in a second, it will be nice to leave the frequency at 1 second per process. As a solution, I’ve decided to run node-red using pm2 and which restarts automatically once node js reaches a certain memory level(2G). With pm2, my project has been running since Friday without a crash or any other setback.
I’m still observing it, though am pretty satisfied as of now.
I would like to suggest that you consider an alternative tool for recording your results. It is likely that a timeseries database such as InfluxDB or the one built into Postgres would be more suited.
If not, you should be using prepared statements if you aren't already. This should significantly speed up the process:
Okay, thank you.
I’ll give it a try…
The fact that it eventually fails even at the slow rate means that it isn't the frequency that is causing the memory leak, the leak is still there even at the slow rate. It just takes longer to become critical.
Did you prove that it is the sqlite node by disabling just that node and leaving everything else running?
Which sqlite node are you using and what version of that node (check in manage palette)?
1 Like
