Analysing Memory Use

[TotallyInformation]

As part of an excercise in restructuring uibuilder which is a quite complex node, I decided I should finally work out a better way of analysing memory use so that I could make sure that I wasn't leaking memory.

Thought it might be a good idea to write this up as an FAQ. Please bear in mind that I am certainly not a Node.js "expert" but I believe that this is useful to get a feel for the size of nodes and whether your nodes and/or flows are leaking memory.

I am sure that there is more that could be written so please to add further comments and corrections. Hope you find this useful.

---

I assume that you are using a version of Node.js >= v12.

First step is to dump out some information when Node-RED starts. The easy way is to add the following code into your settings.js file towards the top, before the module.exports line.

const v8 = require('v8')
console.info(`V8 Total Heap Size: ${(v8.getHeapStatistics().total_available_size / 1024 / 1024).toFixed(2)} MB`)
let mem = process.memoryUsage()
const formatMem = (m) => ( m/1048576 ).toFixed(2)
console.info(`Initial Memory Use (MB): RSS=${formatMem(mem.rss)}. Heap: Used=${formatMem(mem.heapUsed)}, Tot=${formatMem(mem.heapTotal)}. Ext C++=${formatMem(mem.external)}`)

This gives you the total "heap" size available to Node-RED. If your actual use is pushing on this, there is a problem, not least of which will be that node.js will be constantly trying to recover memory which may impact performance. If your use exceedes the total, you will get an error.

Next steps depend on whether you are testing a custom node or just checking that your flow isn't leaking memory.

Checking for flow memory leaks

This is the easy one. Here is a simple example flow with a function node that outputs the current memory use of Node-RED:

[{"id":"da3d35e1314f395a","type":"inject","z":"58aa202c0d7e669e","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":460,"wires":[["1b4aac38a3d0cd92"]]},{"id":"1b4aac38a3d0cd92","type":"function","z":"58aa202c0d7e669e","name":"","func":"let mem = process.memoryUsage()\n\nconst formatMem = (m) => (m / 1048576).toFixed(2)\n\nmsg.topic = 'Memory Use (MB)'\nmsg.payload = {\n    RSS: formatMem(mem.rss),\n    HeapUsed: formatMem(mem.heapUsed),\n    HeapTotal: formatMem(mem.heapTotal),\n    'External C++': formatMem(mem.external)\n}\n\nreturn msg","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"process","module":"process"}],"x":410,"y":460,"wires":[["24ec3a426815da70"]]},{"id":"24ec3a426815da70","type":"debug","z":"58aa202c0d7e669e","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":560,"y":460,"wires":[]}]

Note that this assumes you are using Node-RED v2.

Add the function into your flow anywhere that you want to measure the current memory use. In particular check multiple invocations of your flow (e.g. when multiple messages pass through) to make sure that the memory use does not continue to climb.

Testing custom nodes for memory use

In this case, the next step is to create a function you can call in various places:

    /** Dump process memory use to console */
    dumpMem: function(prefix) {
        let mem = process.memoryUsage()
        const formatMem = (m) => ( m/1048576 ).toFixed(2)
        console.info(`${prefix} Memory Use (MB): RSS=${formatMem(mem.rss)}. Heap: Used=${formatMem(mem.heapUsed)}, Tot=${formatMem(mem.heapTotal)}. Ext C++=${formatMem(mem.external)}`)
    }

This will write to the node-red log every time it is called. The output shows you the actual memory usage (of Node-RED as a whole) at the time it is called.

So now you can use that function in 3 places in your custom node:

1. At the end of the module.exports function - this is called when Node-RED loads your node definition - between the Starting flows and Started flows info messages in the log.
2. At the end of the function you pass to RED.nodes.registerType. This will be shown whenever you (re)deploy any of the instances of your Node.
3. At the end of the function you pass to this.on('input', ... ). This will be shown every time to send a msg into an instance of your Node.

What do the numbers mean?

• RSS: The "resident set" of memory currently being used by the node.js process (Node-RED in this case).
• Stack: This is where your base variables like booleans, and numbers live. It also contains pointers to more complex objects in the Heap.
• Heap: Is the area of memory in the resident set that stores objects, strings, and closures.
• External (C++): Shows the memory usage of C++ objects bound to JavaScript objects managed by V8 (the underlying JavaScript engine for node.js).

As you monitor your nodes and flows, you will notice that the RSS, and Total Heap as well as the Heap Used figures all change. This is because the RSS is dynamic and can (fairly) freely grow up to a size determined by V8. The V8 Heap figure you output from settings.js should give a good indication of how much total memory you can play with before hitting a hard limit. However, the dynamic nature of memory management both by V8 and your operating system makes this rather variable.

As your node.js process continues to run, V8 monitors memory use in the background and when it decides that there is enough recoverable memory, it runs something called "Garbage Collection". This is a fairly intense process and one of the reasons you may see Node-RED pause occasionally, especially if your server is short of memory.

Recoverable memory comes from variables that are no longer needed. Such as a variable created in a function - at the end of the function call, the memory is no longer needed. This is also one of the reasons it is a really good idea to use let and const in your JavaScript code - because it helps limit the context that the variable lives in and therefore may enable the memory to be recovered faster.

You can somewhat control what V8 does with garbage collection by passing the --max-old-space-size variable to node.js. However, in most cases with newer versions of node.js it is better to let it decide for itself.

There are other V8 parameters that you can also use to control the memory available to node.js but these are very advanced topics and should be avoided if at all possible. V8 usually does an excellent job of managing memory for itself.

Optimising memory use

A few pointers as this is a complex subject.

• Try to avoid having multiple wires coming out of a single node output port. Because if you do, Node-RED has to make a COPY of the msg object. Not only is this a relatively slow process, it also increases memory use.

• ALWAYS name your functions, callbacks, and closures. This doesn't save memory in itself. But it does make debugging a LOT easier.

  So use function myfunc() { ... } rather than const myfunc = function() { ... } for example. Though annoyingly, for arrow functions, you still use const myfunc = () => { ... } to get a named function.

  Use db.query( ..., function myfunc() { ... }) rather than db.query( ..., function() { ... }) for closures and callbacks.

• Limit the scope of variables. Using let and const and making good use of functions. If you need to, you can even use an in-line code-block (wrapping code in { ... }), varibles defined inside using let and const will be limited to that block. Though probably better to use a function instead in most cases.

• Control the growth of large arrays and objects. Set them back to empty arrays/objects if needed though better to control the scope.

• Be wary of the res and req variables that are returned from http. They are massive and contian circular references. As soon as you can, take out what you need and then get rid of them. Node-RED will sometimes remove them anyway in a flow for the same reason.