Flow variables - not persisting through a deploy (sometimes)

General
1

I have a node-red app that acts mainly as a go-between between an AIrTable DB and a No-Code front end. There are 23 Airtable tables that I import, augment, then store their records in flow-level variables. I then access these flow variables and perform various selections, joins, etc as the no-code front-end requests via API calls. The set of flows I am using are 95% HTTP nodes and function nodes.

What I am experiencing of late is: following some deploys (or errors in function nodes). The flow variables are gone.

There is no indication of a failure in the debug sidebar, nor in the messages that appear where deploy messages come and go.

I am running Node-red 2.04 on Ubuntu on a Digital Ocean droplet with 1GB of RAM. These node-red flows have been running (and the flows have been growing in size) for 8 months. The backup (settings.json) file is 276KB. I don't know how to get the size of the memory occupied by the app and the flow variables.

I have set up logging to level "trace" and marked true to metrics and audit in the settings.js file logging section (and restarted the droplet)

This seems to be a memory-related issue, and I seem to have misunderstood how to view the logs, so any clues/suggestions are most welcome

2

Have you set the context store to file in the setting.js? If you are using memory every time node-red restarts you will lose the context data.
https://nodered.org/docs/user-guide/context#saving-context-data-to-the-file-system

3

Thanks @E1cid - that is helpful as I could have a persistent set of flow variables, even across restarts.

However, the underlying issue continues: I can't understand why the app loses its value in the first place. I assume that I am experiencing a restart of node-red without any indication of that.

Even a runtime error inside a function will cause the flow variables to be lost. I can't tell if a restart is happening.

Maybe what I need to do is determine if a restart of a node is happening and whether I have a memory/garbage collection issue. Any ideas?

4

You may have memory issues causing a restart, you can use top on umbuntu to see the cpu and memory.

You should avoid context if you can pass the message along the wires, as this avoid huge context memory and race conditions.

5

Thanks, @E1cid. I'd just like to confirm it is a memory issue, and that the node-red service is indeed restarting, rather than locking up memory used in a function due to a silent error that cannot be released to garbage collection.

As for the memory usage, I need to keep these flow variables as they are used in multiple places. They are created separately from when they are used so no opportunity to dd on to msg object.

But you have given me an idea: I could probably release some of the flow variables and the memory they use.

6

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.