How to store variables (best practices)

[olab]

I guess I need a little guidance here for storing variables for use in Node red. I’m working on a system to present statuses for a industrial plant. This could include maybe 100-200 variables to be used in other calculations. So far I use influxdb and grafana for showing trends. The underlaying system can serve data as OPC UA, MQTT or REST.

My idea is that it might be effective to read the data only once and then store them somewhere in Node-Red as variables. My question is how to do this?

• Store them as global variables?
• Store them in a json file?
• Read them from the source every time I need it?

My goal is that it should be easy to pick up a variable and use it...

Any best practices on this?

[Steve-Mcl]

If they are constant, unchanging values, I would store them in q lookup object in flow or global context for quick retrieval. If they charge infrequently, i.e. once per shift then I'd likely reload them by some trigger. If they change often, get them from a database.

[olab]

@Steve-Mcl

If they are constant, unchanging values, I would store them in q lookup object in flow or global context for quick retrieval. If they charge infrequently, i.e. once per shift then I'd likely reload them by some trigger. If they change often, get them from a database.

Values might change aprox. every 10 sec.

[Steve-Mcl]

So I possibly misinterpreted your intention. I assumed when you said variables to be used in other calculations I thought you meant like production constants/recipe values etc.

My guess is you mean these 100-200 variables are production measurements like performance, counts, line status etc? If so, then what i do is write the important stuff to database on change. Anything need for OPCUA and rest interface I would keep a memory copy (flow/global) for quick access and publish them to MQTT for any other systems listening.

The amount you're talking about is fairly insignificant so just store them in context.

[olab]

@olab

I guess I need a little guidance here for storing variables for use in Node red. I’m working on a system to present statuses for a industrial plant. This could include maybe 100-200 variables to be used in other calculations.

I might a bit diffuse. As you said, the variables is mostly measures counters etc.

I have been experimenting with storing variables in global context but found it a bit "messy" to retrieve them. It would probably better if there was a way to group them...

But... it looks like I'm on the correct path at least

[zenofmud]

You can store an object in a global. For example, here is one I create in a Function for a music player:

// Initilize the musicobj
//var songcount = 0;
//var songcount = msg.payload.length;
//var songlist = [];
//songlist  = msg.payload;
//var musicurl = msg.payload;

var musicobj   = {
    fileURL    : msg.fileURL,
    audioURL   : msg.audioURL,
    songcount  : 0,
    songlist   : [],
    songindex  : 0,
    songtoplay : '',
    action     : "initilize"
};
global.set('musicobj', musicobj);
return msg;

and here is how It is used in a change node:

[Steve-Mcl]

Store everything in a JS object and store the object in global

E.g

global.productionData.station1.state
global.productionData.station1.count
global.productionData.station2.state
Etc

Retrieval can be made in most nodes and in function nodes by accessing global.get().

If organised property, you should be able to retrieve items directly by name (or dot notation) without looping or scanning or lookup/switch cases.

[olab]

@zenofmud

You can store an object in a global. For example, here is one I create in a Function for a music player:

// Initilize the musicobj
//var songcount = 0;
//var songcount = msg.payload.length;
//var songlist = [];
//songlist  = msg.payload;
//var musicurl = msg.payload;

var musicobj   = {
    fileURL    : msg.fileURL,
    audioURL   : msg.audioURL,
    songcount  : 0,
    songlist   : [],
    songindex  : 0,
    songtoplay : '',
    action     : "initilize"
};
global.set('musicobj', musicobj);
return msg;

and here is how It is used in a change node:

Now we're close. I have been experimenting on this but haven't been been able to browse in global context. Where do I find this button?

[Steve-Mcl]

@olab

but haven't been been able to browse in global context

Look at the side bar (where debug output is) there is a drop-down button. Choose context and view flow/globals there.

[zenofmud]

Have you read about using file storage for context? see:
https://nodered.org/docs/user-guide/context
and this might help:

A guide to understanding 'Persistent Context'

This is something I wrote up after playing with the context options. I've filed a PR with some of this added to the Node-RED documentation but thought I'd release it here also. I hope people find it useful. Paul A guide to understanding 'Persistent Context' Prior to Node-RED v0.19.0, data could be stored in context as a global, flow or node. This was stored in memory and would be reset with a restart of Node-RED. As of v.0.19.0 you can now store node, flow and global context data in memory O…

[nordseele]

Hi,
Sorry, I hate to dig out an old thread but instead of creating a new one, it seems appropriate to add another question to this one.

---

I’m using Node-Red as an OSC to I2c converter on a RPI. I’m translating OSC messages and sending them on the I2c bus. I’m using a custom function for passing commands to a whole lot of different devices. Each device has its own set of commands. Some of these devices have so many possible I2c commands available that it doesn’t feel right and clean to store all of them in the same “function node”. + I might need to access these commands from another node.

Until now I’ve used a const / object literal to store all these « per device » commands directly in the function node.

Something like that

const modules = {
  er301: {
    address: [0x31, 0x32, 0x33],
    commands: {
      tr: {
        cmd: 0x0,
        arg: [{name: 'port', type: 'u8'}]
      },
      tr_tog: {
        cmd: 0x01,
        arg: [{name: 'port', type: 'u8'}]
      },
      tr_pulse: {
        cmd: 0x5,
        arg: [{name: 'port', type: 'u8'}]
      }
},
another_device: {
...
}
}

The fact is that I want to share this flow as a « project » later.

Is it preferable to store the global const in settings.js like says the documentation or to put them in a function node within the flow as recommended in this thread ? I just want to make sure this const will be shared along the whole project when I put this on GitHub. By the way, is settings.js part of a « project » too ?
Thanks.

[Colin]

@nordseele

Is it preferable to store the global const in settings.js like says the documentation or to put them in a function node within the flow

Either way is perfectly acceptable. There are other techniques too. Since settings.js is just javascript you can put any valid js there, so you could read another file that contains the data. Alternatively, in a function that runs at boot, you could read a file in containing the settings, so the the flows file is not polluted with the data.

When you say you are going to share it as a project are you using the word project to refer to the node-red projects feature? If so then the problem is (I believe) that the only way for someone to access that would be if they used the projects feature, which many users don't. However I may be wrong there as I don't use projects.

[dceejay]

if you are going to share it then storing in the flow is usually less hassle than messing with settings. You could also look to use a subflow which would make it easier to duplicate as needed.

[nordseele]

Thank you both for your replies.
I'm going to explore these solutions.

My first idea was to have each set of commands placed in its own file (device_1.js, device_2.js...) and these files would be aggregated/merged into a constant at Node-Red startup. But if it's too complicated, I will store everything within the flow as you suggest. There are actually so many different possibilities with Node-Red that I still have to think about which solution will be the easiest and most effective to manage

@Colin

When you say you are going to share it as a project are you using the word project to refer to the node-red projects feature?

Yes indeed, the "project" feature of Node-red. But you're right, I forgot that the "project" option needs to be enabled in settings.js
It seems to be the easiest way to handle dependencies, version control etc. I would like to give the possibility to other users to clone on their RPI in a few clicks the flow I’m working on, with all the contrib nodes required and possibly some additional files. I believe that most of these users will not be familiar with Node-red, so it might be easier if it's a kind of "one-click" install.

[Colin]

@nordseele

it's a kind of "one-click" install.

Unfortunately it isn't quite that easy. If the version of nodejs they had installed was different to yours then it would be necessary to rebuild the nodes (which are in node_modules) so generally it isn't a good idea to distribute node_modules.

Also by doing it that way it would not be possible for a user with node red already running to import your flows and add it to their existing system.

[nordseele]

You’re right, I didn’t think of that.
It would only work on a fresh install of Node-Red.
In this case, perhaps I’ll just provide the "flow" and some installation instructions for the users not familiar with Node-Red. The install script of Node-red is already super fast and handy and it's easy to import a flow, so that might work