Saving context - advice requested!

[davej]

I'm quite new to Node Red but know enough Javascript to be dangerous! I'm primarily an Assembler mainframe guy so this is quite different for me.
I'm writing a fairly simple app that requires some initial defaults to be established then allow the user to override those defaults.
I want to save the modified values across a restart.
Simple enough but I'm asking for advice on Best Practices.

Having read plenty of posts and doc etc there seems to be conflicting advice. Some say to minimize (not use it at all) the use of flow.set/get whilst others seem to be in favour. If you minimize or don't use it - how do you pass data between unwired nodes? Seems to be complicating the issue if you simply 'ban' the use of flow.set etc.

For saving user modified values, do you recommend manually maintaining the values and write them to a file whenever a change is detected... Or has the use of contextStorage pretty much become the normal practice of accomplishing this now? I see however that its use still has to be manually enabled via changes to settings.json and not via the dashboard (as far as I can tell) which makes me wonder if it is still in doubt as a viable solution?
I also read that it performs the updates every 30 seconds. Does that mean it writes out to the file system regardless even if nothing has changed?

My own thinking is to use flow.set into an array for user changes to default values and to manage the writing out of those changes myself to a file but only if values have actually changed during that period to cut down on unnecessary file i/o to an SD card.
I realise this is a dumb question but just looking for general advice before getting too deep and having to backtrack.

[knolleary]

Hi @davej, welcome to the forum.

Context is a core component of the Node-RED runtime. It is entirely suited for storing values/state that you want saved across restarts. Yes there are alternative approaches possible, (MQTT, databases etc), and some will suggest you go down that route. But there's no reason not to use Context if it meets your needs.

The reason persistent context is not enabled by default has nothing to do with any doubts about its function. Its simply the case that not everyone wants context to be persistent (and until relatively recently, there was no persistent option).

@davej

I also read that it performs the updates every 30 seconds. Does that mean it writes out to the file system regardless even if nothing has changed?

The default File store behaviour is to keep the context data in memory and only write out any changes at most every 30 seconds. But it only writes when there is something to be written. You can change the 30 seconds in the file store configuration if you want a smaller window. This default behaviour has been designed with wear on SD cards definitely in mind.

[Colin]

@davej

If you minimize or don't use it - how do you pass data between unwired nodes?

You don't. You wire them up, commonly using the Join node where you need to merge data from multiple sources.

The persistent context (over restart) question is different, however, as you need somehow to store the data. If you are using context then the builtin technique for automatically saving to file as described in the docs is the way to go. An alternative is to use retained topics in MQTT, which has some advantages and some disadvantages.
I don't like global/flow context when it is used just to hop data wirelessly from one node to another, rather than use wires. It makes it difficult to follow the data flow.

If you have not considered MQTT then do look into it as an alternative for persistent storage.

[davej]

Thanks for the advice. Much appreciated. I am going to be using MQTT in my app but I have ruled it out for actually saving data as in my use case above. Having used MQ Series back in the day, I would not want to retain data within it. I have this notion that data belongs in a file! I think that will take another look at Context especially as it only writes if the data has actually changed so I don't see any advantage in me reinventing that wheel!

[Paul-Reed]

An example of changing the file store flush period to every 5 minutes, if it helps...

    contextStorage: {
    default: {
       module: "localfilesystem",
       config: {
               flushInterval: 300
               }
      },
    memoryOnly: {
       module: "memory"
      }
    },

[davej]

Yes thanks. I have that!
As I couldn't find it in the doc, I was mainly concerned that it was flushing regardless of the data having changed but that has been confirmed to not be the case.

[zenofmud]

Here is something I wrote back in 2018 which you might find useful

A guide to understanding 'Persistent Context'

This is something I wrote up after playing with the context options. I've filed a PR with some of this added to the Node-RED documentation but thought I'd release it here also. I hope people find it useful. Paul A guide to understanding 'Persistent Context' Prior to Node-RED v0.19.0, data could be stored in context as a global, flow or node. This was stored in memory and would be reset with a restart of Node-RED. As of v.0.19.0 you can now store node, flow and global context data in memory O…

[davej]

Thanks for the link. I had actually come across that whilst searching on the feature a day or two back! Nice write-up!

[dceejay]

if no-one minds I'd like to mark Paul's answer as the solution and move this into the FAQ category. OK ?

[zenofmud]

It's OK by me.

[davej]

Yes of course

[davej]

I have made changes to use Context and found that if I use flow.set("keyx","abc","storeInFile")
that I have to then use flow.get("keyx","storeInFile") to get it back.
Otherwise if I simply use flow.get("keyx") I get undefined. I would have assumed the variable was already in memory and not have to go to the file to get it..?

I haven't found any documentation that stated that was required. Can you confirm this to be so or am I doing something wrong?

[TotallyInformation]

You are correct. You can have a default store though so if persistence is your default then you don't need the store name.

I did think it was in the docs somewhere.

[davej]

This then leads on to how do I use a Switch node and specify in the === setting option and want to use the flow. dropdown and specify a variable that is in storeInFile.

I can't seem to get the syntax correct as to what to enter in the text box.

[Colin]

The drop down at the right hand side lets you choose the store

[davej]

@Colin I thought I was going mad...
My normal machine is Windows 10 but I just happened to be using an old laptop running Ubuntu (latest version) right now. That drop down on the right hand side is NOT present in Ubuntu as you can see here in this screen shot.

I just checked on my Windows machine and yes that drop down is present.

[zenofmud]

You will only see the drop down if you have more than one type of storage defined in settings.js

[davej]

I have already defined an alteranate persistent store in settings.json as I mentioned a couple of posts up.
As I said, when I look at the same flow on my Windows machine, the drop down is present.

Anyway - I have circumvented the issue. I closed my browser and reopened it and the dropdown now appear. Maybe that is something that has to be done often in Node Red.

[zenofmud]

Might have been a browser cache issue....

[knolleary]

@davej

Maybe that is something that has to be done often in Node Red.

If you make changes to the Node-RED settings file that the editor needs to know about (such as how context is configured) you will need to reload the editor to pick up the changes after restarting Node-RED.

But otherwise, no, you don't need to be constantly reloading the editor.