A guide to understanding 'Persistent Context'

Thank for this guide @zenofmud , helped me understand mutch better. Still have some problems setting up.

Running version 0.19.5.
Stoped nodered using ssh command: sudo node-red-stop
Changed the /root/.node-red/settings.js file to include:


Start node red again with ssh node-red-start
This shows in ssh window:

Should this include the File store i made as well?

If i try to write to File store using function i get error:


Also i do not get a drop down menu in the change node.

What am i doing wrong?

Apart from you context issues, what is your need to run as root?
If you haven’t got one it isn’t recommended.

No need to run as root. Just used sudo to shut down node red to be sure it was shut down, dont use sudo when starting it again. Probaly no need using sudo when shutting down but i tought it could not hurt :slight_smile: Still new to all this and will not use it in the Future.

But it is picking up settings file etc under /root/ and your user directory is set to /root/.node-red in your screenshot...

@Lost It should show both the context stores. For example I added your settings to my settings.js (which on my Mac is /User/Paul/ .node-red/settings.js) and this is what I see at startup:

8 Feb 13:18:23 - [info] Settings file  : /Users/Paul/.node-red/settings.js
8 Feb 13:18:23 - [info] Context store  : 'File' [module=localfilesystem]
8 Feb 13:18:23 - [info] Context store  : 'default' [module=memory]
8 Feb 13:18:23 - [info] User directory : /Users/Paul/.node-red

so it is quite conceivable that because you are running under root there may be an issue.

If you have questions about moving your settings from /root/... to somewhere else.Please open a new thread since that will be off topic fom this thread.

U are correct @ukmoose. Realy confused about why it started as root. But restarted the pi and now it works as expected, that also explains why all of my flows disappeared.

@zenofmud Thanks got it working same as you after a restart. Thanks again for the great writeup.

For newbies like me i think something like:
contextStorage: {
File : { module: "localfilesystem"},
default : { module: "memory" }
},
should be in the settings file by default. This would make all the examples for writing to memory still correct but also could add exampe for writing to File without having to change the settings file and restarting node red and refreshing browser.

Thanks

1 Like

+1 for having the two different Storages preconfigured.

I just added another thing to remember so you might want to read 'Persistent Context' variable names and 'Projects'...something to remember

After I set context to write to a file I encountered some problems. Do you have to specify that you want to read from the file context as well?
context.get("data", "file");

See documentation.

You write to the context.
If the context was setup to write to the localfilesystem, it will get/set it from/to the file.

It depends on what you have set for the default. Did you read the documentation and the write up I did ?

13 posts were split to a new topic: How to limit persistent context data created by the state-flow node

Here is the breakdown on how the persistant context is stored,

Here is what the flow.json contains in these examples:
Screen Shot 2020-12-12 at 4.55.53 AM

Bottom line - you would need to save each context item as an object that contained a value AND a timestamp. Then in a flow in your project, you would need to parse thru each of the context items to check it's timestamp and then delete that context item.

While you could edit the file itself and remove the items, you would have to stop Node-RED, edit the file and then start Node-RED again.

6 Likes

Hi, just a quick confirmation if I have correctly understood:

  • the "persistent contexts" are written within the files with a specified timing that can be settled with the "flushInterval" option when "cache" is enabled
  • This is used just as a backup to maintain the value at node-red restart. This means that when I need to access to the file storage contexts, actually I always read them from memory, even if I write for example global.get("my_var","file").
    Is that correct?

With the default configuration of the file context store, that is correct - it uses an in-memory cache. But you can configure the store to not use a cache and any get/set operation is done on the file directly.

Ok. So, to configure the store to not use a cache and have any get/set operation done on the file directly, the "cache" option has to be set to false. Right?

If you change 'cache' to false, Eash time you store a value, the context will be written to storage. As it says in the documentation:

The flushInterval is provided to minimise wear on the underlying storage, such as on a Raspberry Pi’s SD card. Note that if Node-RED is unexpectedly killed, any data that has not yet been flushed will be lost.

So you have to decide what effect continuous writing it to storage will have on your usage case.

Perfect, thanks

Hi [at]all,

so far I what have read about persistent context, I tried to solve this myself with doing the following:
though for me its not a decisive target to have my global context stored practically "realtime". and probably for many others it isnt AW.

since function nodes provide the possibility to set up a start-up and last-will routine, I did the following:

[{"id":"5d2e2929.cfe1c8","type":"function","z":"207dcb76.6bf074","name":"","func":"keys = global.keys();\nfilename = \"/config/node-red/JsonDB/globalContext.json\";\nnewobj = {};\n\nfor(var key of keys){\n    newobj[String(key)] = global.get(String(key));\n}\n\nreturn {payload:newobj,filename:filename};","outputs":1,"noerr":0,"initialize":"","finalize":"// Code added here will be run when the\n// node is being stopped or re-deployed.\n\nkeys = global.keys();\nfilename = \"/config/node-red/JsonDB/globalContext.json\";\nnewobj = {};\n\nfor(var key of keys){\n    newobj[String(key)] = global.get(String(key));\n}\n\nreturn {payload:newobj,filename:filename};","x":360,"y":80,"wires":[["fb1b31b3.1242c","4dc0b987.dd7a98"]]},{"id":"6dd27c6f.a97144","type":"inject","z":"207dcb76.6bf074","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":160,"y":80,"wires":[["5d2e2929.cfe1c8"]]},{"id":"fb1b31b3.1242c","type":"file","z":"207dcb76.6bf074","name":"Write globalContext File","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"none","x":630,"y":80,"wires":[[]]},{"id":"4dc0b987.dd7a98","type":"debug","z":"207dcb76.6bf074","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":710,"y":180,"wires":[]},{"id":"f6de00e2.4ad2e","type":"function","z":"207dcb76.6bf074","name":"","func":"contextFromFile = msg.payload;\n\nfor(var key in contextFromFile){\n    global.set(String(key),contextFromFile[key]);\n}\n\nnode.warn(\"Reloaded GlobalContext Successfully!\")\nreturn;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":820,"y":320,"wires":[[]]},{"id":"918f5183.f6a17","type":"file in","z":"207dcb76.6bf074","name":"Read Global context File","filename":"/config/node-red/JsonDB/globalContext.json","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":370,"y":320,"wires":[["db667674.2721e8"]]},{"id":"d98139ae.7b3418","type":"inject","z":"207dcb76.6bf074","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":"2","topic":"","payload":"","payloadType":"date","x":170,"y":320,"wires":[["918f5183.f6a17"]]},{"id":"db667674.2721e8","type":"json","z":"207dcb76.6bf074","name":"","property":"payload","action":"obj","pretty":false,"x":550,"y":320,"wires":[["f6de00e2.4ad2e","8118fc66.1343b"]]},{"id":"8118fc66.1343b","type":"debug","z":"207dcb76.6bf074","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":380,"wires":[]}]

which at a deploy or restart saves the context.
so far that works with everything i wanted to do so far and i have my sd card written only at restarts and global redeploys.

and reloads them at startup


Question pointing to the admins, is there any restriction in behalf of e.g. globally stored functions (which are injected) or anything? is there something I get completely wrong regarding the benefits of setting this up in settings.js by default?

to store it, it needs to be serialisable... this is generally not possible for actual functions and a few other types of objects (eg maps I think). Basic object type are fine.

1 Like