A guide to understanding 'Persistent Context'

Hi, just a quick confirmation if I have correctly understood:

  • the "persistent contexts" are written within the files with a specified timing that can be settled with the "flushInterval" option when "cache" is enabled
  • This is used just as a backup to maintain the value at node-red restart. This means that when I need to access to the file storage contexts, actually I always read them from memory, even if I write for example global.get("my_var","file").
    Is that correct?

With the default configuration of the file context store, that is correct - it uses an in-memory cache. But you can configure the store to not use a cache and any get/set operation is done on the file directly.

Ok. So, to configure the store to not use a cache and have any get/set operation done on the file directly, the "cache" option has to be set to false. Right?

If you change 'cache' to false, Eash time you store a value, the context will be written to storage. As it says in the documentation:

The flushInterval is provided to minimise wear on the underlying storage, such as on a Raspberry Pi’s SD card. Note that if Node-RED is unexpectedly killed, any data that has not yet been flushed will be lost.

So you have to decide what effect continuous writing it to storage will have on your usage case.

Perfect, thanks

Hi [at]all,

so far I what have read about persistent context, I tried to solve this myself with doing the following:
though for me its not a decisive target to have my global context stored practically "realtime". and probably for many others it isnt AW.

since function nodes provide the possibility to set up a start-up and last-will routine, I did the following:

[{"id":"5d2e2929.cfe1c8","type":"function","z":"207dcb76.6bf074","name":"","func":"keys = global.keys();\nfilename = \"/config/node-red/JsonDB/globalContext.json\";\nnewobj = {};\n\nfor(var key of keys){\n    newobj[String(key)] = global.get(String(key));\n}\n\nreturn {payload:newobj,filename:filename};","outputs":1,"noerr":0,"initialize":"","finalize":"// Code added here will be run when the\n// node is being stopped or re-deployed.\n\nkeys = global.keys();\nfilename = \"/config/node-red/JsonDB/globalContext.json\";\nnewobj = {};\n\nfor(var key of keys){\n    newobj[String(key)] = global.get(String(key));\n}\n\nreturn {payload:newobj,filename:filename};","x":360,"y":80,"wires":[["fb1b31b3.1242c","4dc0b987.dd7a98"]]},{"id":"6dd27c6f.a97144","type":"inject","z":"207dcb76.6bf074","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":160,"y":80,"wires":[["5d2e2929.cfe1c8"]]},{"id":"fb1b31b3.1242c","type":"file","z":"207dcb76.6bf074","name":"Write globalContext File","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","encoding":"none","x":630,"y":80,"wires":[[]]},{"id":"4dc0b987.dd7a98","type":"debug","z":"207dcb76.6bf074","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":710,"y":180,"wires":[]},{"id":"f6de00e2.4ad2e","type":"function","z":"207dcb76.6bf074","name":"","func":"contextFromFile = msg.payload;\n\nfor(var key in contextFromFile){\n    global.set(String(key),contextFromFile[key]);\n}\n\nnode.warn(\"Reloaded GlobalContext Successfully!\")\nreturn;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":820,"y":320,"wires":[[]]},{"id":"918f5183.f6a17","type":"file in","z":"207dcb76.6bf074","name":"Read Global context File","filename":"/config/node-red/JsonDB/globalContext.json","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":370,"y":320,"wires":[["db667674.2721e8"]]},{"id":"d98139ae.7b3418","type":"inject","z":"207dcb76.6bf074","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":"2","topic":"","payload":"","payloadType":"date","x":170,"y":320,"wires":[["918f5183.f6a17"]]},{"id":"db667674.2721e8","type":"json","z":"207dcb76.6bf074","name":"","property":"payload","action":"obj","pretty":false,"x":550,"y":320,"wires":[["f6de00e2.4ad2e","8118fc66.1343b"]]},{"id":"8118fc66.1343b","type":"debug","z":"207dcb76.6bf074","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":830,"y":380,"wires":[]}]

which at a deploy or restart saves the context.
so far that works with everything i wanted to do so far and i have my sd card written only at restarts and global redeploys.

and reloads them at startup


Question pointing to the admins, is there any restriction in behalf of e.g. globally stored functions (which are injected) or anything? is there something I get completely wrong regarding the benefits of setting this up in settings.js by default?

to store it, it needs to be serialisable... this is generally not possible for actual functions and a few other types of objects (eg maps I think). Basic object type are fine.

1 Like

Reading through this topic I assume that deletion of stored data is still not included as default option? Some people mention the change node. How to use this node to delete data?

I would set a variable to the value null personally. Not sure if that deletes it though, can't remember I'm afraid.

I'm not sure what you mean by:

You can use a change node with the `delete optionScreen Shot 2021-04-13 at 4.23.37 PM

One of the funniest, accurate explanations of null vs. delete vs. empty (zero), etc... I have ever seen is below... and I discovered it in this forum!

8 Likes

very helpful for understand how to persistent context,
and this article link should sit inside the template node's help!!!

Welcome to the forum @msongz

The problem is that context can (and is) used with virtually all nodes, not just the template node. In fact I don't think I have ever used it in a template node. The most common use is probably in the Function node.

In case it helps anyone, here is a link to a useful example by @Steve-Mcl
Also read down that thread to find out about saving a context object by 'reference'

2 Likes

7 posts were split to a new topic: Issue with Presistent Context when using q-gate

Hi all,
I am trying to modify the settings.js like in this post:

Therefore I started Node-Red and checked the log where the file is stored:

start
> node $NODE_OPTIONS node_modules/node-red/red.js "--settings" "/etc/node-red/config.js"
4 Jan 09:53:03 - [info] 
Welcome to Node-RED
===================
4 Jan 09:53:03 - [info] Node-RED version: v2.1.4
4 Jan 09:53:03 - [info] Node.js  version: v16.13.1
4 Jan 09:53:03 - [info] Linux 5.10.63-v8 arm64 LE
4 Jan 09:53:05 - [info] Loading palette nodes
4 Jan 09:53:11 - [info] Dashboard version 3.1.3 started at /endpoint/ui
4 Jan 09:53:14 - [info] Settings file  : /etc/node-red/config.js
4 Jan 09:53:14 - [info] Context store  : 'default' [module=memory]
4 Jan 09:53:14 - [info] User directory : /config/node-red/
4 Jan 09:53:14 - [warn] Projects disabled : editorTheme.projects.enabled=false
4 Jan 09:53:14 - [info] Flows file     : /config/node-red/flows.json
4 Jan 09:53:15 - [info] Server now running at http://127.0.0.1:46836/
4 Jan 09:53:15 - [info] Starting flows

As you can see it should be stored in /etc/node-red/config.js, but I can not find this folder. I connected via ssh and logged in with root and the pw.but I can not find the file:

What could be the problem?

Thanks

Have you looked in /config/node-red/?

Also, have you saved any values in file context?

Since you have the default set to memory, you need to specify when you want it to be saved into the file. For example, in a change node you will have to pick the 'File' option


or it will default to memory.

If you want it to default to the filesystem, change the settings.js to

     	contextStorage: {
			default    : { module: "localfilesystem" },
 			memory     : { module: "memory" }
     	},

EDIT: you should also read Working with context : Node-RED
and Writing Functions : Node-RED

It is only there because that's where you told it you wanted it to be when you started Node-RED - line 2 of your output. I don't believe that is a normal option for a default installation of Node-RED. How did you install Node-RED?

What happens if you do cd /etc/node-red? Do you get an error?