๐ŸŽ‰ Node-RED 0.19 released

If that list is part of your object, you have a property called "return" which is a reserved word in JavaScript so I wonder if that could be it?

Thanks Nick.

Just to note that, if you switch the settings for a variable from file store to memory store, the file version remains, the only way currently to delete that is to edit or delete the store file. Both will be shown in the context browser.

This could very easily become confusing since different nodes might end up using different versions of the same variable name. Something to keep an eye on.

Bad news, I have tried to renane this variable but still failing.

@knolleary I've been playing with the new context storage feature and it's really nice.
Maybe the docs should be more clear: we don't have to name the first storage default , we can name it anything like test or myStorage and it will become the de facto default storage. Any additional storage will have to be specifically named to be accessed.

However, a problem occurs when you use two or more storages of the same type. memory storages are considered as being different "memories" which is ok but that's not the case for localfilesystem storages: the data is saved in only one file (one per context type) while their current memory counter-part are saved in different places. Let me explain.
Let's say I have two localfilesystem storages named fs1 and fs2. Each has a global data named myData.

A function node with
global.set("myData", true, "fs1"); will store myData: true in fs1 while
global.set("myData", false, "fs2"); will store myData: false in fs2.

The new context-browser feature (a very nice touch !) will show that the memory version of fs1 has myData: true while fs2 has myData: false. But those storages share the same json file and so a Node-RED restart will read back myData: false for fs1 and fs2 storages.

I also noticed that the write operation to a localfilesystem storage is slightly delayed (10-15 seconds ?) meaning that data might be "lost" if the Node-RED process is interrupted (willingly or not) before it had time to write the data in file (I'm on Win10 here). I supposed that's per design, but it might be worth mentioning in the docs as well.

I like the fact that you updated the TypedInput with a dropdown to select the storage but IMHO it should display the storage name as well even when the dropdown is not expanded.

@tilleul thanks for the feedback.

The documentation for the localfilesystem store is here: https://nodered.org/docs/api/context/store/localfilesystem

It describes how to configure where the store writes its data, its the caching behaviour and how to configure the flush time.

Will look at how to draw more attention to this from elsewhere in the docs.

@knolleary wow ! sorry I missed that information ! You've really thought of everything ! :+1:

@knolleary I have a small (?) suggestion/feature request regarding context storage.
Would you consider adding a fourth parameter to the asynchronous get function of the contexts ? This 4th parameter, only useful with file storage (for now ?) would force a refresh/file read of the context. Let me explain.

With that brand-new context storage, Node-RED now has an effective albeit simple mini-database system. For large amounts of data, they're might be more powerful/adequate solutions like mysql etc but for small amounts, the context storage seems enough.

Up until now, I was more-or-less replicating this behaviour with the various available nodes: a WATCH node to watch a json file for changes, then upon a change a FILE READ node is triggered, then a JSON node to parse the data, and then storing all this in the global (memory) context. Every time the json file was modified (either by Node-RED or another process), it would be read again, the data would be extracted and stored in the global context. Whenever I wanted to use the data in Node-RED, it was already in memory in the global context.

What's great with the file storage is that it is read at the start of NR without having to configure any node and that it can cache its data in memory, which is fast. The only thing that seems to be missing is forcing a context file re-read when a change in the file is detected (or when any other event happens).

I'm suggesting a 4th parameter but it could be of course something else, maybe a real refresh() method ?

Or maybe my suggestion does not raise any interest, which is also fine :wink:

The only thing writing to change the file should be Node-RED... the context is cached in memory and only flushed every 30secs (by default) - or on close - so may not be consistent with what is on disk at all times. Reloading the whole file every time there is a change seems like an awful lot of potentially slow IO to me.

@dceejay I understand your concern but as I said, it would not be for real-time data processing, more for config files etc. The new context storage feature allows to separate memory context and file context and allows multiple file context working together: we could have a "classic" (Im already calling it a "classic" !) 30secs-cached file context alongside zero seconds cached contexts.

I simply see the new feature as a way to easy read/write config files ... config files containing data I would store in NR context in the end anyway after having read them...

I think the point is why would you ever want to re-read the file? The latest values of the context variables is always available without re-reading it.

@tilleul there is already the option to disable the caching - so it has to re-read the file every time you do a get. So you could create a store in that mode just for your infrequently-read config values.

But the exact format the localfilesystem plugin uses to store context to disk should not be assumed. It may change (and automatically migrate) as we find ways to optimise the operations it does. It should not be used as a generic way to read/write arbitrary JSON files.

The context api is generic, intended for different implementations to be plugged into it. We wouldn't add an extra argument to the api just for the convenience of one store.

The full design of the context api includes a not-yet implemented 'run' function that can be used to invoke named commands on the store. This was originally envisaged as a way for a store to support atomic actions on values - for example redis has various incr commands that atomically increase the value of an integer... so you would be able to do something like: flow.run('myRedisStore','myKey','incr') (pseudo code... we haven't designed the api to that level of detail yet). These commands would not be portable between different stores, but I could imagine, if the need really did arise, to have a sync command available on the file store to ensure any pending writes have been written at that point and a load command to refresh the local cache.

Ok I understand, thanks for taking the time to reply :slight_smile:

After updating node-red to 0.19, context tab shows all the values present in "functionGlobalContext" like ports, username, password in UI.
I do not want anyone to see the config details. Failing to disable it.
Is there a way to disable the context tab in UI or configure "functionGlobalContext" ?

Hi @doccirrus - no, there is no way to stop that currently - the only option would be to ensure you have secured the editor to prevent unauthorised access.

But I think you've raised a requirement here we hadn't considered. I'll put something on the backlog and see what we could do for 0.20.

Thank you. But there should be a provision to disable it. Because all the global variables used are exposed.

Indeed - hence going on the backlog to fix. Credentials should of course not be passed around within the global context, as any other node could grab them.

Yes agreed. But do not want params like url, port numbers etc to be exposed.
Thank you.

I think there is a contrib node that lets you put data into the credentials store and pull them into a msg so you don't need to put them into global variables.