❓ DISCUSSION: Should nodes save/restore their own context?

General
81

Actually I've had better longevity with "nonme" cards than SanDisk and Samsung, if you consider Microcenter "house brand" to be noname.

82

Hello,
sorry for injecting this thread (thanks to Paul, that pointed me this out), but i'm also interested in that, because one of my node saves states in the filesystem (booolean-logic-ultimate).

One major issue i'm having with users, ist that most of them use a plug-and-play docker container, ready to go, with node-red preinstalled, or similar solutions, like homeassistant.io with node-red plugin, where node-red is closed in a not so easy reachable "bubble".
They aren't able to ditch into settings file and enable a filesystem context storage, simply because they enjoy and use node-red as is.
Not to mention, the companies including node-red in own devices, where you're unable to touch anything.

I don't know if it has been discussed elsewhere, but having the node-red settings manageable via the web interface, would be very helpful.

2 Likes
83

Some good points there. I think it was also Dave who mentioned that running NR under some cloud environments, you might not have access to a filing system either.

That makes it even more important that we have some alternative context storage methods such as database based ones since some environments may find it a lot easier to provide a managed database than access to the filing system.

Of course, that would also indicate that (unsurprisingly), with those additions, using context storage would, in general, be a better option than using files.

It would also indicate that at least the context store settings should be available to change from within NR even though that requires a restart.

For uibuilder, I have to assume that the userDir folder is available and writable, I think that should be a reasonable assumption? Otherwise I don't know how NR would work at all. uibuilder creates a specific folder either directly under userDir or within the projects folder if they are enabled. All other files exist within there. I provide an editor (though it still needs work to make it more flexible) for text files (html, js, css, etc). I need to add a file uploader really (delete is already possible) and the ability to edit in sub-folders.

It would be nice to have a standard editor component for file editing/loading/deleting/renaming of course - if anyone wanted to take that on as a development :grinning:

2 Likes
84

In case anyone is interested in the code. I've just pushed to node-red-contrib-infocache a very noddy version of my node. However, it includes the processing for:

  • Selecting from any defined contextStorage from settings.js
  • If there is no defined contextStorage, it uses whatever NR defines as the default (always default I believe) - this bit could be improved since adding a new instance sees the node marked as having invalid settings - something to do with the way I'm picking up the defaults I think. But it works anyway.
  • The select box list of context stores is build dynamically and shows whether each item is memory based or otherwise. You also get some warning text.

The whole process is rather clunky because RED.settings.context in the html file does not tell you what the module is. So you have to pass this from the back-end which is annoying:

    RED.nodes.registerType(moduleName, nodeGo, {
        'settings': {
            'infocacheContextStorage': { 'value': RED.settings.contextStorage, 'exportable': true },
        },
    })
85

I'm with you on this. It should certainly mean that the settings.js file could be modified from within NR. I don't have a containerized or embedded instance to work with, so I may have missed a "gotcha," but this experiment worked:

  • Use the file in and file nodes to copy settings.js from ~/.node-red/ to ~/.node-red/lib/functions/
  • Use a function node to read the settings file from the library, edit it, and write it back.
  • Reverse the copy process in the first step with the "overwrite file" option selected
  • Stop and restart NR

Note that I have avoided the exec node. (This was on macOS with admin privileges. I haven't tried anything else.) Unless embedded or containerized systems have additional restrictions, this should be a proof of principle that a contributed node or a new NR feature could do the job.

86

For me it sounds very bad idea.

500×547 80.8 KB

87

And that is something that needs to be carefully considered and designed properly and if it ever happens, should be something provided by the editor itself because there are more use cases than you know.

I will be sad if one of you publishes a node that exposes an option to do this unilaterally. I know we can't stop you, but I would rather someone chose to direct their energies to work with us.

And the first step is to talk to me and Dave. Not to go off and do something without discussing the actual requirements.

  • There are users who require a read-only filesystem.
  • There is a reason why the storage layer is a pluggable component and has been from the very start.
88

That was the whole point in starting this thread Nick. Definitely not to start a mutiny!

A lot of valuable views, opinions & options have been put forward, and also lots of cautionary advice which is equally important, but to try and keep this moving in the right direction, how would you (as Leader) prefer to see this progressing.
We've already seen nodes being developed recently which either make use of the contextStore, or more recently writing data files to the filesystem. Some direction may head off further fragmentation.

89

Someone needs to propose something specific - a design that can be discussed. With details on what it is, how it works, how it would be used etc. By preference that would be via GitHub - node-red/designs: Design documents for the Node-RED project

Alternatively sharing a proposal here would also be a starting point.

It's trying to get beyond the vague statements of "there should be someway of doing X". Let's see a concrete proposal for what that way of doing X actually could be.

2 Likes
90

Oh, me too. I wasn't suggesting that. There is massive potential for damage, confusion and security flaws.

That is also why I was cautious about even thinking about the ability to make other changes to settings.js.

But on the other hand, it isn't unusual for a framework system like Node-RED to have a built-in settings UI. Far from it in fact.

Certainly for something that reaches into the heart of Node-RED. Similarly with anything that looks to provide more standardised approaches to things.

Absolutely agreed. The uibuilder editor is deliberately limited as to where it can reach out to files and that will certainly continue and such a restriction would be one of the first requirements that I would put on a standard component.

Yup, I get that - so far, nobody has asked me about that in connection with uibuilder - I'd be happy to think through the issues and put the appropriate checks in place. As with many things, it simply isn't a priority for me right now and likely won't be unless someone asks.

91

Just to say at the moment that I'm still reeling from the many ideas discussed in this thread. As you say, some caution is required and some deep thinking.

As you know, I'm not a pro-developer so it takes time and exploration to work through design ideas.

In my mind, I see 4 standouts that I would like to see designs considered for - or at least starting requirements gathering for:

  1. Slight change to the front end Editor API for RED.settings.context
    So that the underlying module type could be more easily accessible.
    Though I note that this may have unintended security consequences so although the change might be relatively simple, it still needs careful thought by bigger brains than mine.

  2. A standard control that exposes an Editor UI for the selection of a context store

  3. A standardised Editor UI for file management.

  4. A Node-RED settings UI to enable making settings changes from the Editor.
    Noting that this is a BIG change. But also high-value given the ever increasing number and variety of platforms and environments that Node-RED is being used on.

I'll add a 5th thing that has cropped up recently:

  1. A standardised schema for exchanging authentication, authorisation and session data over websockets.

But as always, that is just my view.

I will continue to plod along thinking through things that related to my nodes and hopefully we see further discussion on what good might look like. And if anyone agrees that we should pursue any of the ideas, I'm happy to contribute what little I can both in thinking and code.

92

I was not suggesting that anyone do such a thing, just pointing out that the tools exist and probably could be used in most if not all NR installations.

Could we start with some requirements? I'm not clear on what we would design against.
Sorry, I see that @TotallyInformation has just pitched in with a list. I'll shut up and think.

93

Has this moved to a new topic or stalled? Having just run into a case where these ideas would help if implemented I now have a little skin in the game and an update on where this stands will be appreciated.

94

Not aware of further discussion - is it really 7 months since this thread!!

95

Yes, it really is. So I take it I shouldn't be looking for this anytime soon. :wink: I'm still glad it's been kicked around though.

96

It will come up again... but it just depends on the trigger for it to emerge again. The concepts outlined are good, have merit, just need to find the 'right' idea to enhance the feature set. And find the time and resources to do it as well.

1 Like
98

A post was split to a new topic: Ui Context placeholder