Save/restore state of dashboard input elements

[pacosta]

I apologize if this issue has been raised before, I searched the history and could not find any relevant discussion on the topic.

My proposal centers around modifying the dashboard input notes (numeric, switch, etc.) to make it easier to save and restore their state on restarts, or whenever the nodes are (re)created. I am aware of how to implement this with existing nodes (see Dashboard backup for example), but with many widgets the flows get, in my opinion, unnecessarily large and complicated.

Taking the switch node as an example, I propose to add a checkbox, say, below "Wrap value from max to min and min to max", which could say something like "Save and restore state from file context". This checkbox would be disabled if there is no file context (or it could be hidden altogether).

When the user changes the state manually, or via an input message, that state would be saved in the global file context, under the key "dashboard-state", which would be an object whose keys would be the dashboard widget node ids, and the values would be the corresponding state of the associated widgets.

When the nodes are created, if the "Save and restore state from file context" is checked, then the state would be retrieved from the global file context, defaulting to the current widget default if there is no saved state.

I have a proof-of-concept working for the ui_switch node, but I have not looked at the other nodes to see if this can be extended to all the other input widgets. I may also be missing important implications or ramifications of this proposal, so I am keen to get feedback from the community.

Thank you!

Pablo

[Paul-Reed]

@pacosta

When the user changes the state manually, or via an input message, that state would be saved in the global file context

Yes, node persistence has been discussed previously, and slowly but surely many of the contrib nodes are adding this as an option.
It's something that I have raised a while ago, see ❓ DISCUSSION: Should nodes save/restore their own context?

@pacosta

but I have not looked at the other nodes to see if this can be extended to all the other input widgets.

I believe that the developers have previously said that they would want some form of consistency across the relevant nodes, instead of being implemented piecemeal.

@pacosta

I may also be missing important implications or ramifications of this proposal

Isn't there always

I think that I've found your repo (Dr. Pablo Acosta?), but not your PoC. It would be great to know more about your proposal, and more detail of how you see it working.

Thanks for posting.

[hotNipi]

Just a note as in every discussion on the topic - the persistent storage option is not always available. And there is no way the node can figure out the type of storage in use so it can't guarantee the functionality.

[pacosta]

@Paul-Reed

Yes, node persistence has been discussed previously, and slowly but surely many of the contrib nodes are adding this as an option.
It's something that I have raised a while ago, see ❓ DISCUSSION: Should nodes save/restore their own context?

Thank you for the historical context. One point is that I am proposing to store the state of only the input nodes, which should allay some of the concerns about states being outdated.

@Paul-Reed

I believe that the developers have previously said that they would want some form of consistency across the relevant nodes, instead of being implemented piecemeal.

I totally agree with this, to me it only makes sense to have the same UI and functionality in all the input nodes or none. Otherwise it would be too confusing and not a good user experience.

@Paul-Reed

I think that I've found your repo (Dr. Pablo Acosta?), but not your PoC. It would be great to know more about your proposal, and more detail of how you see it working.

The PoC is here: GitHub - pmacostapdi/node-red-dashboard: A dashboard UI for Node-RED, the ui_switch node implements the proposal.

Regards,

Pablo

[pacosta]

Yes, as far as I can tell the editor can only get the context name, but not what type of context it is. On the node runtime side, my "solution" is to pick the first context that uses the "localfilesystem" module in the settings.js file. If none is found the node emits a warning and the functionality is disabled.

[hotNipi]

@pacosta

If none is found the node emits a warning and the functionality is disabled.

Not very useful. If the user can't use any file- (what ever kind of persistent) based storage , it is just a waning for nothing. Stressful.
It has been discussed and nothing better found but the explanation that node tries to store if it is made available via modified settings and even then with excuses.

Functionality without reliability is not something to provide with standard dashboard nodes.

[Paul-Reed]

@pacosta just trying your fork which works well when the localfilesystem is defined in node-RED settings, but if it is not defined (as a fresh installation would be), then an error is introduced, and flows are stopped

27 Feb 09:59:54 - [warn] ------------------------------------------------------
27 Feb 09:59:54 - [warn] [node-red-dashboard/ui_switch] ReferenceError: config is not defined (line:40)
27 Feb 09:59:54 - [warn] ------------------------------------------------------

When it is working, and considering @hotNipi 's post above, would it make more sense to hide the 'Save and restore state from file context' option, if local filesystem was not enabled, so it would not be selectable?

Also, if the saved state is restored, it would be nice to see the 'node status' also displayed (as a visual indication what the current state is).

[hotNipi]

@Paul-Reed

hide the 'Save and restore state from file context' option, if local filesystem was not enabled, so it would not be selectable?

If there is no changes since then - It will be mission impossible.

DISCUSSION: Should nodes save/restore their own context?

It seems like there is no legal way to get information about the module configured for context store(s) in contextStorage if asked from nodes configuration page. (RED.settings.context) Making smart decisions how to offer the persistence is not possible.

[Paul-Reed]

@hotNipi

If there is no changes since then - It will be mission impossible

Have I misread @pacosta 's explanation;

@pacosta

my "solution" is to pick the first context that uses the "localfilesystem" module in the settings.js file. If none is found the node emits a warning and the functionality is disabled.

[pacosta]

@Paul-Reed

@hotNipi

If there is no changes since then - It will be mission impossible

Have I misread @pacosta 's explanation;

@pacosta

my "solution" is to pick the first context that uses the "localfilesystem" module in the settings.js file. If none is found the node emits a warning and the functionality is disabled.

Yes, this is on the runtime part of the node. On the editor part, that I can see, there is no way to know which type of contexts are available, only their name . Hence the only option I could come up with was to emit a warning and disable the functionality.

[pacosta]

FWIW, I fixed the bug and added the initial status indicator, in the PoC repo.

[Paul-Reed]

@pacosta

I fixed the bug and added the initial status indicator

Working fine now

@dceejay would you support this, or maybe accept a persistence model similar to node-red-contrib-queue-gate which writes to the default store, whatever that may be, and add a paragraph to the readme explaining it's limitations.
It's not a perfect solution, but at least it's moving in the right direction, and if a better solution becomes available, it could be implemented possibly without breaking flows.

If not @pacosta, you could always create a contrib node... node-red-contrib-ui-switch.
I'd use it, and I'm sure others would, because it would reduce the number of nodes & complexity in flows.

[dceejay]

While basically a good idea - I'd rather we flush out these limitations with context first - and then maybe get them addressed in a consistent manner before we implement something and then have to unwind it or try to maintain it working in the "wrong way" when a better solution comes along.
How would this work in cloud where there is no file storage ? Would it just not be offered ? Or if there was another (eg database storage) how would it know to pick that ? I agree with @hotnipi that the option should not be visible at all if not available and no warning to scare the naive user. I think states would need to be stored in node context (not global or flow) so there was no possibility of clashing with another node.

[drmibell]

@Paul-Reed

maybe accept a persistence model similar to node-red-contrib-queue-gate which writes to the default store, whatever that may be

Just a small note here: since version 1.5.0, the queue-gate node has not been restricted to using the default context store. The user can choose any of the available stores, and if a persistent one is available it can be used regardless of which is the default. node-red-contrib-multiple-queue also has this capability, and it will be added soon to node-red-contrib-simple-gate. This question of how (or whether) to maintain node context is a "persistent" one, and I have been following the discussion closely. I still think it is reasonable to expect the user to understand how context works if he/she is going to use it, but some accommodation for the naive user may be needed.

[Paul-Reed]

@drmibell

since version 1.5.0, the queue-gate node has not been restricted to using the default context store

I'm using q-gate, but not looked in the config for some time!!

I assume you are getting the available stores from;

oneditprepare: function() {
            var node = this
            RED.settings.context.stores.forEach(add)    // list context stores
            function add(item){
                $('#node-input-storeName').append(`<option value="${item}">${item}</option>`)
            }
            $('#node-input-storeName').val(node.storeName)  // disable if not checked
            $("#node-input-persist").on("change", function(e) { 
                $("#node-input-storeName").prop("disabled", !$(this).prop("checked"))
            });
        }

[drmibell]

That's right -- thanks to a tip from @knolleary and code from @Steve-Mcl .

[Paul-Reed]

@dceejay

How would this work in cloud where there is no file storage ? Would it just not be offered ?

Why not? In Mike's post above, he is able to select either memory or filesystem context (if it is available).
I don't see the point in not offering both options, because users with no file storage could use memory, which although context would not survive a stop/start, it would certainly be better than nothing at all - so long as users are made aware of the constraints & risks.

I've not seen any forum posts or git issues raised by users being confused about context persistence in node-red-contrib-queue-gate, so maybe we're underestimating the awareness of node-RED users....

[dceejay]

@Paul-Reed

I don't see the point in not offering both options

Why would you offer an option that didn't exist ? that makes no sense to me at least.

and what happens when you share a flow ? If you have persistent context called "DiskA" and I had one called "SSDcard" would it work ? or would I have to hand edit all the nodes that now pointed to a non-existent store ? if so that would really mess up sharing of flows if people start using context.

[Paul-Reed]

@dceejay

Why would you offer an option that didn't exist ?

We would be offering 'memory only' as the only option, which of course would exist.
Filesystem storage would not be offered if it wasn't enabled.

@dceejay

and what happens when you share a flow ?

Fair point

[drmibell]

@dceejay

and what happens when you share a flow ?

I can't test this right now, but my recollection is that if you request a context store that doesn't exist, NR uses the default and posts a warning that it is doing that. I suppose this could be an issue any time you a share a flow that uses a named context store.