How to sync the FULL config between server and client side state

[joepavitt]

My main concern here is when is it appropriate to store, and when is it not. Let's take an example for UI Notification (as I explain here too):

1. Send msg = { color: 'red', payload: 'Hello'}
2. Notification shows with that color/message
3. Send msg = { payload: 'Goodbye'}

What colour should the notification be for "Goodbye"? I would argue it should be the default colour, not the color defined in a previous message.

[TotallyInformation]

And indeed - is it even appropriate for notifications to be stored? Shouldn't they be transient anyway?

[arturv2000]

@joepavitt

What colour should the notification be for "Goodbye"? I would argue it should be the default colour, not the color defined in a previous message.

Just my opinion, but that would depend on the type of widget. For a notification can agree that color if not specified should return to default. But for a UI-Text or UI-Button would expect the color to remain until a new msg.ui_update.color is sent.

Now, if I understand correctly, and everything that is related to widget configuration ui_update such as label, icon and others that may exist is saved and restored correctly uppon widget load/reload, and if the question is on what to do with properties that are not present in the ui_update configuration, can agree that it should return to default and the user should handle those states if necessary.

Although msg.payload should also be preserved like another configuration.

[Colin]

@joepavitt

I prefer the idea of having state for anything linked to the Node-RED config (and relevant overrides), and then the data store linked to data it renders, e.g. msg.payload.

Re-architecting the datastore to support both use as a msg history/latest store, and a place to store an accumulated picture of the live data that a node is using feels very sensible to me?

There is indeed some logic to that, though if the api to access them will be the same I am not sure what is the benefit of having two stores. Is not the only difference how the state is fed to the node on connection?

@joepavitt

What colour should the notification be for "Goodbye"?

That is a question about the spec of the node, rather than about how the stores are organised.

[BartButenaers]

Thanks everybody for joining
All these ideas are very illuminating, since everybody has his own use specific cases. .

@arturv2000

but that would depend on the type of widget

Yes absolutely true. I had used the ui-notification as example, simply because I was stuck last night on the msg replay mechanism for the ui-notification node. But for example for my ui-svg node, there will be a lot of state that users want to store, and lots of state that people don't want to store.

Imho I would even go a step further: it doesn't only on the type of widget, but on what the user wants to do with the widget. The user wants to store the color of a widget for use case A, but not for use case B. I think we should only offer a list of supported properties per widget, but is up to the user whether he wants to store the injected property values or not...

@joepavitt

My main concern here is when is it appropriate to store, and when is it not.

That is a VERY good one! Hadn't seen that one coming
How stupid of me to forget about transient property values in my proposal.
Again back to the drawing table

So the user should be able to finetune following information, to determine the widget state:

1. The static properties, as specified in the flow editor via the ui node config screen.
2. The dynamic properties, as specified in the input message.
3. The 'flags' that determine whether these dynamic properties are persistent (i.e. stored in the state store) or transient (i.e. not stored in the state store). The latter ones will only be send once to the client(s), and will not survive a refresh.
4. The 'socketId' that determines whether these dynamic properties are to be send to a single client or all clients.

Suppose the msg.ui_update would be replaced by msg.ui_stored (for persistent dynamic properties) and msg.ui_temporary (for transient dynamic properties). Then we could have a setup like this:

That way the user can decide completely on his own what to do with the properties offered by the widget. And (imho) for developers it is quite easy to understand how stuff works...

Shoot

[arturv2000]

@BartButenaers

Suppose the msg.ui_update would be replaced by msg.ui_stored (for persistent dynamic properties) and msg.ui_temporary (for transient dynamic properties). Then we could have a setup like this:

My one possible issue with this approach (if my understand is correct) is that (not taking in consideration the use of client_id) if any of the ui_temporary properties affects the UI, opening a new browser window will render a different page, which could be intended.

If any property can live either in ui_temporary or ui_update is not an issue, since is the user/programmer responsibility to handle it.

One thing i would change / add to this concept, the squares on the right shouldn't be only associated to existing clients or new clients, should be also associated to widget-load/mount specially the top right one.

[BartButenaers]

@arturv2000

if any of the ui_temporary properties affects the UI, opening a new browser window will render a different page, which could be intended.

Yes you are right, that is something that should certainly be added to the documentation (if this proposal would ever see the daylight). Because this design offers lots of flexibility to the users, but of course they should get some simple examples to explain it, otherwise they would have a lot of headache.

• If you inject a msg.ui_temporary.color=red then only this message will result in a red notification.
• If you inject a msg.ui_stored.color=red then all messages from here on will result in a red notification.

I don't think a dashboard contributor or ui node developer should constrain this flexibility too much.

@arturv2000

One thing i would change / add to this concept, the squares on the right shouldn't be only associated to existing clients or new clients, should be also associated to widget-load/mount specially the top right one.

That is something that is not clear from me in the current design of the dashboard. Do you mean that the widget-load not only happens for a refresh or new client connection, but also at other event? Perhaps at a tabsheet change or something like that?

[arturv2000]

@BartButenaers

That is something that is not clear from me in the current design of the dashboard. Do you mean that the widget-load not only happens for a refresh or new client connection, but also at other event? Perhaps at a tabsheet change or something like that?

Yes, the event I am speaking is in case of tab change. When a widget is mounted to appear on the GUI (webPage) it will send the widget-load event and is supposed to receive the configuration at that time. And when changing tabs is necessary to resend stuff that was sent while the widget was not visible.

I think (for what I can see on the websocket connection) that the widget-load only happens when the widget is mount, not when the page is refresh.

There are some issues with that:

Property `class` not being updated when changing tabs

And i made this PR with an attempt to solve the issue, at least for the UI-Button:

UI-button: Make `ui_update` restore even after tab change / Page refresh

[Colin]

@arturv2000

I think (for what I can see on the websocket connection) that the widget-load only happens when the widget is mount, not when the page is refresh.

Widget load occurs on a refresh, new connection (which is identical to a refresh) and tab change.

I am confused about the ui_temporary suggestion. Is the suggestion that it should change the property for the current session, but not for other sessions, or is it that it just applies to this message?

[arturv2000]

@Colin

Widget load occurs on a refresh, new connection (which is identical to a refresh) and tab change.

I think that is only valid if the widget is in the visible tab at that time.

@Colin

I am confused about the ui_temporary suggestion. Is the suggestion that it should change the property for the current session, but not for other sessions, or is it that it just applies to this message?

From my understanding it would only be for that specific message, in this case i don't think it is even necessary to have a store.

[Colin]

@arturv2000

@Colin

Widget load occurs on a refresh, new connection (which is identical to a refresh) and tab change.

I think that is only valid if the widget is in the visible tab at that time.

Yes, I should probably have written
Widget load occurs on a refresh, a new connection (which is identical to a refresh) and a page change to the page containing the widget.

[TotallyInformation]

@Colin

page change

Page change or page-tab change (e.g. an SPA route change)? On an SPA route change, does D2 unload the widget from memory or not?

[Colin]

@TotallyInformation

Page change or page-tab change

Page as in the D2 definition of a page. The equivalent of a tab in D1.

@TotallyInformation

On an SPA route change, does D2 unload the widget from memory or not?

Someone else will have to answer that, but when the user selects a page it 'mounts' the widgets on that page, and remounts them if the user goes away from that page then back again.

[TotallyInformation]

@Colin

page

OK, probably worth ensuring clarity since D2 supports both actual pages and SPA "pages" and the two will have two very different paths and requirements to caching.

@Colin

it 'mounts' the widgets on that page, and remounts them if the user goes away from that page then back again

OK, so I think that implies that just changing an SPA page requires the cache to resend each time?

[Colin]

@TotallyInformation

changing an SPA page requires the cache to resend each time?

Do you mean the data and state stores? If so then yes.

[arturv2000]

@Colin

@TotallyInformation

On an SPA route change, does D2 unload the widget from memory or not?

Someone else will have to answer that, but when the user selects a page it 'mounts' the widgets on that page, and remounts them if the user goes away from that page then back again.

Can confirm, there is a mount event when the user enter the tab/page and an unmount event when the user exits the tab/page.

[Colin]

Has anybody worked out what the client side vuex data store should be used for? I don't see the point in it.

[BartButenaers]

Very valid question Colin. I started a new ui node this week, but I have now 3 unfinished ui nodes. And I realize now that all 3 ui nodes are stuck at similar state management issues (refreshes, ...). I had hoped that by drawing the design above, I would get enough knowledge to get back on track. But with my current free time, I find it a bit too complicated to be honest. So I really hope that things can be simplified a bit. And I also don't know if my last suggestion above does make sense at all... Because I still don't understand why the datastores are needed, both server-side and client-side.

[Colin]

I have switched to using the state store for state (!) and the data store only for replaying the most recent message (where needed). Client side I use local data for storing any state that is not in props. It has worked out pretty simple. If you want to see how it has turned out you could look at GitHub - colinl/node-red-dashboard-2-ui-dropdown-cdl: A dropdown node with background colour change while waiting for confirmation via input msg which is a simple replacement for the dropdown node, with a tweak. After the user makes a selection it changes the background colour, reverting it when the node receives a confirmation that the value has changed. It is for the situation where the dropdown is making a change in a remote location and I want to know that it has got to that location and a confirmation come back (via MQTT in my case). I haven't got any plans to publish it at the moment, hence the lack of help text.

I am in the process of updating my flow that creates a basic ui node, to include the changes.

[BartButenaers]

@Colin

I have switched to using the state store for state (!)

Hi Colin,
Thanks for sharing your workaround for our state issues!
Very useful indeed!!
Had only a very quick look at your code, but there are some things I don't get yet:

1. When an input message is injected, you update the State store. But how does these State store changes arrive in your Vue frontend?

2. You don't send your message to the next nodes in the flow. Does this mean there is no passthrough of input messages possible?

3. You watch a value in your frontend. How does this work for you?