How to sync the FULL config between server and client side state

The message goes to the frontend and then, basically, I repeat the code that is in the server to update the state in the client. So in processMsg() I have, for example,

            if ("enabled" in msg) {
                // update our local copy of props
                this.props.enabled = msg.enabled
            }

I have been thinking about how to avoid that. When the widget is mounted the server sends the props object to the client. The props object is effectively the state store and node config merged . One option would be to attach that to each message sent to the client. Do you know whether props is available via some API? If so it would be easy to add it into the message as msg._props. I don't know if there is any down side to doing this other than bandwidth. In fact it would only be necessary to include overridden properties and data in the state store.

Correct, I virtually never use pass through. If ever it is needed it is easy just to wire round the node and achieve the same thing.

value is updated by vue when the user makes a selection in the dropdown, via v-model="value" in the v-select. So the watch triggers when the user makes a selection. The complication is that if the selection is changed by an incoming message then that is also picked up by the watch, so I have to use the rather clunky technique of using flags this.fromManual and this.valueFromMsg to track where the change originated. I need to revisit that as I feel it shouldn't need two flags, but at the time it did seem necessary. If I do need two then at least the naming should be consistent.

That might be useful for simple widgets with only a small amount of state. But if I have an svg floorplan visualizing a large serie of sensors, I cannot afford to send the entire state (incl the static static from my config screen) to the clients every time. I am now looking for an easy way to determine the delta and send that to the clients. Of course a single input message will result in multiple changes to the state store, and I would like to send all these changes as a single event/transaction/batch to the clients.

And afterwards replay that delta on the client side state. Because like you say it is not convenient if you have to write the same code twice, once for the server side and once for the client side.

No idea whether e.g. Vue offers something out of the box, or that I need to implement it myself.

Hmm I would like my ui nodes to behave similar to the core nodes, for features like e.g. passthrough of messages. Therefore I am not tempted to manipulate the input messages, in order to be send my state store delta's to the clients. I just want to copy data from my input message to the state store, and then have a separate event to push the state-change to the clients via the socket. That way my input messages are untouched, and can optionally be passed through the output.

Good point.

You could call send(msg) to send it on before you add in your delta definition. In fact it might be necessary to clone it before calling send().
Alternatively I think you could use a custom event to send it separately from the message, but I don't know exactly how to do that.

Can you envisage cases with the svg node where passthrough is actually useful?

Good point!
Honest answer: No.
But: I am pretty sure that after a month some users will popup to ask for this feature :wink:

Tell them to wire round the node. Pass round rather than pass through. I can't see any point adding complexity to the code and the ui for a feature of very little or no use. There have been many instances here where passthrough has caused problems, particularly infinite loops.

For my svg node it might be not much of a use perhaps. However perhaps it is useful for my other ui nodes. Moreover the passthrough feature is a standard feature in the dashboard (and most core ui nodes), so I want the state store syncing mechanism also to support it. And I can't do that when I start manipulating input messages.

This works in onInput to passthrough the incoming message then build the deltas data and pass that to the client.

                // passthrough incoming message
                send(RED.util.cloneMessage(msg))
                // build config delta
                msg._propsDelta = ...

If I understood the question correctly - you can define a custom event handler in the server node, which will reflect the msg to all client widgets.

// Msgs from the client
['clientMsgs'+node.id]: function (conn, id, msg) {
	switch (msg.msgType)
	{
		case 'userEditSync':
			// save the update in the datastore
			// base.stores.data.save(...);

            // update all other client widgets, by having the node send the msg to itself,
            // forcing replication to all
			node.emit('input', msg);  // 
			break;
      .....

you can have the client embed its _client identifier in the msg, so that it can ignore that msg once it loops back to itself

That is not quite the requirement, it is to have a message from the server sent to all clients (subject to _client), but it is to happen in the onInput event prior to the message itself being sent.

Not sure I understood the requirement. You want to receive a message in the server node, and within onInput create & broadcast another message to all clients, before the original message is sent?

As far as I checked, onInput enables you to read/change the incoming msg before sending it to the clients. So you can just enrich the original msg with any new data you want.

If you want the new data to be sent in a separate msg before the original msg, then until @joepavitt gives us proper server node APIs for input handling and client broadcasting, you can use the following workaround (ugly but works).

  1. Receive the incoming msg ("msg-1") in onInput
  2. Set some "ignoreMe" flag on msg-1
  3. Create your new message (msg-2)
  4. Send msg-2 to your node's own input port: node.emit('input', msg-2)
  5. If the original message is required, clone the original msg-1 to msg-3, and send it as well

Once you exit onInput, messages 1, 2, 3 will be broadcasted in this order to the clients. The clients should reject msg-1 per the "ignoreMe" flag.
You will also need to flag msg 2 & 3 so that they don't get reprocessed on the server node (endless loop).

My solution to the problem in the old AngularJs dashboard was to store all the needed variables as flow context, then create a function that has only return {payload:""} connect that to a change node that pulls all the flow variables needed for the full state into msg.payload and connect that to my dashboard ui_template node so it could only receive messages with the full state no matter the incoming message. I also put a ui_control node before the function to set the full state when any event is triggered.

Yes, that works, but the result is that it pollutes the message passed on if message passthrough is enabled. In practice this is unlikely to be a problem, but it is undesirable. To avoid that, the message can be cloned and the cloned message passed through to other nodes, then the original message can have the additional data added to it and sent to the clients. It works but again seems messy.

The example node has in the server code

            onSocket: {
                /*
                'my-custom-event': function (conn, id, msg) {
                    console.info('"my-custom-event" received:', conn.id, id, msg)
                    console.info('conn.id:', conn.id)
                    console.info('id:', id)
                    console.info('msg:', msg)
                    console.info('node.id:', node.id)
                    // emit a msg in Node-RED from this node
                    node.send(msg)
                }
                */
            }

Which shows how to listen on a custom event. Can I not send a custom event from the server and listen for it in the clients?

We are talking here about a custom node, not a ui-template, so the user should not have to add anything to his flows to achieve the result.

Another reason for sending the data separate from the message is that it may contain information that must not be replayed on page refresh.

I assume you could add a dedicated listener on the client node, something like

    mounted () {
		// set msg listener
        this.$socket.on('my-new-listener:' + this.id, (msg) => {
			...

However, in the server node, you will need to know the socket & client Ids, which AFAIK are not exposed to us at the server node.

But I see no issue with using node.emit('input',msg), telling the server node to broadcast the msg like any other incoming flow msg. You just need to ignore this self-sent msg when it goes through your server node's onInput

One thing to bear in mind here, is that the msg is submitted to the server node's input queue and sent asynchronously, i.e. if there are other messages already in the queue, your new message will be sent after them.

.

Good point, I had neglected to consider the fact that the message must only be sent to the clients showing the node.

That does work, in fact it is rather easier than I had thought, as node.emit appears not to add a message to the servers input queue, but to directly call onInput. So it is not necessary to worry about messages already in the server queue, and also a message sent using node.emit is sent to the client before the original message. So this appears to work, in the js file:

            onInput: function (msg, send, done) {
                // ignore this if msg._updates present so that it gets sent directly to the clients
                if (msg._updates) {
                    return
                }
                // process msg as necessary
                // send the updates back to onInput for sending to the clients
                node.emit('input', {_updates:  {/*update data*/} )
                // return so that the original msg is sent to clients
                return
            }

Thanks for suggesting this, which will do what I need for the moment. It is messy, however, what is needed, I think, is an API to allow sending to the clients. I will raise a feature request for this.

@BartButenaers I think this technique should allow updates be sent to the clients in a form that minimises the requirement for duplicated code in the server and in the client. I am going to try and implement it in my dropdown node.

I have submitted a feature request to add API methods to allow data passing between client and server side of the node. Add API to allow third party ui nodes to pass data between server and client code Ā· Issue #1116 Ā· FlowFuse/node-red-dashboard Ā· GitHub

Hi @Colin,
Could you please explain how we can avoid duplicating code on client and server this way? You are still forwarding messages to the client, which contain only partial state, or am I mistaken? I would think we need to update the server side state, and that the dashboard framework makes sure our client side state is updated, without every ui node developer having to do it himself...

I have to admit that I currently lost a bit the Node-RED fun factor. Because beside a few partial migrated ui nodes, I got nowhere last year. Turning round and round in circles.... So not following Discourse in depth anymore at the moment. Hopefully the fun comes back soon...

I am not sure that is practically possible, but that may be because my imagination is not up to it. The server cannot know the state of all attached clients. I am hoping the technique I am thinking of will end up with a handful of lines of code at each end.

I am going to try and implement it in my dropdown node and see how it works out in practice.