In an attempt to understand how msg._client is supposed to work with Dashboard 2 I enabled Client Data for ui-text in the dashboard Client Data tab.
I configured a ui-button to send a string to a text node on the same page and opened two browser windows on the page. As expected, when I clicked the button, the string only showed in the text node in the window where I clicked the button. If I refreshed the page, however, the string disappeared. Is that expected? I am using dashboard 1.11.1.
Yes, that is expected.
It's something that a lot of people are experiencing for the first time, and it's not intuitive at all, especially given how it "normally" works, but it's very difficult to justify having it work the same way.
It boils down to technical reasons, not usability:
-
Without Client Data: If we have no
_clientfilter, we send it to all clients, and store, in-memory of Node-RED in our "Data Store" that message, so that any new clients connecting can retrieve the latest state. When you refresh, it's a new client (socket connection) and so we re-load from that store. -
With Client Data: When we filter data to a specific client/user, we send it to that client, but do not store that anywhere. Reason being, if we did, we'd need a data store that can map a
msgfor every node, and for every client/user, the memory usage here would grow unmanaged and very fast, and likely cause a lot of performance problems.
As such, I've provided two recommended Design Patterns:
- Single Source of Truth: All users of your Dashboard will see the same data. As you're used to working
- Multi Tenancy: Makes the most of
ui-event, whereby you can retrieve the relevant data on load of the page from a context store or database.
I do appreciate it's new, and not intuitive. We may end up biting the bullet and just implementing per-client statestores, as that would just provide the functionality, but the automated management of this for all node types (think ui-chart where we could have tens of thousands of data points stored, per client) and memory bloat was too significant for us to go down this route a first port of call.
@SynoUser-NL has been hitting this very recently too, and may want to weigh in with ideas, and feedback on how they're getting on with alternative approach
2 Likes
This was a walkthrough I did when explaining to Den: https://www.youtube.com/watch?v=BWYpAR1xS70
I can't see how true multi-tenancy (assuming that means multiple users each seeing their own version of the database) will work without per-client datastores.
[Edit] That should read 'each seeing their own version of the dashboard'
Hi,
We ran into that problem too. And after much testing and learning (thanks to Joe) we are now storing the information we need on a per (Flowfuse) user & per session (socketId) base in global memory, after user input.
Combined with a flow that triggers on page selection, we now inject a message with the relevant information to the input nodes that need it on the selected page.
This works pretty well for our DB2 users.
One issue I'm seeing is that, over the course of the day, Global memory tends to fill up with user & browser sessions. In part of course due to us doing both User and session. And I have not yet looked into a way to delete unused entries. So we'll probably have to do that by flow trigger every evening. Or maybe on session close\disconnect. Or both to prevent ghost objects in memory.
With this solution you can see what is stored in Global memory over time, even with what little amount of information we're storing. And it also makes visible what an enormous job it would be to have DB2 handle all this automatically.
While that would definitely be very nice to have, the solution provided by Joe works fine. And while you could of course save entire objects or arrays in Global memory, being cautious with what you're storing gives you more control (i.e. you can check and modify the data that is stored) and is less of a security risk I'd imagine (the less you save in memory, the better as the Global information is shared by all users\sessions).
A kind of in between solution would be for NodeRED to have a User memory store (in addition to the Flow & Global ones) when running on Flowfuse. Having this would have benefits for our use case, and may simplify the work needed to have the multi-tenancy done by DB2 as well.
But that's probably an entirely different and also bigger story..
The Walktrough by Joe pretty much explains it all, but if you have quesions feel free to ask.
DenW
Do you need it on socketId as well as user?
By global memory do you mean the server side data store?
Do you need it on socketId as well as user?
No, you don't. You can do it on user only or on socketId only.
With User only information you can only have the information available per user. So every browser session with the same user will use the same information.
With socketId the information is available per browser session. Having multiple users with the same browser socketId is highly unlikely, so this is purely to have a bit more overview when you (as an admin) are looking in Global memory (right pane Context Data in NodeRED).
If you don't care about that you can ommit that.
By Global memory i mean the global. memory location you can choose in the Change node, i.e.:
Set: data[msg._client.user.username][msg._client.socketId][msg.topic]
To: msg.payload
Will create an object in global memory with keys named "topic" for each user for each socketId.
So you will need to have a msg.topic and a msg.payload in your message, as well as the msg._client object.
Again: if you don't care about the user info you can discard [msg._client.user.username].
DenW
That makes sense.
OK, global context. I should have realised that as you are using a node-red level wrapper around the dashboard rather than features built into the dashboard.
To reduce your memory bloat you could possibly monitor the node red log. When a client disconnects the dashboard emits a message such as
17 Jun 13:35:38 - [info] [ui-base:Dashboard] Disconnected FS7gCxfKuHd0LduOAAA5 due to transport close
where the random string in the middle is the socket id that has disconnected, so you could clear out the data for that node. Perhaps @joepavitt could provide a ui-event message when a socket connects or disconnects, which would make that a bit more reliable.
Olease open an issue and I shall make it so.
The challenges Den is seeing here with bloat, even on the personalised/optimised route is precisely why we donr handle this automatically in the core of Dashboard
Just because the user-specific data doesn't exist in the Dashboard stores, doesn't mean it cant exist at all.
It should live in Node-RED's context memory (as @SynoUser-NL has done) or in a Database.
When building traditional, coded applications, you would never have temporary state stored server-side, it would all be managed on the client. Permanent data you want to recall regulsrly would then be stored in a database.
Our model here would alloe you to go from ui-event, get the user.email (for example), retrieve some data from a database (outside of D2.0) and then display the information in a Dashboard
@joepavitt : what are the chances of this happening? Is it worth putting in a Feature Request (on Flowfuse or NodeRED) for?
It would be an underlying Node-RED/FlowFuse feature, so if youre able to open the issue on the FF repo, that wouod be best.
Another equally valid solution here though is, you want a NoSQL/Unstructured database...
If you can detail the problem you're facing, and what you need from a solution. Propose the user context stores as an idea, and we can go from there
How do you cope with socket id changes - e.g. if a client PC goes to sleep or its browser puts the tab to sleep, the socket.io connection will drop. When it comes back it will have a different socket id.
@joepavitt : Yes, but that would mean setting up a database for storage if little bits of info which is essentially temporary. A bit overkill imho. Besides, the way it is now, that information is still accessible for all editors once in memory.
@TotallyInformation : We don't. I'm not aware of any way to do that.
We deal with it by communicating to our users that this is how it works. As they do have some technical skill and understanding, that is accepted. 
I know you are aware Joe, I wasn't sure whether @SynoUser-NL or other people knew. 
What are you referring too?
It wasn't me that typed that message, it was Den in response to my message
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
