Uibuilder access context data

Dashboard
1

Hey there,

I have not found any info about this topic so far.
For the current project I am heavily relying on context data to store and process (flow and global context data stored in RAM and file storage).
Up until now I used the Node RED dashboard to display the data.
But as the project became more and more complex I needed more flexibility for the GUI.
That's why I switched to uibuilder recently.

Forgive my ignorance in this regard, I am not a web dev. by any means.

I am looking for a way to access my context data from within Node-RED directly and use it in uibuilder. Sending all context data continuously to uibuilder via messages is what I'd like to avert.
in the Node-RED dashboard you have a node for each button, text field, display and so on. In uibuilder you'd have hundreds of msg objects to be received and parsed by the single uibuilder node.

Not sure if I am on the wrong track here, but could I somehow access a variable somehow like I am able to do in the function node? I mean by using something like flow.set() or flow.get(), or global.get()?

As far as I understand the Node-RED is all based on node.js and the data must be stored somewhere. So why not read the data directly but use cumbersome messages for everything? I am fine with the flow system when it comes to the general structure and build. But I like to keep it simple and use a lot of function nodes to process data in objects and structures at once instead of consecutively.
Is this the way to go?
Honestly I don't know, it's just the way I am used to program (I have a background in c/c++ and plc programming).

Let's use a very simple example here (this is just a very small part of my context data):

[
	{"number":4751,"creation_time":1600700057133,"positions":
		[
			{"lid_type":1,"body_type":1},
			{"lid_type":2,"body_type":2},
			{"lid_type":3,"body_type":3},
			{"lid_type":1,"body_type":3},
			{"lid_type":3,"body_type":1}
		],"start_time":"","finish_time":"","qa_passed":false,"state":"new"},
	{"number":4752,"creation_time":1600700059531,"positions":
		[
			{"lid_type":1,"body_type":1},
			{"lid_type":3,"body_type":1}
		],"start_time":"","finish_time":"","qa_passed":false,"state":"new"},
	{"number":4753,"creation_time":1600703634559,"positions":
		[
			{"lid_type":1,"body_type":1},
			{"lid_type":3,"body_type":1}
		],"start_time":"","finish_time":"","qa_passed":false,"state":"new"},
	{"number":4754,"creation_time":1600703671892,"positions":
		[
			{"lid_type":1,"body_type":1},
			{"lid_type":3,"body_type":1}
		],"start_time":"","finish_time":"","qa_passed":false,"state":"new"},
	{"number":4755,"creation_time":1600890617062,"positions":
		[
			{"lid_type":1,"body_type":1},
			{"lid_type":3,"body_type":1}
		],"start_time":"","finish_time":"","qa_passed":false,"state":"new"}
]

Let's say I want to display all the information in a table and only 3 values in this array of objects change.
The ideal way would be to update the related table cells only, not sending the whole array over and over again with every change. I am looking into the tabulator tables front-end library right now. It enables me to only update changed values. But somehow I need to give tabulator access to this data to let it examine it.
Would I really need to send ALL data continuously to the uibuilder node, let tabulator examine the data and in the worst case update a single cell, or even nothing at all and trough the rest into the void?

Web development with all the different html, css, js, php files, virtual dom, dom and so on is something I am still trying to get my head around. What framework or front-end libs to use, how to access data, process it and send it here, or there... still learing this stuff. Right now I just what to display lots of data compact and well-arranged. The dashboard is just not doing it for me.

Maybe you guys can somehow point me at the right direction.. :wink:

Maybe you'll even tell me to use a database instead :crazy_face:

Thanks and cheers
newbow

PS: I am not sure if my text is properly conveying the my weird train of thoughts. I hope it somehow makes sense :thinking:

2

Hi, I'm afraid that you are not quite on the right track.

What you have to remember is that Node-RED is a server, call this the "back end". Whereas your web UI is the "front end", it runs in your browser whereas Node-RED runs on the server.

Your context/flow/global variables are kept in Node-RED on the server.

So you only have 2 options for getting data from Node-RED to your browser. You could include information in the initial HTML page and you can send data back and forth using the built-in websockets interface.

In the example you give, you only need to pass the data that has actually changed from Node-RED to your front-end ui. Sometimes though, it can be more difficult to separate out changes than it is worth and may just be easier to send the full data again. Hard to know without understanding the full details of the data.

For example, I'm building a new interface for my Drayton Wiser home heating. There isn't a published API and I have to poll the data regularly and then send this to the browser. That's quite a lot of data - I poll it every 60sec. So I've built a node.js module that does the poling and identifies the changes so that I can just send the changes to the front end. Still, the front-end retains the full data in memory (remember that it is a completely different device than Node-RED, the back end) but I only need to pass the changes which are generally relatively small. Even so, my previous interface worked just fine, even though it was passing a very large and complex object variable every 60s.

No, not necessarily - however, to avoid it, you have to separate out the changes at the Node-RED end. Whether this is worth the effort is something you need to work out.

The advantage of using something like VueJS for your front-end is that it takes care of updates for you. You pass it the data and it works out what to do to the UI. Generally, this is efficient enough for most purposes.

It is complex, I would recommend simply using VueJS and let it do its stuff for you. Then you only need to remember some fairly simple rules. The main rule being: create your data objects up front so that Vue will react when you change the data.

A database might help but it doesn't change the fact that you either have to re-send all data when it changes or you have to filter out just the changed data and send only that.

My advice is, don't worry too much about sending chunks of data from Node-RED to your UI to begin with. However, bear the issue in mind and think about how you can reduce the data exchange sizes once you have the basic display logic in place. This is because you might find that exchanging large amounts of data might not even be noticeable in your situation. If you are using a local connection (e.g. a Pi to a laptop over local WiFI), you may be able to send several MB without any noticeable delay. Only optimise when you have to. If your data is only a few KB, unless you are using a $$ per-KB connection, it might not be worth even thinking about.

uibuilder already has significant performance optimisations over Dashboard so that is also in your favour. With Dashboard, most things - both data and interface - are sent over websockets. With uibuilder, all of your core is loaded using standard http(s) and so is cached and optimised. Only the dynamic data is sent over websockets.

1 Like
3

Hey there,

Thank you for the detailed response, really appreciate that.

Maybe I should describe the process in more detail or go into a little more detail here.

The purpose of this project is to send and receive data to a plc via OPC UA for it to process it.
The machines PLC is a standalone device, capable of running on it's own.
It does not require Node-RED to run.
But the task is to be able to access a web interface which displays the status of the current production and general information about the machine.
Additionally the web interface has a form to fill in new orders and attach it to the queue.

Most of the data being received from the PLC are OPC UA subscriptions to the plc data structures. Meaning the data is being received in Node-RED whenever it's undergone a change.
So the PLC is just doing its thing, while the OPC UA interface is monitoring the subscriptions for a change in value. Context data of Node-RED is used to buffer all this information, process and store it.
All data is structured in JSON objects for easier readability.

Now I'd like to redesign the web interface to better suit the needs.
I think a table is optimal for displaying lots of data. The Node-RED dashboard is just not made for dense display of data. I need to be able to freely size and arrange my objects (buttons, dropdowns, etc.) in a way to utilize the maximum screen space.

As you mentioned, I'd like to use VueJS for the front-end, as it's supposed to be very lightweight and responsive. Additionally I am thinking about using the tabulator front-end library, it has very useful functions for tables.

So, now I just need to feed all data from the back-end to the front-end as you mentioned.
And of course vice versa, as the front-end needs to pass the order data back to Node-RED to be send to the PLC.

What would you suggest in my case?
Should I use these web sockets?
Because if I put all data in the HTML code, doesn't it mean it is static data?
Still I don't fully get the idea of the interface to pass data from back-end to front-end... I am sorry.

Node-RED and the front-end are running on the same system, so basically there should be minimal overhead for the data exchange.

To keep everything organized I already build JSON structures with arrays and objects. So how, for example, would I go about passing the data I posted above? It would come really handy if I could send big chunks of data to the front-end and parse all that there.

Thanks
newbow

4

I hope you mean Javascript objects not JSON (a JSON object is a string).

This post should help to clarify the difference between Javascript objects and JSON.

1 Like
5

Does this matter much? Isn't JavaScript object and JSON sort of interchangeable with each other?
I thought JSON is more feasible for websites, as at your link it states:

"It is often used when data is sent from a server to a web page..."

I could easily convert it to a JavaScript object if it's required though...

6

Oh, sorry, I assumed that you thought that they were the same thing. If you are manipulating data then you definitely want it as objects not as strings. Extracting data from strings is very difficult. I don't mean it should not be sent up the front end as json, but that should be automatic i would have thought.

1 Like
7

Well tabulator works just fine in uibuilder with or without Vue. I think that I responded to something on that a few days ago and showed how to do that.

In the flows that save your incoming data to Node-RED variables, send the data direct to the front end as well. In fact, re-arrange your data storage using the caching concept that you will find examples for in the uibuilder library.

That way you can save the data for use elsewhere if you need it but you also store it in a way that it will be retrieved and sent to uibuilder when a new client connects or when a client reloads the page.

Yes. Only if you find that the initial data load is too much should you consider anything else. Even then, you can optimise an initial data send by breaking it into logical sections if needed. More likely, you will want to use tabulator to take more control of the loading process by initially only sending the first page or two of your tabular data on initial load then, when tabulator receives a page-change event, send a msg back to Node-RED to tell your cache node to send the next cache page (a certain number of records from the stored data).

If you can do that, you wont need to worry about a massive data load.

Take the cache example and change it so that it only sends a set number of records from your object at any one go. In the front-end, use tabulators event handlers to send a request back to Node-RED when a page fwd/back button is pressed. In Node-RED, send that request to your cache node (which is a function node) so that it sends the next/previous page (set number of records).

1 Like
8

No problem, it's just that the topic is pretty new to me and I'm trying to find my way around it somehow.
Except for the topic of connecting the front-end to the back-end, I actually understand that pretty well.
That is actually the reason why I opened the thread here.

9

Thanks a lot for the answers.
I successfully tested the web sockets and I am able to send and receive data back and forth.
But it's a very very simple setup and I have still some questions if you don't mind..

I used this example flow here:
https://flows.nodered.org/flow/8666510f94ad422e4765
And changed few values to try out some things.
It even takes my whole JavaScript object and displays it properly as a string.
The beauty of my OPC UA setup is, that the subscription takes care of the changed values, so only a change in value triggers a message. I can then forward it to the web socket connection and sort of get a live update of any change. Within the realms of latency of course.

So but now I am a bit confused as to how am I able to use and distribute the incoming data at the front-end.
Let's say I have the following JavaScript object coming in at the web socket:

{
	PLC_ready:"false",
	PLC_heartbeat:"true",
	PLC_busy:"false",
	PLC_done:"false",
	PLC_error:"0",
	PLC_light_RED:"true",
	PLC_light_YELLOW:"false",
	PLC_light_GREEN:"false"
}

In the example there is this line:
document.getElementById('messages').innerHTML = line;
which places the data in the messages div <div id="messages"></div>, got that.
But how can I distribute the incoming data to different divs? Let's say I want to send the "PLC_ready" value to a location, and the "PLC_busy" to a completely different spot on the website.
If I could somehow buffer it somewhere maybe I could reference it in my code at different lines.
Or would I need multiple web socket connections at once to distribute the data?
In Node-RED one could use the switch node to split data into separate messages and distribute them via multiple outputs.
Again please ignore my ignorance in this regard. I watched several YoutTube videos and read tutorials, but this is still a riddle wrapped up in an enigma for me :thinking:

If you guys don't mind I'd still use this thread even though the original question has somehow been answered... yet the general problem persists.
I could of course open a new thread, but somehow I wanna keep all together here for others to find if they have similar problems.

Lets use tabulator for an example:
Quickstart guide of tabulator states that I place my div where I want my table to appear.

# 1 Create your DOM element
 <div id="example-table"></div>

Then I have to define the table contents:

// 2 Define some data for the table

 var tabledata = [
 	{id:1, name:"Oli Bob", age:"12", col:"red", dob:""},
 	{id:2, name:"Mary May", age:"1", col:"blue", dob:"14/05/1982"},
 	{id:3, name:"Christine Lobowski", age:"42", col:"green", dob:"22/05/1982"},
 	{id:4, name:"Brendon Philips", age:"125", col:"orange", dob:"01/08/1980"},
 	{id:5, name:"Margret Marmajuke", age:"16", col:"yellow", dob:"31/01/1999"},
 ];

And this is were my issues start.
First, where should I put this?
In a separate .js file? at the bottom of the html file tagged as <script></script>?
Second, how do I now populate this tabledata with my variable data coming in from the web socket?

Thanks!

10

That isn't uibuilder though. uibuilder creates you a web interface AND a matching websocket interface all in one node.

You seem to be over-thinking it. If you send a msg into your uibuilder node, it arrives in your front end - a bit of magic if you like. You don't need to think about it, that's the point. All you need in the front end is the uibulder.onchange('msg', function(msg) { .... }) function. That function is passed the msg so that you can do whatever you like, such as changing the data in your table.

You don't need any of that in uibuilder unless you really want to do things by hand. By default, uibuilder comes with VueJS which is a dynamic web UI framework. Your HTML links to Vue data variables, when you update the variable, your HTML updates. Another bit of magic.

Please look again at the default template for uibuilder and try sending it some data. Then try one of the examples in the library so that you get a feel for the flow.

It is much easier than you are thinking.

The tabulator documentation is very much less than helpful because it assumes that you already know a lot.

Please see my response on this question about tabulator, Vue and uibuilder. I give a worked example that you can try for yourself.

1 Like
11

Damn, I just noticed you're actually the big brain behind uibuilder :astonished:
Sorry I bombarded you with my stupid questions here :grimacing:

So to be clear, basically uibuilder has the web socket thing build in. Meaning if I send a Node-RED message to the uibuilder node it is somehow sending this msg to the front-end via a web socket connection behind the curtains? That's great!
If this is the case, then I did misunderstand you earlier in regard to the web sockets.
I though that you have to use some separate web socket nodes to send and receive.

As of how to use the uibulder.onchange('msg', function(msg) { .... }) function...
Is that roughly what you meant?

// If we receive a control message from Node-RED, we can get the new data here - we pass it to a Vue variable
uibuilder.onChange('ctrlMsg', function(msg){
    //console.info('[indexjs:uibuilder.onChange:ctrlMsg] CONTROL msg received from Node-RED server:', msg)
    app.msgCtrl = msg
    app.msgsControl = uibuilder.get('msgsCtrl')

    //table data
	let tabledata = msg
})

If I need to separate my data based on the topic I would need to parse the msg.topic in there and differentiate based on that?

Sorry if this is a stupid question, but am I able to use the dot notation here to get to the sub properties of the message received? Or do I have to parse it somehow beforehand?

12

No problem.

Yes. It uses the Socket.IO library to manage the connections. So strictly, it is capable of more than just websockets - but in short, yes. It also provides the connection as an isolated one, each uibuilder node has a separate, isolated channel. So you can have multiple uibuilder nodes running independently of each other if you want to.

Yep.

Yep. :slight_smile:

Not stupid at all.

You get the data at the front end exactly as you would in another Node-RED node - as a proper JavaScript object.

That was one of the design goals of uibuilder. That it should be as easy to use as handling messages in a Node-RED flow. That is also why you can send data back to Node-RED from your front end simply by using the node.send() function just as you can in a Node-RED function node.

uibuilder lets you treat your front end as much as is feasible like Node-RED itself.

Seriously though, try the example flows out and you will see exactly how it works. Also check out the WIKI which has loads more information. You can also look at the technical documentation - that is now accessible from a button in the node.

13

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.