Uibuilder: to subscribe or not to subscribe?

Hi,

I will create a separate thread for the discussion

Chris

1 Like

On seconds thoughts -given the amount of content already in this thread, will keep this open. Will create new threads for side topics. @tve apologies for hi jacking your thread, but the discussion is absolutely consistent with your thread. Let me know if you want us to stop using this

In the place where I do all my best thinking, I just realised the while creating a new dashboard from existing flows is I am hoping a sensible problem to solve, I just realised without changes to the UI nodes we will not be able to provide a seamless experience to user.

A mechanism to do this, and I don't know the UI nodes at all, would be for the UI nodes to have an option to use UIBuilder (as the primary or secondary UI). This would require a change to the UI nodes to provide a mechanism to optionally add a transparent pass through to the UIBuilder node. So the text node for example would have an option in config to provide info to UI Builder. This would enable a output on the text node which could be connected to UIBuilder. Alternatively there could be s UI Builder server node config which could be configured on the UI Nodes that would handle the communication.

@knolleary, while this is still obviously in early thought, is a PR to the UI nodes to add an additional path a sensible approach ? If not then the concept of using existing dashboard infrastructure to extend UI Builder falls a bit flat

@dceejay Apologies, I understand you maintain the dashboard nodes. @knolleary sorry for the spam

The problem here is that I think that Dashboard works in a totally different way to uibuilder. AFAIK, Dashboard nodes deliver the code to the front-end whereas uibuilder uses standard files for its components.

We really don't want to make Node-RED compose and send code over websockets and then send data, that isn't terribly efficient and isn't very scalable or cacheable. The components should be parameterised bits of display code that only need standard data sending to them and nothing more. The display code can be in files and delivered using standard web server methods.

Check out the example VueJS components in this repo for an example of what I mean. These have been designed so that you can simply send pre-formatted messages from Node-RED to make use of them, all you need to do in the front end is include the library and add a blank element to your HTML.

We are in agreement. I absolutely agree that we do not want to send code via ui builder. My point and probably poorly made, is a change to the UI nodes to transparently pass any data it receives to UIBuilder as well as doing its current dashboard functions (sending code to the dashboard)

This would enable the two dashboards to run in parallel

Chris

2 Likes

Had a fun night last night (clearly I am not a movie maker nd have no intention to be)

Created a trivial flow which would extract the dashboard components via the admin api (limited to 1 tab, 1 group and ui_text and ui_button) , does some basic source code generation for dart ,saves the file to the flutter library and then reload using flutters awesome hot reload feature

The flutter UI clearly needs work and this is nothing more than a very first step to understand what sort of work flutter UI creation from a dashboard config would be. My view is while there Is non trivial work it is feasible especially for the core nodes.

My next step in proving the technology will be to hook it up to a naive WS connection to allow 2 way comms between the flutter app and node-red.

2 Likes

Interesting indeed.

Have you tried the integration between Flutter and uibuilder? If that works OK then you may be able to remove the manual steps and get the ws connection via uibuilder.

Hi,

so a little bit of progress. Have got a WS connection between NR and the Flutter app. It will refresh the flutter app on deploy. The next step over the weekend (family permitting) will be to get the data flow going (currently only control flow for sending updated flows) and adhoc data back from button presses. The next step is a bit of SMOP (simple matter of programming) so am not expecting much hassle.

BTW the cool thing about flutter is getting the native app for nothing. I will try and release the flutter app to GitHub over the next week or so.

The flow that does the extract from Node Red

[{"id":"d2ad6df4.275668","type":"http request","z":"6874eab9.ded86c","name":"flows","method":"GET","ret":"obj","paytoqs":"ignore","url":"http://localhost:1880/flows","tls":"","persist":false,"proxy":"","authType":"","x":550,"y":180,"wires":[["94635674.322ce"]]},{"id":"94635674.322ce","type":"function","z":"6874eab9.ded86c","name":"Filter and reorg UI","func":"let flows = msg.payload\nlet tabs = flows.filter(elm => elm.type == \"ui_tab\" )\nlet uigroups = flows.filter(elm => elm.type.substring(0,8) == \"ui_group\" )\n\nlet uiTabs = []\n\nfor (let i =0; i < tabs.length; i++){\n    tab = tabs[i]\n    let elms = uigroups.filter(elm => elm.tab == tab.id)\n    elms = elms.sort((a, b) => a.order - b.order)\n    console.log(`elms ${elms.length}`)\n    let groups = []\n   \n    for (let j = 0; j < elms.length; j++){\n        group = elms[j];\n        let groupElms = flows.filter(elm => elm.group == group.id)\n        let cleansedElms = []\n      //  cleansedElms.push(groupElms);\n        for (let k =0; k < groupElms.length; k++){\n            elm = groupElms[k]\n            delete elm.wires\n            if (elm.type ==  \"ui_button\" || elm.type == \"ui_text\") cleansedElms.push(elm)\n        }\n        cleansedElms = cleansedElms.sort((a, b) => a.order - b.order)\n        groups.push({id : group.id, group_details : group , element : cleansedElms})\n    }\n    uiTabs.push({id : tab.id, tab_details: tab, groups : groups})\n}\n\n\nlet control = {\n    type : \"control\",\n    scope : \"tab_build\",\n    tabs : uiTabs\n    \n} \nmsg.payload = control\n\nmsg.control = control\n\n\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":730,"y":180,"wires":[["86a2cebe.401fe8","a5bf0d6d.6cefe"]]},{"id":"9983e29b.6844e8","type":"websocket out","z":"6874eab9.ded86c","name":"","server":"706934a9.4651ac","client":"","x":740,"y":240,"wires":[]},{"id":"86a2cebe.401fe8","type":"json","z":"6874eab9.ded86c","name":"","property":"payload","action":"","pretty":true,"x":570,"y":240,"wires":[["9983e29b.6844e8"]]},{"id":"fdaf4382.019728","type":"inject","z":"6874eab9.ded86c","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":390,"y":240,"wires":[["d2ad6df4.275668"]]},{"id":"12dfd1d3.616c2e","type":"status","z":"6874eab9.ded86c","name":"","scope":["8c963eb6.05be18","29b60332.e886a4"],"x":180,"y":180,"wires":[["588e1b39.ca3d24"]]},{"id":"588e1b39.ca3d24","type":"change","z":"6874eab9.ded86c","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"{\"type\":\"Control\",\"scope\":\"flows\"}","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":380,"y":180,"wires":[["d2ad6df4.275668"]]},{"id":"a5bf0d6d.6cefe","type":"debug","z":"6874eab9.ded86c","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":680,"y":120,"wires":[]},{"id":"1fb5cdef.38c5f2","type":"websocket in","z":"6874eab9.ded86c","name":"","server":"706934a9.4651ac","client":"","x":240,"y":120,"wires":[["d30b5acb.34de58"]]},{"id":"d30b5acb.34de58","type":"debug","z":"6874eab9.ded86c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":450,"y":120,"wires":[]},{"id":"706934a9.4651ac","type":"websocket-listener","z":"6874eab9.ded86c","path":"/ws/simple","wholemsg":"false"}]

1 Like

So have very nearly got all I need for proof of technology for a flutter drop in for an alternative for the dashboard (particularly core nodes)

The demo can

  1. Generate UI from flow data on deploy
  2. Receive data from Node-Red,
  3. Send data back to Node-red.
  4. Crude re use of existing UI nodes for definition and data in
  5. Run in web, iOS, macOS (yep it just worked) android have not tested android but should just work also. When I publish to Github later next week, I suggest someone builds a window version. I don't have a Linux desktop so won't build for that.

It all seems pretty sensible and from what I can see no reason why it can not be stable. I am not suggesting that this a replacement path for the dashboard but just a concept of generatable UI from the existing UIs.

Before I publish the code I will add support for data-intensive widgets like graphs, charts etc. I cant see a problem with it it is just slightly tricky work to get the architecture right. I think I would like to put some side caching in place for that but will think it through a bit before I hack it together

Regardless, the next steps are pretty clear to me. I have a pretty good view of what I would like to see in a set of interface contracts (old term but still works) between the existing node-red UI nodes - a UIbuilder NR interface and the front end uibuilder interface. I know @TotallyInformation would probably prefer that we would just use the uibuilder-fs.js however at least in the flutter case this is not possible. If flutter is truly the exception I am ok with this just being documented in code but I still feel it is worth the intellectual effort to document it

If people are keen I would like to jump on a call to talk about scoping the document and then look at a divide and conquer approach @dceejay I would think that this is a useful step regardless of technology for the next steps with the dashboard.

Over to you guys. I will keep hacking the POT a bit to add the data-intensive widgets and tidy up a bit by NR interface -Quite possibly I might end up deploying the flutter stuff for some internal dashboards at work as multi-user should be relatively straight forward. (and I could justify some work hours on it)

BTW if there are any flutter people out there who would like to help (i was left without a FE design bone), would love to hear from you

2 Likes

Hello @chrisn-au,
That looks very promising. Please keep up the good work!

I'm not really experienced with frontend frameworks, so I was wondering what you guys mean with not sending code to the client. Is that only related to sending live code via websockets, or more general? Some examples:

  • Does this mean it is not possible anymore to write custom UI/Javascript/CSS in a Template node? Hopefully this is not a very stupid question ...
  • My UI nodes contain a large amount of client side code, quite heavily depending on the AngularJs scope variables. Is there an easy way for me to adapt my ui nodes in the near future to run your dashboard, or do I need to start again from scratch?

About custom UI nodes, I have another question. The current dashboard has a lot of parameters:

ui.addWidget({
                    emitOnlyNewValues: false,
                    forwardInputMessages: false,
                    storeFrontEndInputAsState: false,
                    persistantFrontEndValue: false,

Will those still exist? I assume not. Just asking because I have spend a LOT of free time in the past to optimize those settings, and hope not having to go through that hell again if I ever have to migrate my UI nodes ....

Is that a web or native app? If the latter, do you think we could make use of functionality like push notifications, getting geographic location ... like other native apps?

Bart,

great questions

I have zero experience with the

The flutter approach would NOT support this. Not to say there are not alternatives but you will not be able to send JS/UI/CSS in a template node in the flutter path. This may be a reason why the approach dies. I would expect a more HTML approach may still be able to support this but I am also out of my depth

Migration of existing Dashboards will need to be a consideration. I have no answers now though

This is an interface we need to get documented regardless of approach, It needs to be consistent and logical (not saying the current approach isn't - but the documentation does need to be)

In the flutter app approach, it will be native IOS Android, MacOs Windows, Linux, Web. It will support notifications, geo stuff etc. I suspect better than some of the Web technologies today

From my point of view, if the flutter code-gen approach has a role, it will be for the user who just wants to use core widgets out of the box and accept most layout restrictions or the POWER user who wants to write their own front end. The segment in the middle may be underserved and hence the whole approach may not be feasible

1 Like

@BartButenaers Can you send me a simple ui_template example (a hello world ideally) and possibly one little more complicated

Thanks

Oh, that is a pity :slightly_frowning_face:. Because that is being a LOT nowadays ...

Temporarily I don't work with Node-RED, but you can find a lot of examples (like e.g. here).
Thanks for at least having a look at it!!!!

Looking forward to working on this.

It is more that I am concerned we might be swapping one fairly closed case (the complex Angular v1 of Dashboard) for another fairly closed case (using a Google language and toolset). Rather that opting for a truly open case.

However, as always, what is important is that we agree on the interface. That way, everyone has the opportunity to work with it and produce innovative solutions.

Already said that I'm happy to do that.

What I've always meant is that you should not be reliant on the back-end sending code over dynamic interfaces (e.g. websockets). This simply isn't efficient and tends to prevent the standard efficiencies of http from being used.

Most of the code is static and therefore doesn't need to be sent over a dynamic interface but instead can be cached using web services working with the browser.

However, as in uibuilder's case, that does not prevent you sending code dynamically. You just shouldn't have to do it as standard.

This is easily converted to static code. But the scope variables would need to be converted in accordance with whatever front-end framework is in use.

And this is my worry with the Flutter approach - how many people know DART? Speaking for myself, there is no way that I have the time or energy to learn a new language just to deal with this case. Not after spending so many years working with JavaScript. Which I chose because I can work on both front- and back-end code in the same language and have it run on pretty much every platform.

If we can separate out the Dashboard extract code from the Flutter front-end, then it should be fairly straight-forward to integrate to different front-end libraries.

I was especially pleased to see how easy it was to work with Svelte for example (though it requires a build step so possibly not the best choice as a default for uibuilder, still it "just worked"). While I am certainly no Svelte expert, the concepts and the code were really easy to work with and result in a pure JavaScript output. The whole thing being efficient and fast. So I could see that being a good framework to have as an alternative whether or not uibuilder was used as an integration "partner". I think that VueJS would also be amenable but then I expect REACT would as well.

Totally agree here. This is the crux of any next-generation Dashboard for Node-RED. A standardised interface.

1 Like

Yes that is unfortunately also the case for me. One of the advantages of Node-RED for me is that I can use the same programming language in flow editor, backend and dashboard at this moment...

1 Like

I just looked at Svelte :slight_smile: seriously, I concur with the new lang/env hurdle (and just for clarity you will find dart quite javascript like but that is not the point)

Looking at SVELTE it looks interesting. Still getting my head around it. I might try to generate my example dashboards from the flutter case in Svelte. Possibly a good way to learn.

Would the Svelte intention be that for a zero-config dashboard replacement, that we would generate the dashboard on deploy and then compile it? I assume the compile for small dashboards would be quick - The compile step could be orchestrated so while not an ideal soln not a show stopper from my perspective

Some time ago there was a very nice discussion about a building a new dashboard. That discussion was initiated by @andrei-tatar, the author of the current AngularJs dashboard. Unfortunately the ideas from that duscussion have never resulted in a new dashboard. But I see that Andrei has mentioned also Svelte in that same discussion...

Another noob question: I assume we can never stick to a single language (Javascript) for all our tools (flow editor, backend, dashboard...) when we also want to have a native app for both Android and iOS?

Personally, I would rather generate the components or maybe a collection of common components and send configuration to produce the display. So that you don't need to compile just to add a component to the display.

Well that is the $64,000 question. Again, personally, I would prefer to TRY to stick to HTML/CSS/JavaScript for all UI's. This is the standard I've set for the organisations I am the head of architecture for.

It is true that you can't reach 100% on these things but I would rather people focus on the set of international standards that cover the largest base.

2 Likes

Can I have your entire flow please?

I'm finally finding some time to circle back to this. I've been doing some tests with uibuilder + vue + vuetify to get a feeling for what I can get out of the box vs. have to build. I'm not a UI whizz so I periodically need a refresher :slight_smile:

I'm trying to grok the various discussions about a new dashboard. It seems to me that the top dividing line is around how the content of the dashboard is defined. In the current dashboard the content of what you see is defined in node-red using node-red nodes. The alternative is what uibuilder steers towards (while remaining flexible) is to define what you see in html/js, the way "normal" web UIs are done.

There are clear advantages to both approaches and I'm really happy that uibuilder enables the second option. I'm now also understanding a bit of the difference between a component in vuetify (or bootstrap-vue for that matter) and a component in a dashboard attached to node-red. Vuetify components (and web components in general) are configured and driven using a combination of html tags and data values. Adding node-red to the mix introduces messages, so the question is how should messages reach components and what should they be able to do to them?

In the context of uibuilder, or more precisely, in the context of a dashboard whose content and layout are defined in html/js I don't see the point of using messages to drive the looks of components. For example, I would argue that while messages obviously need to set the value of a gauge, I have no need to set the diameter of the gauge via a message from node-red because I already decided that I'm doing layout using html/js/css.

So looking at uibuilder-vuejs-component-extras/gauge at master · TotallyInformation/uibuilder-vuejs-component-extras · GitHub I would say that there's too much functionality there! :wink: A gauge should only have messages to set the value, maybe also to set min/max values, but that's it. The rest should come from the html/js/css. I believe that less is more, simpler is better, stay focused, etc. (Someone who really wants to double the size of a gauge if a certain threshold is reached can always write some javascript to make that happen.)

Something all this has clarified for me is that there really is a need for two different types of dashboards: one fully controlled through node-red nodes and one that uses std html/js/css to control the content of the page and only exchanges data with node-red.


As I think some more about the original "to subscribe or not to subscribe" topic I want to pick some more on uibuilder-vuejs-component-extras and specifically the gauge example. I don't think I want that! :slight_smile: (Please take all this as a discussion, as brainstorming, not as a value judgement! If you come to the opposite conclusion then we both win :slight_smile: , 'cause clarity is my goal, not imposing one approach.)

The reason I don't want the gauge in that repo is that I think it's beneficial to separate web components and widgets that talk to node red. Lemme explain. For dashboards one commonly uses card UIs, or at least UIs that have multiple rectangular containers on the page. Within each card one tends to find multiple web components, for example to display a couple of values and some input buttons/controls. At that point I think it's really beneficial to use the card abstraction to separate how the card content is constructed using the web components at hand from the messages that are exchanged with node-red. I.e., I'd like messages from node-red to reach a small snippet of code attached to the card and from there to plug the data into the right web components.

In general what I described isn't exactly what I want 'cause of nesting. Sometimes the visual card on the page should receive messages but sometimes the visual card contains a few sub-containers that should receive the messages. But that's a detail.

Continuing to explore...

1 Like