Uibuilder: to subscribe or not to subscribe?

This is something that could easily be added to uibuilderfe. A mapping from an incoming topic to a browser variable (a Vue variable in this case but that is irrelevant). That alone would eliminate the need for much of the Node-RED logic since everything could simply be sent straight to the front-end and anything not relevant would simply be ignored, anything relevant would automatically update the variables and hence the UI.

That could also be used to trigger the initial layout as well. On initial startup, a new client could be sent both the layout configuration and the initial data settings.

That is really how uibuilder works anyway. But at the moment, you are responsible for mapping the incoming msg's to UI components. Auto-mapping would be one less thing that people needed to do and would nicely simplify the UI javascript particularly when combined with Vue's generic v-bind.

The @click of course already has a helper that lets you assign a click method directly to a uibuilder method so that it becomes a 1-liner.

Can you elaborate? If I have 10 gauges in the UI I still need to specify which piece of data each gauge is supposed to show, right?

Ah, I had forgotten about eventSend, yup, something like that is what's needed.

Having formulated the loose coupling between node-red and the UI, the next step kind'a falls right out: a self-configuring dashboard UI!

In the current dashboard the configuration happens on the node-red side: there are fields in each UI node and a dashboard designer side bar to configure everything. Now that node-red no longer needs to know about the UI it frees the UI to configure itself.

The idea is that the content area of the UI consists of a grid (duh) and that grid has a little +add button tucked into a corner. Press that button and a modal comes up with a drop-down selector of components and you can choose which type you want to add. Once you choose the component type the modal expands with the UI options offered by that component, one field being the name of the state variable to bind to the component. Hit submit and the component is added to the grid. Under the hood the grid is bound to a state variable just like a gauge or switch would be and that state variable contains the configuration for all the components in the grid. This way the configuration state of the grid is persisted via node-red.

Of course a bit more is needed beyond just an +add button 'cause one has to be able to reorder components, delete them, and edit their configuration. Also, the whole page shouldn't be just a grid: it would be nice to be able to pick components for top-nav, left sidebar, bottom-bar, and in the main area one could have multiple grids one after the other that each lay components out differently, including full-custom grids.

The bottom line of all this is that a UI designer as a separate application or as a sidebar in node-red is not needed. The UI frame can be written to include its own designer functionality, which incidentally can be totally live, i.e., while the UI and all the components are running and showing live data. This is turning out a lot more interesting than I had anticipated!

NB: the existing NR UI components could be ported to this model. It would take a grid that lays components out the way the current UI does and instead of live editing the config would be prepared by the NR UI nodes.

Sorry, I'm more thinking about the little guy rather than your use-case. Just something to make it even easier to link data from Node-RED to the UI components without needing code:

uibuilder.automap(this, [
    {var: 'var1', topic: 'sending_var1'},
    {var: 'var2', topic: 'sending_var2'},
    {var: 'var3', topic: 'sending_var3'},
])

Where this would be your Vue app reference - or could be window for example if you wanted to use global vars (e.g. not using a framework). The array maps a msg to a variable via the msg.topic. In this case, you would define var1 etc in your Vue data section. If a msg is sent from Node-RED with a topic of 'sending_var1', the uibuilderfe library would automatically apply the msg.payload to the appropriate variable.

Not quite decided whether this is really enough of a benefit to be worth while. But it kind of beats having to do an onChange function with a select/case in it.

1 Like

The saga continues, now with a proof-of-concept dashboard designer UI. Here's a UI with a tab that has a grid with zero components and an add button in the upper-right:

Clicking on the add button brings up a modal with a drop-down list of available components (the drop-down looks awkward in this screen shot without cursor or highlighting):

Selecting a component expands the modal to show the properties that can be defined. Each property can be set to a literal string or it can be bound to a reactive state variable using a drop-down of existing state variables:


Hitting the add button adds the component to the grid, and it's live (I added two components):

Reloading the page and keeping the added components works too: the component config is persisted in node-red.

Baby steps, but this looks promising to me! :horse_racing:

2 Likes

Hi Thorsten,
That looks very nice!
I have to admit that this discussion goes a bit too much into frontend framework detail for me, so would be nice if you could answer my noob question below.
I have develped a series of custom UI nodes in the past for the current AngularJs dashboard. Will it also be possible to develop such nodes in your dashboard concept? I mean UI nodes that can be installed separately from the palette, and dragged into the flow. Like the current dashboard allows us to do at the moment.
Or is that not the purpose of your setup?
Thanks for the clarification!
Bart

Yes, "anything is possible" :slight_smile: That being said, unless others get excited and chip in, what I'm doing is unlikely to reach "production quality".

In a bit more detail, the FlexDash expects that each widget is a web component that has a certain interface (details still being worked out). So you can author additional web components. That is similar in effort to authoring new node-red nodes but different.

It may be possible to create a widget similar in spirit to the uitemplate node where the user can compose wimple web components without having to install tooling. That would be useful, for example, to create custom arrangements of buttons, indicator lights switches, drop-downs, text fields, etc.

Note that with FlexDash you really don't drag UI components into the node-red flow. Instead you send messages with a topic and payload to a UI state node. Then, in the dashboard itself, you drag a widget onto your grid and you attach it to the same topic.

1 Like

One thing this discussion has prompted me to do is to start refactoring some of the uibuilder core js.

In particular, I was reminded about the Singleton mode for node.js modules and classes. And so I've started moving all of the Socket.IO logic into a separate module.

The interesting side-effect of this will be that we can reference this singleton module from other modules - including other nodes. Which immediately makes it possible to have more nodes in the uibuilder "universe" using the same Socket.IO channels.

It is a fair chunk of work so won't happen overnight - v3.3.0 will be released shortly with some fixes and a new feature that lets you send a msg that will reload the browser page. Phase 1 of a proper development server capability for uibuilder right inside Node-RED.

The refactoring of the Socket.IO functions will let me send a message to the front-end from more places in the code, including the API's which are currently defined outside the instance and therefore do not currently have access to Socket.IO and therefore cannot send a message to the front-end.

That will mean that I can enable a flag in the uibuilder file editor panel that will trigger all connected clients of a uibuilder instance to reload when a file is saved.

A later improvement will be to add the option of using chokidar to watch the instances folders for changes and automatically trigger a reload.

Doing this refactoring should also give me the knowledge to do similar work on other areas of uibuilder so that we can make it even more flexible.

For me it would be great if there was a core module that only handled the basic socket.io connection without all the helpers to inspect or auto-handle stuff...

As long the helpers don't create any functionality the bare core needing user needs to suppress, I cant see any problems. The thing is - the users are at wide range of knowledge base so all meaningful helpers are very welcome thus must be also respected by any user. The thing lives in world of openness. So strict rules can't always to be applied.

For all the reasons previously discussed, this is not possible. It would work for you but not for anyone else. It needs to work for everyone.

Correct indeed.

As previously said, there is a lot of refactoring that can be done in uibuilder to make it better, faster, etc. But it won't be losing any features - at least right now.

Once refactoring has progressed, it may be possible to do more and with the work I'm doing on the Socket.IO part right now, it may well be possible to have a separate node that only provides Socket.IO. But there is so much more to do in other areas, I can't see it being top priority for a while.

Exposing a core module that just manages the socket.io connection doesn't mean there can't be a separate module that wraps a ton of conveniences on top. It's just more modular. For example, there couple be vue-focused wrapper and a separate react-focused wrapper.

See the final para of that last message.

Sorry for hijacking this thread, but when I designed GitHub - cinhcet/node-red-contrib-component-dashboard: Flexible dashboard for Node-RED based on web-components I have thought about a lot of the things you are discussing here.

  • Dashboard widgets are web-components
  • Each dashboard widget has its own node
  • There is a mechanism that enables to define the appearance etc. of the widget from Node-RED (I am not using it, but it is there)
  • There is a mechanism to create new components from existing ones (sub-components).
  • You can indeed create a "meta" widget that defines a grid where you name the widgets that should be placed there. Then everything could be configurable from Node-RED, no coding needed. That would be a lot of work, but possible. I am not interested in doing that, however, since for me it is faster to just write the html code.
  • Widgets can be published as NPM modules (web-component and node)
  • Buffering, replay etc. is handled for you like in the old dashboard
  • No dependency on any web-framework

Just for your info....

1 Like

Haha! This is the hijack thread :laughing:

I did look at your repo, but at the time I didn't want to build any of this stuff... And also, it's bare JavaScript, which I'm not ready for. I need more hand-holding... :crazy_face: But it does look nice!

It's actually only once I formulated the stuff I did in the last 5-6 posts that I decided to try and build a dashboard...

That crazy dashboard of mine keeps evolving :crazy_face: a big refactoring brought the code size down and it's all becoming much nicer and cleaner. I spent quite some time improving the editing and adding a few widgets:

Editing is now live, no longer a modal. Any changes happen immediately and the rest of the dashboard remain live too. Widgets can now be resized and moved, although the UI for that needs improvement (I just don't want to dive into drag and drop right now):

The way the editing of a widget works is that each of the component's properties can be bound to either a literal string, a literal number, or a data value coming in from node-red. The output of switches, buttons, etc can similarly be bound to a data value that goes back to node-red.

So far it all continues to look very promising but every single piece needs work! Except for some trouble I'm still having with the reactivity in Vue it's actually becoming functional :slight_smile:

The widgets themselves are surprisingly simple. They're basically just a remapping of props with a little bit of logic around the vuetify (or other) base components. The sparkline, for example, is less than 60 lines of (quite sparse) code.

Another interesting development is that the entire thing is completely independent of node-red. It could be used for anything that can send it data over the websocket. Could even be served up and fed by an esp32...

4 Likes

This is pretty much what I was expecting to achieve with Vue and uibuilder where uibuilder is just acting as the conduit. Vue does most of the heavy lifting for the front-end design. Node-RED provides the data via the uibuilder channel.

2 Likes

Time for a demo :yum: If you have a moment, head over to FlexDash and play. To edit stuff you need to turn on editing mode in the gear menu at the top-right. This demo runs stand-alone with random data generated internally, there is no server connection (and edits are not saved across browser refreshes). I did start a readme with some high level architecture info and I've tried to be generous with comments in the source. Feedback appreciated.

5 Likes

Looking seriously good

1 Like

I know you like uPlot but have you considered using pivottable library at GitHub - nicolaskruchten/pivottable: Open-source Javascript Pivot Table (aka Pivot Grid, Pivot Chart, Cross-Tab) implementation with drag'n'drop. ?

In my opinion it is more suitable for monitoring sensor data from network of devices with very little coding.It include tables and graphs in same package. And unlike other graphing software it is easy to render table or graph with drag and drop selectors. It could be standard widget for FlexDash so user do not need any other separate widget for graphor table. I use following code for my purpose:

<script type="text/javascript">
    // This example loads sensor data from a ESP32 gateway which collect data from remote sensor devices and store it in a CSV file.
  

    $(function(){
	    
        var renderers = $.extend(
                    $.pivotUtilities.renderers,
                    $.pivotUtilities.plotly_renderers,
                    $.pivotUtilities.d3_renderers,
                    $.pivotUtilities.export_renderers
                    );
       
        Papa.parse("http://192.168.0.3/sensors.csv", {
            download: true,
            skipEmptyLines: true,
            
            complete: function(parsed){
            for (let i = 0; i < parsed.data.length; i ++) {
            
            var d = new Date(parsed.data[i][0]*1000);
            parsed.data[i][0] = (("00" + d.getFullYear()).slice(-2) +  ("00" + (d.getMonth() + 1)).slice(-2) +  ("00" + 
            d.getDate()).slice(-2) +  ("00" + d.getHours()).slice(-2) +  ("00" + d.getSeconds()).slice(-2));
            parsed.data[0][0] = "Timestamp";
					  
					  if (parsed.data[i][1] == 6) {
					    parsed.data[i][1] = "Livingroom";
					  } else if (parsed.data[i][1] == 16) {
					    parsed.data[i][1] = "Kitchen";
					  } else if (parsed.data[i][1] == 26) {
					    parsed.data[i][1] = "Bedroom1";
					  } else if (parsed.data[i][1] == 36) {
					    parsed.data[i][1] = "Bedroom2";
					  } else if (parsed.data[i][1] == 46) {
					    parsed.data[i][1] = "Bedroom3";
					  } else if (parsed.data[i][1] == 56) {
					    parsed.data[i][1] = "Bedroom4";
					  } else if (parsed.data[i][1] == 66) {
					    parsed.data[i][1] = "Bathroom1";
					  } else if (parsed.data[i][1] == 76) {
					    parsed.data[i][1] = "Bathroom2";
					  } else if (parsed.data[i][1] == 86) {
					    parsed.data[i][1] = "Bathroom3";
					  } else if (parsed.data[i][1] == 96) {
					    parsed.data[i][1] = "Bathroom4";
					  } else if (parsed.data[i][1] == 106) {
					    parsed.data[i][1] = "Laundry";
					  } else if (parsed.data[i][1] == 116) {
					    parsed.data[i][1] = "Boiler Room";
					  } else if (parsed.data[i][1] == 126) {
					    parsed.data[i][1] = "Workshop";
					  } else if (parsed.data[i][1] == 136) {
					    parsed.data[i][1] = "Garage";
					  } else if (parsed.data[i][1] == 146) {
					    parsed.data[i][1] = "Office";
					  } else if (parsed.data[i][1] == 156) {
					    parsed.data[i][1] = "Tank";
					  } else if (parsed.data[i][1] == 166) {
					    parsed.data[i][1] = "Solar";
					  } else {
					    parsed.data[i][1] = "Unknown";
					    parsed.data[0][1] = "Location";
					  }  
					  
					//console.log(parsed.data[i][1]); // (13)arrays of values by sensor type
				}                
                
          $("#output").pivotUI(parsed.data, {
                 
					rows: ["Location"], cols: ["Timestamp"],
					aggregatorName: "List Unique Values",
                    vals: ["Temperature"],
					rendererName: "Line Chart",					
                });
            }
        });
         
        var dragging = function(evt) {
                    evt.stopPropagation();
                    evt.preventDefault();
                    evt.originalEvent.dataTransfer.dropEffect = 'copy';
                    $("body").removeClass("whiteborder").addClass("greyborder");
                };

                var endDrag = function(evt) {
                    evt.stopPropagation();
                    evt.preventDefault();
                    evt.originalEvent.dataTransfer.dropEffect = 'copy';
                    $("body").removeClass("greyborder").addClass("whiteborder");
                };

                var dropped = function(evt) {
                    evt.stopPropagation();
                    evt.preventDefault();
                    $("body").removeClass("greyborder").addClass("whiteborder");
                    parseAndPivot(evt.originalEvent.dataTransfer.files[0]);
                };

                $("html")
                    .on("dragover", dragging)
                    .on("dragend", endDrag)
                    .on("dragexit", endDrag)
                    .on("dragleave", endDrag)
                    .on("drop", dropped);
        
     });

</script>

Content of sensors.csv file is as follows:

Timestamp,Location,Voltage,RSSI,Temperature,Humidity,Pressure,Light,Open/Close,Level,Presence,Motion,Custom
1621906615,26,6,-37,51,78,232,86,0,0,0,0,0
1621906647,6,6,-37,49,44,225,55,0,0,0,0,0
1621906678,6,146,-37,46,45,228,72,0,0,0,0,0
1621906691,26,6,-36,46,45,228,72,0,0,0,0,0
1621906723,6,146,-39,54,88,250,66,0,0,0,0,0

Dependencies are as follows:

Thanks.