FlexDash: towards a new dashboard for Node-Red

Over the past few weeks I've been working on a new dashboard. Initially my goal was to throw something together I could use to replace my use of the standard NR dashboard because I've outgrown it. But as I dove into it, it became a fun project and so I'm ending up building far more than I initially expected. In part, I also realized that if I want to use this for the next decade (I've been using NR for 5-6 years by now) I better build it so its use doesn't require much javascript, html, or css knowledge, because I'm bound to forget it all again over that time frame :slight_smile:

Currently there's a demo available at FlexDash which runs entirely in the browser (no node-red connection) and displays random data. This demo has all the code to connect to NR but for now it's easier to work with generated data. Keep in mind that this is very preliminary.

I thought I'd start this thread as a way for me to record design decisions and perhaps to get some input. In the end, what I'm trying to end up with is three-fold: a new dashboard for myself, a dashboard that is not tied to node-red so I can use it elsewhere too, and a dashboard that others can use and contribute to as well. As you can see, I'm quite selfish :innocent:

Before diving in I want to acknowledge that nothing here is new. It's all existing ideas rehashed, I'm just documenting which ideas I decided to borrow. :yum:

Self-contained Dashboard

The most important design choice I made is that FlexDash is self-contained by which I mean that the entire life-cycle, from creating, configuring, to operating happens in the FlexDash code in the browser. A server (or multiple servers) primarily just feed data to be displayed and receive user input events in return when the user flips a switch or moves a slider. A server is also needed to save (i.e. persist) the configuration of the dashboard, but in principle this could also be done by saving the config in local (browser, etc) storage.

This design is in contrast to the current dashboard where the configuration is edited in Node-Red and, once saved, is pushed to the browsers displaying the dashboard. In that model there are two somewhat distinct pieces of code: the code to display the dashboard and the pieces of code attached to NR UI nodes to configure how these nodes are to be shown.

The reason for the new architecture is two-fold. First it makes the dashboard more independent of NR, which has benefits such as being able to display data coming from other sources like MQTT or directly from a microcontroller. Second it makes for a more interactive and responsive user experience in that the user can manipulate the UI components directly on-screen and immediately see what the effect of a change is.

An obvious downside of this architecture is that editing no longer happens entirely in NR and, perhaps more significantly, that the dashboard is conceptually more removed from NR than with the current model. Specifically, the user no longer places specific UI elements, such as switches, charts, or gauges into the NR flows. How data in NR is linked to the dashboard leads to the second most important design decision, which is how data flows to the dashboard.

As an aside, the design doesn't entirely preclude having specific UI nodes in NR, it just makes it unnecessary and perhaps more cumbersome than not having them. Ultimately this is probably mostly a matter of personal preference. :roll_eyes:

Data flow via a topic hierarchy

Data produced in Node-Red is bound to UI components displaying that data using a topic tree very similar to MQTT. Many Node-Red messages already have a msg.topic so this notion should feel familiar to NR users. The basic mechanism is that a NR flow sends a message with a topic and a payload to a generic dashboard node and a corresponding UI component displays the payload value by being "bound" to the same topic (FlexDash uses the term binding inherited from the reactive data binding in Vue but one could use subscription just as well). An example might be a temperature that the flow sends to msg.topic = "home/livingrm/ceiling" and that a temperature UI element visualizes. :thermometer:

A major benefit of using a topic-based binding is that it becomes possible to vary the number of UI components based on the actual data. For example, a flow could produce values of the form home/temperatures/<room name> and a special array UI component could be configured in the dashboard to instantiate as many thermometer UI components as there are values under home/temperatures.

The dashboard design is such that for each input to a UI component the user can choose whether to assign a literal value or whether to bind it to a topic. This means that any input exposed by the UI component can be controlled from NR. :rocket: An example might be to bind both the title and value of a thermometer to topics so one can rotate which temperature is shown over time. Some of this is also possible in the current NR dashboard, but only where explicitly programmed by the UI component developer.

All this leads in to the implementation architecture...

Implementation architecture

FlexDash is written using the Vue2 web design framework together with the Vuetify component library. For a highly interactive web app the "reactive" frameworks seem to be the way to go and Vue is both widely used and very approachable. The latter is important so users can develop their own UI widgets easily. Vuetify works well with Vue and has a large selection of components that are used to create the dashboard's UI (toolbars, buttons, menus, etc). Bootstrap-vue would probably have been a good choice too. Instead of Vue other frameworks could also have worked. I briefly looked at React and Svelte and wasn't convinced that they would be better for this project but I can't claim any authority on that point.

In the future, it would be great to port FlexDash to Vue3 because that solves some reactivity issues that I've bumped into and that lead me to look whether the grass is greener on the React or Svelte fronts. :smirk: More importantly Vue3 improves the way components can be written (the "Composition API"), but currently Vue3 is not supported by Vuetify so this has to wait at least a few months.

The design is entirely based on web components as implemented by Vue. There are four levels to the user interface:

  • the dashboard as a whole ("the page" or "the app")
  • a number of tabs, represented in a conventional manner across the top of the page
  • one of multiple grids filling the content portion of a tab
  • widgets that are placed into a grid and that display the dashboard data

These UI levels are implemented more or less by corresponding Vue components:

  • the dash component, which implements the top-level app frame, the tabs and shows icons for the server connections
  • a selection of grid components that implement different strategies for arranging widgets (currently only one is implemented :lying_face:)
  • a widget-wrapper component that implements the editing of inputs and the binding to topics
  • the actual UI widgets that display data

The last level is often comprised of two layers: a web component or library that someone publishes on the web, such as a charting package or a fancy data table implementation, and a Vue component wrapper around it that integrates it into FlexDash.

Widget API

The most important interface inside the FleshDash architecture is the widget API. This is the contract between widgets that display data and the "upper" layers of FlexDash. Currently the API is still evolving rapidly and not formalized at all. (I also want to add that I'm only familiar with Vue components and ignorant of the web components spec, so if there's something FlexDash should do so it can include non-Vue web components then that would be an interesting topic for discussion. :thinking:)

An important decision in FlexDash is that components are integrated by introspection. What this means is that FlexDash loads a bunch of components and then queries what they expose to Vue in order to determine how they can be configured in FlexDash. This is a bit dirty in that it reaches into Vue but it seems to be the best simple way to go. Most likely a custom loader that inspects the components before registering them in Vue would be cleaner and could implement more features, but it's more work with unclear benefits at this stage. :sleeping:

The primary effect of the introspection is to drive the layout of the editing panel that the widget wrapper sets-up. It currently looks like this:

The main aspect of a widget component that FlexDash inspects are the properties. These are variables in the component to which Vue can feed values and are used to specify title, color, value, etc. More specifically, a Vue parent component can bind the child component's "props" to its own variables and thereby feed data into the child component. In FlexDash this parent component is the widget wrapper that uses these Vue bindings to feed either literal values or dynamic values from the topic tree into the widget component.

The widget wrapper inspects each individual prop as follows (Vue calls the properties "props"):

  • the name of the prop is the label the user sees in the widget editing panel
  • the default value is shown as initial value and as tip
  • the type declared for the prop drives the type of input element used by the wrapper to configure literal values (e.g. string, number, boolean, color, array, object)
  • the type is also used to convert dynamic values, in particular from string to number or to boolean
  • the validator is used to show "invalid input" errors to the user while configuring

In the code of a widget this looks as follows:

props: {
  arc: { type: Number, default: 90,  // degrees spanned by the arc of the gauge
         validator: (v) => v>10 && v<=360 },
}

In addition to the Vue standard prop attributes, it seems attractive (i.e. not yet implemented :grin:) to add custom attributes for the following:

  • a help string for a larger tooltip explaining the prop
  • ordering and grouping directives to drive the layout of the widget editing panel

In addition to inspecting the props of a widget component, "it seems attractive" to add custom options to the component for the following:

  • specify output events for the data values the widget can emit to be sent back to the server so the user can bind these to topics (right now the editing panel adds a phoney "_output" prop for that purpose)
  • provide a custom rendering function to display a custom editing panel instead of the generic one provided by the widget wrapper

All in all, so far writing a widget wrapper around some existing component or library is rather simple and can be done in one sparse page of code, often less. The biggest headaches and time sinks have actually been dealing with HTML layout tricks to ensure that the visuals fill the available area nicely.

Config change implementation and undo

After coding a couple of "save/cancel" and "do you really want to delete?" interactions I reached the conclusion that I'm better off biting the bullet and implement undo for the configuration edits. This way all actions can be simple and immediate and the user can hit "undo" if something unexpected happened.

The configuration is represented within the topic tree just like other dynamic values but under a special $config top-level branch. The configuration is organized as a denormalized hierarchical structure: there is a map of tabs, a map of grid, and a map of widgets. The keys of each of these maps are IDs (such as w00233 for a widget) and the values hold the config of an element. The links between the types of elements are by IDs, so for example a grid has a widgets array that holds the IDs of the widgets that are placed into the grid. A simple config for a dashboard with one tab, one grid, and two widgets might be:

$config.dash = { title: "Home Dashboard", tabs: ["t00001"]}
$config.tabs = {
  t00001: { icon: "house", grids: ["g00001"] },
}
$config.grids = {
  g00001: { width: 200, height: 120, widgets: ["w00001", "w00002"] },
}
$config.widgets = {
  w00001: { kind: "Gauge", title: "Living Room", unit: "F", value: "home/temperatures/livingrm" },
  w00002: { kind: "Gauge", title: "Bedroom", unit: "F", value: "home/temperatures/bdrm" }
}

In FlexDash all data items, including the configuration, are held in a store object which exposes mutations, such as addWidget or changeGrid. To implement undo, each mutation is coded to create a set of operations to mutate the topic tree to implement the mutation as well as a set of operations to revert it. For example, to add a widget the widgets array of the grid needs to be updated and the new widget's config needs to be entered into the widgets map. To undo this, the grid's widgets array needs to be set to the old value and the widgets map entry deleted.

In terms of server communication, after trying a few approaches the best seems to be to push the mutations asynchronously to the server (although typically the server responds instantaneously). The conceptual model is borrowed from Google Apps: the user sees the change immediately locally while it's pushed to the server in the background and an unobtrusive UI element shows something like "saving..." or "config is saved". If there is a problem, the UI element becomes more prominent and shows some information about the problem. At that point the user can fix the problem or decide to proceed with edits at the risk of not being able to save them.

In addition to the above machinery, it should be easy to add the ability to copy the config to the clipboard as well as import a config from the clipboard, just like in the NR admin for flows.

From the server's point of view, the config updates are mutations of the topic tree indistinguishable from "data" mutations coming in from NR nodes that update values to be displayed or from UI widgets representing user actions. It would be a fairly localized and simple code change to split the config from the data such that data and config can be sent to different servers. However, by combining them it is possible to modify the config programmatically from the server end. For example, the array of thermometer widgets described previously could be implemented by updating the config from the NR end instead of using an array construct on the dashboard side. Editing the config from the NR end also opens the possibility to implement or port the existing UI nodes in that each such node could create and manage its UI counterpart by editing the config.

The initial undo implementation does not take distributed editing into account, i.e., there is no attempt to guard against or merge config changes that happen simultaneously on different browsers or come in from the server end. Given the nature of the dashboard the assumption is that this simplification is OK, at least until proven otherwise.

Customizing FlexDash

The main way to customize FlexDash is expected to be by writing new widgets. In principle these can be loaded from anywhere that the browser can communicate with subject to the standard browser security restrictions. The dashboard itself is loaded from static files and additional widgets can be placed next to those. But it appears possible to confgure Vue such that it includes its "compiler" in the application, which would enable loading additional components dynamically from source code, i.e. with the standard Vue html-javascript-css single-file-component format. This could lower the barrier of entry to developing new components quite low. The biggest limitation to that will be loading dependencies that are used in the component.

The grid is another place where FlexDash can be customized. The initial grid uses the HTML native grid layout. An alternate grid that uses flex layout in the way most web pages are structured could be written as well. But perhaps more interesting would be to write a full custom grid that may or may not use widgets. In the end, all it needs to do is to render html and reach into the store to retrieve the data needed for display plus to update the store for any user input. Given that a tab can support multiple grids that are tiled one below the other, it is also possible to combine standard grids and full custom grids on the the same tab.

The UI styling of FlexDash is kept fairly neutral and could be customized (this is currently awaiting the implementation of a color picker...) :crazy_face: The overarching concept is that the visual excitement should come from the data displayed in the widgets and not from the dashboard frame. As a result, all the page elements are kept in whites for the light version and in dark grey for the dark version. A single "primary" color is used to highlight UI elements primarily in editing mode and currently Node-Red's red is used. The plan is to let the user change the primary color as well as inject some color into the title bar. For better or worse, the overall style follows the material design theme in part because that's what Vuetify supports.


Phew, that core dump ended up a lot longer than I planned :nerd_face: :thinking: :crazy_face: :rofl: But better write things down now while they're still fresh than struggle later...

9 Likes

Hi Thorsten,

Thanks for all the time you have spend to build this dashboard!!!

The community has elected me for asking noob questions about FlexDash, so here we go :wink:

I would like to repeat of your quotes from the original discussion, to make sure I have understood it correctly:

Is this still something that is possible in the current framework? In the assumption that you would have enough free time to spend on this of course ...

What I understand now is that you have a single general node in the flow, to which all the data is being wired. I must admit that I'm rather fond of the way of working with the old dashboard: that you can install UI nodes from contributors, completely identical to how you installed the non-UI nodes. And that each UI node had it is own config screen, just like with non-UI nodes, and so on ...
So there was very little difference (visually) between a UI and a non-UI node for users.
Is something like that not possible at all anymore? I can understand that you want to have a dashboard completely separated from Node-RED, but it would be nice if there were e.g. wrapper UI nodes. Again - assuming you would have enough free time - is something like this possible, or not at all?

[EDIT]: another reason for me is that I have spend a 'massive' amount of free time on my current UI nodes. If something similar would be available, then I can consider to migrate those nodes once. But rewriting them from scratch is unfortunately not an option, so then I'm doomed to keep using the AngularJs dashboard as long as possible ....

Thanks for all the effort. I feel this is closer to what i would like to use. It is very counter-intuitive for me to add nodes to node red admin panel and expect it to appear in the dashboard. Have you checked the freeboard project? May give you some ideas even though its pretty outdated. That allowed me to build a multi-tenant multi-dashboard solution with the node-red as backend.

Thanks for the encouragement! I didn't know about freeboard, am looking at it now. I'm discovering one new ned-red dashboard a day :smiley:

Yes, this is still on the horizon but I'm not really clear on what it means yet. The cleanest way to create a widget would be to write a .vue file. A good non-trivial example is the implementation of the gauge widget. If you look at that, bear in mind that 95% of the work has to do with getting the SVG drawing to conform to the enclosing widget card and to position the title and value on top of that. I used vuetify classes for some of the text stuff but it may actually have been cleaner to use plain html and css. The rest is stuff you can learn in 10 minutes and is covered in the most basic Vue tutorials.

I believe such a Vue file could be authored using your favorite editor if it can save to the static file area of node-red or it could be authored directly in node-red given some of the functionality in uibuilder.

So there are a bunch of things you're touching on. The "single general node" thing is a uibuilder concept. For FlexDash it's really "a single topic tree". I have not yet gotten to writing node-red nodes for FlexDash, but so far my experiments tell me that one single node is usable but not so great.

On the output side (from NR to FlexDash) there are a number of things that need to be done, two of which are assigning a topic to messages that don't have one and sending the last remembered value to new browsers connecting. On the input (from FlexDash) one has to filter the topics and only forward the pertinent messages. All this can be done with conventional NR nodes but this is how my test flow looks like and I only have a couple of topics:

My thinking is that I would like to have a couple of nodes that cover a few output and input use-cases using single nodes without links or anything. So if you have a node that outputs a message with a payload you can drag in a flexdash node, wire one into the other and set the topic on the flexdash node and be done. This is very similar to the way the UI nodes work, except that you decide which widget displays that value in flexdash not in node-red.

(Hmm, some crazy thoughts come to mind, such as loading a portion of flexdash showing the widget and the edit panel into an iframe in the NR editor...)

In terms of "installing" that's still entirely open. Most likely widgets can be installed by dropping some files into a static web server directory (node-red provides one). That means that in principle, you could install a node-red-contrib-xxx package that all it does is drop some files into the right directory. Not sure how the user experience ends up looking like... With FlexDash it might even be possible to load widgets without any installation, just by typing a URL and asking it to load that. I think it's too early to determine what is going to work for people and what not.

In terms of configuring, FlexDash deliberately differs from the way it currently works in node-red. Take the example of configuring the colors and thresholds of a gauge (oops, I haven't really implemented the the color thresholds yet, so this is a bit forward-looking). In NR you do the config in a panel, then you deploy, then you switch to the dashboard, if you're lucky it's showing the effect, often I have to refresh and wait, then you don't quite like it and repeat. In FlexDash you click on the edit pencil on the widget, then as you click on a color in the color picker the widget instantaneously shows the effect. If you want to see how it looks with 95 as input value, the moment you type "95" into the input box you see how that is rendered. So the feedback is instantaneous, no "edit-deploy-refresh" type of cycle. That being said, I'm sure some people will prefer the current way :roll_eyes: :smiley:.

Given that FlexDash barely exists it's a bit early to think about migration :grin: and I really haven't spent time thinking about it. I believe there are a number of different options but they're all very speculative at this point.

Thanks for chiming in!

Can you elaborate on what "multi-user" means in this context? Do they have access to different flows? Or different portions of the data? Or do they each see the widgets they configured for themselves? Or...?

They can design their own dashboards (combination of widgets and data sources) and data sources supply only the authorized portion of the data. Do you have a similar use-case?

I don't have a multi-user use-case, but it's interesting to know and since you actually built something your input is valuable. Sounds like something like that could be made with FlexDash but it would take some wiring... The notion of multi-user dashboard has come up many times over the years so I'm sure you will not be the last to mention it :laughing:.

How do you represent the "authorized portion of the data" on the node-red side? By flows? By a special property in messages? I can't quite tell how freeboard connects data in NR with the dashboard...

Dashboard is frontend, data sources are backend http endpoints. Dashboard asks for login, after login you get a secure cookie (jwt). It is presented back to backend with each request. Backend checks / validates cookie and understands who you are, then returns only relevant data.

I have also implemented real time data by adding Azure SignalR Service into the equation which also allows scale and secure isolation of users. Dashboard connects to signalr, node red publishes to signalr

I see. Does the dashboard connect at all to node-red then?

login endpoint, data-supplying endpoints, signalr authentication... all built by node reds http-in nodes.

1 Like

Oh you should not worry at all that I'm thinking of migrating in the near future...
But your title "FlexDash: towards a new dashboard for Node-Red" sounds promising :wink:

I was just wondering whether your dashboard "could" offer UI nodes, for those who want to build a dashboard without having to learn web frameworks (due to lack of knowledge or time). Users like me that want to build a dashboard the Node-RED way: install UI nodes via the pallette, drop those nodes in their flow, configure them via their config screen, and deploy.

I completely understand that some people don't like this mechanism, but then they simply don't have to install/use it... However stop supporting UI nodes because some users don't like it, would seem "very" weird to me :thinking:. Would be ideal if there was a way to support a mix of both ways...

Summarized: just wondering if there are technical limitations that would prevent your dashboard from supporting UI nodes. And when it is technically possible, but just too much work: so be it. It is your free time, and we cannot tell you how you have to spend it... I am already grateful that you develop stuff for us for free :clap:

1 Like

I pushed an updated demo to FlexDash

At first glance it will not look terribly different, however, the innards got a huge overhaul. The editing is now fully instantaneous for everything. Instead of "save" buttons or confirmation dialogs I implemented undo, so if you hit an unintended button it's easy to revert. By doing it this way I also got to delete a ton of code, a lot of functionality is now so simple and straight-forward.
I also moved all config transformations into the "store", which is almost fully unit tested.

I believe I now have almost everything necessary for basic editing. There are lots of improvements possible, but at least stuff functions. The next step will be to re-implement the server connection stuff so the dashboard can connect to something (kind'a useful)...

If you spend some time clicking around in the demo and see stuff that is just bad, please let me know.

Excellent job.

Thank you for your hard work.

1 Like

Thank you for sharing your Flexdash and all of your hard work to get it to its current stage. As someone fairly new to Node-RED it does look like a very nice option for a dashboard that may be easier to set up. I look forward to a working version being released.

1 Like

A new FlexDash demo is available! Still at the same location: FlexDash

A big step forward is that this demo can connect to Node-Red :tada:: I implemented a simple websocket transport which means that anyone can try the demo with Node-Red without installing anything!

Yup, just have your Node-Red server handy and point your web browser at the demo. Then switch to the "websock" tab and follow the instructions, which will walk you through adding the websocket nodes to Node-Red (two nodes to send to the dashboard, and 4 nodes for the round-trip toggle switch to light status demo). There's also a link at the end for a packaged flow...

In addition some of the noteworthy improvements are:

  • ability to launch the dashboard to a specific tab using a URL anchor
  • ability to launch the dashboard with pre-configured websocket address via URL query-string
  • help text for all the widgets in their edit panes
  • tooltips for most of the widget inputs in the edit panes
  • a new TimePlot widget that doesn't require study of uPlot's options structure for many simple use-cases and fixing of the legend and tooltip
  • refactoring of the demo so random data is not produced in some hidden code but instead uses a set of "RandomXxx" widgets, this makes it easy to inject random data into any widget, for example to experiment with different settings
  • ability to to expand the markdown and time-plot widgets to full-page
  • ability to expand text inputs for widget editing to full-page

There's still a healthy road ahead :thinking::

  • saving the dashboard config via websocket
  • uibuilder/socket.io
  • editing of array and object config inputs to widgets
  • selecting hierarchical topic bindings when editing widgets
  • color picker for widget inputs
  • dynamic loading of custom widgets

4 Likes

I just stumbled into a really cool new feature for FlexDash: embedding the std node-red dashboard efficiently! That makes it easy to migrate from one to the other by having one page with some of each :smiley:. Here's how it looks in my preliminary hack:

What's really cool is that I have the "NR x" tabs at the top, with the "regular" FlexDash tabs, and clicking on the "NR x" tabs brings up the corresponding NR dashboard UI tab by navigating the iframe. I.e., even though there are a whole slew of NR dashboard tabs the dashboard is only loaded once, the iframe simply navigates from one tab to the next! :rocket:

My screen shot still has the green title bar and nav button of the NR dashboard, I would eliminate that and only use the FlexDash tabs. Also, the "NR x" tab names aren't exactly user-friendly, my current hope is to somehow get the list of tab names :thinking: and send that to FlexDash so it can put the right names below the icons.

NB: this iframe stuff is not in the current demo...

1 Like

I was having some ideas about making FlexDash multi-user... First it needs auth, but beyond that, I'm wondering whether the following set-up can be simple & sufficient for most:

  • each user belongs to one or multiple groups
  • each group has either its own dashboard config, or has a config that is a subset of the tabs in some other group's config
  • each group has access to selected subtrees of the topic namespace

By default all users (or the unauthenticated user) belong to a default group where the full dashboard config is held, i.e., the whole machinery is invisible. Then, as an example, suppose that some new user should only have access to a subset of sensors. This could be implemented by creating a user account and a group with that user, identifying the tab(s) that user should have access to, ensuring that all the data consumed by that tab uses a common topic prefix, and limiting the group to only that topic prefix.

Even without auth this could be useful to create subsets of the dashboard. For example, I may want to have just a couple of common tabs on my home-control tablet and not all 20+.

All this seems relatively simple to implement and, it seems to me, would go a long ways to proving "others" access to data and controls in a controlled manner. Thoughts?

1 Like

Hi Thornton, loving the work.

Regarding what a user or group config holds, your proposal looks good so far. May I also suggest also a concept of page and/or widget visibility by group and that one of the options be private (i.e. user only - for personal viewing)?

Tab == page, basically. Per widget config seems excessive, that's a lot of config everywhere, plus, the key thing is really the data. For access control the main thing is to limit which data, i.e which portion of the topic space gets sent to the browser. Hiding the tabs that can't show anything 'cause the data ain't there is pure cosmetics.