Alternative of current ui nodes in the FlexDash dashboard

General
21

Thanks Steve. However, it would surely be even better to also define a standard so that we can potentially adopt it across multiple dashboard-like nodes? After all, using the config panel is generally not the only (and certainly not the most flexible) way to pass such data to a node.

22

Bart has been badgering me about some of this since day one :laughing:

The way I've been thinking about it is the following, correct me if I'm not capturing your use-cases:

  • fd-quickie NR Node to replace the ui-template node: you can write a simple Vue SFC (single file component) using a text editor pane in the Node-Red admin.
  • FD nodes: a Node-Red node package that contains NR nodes can have a subdirectory that contains associated Vue components that are loaded into FlexDash as additional widgets.

The first option is for quick prototyping and simple stuff. I know how to implement that using vue3-sfc-loader. I believe it would be better to have a server-side implementation of the compilation and packaging. So far I haven't looked a ton into that stuff, there's a lot of black magic...

The second option is really about being able to publish FlexDash UI node packages and have others import them. That ought to be easier than the first option. But it does require diving into that whole packaging mess...

In terms of the UI functionality opened up by the second option the obvious one is to provide additional widgets. But there are more options. This comes down to the structure of a page in FlexDash:

  • Top nav bar (plus left slide-out menu on small screens) exposing multiple tabs
  • The content of a tab consists of one or multiple grids stacked vertically (i.e. each grid occupies the full width of the page), there are multiple types of grids
  • The standard grid is populated with a grid of widgets
  • The iframe "grid" is populated with an iframe that loads some URL (i.e. this is not really a grid)
  • It would be easy to add a full-page "grid" that populates its entire content area with a single widget

The latter case would allow users to take over a full page with a Vue component. (It would also be possible to provide a custom grid, but right now the interface around a grid is not well abstracted in in FlexDash leading to a ton of code duplication. It's V0.x after all...)

The whole notion of loading additional widgets/components raises the question how does one develop widgets/components. For me, there's no question that anything that doesn't provide hot module reload and the Vue debugger is a non-starter. But the overall model isn't clear to me yet. I'll write a follow-up post on this.

I'm sorry, I don't understand what you are referring to. Are you referring to the typedInput widget one can use in a node's HTML file for its input fields? I don't know what that has to do with FlexDash... Maybe some confusion about what I wrote regarding msg.set (which I implemented as msg.params)? Let me try to clarify:

  • Consider a simple gauge widget that has the params: value, min, and max.
  • The fd-gauge node would have configs shown in the NR edit pane for min and max, presumably these are integers
  • A message sent to the fd-gauge node can have a msg.payload, this would set the value (how the payload is interpreted is really under the control of the node as you can see from the code I pasted above)
  • A message sent to the fd-gauge node can also have a msg.params object to set other fields, for example, { min: -10, max: 10 } sets the min and max params.

NB: I now realize NR nodes have properties and Vue components have params, so maybe it should be msg.properties since it's intended to override the properties of the NR node.

NNB: I just happened to read the docs for the delay node and it uses plain msg fields to essentially override properties: msg.delay, msg.rate, msg.reset, ... So maybe the fd-gauge should accept msg.min, msg.max, msg.value, and msg.payload as an alias for msg.value.

23

Are you talking about standardizing how a message can override properties of a NR node in general? That would be interesting and worth a separate thread. But maybe I'm misunderstanding?

24

No, I'm talking about how we define the data that might be sent using a msg to a front-end component. Or to a node that creates a front-end component. Depending on how the dashboard feature works.

25

I'm trying to figure out what developing custom FlexDash nodes ideally looks like. My thought so far:

  • Start with the fd-sample node, this node would have the following:
    • a NR node that essentially passes messages through: node input to widget and widget output to node output
    • a bare bones widget that displays the content of messages coming in
    • a button in the widget to produce some sample output
  • Looking at Creating your first node : Node-RED one would clone the fd-sample repo to a local directory and link it into Node-RED using npm install <folder>.
  • Developing the NR node part would occur "normally", as one would develop any other node.
  • Developing the widget part would begin by flipping a development mode switch in the FlexDash config node.
  • This would install the FlexDash development dependencies (vue compiler, vite, ...) and would also cause FlexDash to be loaded in development mode using vite to support hot-module-reload.
  • Editing the widget source can be done using the same editor the user uses to edit the NR node source as the widget source is just in a subdir. Saving a file would cause a hot-reload in FlexDash so the edit-test cycle is really fast.
  • When done developing, a "bundle" button somewhere would perform a build of all the Vue code in the NR node package (this could contain multiple nodes and widgets) allowing the node & widgets to be used in non-development mode.
26

FlexDash doesn't really has the notion of sending data to a widget/component :nerd_face:. It uses reflection. You insert data into a data structure (a tree) on the Node-RED side. That is reflected to a tree in the front-end. A widget is then configured to look at specific nodes in that tree to get its values. The configuration itself is also in that tree.

You may recall that we had a long discussion about different models about a year ago :wink: and I'm always happy to continue that discussion and also to change FlexDash. So far the reflection model has worked well and I believe it solves some of the persistence/initialization/caching issues that Bart mentioned in the first post in this thread.

27

Please try out the FlexDash test nodes and test flow I showed above:

  1. You must have Node-RED >= 2.0.6 with Node.js >= v14 (v12 chokes on require('./foo.js')), see below for docker instructions if that's of interest
  2. In Node-RED admin go to 'manage palette' and flip to the 'install' tab
  3. Search for flexdash and install node-red-flexdash and node-red-fd-testnodes
  4. Paste the following nodes into a flow and deploy
[{"id":"2616a723dfdcdc69","type":"flexdash testbutton","z":"56494f06ecf796f0","fd":"ad4328.f698dcd8","name":"count up","enabled":true,"color":"#f66151","output_value":"up","icon":"mdi-arrow-down-bold","title":"UP","x":200,"y":140,"wires":[["581055fc8e921b29"]]},{"id":"d1260f509d6e3258","type":"flexdash testbutton","z":"56494f06ecf796f0","fd":"ad4328.f698dcd8","name":"count down","enabled":true,"color":"#99c1f1","output_value":"down","icon":"mdi-arrow-down-bold","title":"DOWN","x":190,"y":200,"wires":[["581055fc8e921b29"]]},{"id":"581055fc8e921b29","type":"function","z":"56494f06ecf796f0","name":"counter","func":"switch (msg.payload) {\ncase \"reset\": context.set('incr', 1); return { payload: 0 }\ncase \"up\": context.set('incr', 1); return null\ncase \"down\": context.set('incr', -1); return null\ndefault: return { payload: msg.payload + context.get('incr') }\n}\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":400,"y":140,"wires":[["1cf748462f03af58","ae20d27612ba8e14","93948bfd85d0c12b"]]},{"id":"76782e8e2cd1a96d","type":"inject","z":"56494f06ecf796f0","name":"","props":[{"p":"payload"},{"p":"reset","v":"true","vt":"bool"}],"repeat":"","crontab":"","once":true,"onceDelay":0.1,"topic":"","payload":"reset","payloadType":"str","x":190,"y":80,"wires":[["581055fc8e921b29","ae20d27612ba8e14"]]},{"id":"1cf748462f03af58","type":"flexdash testgauge","z":"56494f06ecf796f0","fd":"ad4328.f698dcd8","name":"counter gauge","value":null,"unit":"","title":"count","color":"#00ee00","arc":90,"min":0,"max":100,"x":620,"y":140,"wires":[]},{"id":"ae20d27612ba8e14","type":"delay","z":"56494f06ecf796f0","name":"","pauseType":"delay","timeout":"1","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":400,"y":220,"wires":[["581055fc8e921b29"]]},{"id":"93948bfd85d0c12b","type":"debug","z":"56494f06ecf796f0","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":610,"y":220,"wires":[]},{"id":"ad4328.f698dcd8","type":"flexdash config","port":"1881","options":"","path":"/flexdash","redServer":true,"saveConfig":true,"allOrigins":true,"ctxName":"default"}]
  1. Open the dashboard: http://localhost:1880/flexdash if you have a vanilla install, and if not, open the flexdash config node (config nodes side-panel) and use the link below the 'path' input field.
  2. Observe the count gauge counting up, press the 'down' button and it will count down.
  3. If you go back to the flow-editor and look at the debug pane you can see the counts as well
  4. The flexdash nodes can be found in the palette in a flexdash category, please ignore the flexdash in and out nodes for now

Sources

Docker

If you have docker you can try out the steps above painlessly without disrupting any of your existing Node-RED, just run

/usr/bin/docker run --rm -ti -p 1990:1880 nodered/node-red:2.2.2

and point your browser at http://localhost:1990/ (the dashboard will be at http://localhost:1990/flexdash). Of course, you can use a port other than 1990...

1 Like
28

Which is why I included the 2nd sentence. If there is a way to send a message to create or change a front-end UI - no matter what the actual mechanism, it would be good to see if we can standardise that.

In the same way that it would be good if the tree config that you create could be standardised so that it will work with things other than Vue. Since relying on vue v2 already means work to change to v3 and who knows what in the future. Some people may prefer to work with Svelte, REACT, Angular, .... depending (at least in commercial circles) what skills and knowledge and standard dev stacks they have.

I'll certainly be taking a new look at how you are doing things to see if there is anything that can be adopted into future versions of uibuilder.

I do remember and there are many approaches that may work - reflection being one. But if we can share standardised data structures, we multiply our ability to develop and improve across more people and projects.

29

Hey Steve,
I also like the typedInputs, but I have some questions about there usage. Let's take the "button color" field for example. You would have two options in the typed input:

  • a msg.xxxx option to specify the color in the input message (with e.g. 'set' as default value for xxxx).
  • a color input field to specify the color in the config screen

However:

  1. Both options need a different type of html input field: color or text. Is this possible? Because a lot of UI nodes will need to specify colors on their config screen...

  2. I like the typedinput options because you immediately see where the color will be coming from. Because currently I always forget how the nodes work:

    • Do they accept values from the input message only when no value has been specified in the config screen.
    • Do they always accept values from the input message, even when a value has been specified in the config screen. In the latter case the value from the config screen, that can be overwritten by a value in the message.

    With the typedInput options on the other hand, it is very clear where the value is coming from. However suppose you select the 'msg.xxx' option, is it then also possible to specify a 'default' color in the other non-selected color option?

Never heard of the word "badgering". And Google translate doesn't tell me if I should interpret this in a positive or negative way :wink:

What is not clear to me yet: how do you specify the layout of a page? Is that via a layout tool, or do you need to enter manually this via a text editor, or ...

Are you referring here to your topic tree? Am I correct that this tree would have to solve our current message replay issues?
To understand your setup a bit better: so you have one big tree, where a.o. the config of all UI nodes is being stored. And that tree is stored on the backend or the frontend side? If it would be on the frontend side, then I don't understand how a new client would get automatically a copy of that tree...

1 Like
30

Synonyms...
Bugging,. Hounding. Telling me about. Harassing me. Bugging me :joy::joy::joy:

But in a nice way :ok_hand:

2 Likes
31

With you it's all positive :sunglasses:

The layout is an HTML grid. Very similar to the current UI dashboard, except built into the browser and better. The widgets in the grid are just a linear list and get placed left-to-right and then down the page, row by row.
All you can control is the size of each widget in grid units and the order of the widgets. Currently you do that using the edit mode in the dashboard itself (click on the pen in the top-right of a widget and use the up/down arrows to move it in the order or change the number of rows/cols -- this should be drag&drop... someday...)

You can think of the topic tree as a pub/sub tree like in MQTT, or you can think of it as a big JSON data structure (tree). Maybe the second model works best here, so I'll use that.

The tree originates in Node-RED, that's where the master copy is. When a dashboard connects it gets a copy, i.e. just transfer the JSON. If a node changes some value, the code in NR broadcasts a change message that has [json_path, new_value] to tell the dashboards how to update the tree.

Let's make it concrete using the simple gauge from earlier (having min, max, and value). So we might have the following in the tree:

{ outdoor_gauge: { min: -20, max: 50, value: 25 } }

The gauge Vue component is bound to these three leaves of the tree, if they change, the gauge redraws. So if, on the NR side, we call fd.set('outdoor_gauge.value', 28) then that gets propagated to all dashboards and thus all gauges.

The way is this helps with new browsers connecting is if you think of the tree holding the widget state and widgets being otherwise state-less. In more mathematical terms, widgets should be a function that transforms the tree into pixels on the screen. This way a new browser gets the same tree as current ones and draws the same pixels. What this solves is that it doesn't matter whether the last message updated just min, or just value, or perhaps all three.

A canonical example that is more difficult is a time chart. Suppose a simple chart widget that has three params: min, max, and data. Min and max define the range of the Y axis. Data is an array of [x_timestamp, y_value ] pairs. So we might have:

{ temperatures: { min: -20, max: 50, data: [
    [ 1646409600, 25 ], [ 1646413200, 26 ], [ 1646416800, 28 ]
]}

Here we assume that temperatures.data encodes the entire data shown in the chart, so a new browser will get the full chart, not just the last value or something like that. If a new temperature value comes in, the NR side appends it to data and it all gets reflected to all browsers.

Now you might ask about efficiency, specifically, is the full data sent each time a new value is appended? That's a matter of optimization :slight_smile: . For example, there could be operations such as fd.append('temperatures.data', [ 1646420400, 29 ]) that could be made efficient.

The config of the dashboard is stored in the same tree under a $config top-level key. It has sub-keys for tabs, grids, widgets, and the dashboard as a whole. Those contain all the config info and are shipped to the browser when it connects, just like everything else. If anything in that $config sub-tree is changed, like adding a widget, all browsers immediately show the change.

The final piece is that the content of $config is persisted in a (hopefully) persistent Node-RED context store so it's there for NR restarts while the rest vanishes.

I hope this explains things at the risk of being long-winded...

4 Likes
32

Very clear. Now I finally understand why you were asking about persistant context a few days ago...

Now that will be a huge improvement for me. I had to reinvent mechanisms for every of my ui nodes, and in a lot of cases users had to implement a solution themselves by adding other nodes in their flows. Would be really nice if we could avoid all of that, by having a centralized solution in FlexDash.
Very useful!!
Thanks a lot for explaining the internals...

1 Like
33

Not at all. thank you for taking the time to expound on it!

2 Likes
34

I did some basic tests (via Docker) and I can summarize my tests like this: it is VERY EASY to setup Flexdash and FD UI nodes. With such a dashboard, creating a dashboard in the futrue will continue to be possible for ALL kind of users in our community (even those with no web development skills). Kudos to @tve :pray:

Below my first impression feedback:

  1. I first got this screen when opening Flexdash:

    image

    I was not sure whether I perhaps had to restart Node-RED, to activate Flexdash somehow.
    After importing your test flow, the dashboard came available.
    It would be nice if you could - like in the current dashboard - show a page that indicates that there are no UI nodes available. Perhaps that was already on your TODO list...

  2. When I imported your flow, I manually had to reset the browser page in order to see the dashboard. The current dashboard will automatically refresh after deploy. I assume that will be available in the future, when I see you talking above about "hot-reload of modules"?

  3. Your node set contains - besides your two UI nodes - also a FlexDash-In and a FlexDash-Out node:

    image

    Do those nodes have any use to me when I use UI nodes?

  4. When I look at the config screen of one of your UI nodes, the common fields ("name" and "server") seems to be at the bottom of the config screen:

    image
    image387Ɨ511 10.3 KB

    In the current dashboard those fields are at the top of the config screen. Imho on top seems better to me, since that is stuff that all UI nodes will offer.

  5. Perhasp use another label for the "Server" config node? Isn't e.g. "Webserver" more appropriate in this context? Or simply "FD Dashboard" because it refers to some specific dashboard...

  6. You need to specify the context store on top of the config node screen:

    image

    I assume that is required to make your JSON tree persistent? Perhaps this is a really good way of working, but you confused me now. Can't remember that I have ever seen such an input field on any Node-RED node until now. But I might be completely mistaken. Would be nice to get some community feedback about this...

  7. You can save the FlexDash config:

    image

    Isn't this somehow related to the context store name?

    • When my default context store is in-memory, then my FlexDash config is not persistent.
    • When my default context store is e.g. filesystem-based, then my FlexDash config is not persistent.

    So I would expect that you always save your flexdash config in the selected context store. And that the context store determines where the config will be stored. Or have I misinterpreted this checkbox?

  8. When I uncheck the "Save FlexDash config" checkbox, then I get this popup when refreshing my dashboard page:

    image

    So it looks to me that you need to save the config, otherwise it won't work. If that is indeed the case, it might be better imho to remove the checkbox and store it always?

    BTW perhaps this is all nice explained in your Github readme page, but I am now just using it like a less-technically skilled user...

  9. I was not sure if I had to enter something in the "Custom socker io options", because it has a red border by default since it is empty and required:

    image

    Since it is only for advanced usage, it seems appropriate to me if you specify this field as non-required.

  10. When I refresh the FlexDash browser page (via the Chrome refresh button) the counter is becoming alive without me doing anything:

    flexdash

    When I deploy my flow again, the counter becomes stable again. However then the buttons don't work anymore:

    flexdash2

    This might be more difficult for you to solve...

2 Likes
35

That's a good one, @BartButenaers .

I am occupied with family business today, however, I will definitely test it soon as well.

The good thing ... I have no clue about anything what @tve has technically created here. Hence I might very well represent the "techno-noob" who just wants to have a simple yet powerful option to build a dashboard :grinning:

@tve a big Thank You!

EDIT: I couldn't resist. I just wanted to prep my test VM until my family is as well ready to start and I am already done. YES :grinning:

I can confirm on both macos Safari and Opera that after re-deploy the counter stops and neither of the button work anymore. (see EDIT II)

EDIT II:

If only I wait for 10 secs or so counter and button work again. Counting up, starting from zero.

EDIT III:

Changing minimum value in counter gauge to "-100" does not change the gauge's optics. It remains at "zero to 100"

2 Likes
36

BTW I don't think that somebody who has a test VM is a real techno-noob :joy:
Anyway very kind of you to join our FlexDash test department!

2 Likes
37

@BartButenaers and @jodelkoenig thanks for the great feedback! :blush: Some things need fixing and the confusing stuff should all be explained in the 'help' side-panel, but I know that nobody looks there 'cause it's usually closed or showing a different tab. Also, it's an intimidating wall of text... I think I need to make the 'advanced' section collapsible and put a warning at the top. Some of it can also go away...

WRT persistent context, I need to replace that with some other solution, either store the dashboard's config in a file or in the flow. Right now I'm not sure which is best.

I'm excited that you managed to try it all out! I was busy with the next steps yesterday and got hot module reload working for flexdash itself, so one can now serve-up a dev version of FD from within Node-RED, modify a line in the source, and it all reloads! Now I believe I just need to massage some paths to pull in files from another directory and it'll be possible to develop new widgets. :tada:

NB: the connections stuff in FlexDash is too convoluted and unfriendly. I need to simplify it which will result in better behavior. It's definitely on my radar. For now, the browser's reload button is recommended...

NNB: after installing FlexDash do not bother opening a web browser pointing to it until you have imported and deployed the demo flow. You will just get a "Cannot GET /flexdash" message. The reason is that the FlexDash server is in a config node and that doesn't get loaded/started until a flow with a regular node using it is deployed.

2 Likes
38

Yes indeed. I confess guilty. The male species DNA is not programmed to read manuals...
The config screen itself should be as self-explaining as possible. Especially since your first nodes will be used by UI node developers as reference...

I seem to remember some cases from the past where people had problems with config saved in a file. Unfortunately I don't remember anymore what was the case (cloud backend, permissions...). Hopefully some other folks join this discussion to give you some advice!

I have to admit that I had not much time this morning to test. But it was the least I could do in return for your hard work. However the setup was so damn easy that it was done in no time. Couldn't be better...

That would be cool. Then my feedback issue nr 2 should be solved...

I was expecting that this would be the answer. The reason why I proposed a few days ago to use a sidebar panel, is that these panels automatically create a config node when it is not available (to store the info that you enter into the sidebar panel). But in the case of FlexDash a sidebar panel might be complete overkill, since until now I have not felt the absence of it...

You made some incredible progress within a single week. Damn I have the impression now that you will completely mess up my personal development roadmap for 2022 :wink:

1 Like
39

Same here, that's why I haven't changed anything. But I suspect persistent context doesn't work in those cases for the same reason...

Your first feedback item ("Cannot GET /flexdash") is because you openened the browser before deploying the test flow. FlexDash wasn't even loaded yet. The second issue was a corollary of the first one. The only simple fix is to point this out in the getting-started instructions.

You also had a problem with the dashboard stopping when you restarted Node-RED. I managed to reproduce: it only happens on the first re-deploy after a fresh Node-RED restart, or something like that. Seems to be a socket.io client bug, I'll have to investigate, sigh.

1 Like
40

This is having me banging my head against the wall repeatedly... :face_with_head_bandage:

I have the feeling this happened because of the reconnect issues, but it brought to light another issue. FlexDash widgets really like to have their inputs be of the correct type, e.g., the min/max should be numbers, not strings with digits in them. But the NR config editor basically produces strings unless one defines an oneditsave function and fixes up all the types. That's just a giant PITA. Here's the source code from the gauge that defines its params:

  props: {
    value: { type: Number, default: null, dynamic: "$demo_random" },
    unit: { type: String, default: "", tip: "superscript after the value" },
    title: { type: String, default: "Gauge" },
    arc: { type: Number, default: 90, tip: "degrees spanned by the arc of the gauge",
           validator: (v) => v>10 && v<=360 },
    min: { type: Number, default: 0, tip: "minimum value" },
    max: { type: Number, default: 100, tip: "maximum value" },
    color: { type: String, default: 'green', tip: "color of filled segment" },
    low_color: { type: String, default: "blue", tip: "color below low threshold" },
    high_color: { type: String, default: "pink", tip: "color above high threshold" },
    low_threshold: { type: Number, default: null, tip: "threshold for low_color, null to disable" },
    high_threshold:{ type: Number, default: null, tip: "threshold for high_color, null to disable" },
    base_color: { type: String, default: 'lightgrey', tip: "color of unfilled segment" },
    needle_color: { type: String, default: 'white', tip: "color of needle" },
    radius: { type: Number, default: 70, tip: "inner radius, outer being 100" },
    stretch: { type: Boolean, default: false, tip: "false: 2:1 aspect ratio, true: stretch" },
  },

As you can see, all the information needed to produce the config edit pane is already there. If you edit the gauge in FlexDash itself it automatically produces an edit panel (which could use some enhancements, no doubt):

image
image1186Ɨ930 74 KB

I'm wondering whether I should just iframe that edit panel into the NR editor, including the live preview of the widget, and save everyone a ton of time recoding edit panels for NR... Or maybe load the Vue compiler into Node-RED and autogenerate these edit panels (if only it wasn't Angular...).

Thoughts?