🎉 [uibuilder new release] v5.1.1 - Some nice new features and illustration of future features

Share Your Nodes
1

Hopefully you will be as excited as I am about some of the new features in this release!

Most notably, a brand new client library in both ESM and IIFE (UMD) versions so that you can start experimenting with browser-based ES Modules if you wish to or you can carry on using the simple script tag to load the more traditional IIFE version if you prefer.

The new client library introduces the concept of configuration-driven UI's. Create a UI partially or fully from JSON data either from files, web API's or simply from messages sent from Node-RED. Completely up to you. It works with standard HTML elements but also with ANY framework! Including Web Components, VueJS (v2 and v3), Svelte, etc. Check out the client documentation in the Tech Docs: Using the modern front-end client (totallyinformation.github.io).

Along with the new UI feature comes the first experimental UI node. uib-list allows you to dynamically create a list on any web page connected to uibuilder simply by sending an array or object of data. The data is also cached and so is automatically sent when a client connects. A little simplistic in this version but please check out the roadmap for some of the goodies to come: Roadmap (totallyinformation.github.io).

There are also the usual updates and bug fixes. A potentially useful update for some is the addition of a pageName property on the client connect/disconnect control messages. This allows you to make decisions about whether to respond based on the source page - useful in multi-page apps.

Also new is the connections counter on control messages that let you know whether a client is a fresh page load or a reconnection. This is already added as an option to the uib-cache node so that you can optimise cache replay events. It will be added to other caching nodes such as uib-list in a future release.

As always, please let me know what you think and about any bugs.

Many thanks to the various people who have helped me with this release both with ideas and with testing.

Regards, Julian.

Note that this is likely to be the last of the v5 series (barring small fixes). The next main release should be v6 with a move to Node.js v14 LTS as a minimum version. That will be released after Node-RED v3 is released.

Changelog

Fixed

  • Improved module path search to allow an array of locations. Removes spurious warning about socket.io client not being found.
  • /uibuilder/ping now correctly returns 204 (no content) status not 201.
  • web.js::buildHtmlTable - over-optimised regex broke the table cells, now fixed.
  • Connected control message now correctly contains the client id and client IP address. The client id does not change as much as the _socketId, it is saved in browser local storage so will be the same across sessions and multiple windows/tabs.
  • uib-sender was not using topic defined in settings. If present, that overrides msg.topic.
  • uibuilder node - Editor: Add library had been over-optimised and wasn't working in the right order. Now fixed. This also fixed a problem with url rename.

Changed

  • clientId is now session stable. That means that it does not change unless the client browser is restarted. It is now also included in more messages. For control messages, it will be found as a msg property. For _ui related messages, it will be a property under msg._ui. Any other uses will appear under msg._uib.

  • All code now Linted to "Standard JavaScript" with node.js v12 and front-end to ECMA2019. Null/undefined guards put in place.

  • Package.json: Changed homepage to point to Tech Docs on github.io.

  • Client libraries and css available on ../uibuilder/ path as well as on ./ path for consistency with other server paths.

  • Client connect, disconnect and error control messages (uibulder node output port #2) now contain more information. Includes: client version, clientId, Client IP address, page name, number of (re)connections.

  • express-session and jsonwebtoken dependencies removed as no longer in use.

  • Editor:

    • Added stylesheet containing a class of emoji which provides nicer, cross-platform, colour emojis.
    • Libraries tab:
      • Change "URL to use:" to "Estimated link:" on the Libraries tab to make it clear that it might not be correct (down to the library author).
      • Added info emoji to package name (links to package homepage).
      • Added url link to estimated library to make it easier to find out if it actually exists and exactly where.

  • uib-cache node

    • Added option to not replay the cache if the client connection isn't actually new (e.g. if the client is a reconnection after restarting Node-RED).If the control msg recieved contains msg.connected and it is >0, that means that the client is reconnecting and this isn't a client page load. Note that currently, only the new ES module client library populates the msg.connected value. This option is selected by default.
    • The context store type (node, flow, global) can now be selected (only node was previously used).
    • The variable name can now be changed. May be needed if using flow/global stores with multiple cache nodes.

  • package-mgt.js:

    • Rewrite root package.json and package details processing for more efficiency + prettify package.json output
    • Add outdated (current/wanted/latest) to uibRoot/package.json>uibuilder.packages in prep for update display in Editor

  • Old client library uibuilderfe.js

    • Connection count, and page name added to initial "client connect" message in line with the new client.
    • The old uibuilderfe.js client library is now "functionally stable". It will no longer be updated. Please consider moving to the new library (see below). When v6 is released, the old client library will be deprecated.

New

  • uib-brand.css - will eventually be the new default uibuilder CSS. It is light/dark switchable both manually and by browser preference. Still under development, this WILL CHANGE, probably quite a bit. There are still some variables that are needed in order to be able to sensible control things like spacing and sizing.

  • New ES Module front-end client library (uibuilder.esm.min.js)

    The new library will only work with web browsers from early 2019 or later (only really impacts if you are stuck on IE11 or on an Apple device that doesn't support iOS 12 or later). It uses the new brand CSS by default.

    Key differences from the old library:

    • Supports the new configuration/message-driven features that let you both build and modify web pages dynamically.
    • Can load an entire UI from a JSON file (or JSON response from a web server). Can also do incremental loads and dynamic changes.
    • Incorporates the socket.io client library so you don't need to worry about it ever again!
    • Requires a modern(ish) browser.
    • Has to be loaded as an ES Module.
    • No built-in VueJS features, use the new msg._ui features instead.
    • No need for uibuilder.start() in your code any more (nearly always). Often no code needed at all in fact! (Other than loading the library of course).
    • uibuilder.eventSend() now has a lot more information attached. It also now uses the msg._ui property to hold all of the information (except for the payload which is as-before). This brings it into line with the other _ui handling. Attributes, classes, clientId and custom properties are all now included.
    • New function uibuilder.ui({...}) allows passing the same data as msg._ui from front-end code.
    • The "client connect" uibuilder control msg that is output on port #2 when a client (re)connects, now has additional details from the client: client version, clientId, client ip, number of connections since the client last (re)loaded the page. Use to work out whether the client is new or a reconnection. Used by the updated uib-cache node.

    See the uibuilder.module.md page in the tech docs for all of the features and details for the new library.

    • Clients now report how many times they have connected since last page load. This lets uibuilder know whether the client is reconnecting or connecting for the first time.

    Please use the uibuilder.esm.min.js in preference to the uibuilder.esm.js version which is only for people needing to do their own bundling. The min version includes a map so that debugging is as good as (actually better than) using the non-min version.

  • New IIFE Module front-end client library (uibuilder.iife.min.js)

    This is the same as the ES Module client version above but is wrapped as as a standard JavaScript IIFE function which means that it can simply be loaded as a script link (as per the orgiginal uibuilderfe.js client library). You should not attempt to load this version of the new library as an ES Module.

    As with the module version above, when using this version, you should no longer manually load the Socket.IO client library and should hardly ever need to call uibuilder.start.

    All of the features of the ESM version should work as expected but please note that testing on this client has been limited so far. Please report any errors so that they can be corrected.

  • New Node - uib-list

    Consider this to be the first experimental node in what will hopefully be a series. It can be sent a message who's payload is an array of strings or an array of an array of strings.

    The node creates a new <ul>, <ol>, or <dl> HTML list according to the settings. In the case of it being sent an array of arrays, the outer array creates a new list entry and the inner array is joined as a comma-separated list. A <dl> list however must be given an array of arrays and the first entry in the inner array becomes the <dt> entry with the 2nd becoming the <dd> entry.

    The node also requires you specify the uibuilder URL that the node links to along with a required HTML element id that is used to identify the element. Optionally, you can also set a parent element by specifying a CSS selector, the list will be attached ot the end of that parent as a new child. Note that if the selector is not unique, only the first found element will be used.

    Instead of outputting to the uibuilder node, you can instead output a message that contains the appropriate msg._ui configuration used by the new front-end ES Module library. You can use this to help you build larger and more complex UI's and to help learn about how the configuration-driven UI features work. Such a message could be further processed and then sent to an appropriate uibuilder node.

    One additional feature is that the configuration is retained in the node (until Node-RED is restarted or you re-deploy the node/flow) and whenever a new client connects to the matching uibuilder instance, it will send the complete configuration to the new client. This ensures that client browsers connecting after you have created the configuration will all receive it and so will have matching UI's.

  • Runtime

    • Each uibuilder node instance issues a tiEvent when:

      • Instance setup is completed (node-red-contrib-uibuilder/${url}/instanceSetupComplete). The node object is passed as data.
      • A client (re)connects (node-red-contrib-uibuilder/${url}/clientConnect). The control msg is passed as data.
      • When a client disconnects (node-red-contrib-uibuilder/${url}/clientDisconnect). The control msg is passed as data.

      These can be used by any other node that uses the @totallyinformation/ti-common-events module. Such as the experimental uib-list node.

8 Likes
2

Also forgot to say that there are a couple of new front-end code templates that illustrate the use of the new client library and a new example in the import library that illustrates not only the new client libraries but also the new uib-list node.

3

Thank you!
Is there some code-snippet to look at how the new functions/methods have to be used?

4

Morning Julian, just installed this on an instance of v3-beta.3 that has never had uibuilder installed. Got this Uncaught Exception...

======================================= [UIBUILDER PREINSTALL] =======================================
c:\users\Stephen\node-red-1881\node_modules\node-red-contrib-uibuilder\bin\uibpreinstalljs:10
Please ignore warnings about missing peer dependencies for jquery
c:\users\Stephen\node-red-1881\node_modules\node-red-contrib-uibuilder\bin\uibpreinstalljs:11
==================================================================================================


c:\users\Stephen\node-red-1881\node_modules\node-red-contrib-uibuilder\bin\uibpreinstalljs:12

added 66 packages, and changed 1 package in 12s
c:\Program Files\nodejs\node_modules\npm\lib\npm.js:366
29 Jun 07:47:54 - [info] Installed module: node-red-contrib-uibuilder
packages/node_modules/@node-red/util/lib/log.js:94
C:\ProgramData\npm
c:\Program Files\nodejs\node_modules\npm\lib\npm.js:366
29 Jun 07:47:57 - [error] [uibuilder:package-mgt:readPackageJson] Failed to read package.json file from  c:\users\Stephen\node-red-1881\projects\v3-testing\uibuilder
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [warn] [uibuilder:package-mgt:getUibRootPackageJson] Could not read c:\users\Stephen\node-red-1881\projects\v3-testing\uibuilder\package.json. Creating minimal version.
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [warn] [uibuilder:package-mgt:getUibRootPackageJson] No uibRoot/package.json file, creating minimal file.
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [info] Added node types:
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [info]  - node-red-contrib-uibuilder:uibuilder
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [info]  - node-red-contrib-uibuilder:uib-sender
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [info]  - node-red-contrib-uibuilder:uib-cache
packages/node_modules/@node-red/util/lib/log.js:94
29 Jun 07:47:57 - [info]  - node-red-contrib-uibuilder:uib-list
packages/node_modules/@node-red/util/lib/log.js:94
{
  "version": "5.1.0",
  "name": "uib_root",
  "description": "Root configuration and data folder for uibuilder",
  "scripts": {},
  "homepage": "",
  "bugs": "",
  "author": "",
  "license": "Apache-2.0",
  "repository": "",
  "uibuilder": {
    "packages": {}
  },
  "_id": "uib_root@5.1.0",
  "extraneous": false,
  "path": "c:\\users\\Stephen\\node-red-1881\\projects\\v3-testing\\uibuilder",
  "_dependencies": {},
  "devDependencies": {},
  "peerDependencies": {}
}
c:\Program Files\nodejs\node_modules\npm\lib\npm.js:366
29 Jun 07:47:59 - [red] Uncaught Exception:
packages/node_modules/node-red/red.js:501
29 Jun 07:47:59 - [error] TypeError: Cannot convert undefined or null to object
    at Function.keys (<anonymous>)
    at UibPackages.updateInstalledPackageDetails (c:\users\Stephen\node-red-1881\node_modules\node-red-contrib-uibuilder\nodes\libs\package-mgt.js:297:22)
    at runMicrotasks (<anonymous>)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
packages/node_modules/@node-red/util/lib/log.js:94
Process exited with code 1

Line 297 points to this...

        // Make sure we have package details for all installed packages
        await Object.keys(lsParsed.dependencies).forEach( async pkgName => {
            await this.updIndividualPkgDetails(pkgName, lsParsed)
        })

UPDATE...

In order to get node-red running once more, I had to adjust the code...

        await Object.keys(lsParsed.dependencies || {}).forEach( async pkgName => {
            await this.updIndividualPkgDetails(pkgName, lsParsed)
        })
5

Use the Node-RED import menu, go to examples then look for uibuilder. You will find the uib-list example there, import it and it should give you examples of both new client libraries along with the uib-list node.

6

Thanks Steve, will get a fix out. Andy also spotted it.

If anyone needs a work around, from a server command line:

// go to the uibRoot folder
cd ~/.node-red/uibuilder
// Install the dummy `aaa` package
npm install aaa --save
// restart node-red

That should do it.

I'll get a fix out ASAP.

7

Once again, thanks Steve for the pointer. v5.1.1 with the fix has now been published.

1 Like
8

Tested uib-list. Cool!
I suppose it can also be re-worked a little to have a new node: uib-chart, or maybe with a little of code on the fronted the uib-list node can also be used to populate a chartjs.

9

:slight_smile:

The reason I started with list is because it is a set of native HTML tags. I'll undoubtedly do a uib-table next because people find tables hard to build by hand.

A chart is more complex because it needs an external library. I really wanted to do a web component for a chart but it turns out that most chart libraries really don't play nicely with ESM's. Also, there is a lot of complexity to a chart.

Having said that, yes, a chart is certainly needed and is on the cards at some point.

Not quite how the node works. I need to give some more thought as to how a chart node should work and the best way to include the external library.

10

Yes, of course, but this is not a problem of uibuilder. I thought at the chart because you need a node to send the data from NR to the frontend. Once the data is received (and I think uib-list is good for this) then it's a job of the frontend designer.

11

I think you may have misinterpreted how the uib-list node works. It sends a set of instructions to the client that consist of tag names and properties. It literally sends the instructions to the uibuilder client library that gets it to do what you would have to do to build the same thing using code.

Click the checkbox in the node that sends the output as a message instead of to uibuilder and add a debug node and you will see the output. Here is an example:

{
    "_msgid": "62fa1d879a358f1a",
    "topic": "auto-create-list",
    "_uib": {
        "originator": "ee38039276e446bb"
    },
    "_ui": [
        {
            "method": "remove",
            "components": [
                "#li1"
            ]
        },
        {
            "method": "add",
            "components": [
                {
                    "type": "ol",
                    "id": "li1",
                    "attributes": {},
                    "components": [
                        {
                            "type": "li",
                            "slot": "LI One"
                        },
                        {
                            "type": "li",
                            "slot": "LI Two"
                        },
                        {
                            "type": "li",
                            "slot": "LI Three"
                        },
                        {
                            "type": "li",
                            "slot": "LI Four"
                        }
                    ],
                    "parent": "#aparent"
                }
            ]
        }
    ]
}

You can see that it first removes the existing list element if it exists then rebuilds it by sending each element starting with the parent ol tag and then a series of li tags as children.

So a chart would need a completely different set of instructions.

You can also see that this isn't really a terribly efficient way to do things. which is why I've been looking at web components. With a web component, you can build the UI structure and logic into the library and Node-RED only has to send the actual data. More compact, less data.

The node method is OK for a lot of things but for something really complex, I'm not convinced it is quite the right approach. But is IS simple and very node-red like. It is also incredibly flexible.

That is indeed the better approach but it needs front-end libraries or frameworks to make it easy to consume and as you've recently discovered, ones that are both easy to use and look good are hard to come by.

I very much doubt I'll have the time for some years yet to be able to build out even one of these approaches on my own let alone both. So unless others join in, it will remain a slow and steady progress. One of the purposes of the uib-list node is to build an exemplar for this approach so that others can also contribute fairly easily. That is true for the web-components library as well which hopefully will become an exemplar for developing the other approach.

12

Yes, I did! Thank you for clarification.

But now that it is clear, I have some consideration....

With uib-list you are moving the code from the frontend web to uibuilder. This is certainly useful but my question is: is this really within the scope of uibuilder?.

It is certainly convenient to have a ready-made list structure, but I'll still have to put my hands on it for specific customizations of styles/ordering/multi-selection/etc.

In addition, it really depends upon the framework the user is using. During my tests with Quasar, for example, I made a List with this code:

   <q-list bordered separator>
      <q-item clickable v-ripple>
        <q-item-section>Single line item</q-item-section>
      </q-item>

(that is not compatible with the format received from uib-list, but you can add icons, multi-line captions, styles, etc.)

Again, during my tests, I have realized that the hardest effort is not in drafting the main layout, but in handling each widget (button, list, led, dropdown, switch, gauge, etc) with a model/data/topic in index.js. (uibuilder.eventSend is on this path, but instead of sending a "click" event, it should send the associated data (true/false, list-selected-item, slider-value, etc. ) With the help of vue-components (to load repetitive snippet of html code) or router-view (to load full pages into the layout), the html is not a so big problem. The PITA remains in the js. IMHO, efforts should be focused on this: providing the user with a ready-made structured model to manage his widgets. The frontend layout should be left at his free choice.

You made an exceptional tool of great potential, congratulations. Giving the user the ability to communicate to and from Node Red is unvaluable. I could make a full web-app with just uibuilder.send and uibuilder.onChange()!

I have not yet investigated web-components, but if they go towards a simplification of the model handling code, it will be the target.

I think the whole community should be grateful for the work you are doing. I'm sorry I can't help you with developing, but I don't think my coding skills are up to par.

13

That is an excellent question :slight_smile: and the truth is that I'm not completely convinced. But that is why uib-list is marked as experimental. I absolutely want feedback like this because in the past there have been lots of calls for uibuilder to have a similar approach to Dashboard, e.g. providing lots of nodes that create UI elements.

Yes indeed. So where should the line be drawn between something that is quick and easy - especially for folk with no HTML or coding experience at all and something that helps people with coding experience but who need tools to speed up their design processes? And everything in-between.

Also, the reason that I added an option to output the msg._ui instead of sending it direct was to give people a quick-start from which to work should they need/want to do things that the node doesn't do.

Truth is that there isn't a one-size-fits-all. You can see that with Dashboard. As people develop more complex Dashboards, they eventually get to a point where they have to start coding and Dashboard's internal complexity actually makes that relatively hard (you need to know not only HTML/CSS/JS but also how Angular v1 works AND how Dashboard itself works).

No but uib-list is aimed at people who need a simple list but either don't have the coding skills or need something fast. And it isn't a complete example of course, only an experiment.

What you CAN do with the new clients is still generate that UI element purely from Node-RED if you want to. After all, that's actually what uib-list does but it works with <ol> and <li> tags instead of the Quasar equivalents.

The data schema that msg._ui uses can build ANY UI element from any framework. uib-list is simply an example of a wrapper around that data schema. It illustrates a possible approach for low/zero code UI creation.

:slight_smile: Yup!

Ah, so now you are getting close to one of the reasons I also started writing some web components. How to capture and return the "interesting" data from each component. Given that there are 10's of thousands of components in the wild across dozens of frameworks, all of which work in their own way.

We are back to the problem that uibuilder either has to back a specific framework (which experience shows is a bad move because frameworks don't always age well) or has to find ways to work without specific framework lock-in.

The "ideal" solution to this would be for someone else to build framework-specific solutions around uibuilder as a core.

:+1:

Well, many thanks for your kind words. And don't worry about coding, it is great that you are taking an active part in the discussion and testing, it really helps clarify things and drive out issues.

14

they should go with Node Red Dashboard, no question here.

IMHO this is not the scope of uibuilder. It is a professional tool for those (skilled) who want specific customization in their NR.

I support this.

15

Except that the current Dashboard is closing in on end-of-life. The underlying framework is no longer under active support. Currently, there is no obvious alternative.

16

Hello Julian ..

I switched today to the new uibuilder.esm.js .. all is good with my application that uses a build step.
One thing i noticed is that because i wasnt loading a css file (all css was in each .vue file)
the uib-brand.css file was loading but clashing with existing bootstrap-vue css styles.

So i tried to pass the loadStylesheet option setting it to false .. but that didnt make a difference.
As a workaround i loaded a fake css so the code responsible of loading uib-brand.css wouldnt kick in ?

// uibuilder.debug(true);
uibuilder.start({ ioNamespace: "/navbar", loadStylesheet: false });

Also uibuilder.debug(true); doesnt seem to enable debug msgs

1 Like
17

Great! Worth a WIKI write up maybe?

oops, a bug :slight_smile:

Ah, missed documentation change.

Using the modern front-end client (totallyinformation.github.io)

Internal logging is much improved over previous versions of this library. There is now a dedicated internal log function which adds colour highlighting to browsers that support it in the dev tools console. That includes all Chromium-based browsers and Firefox.

You can alter the amount of information that the uibuilder library outputs to the console by changing the logLevel with uibuilder.logLevel = 4 where the number should be between 0 and 5. you can set that at any time in your code, however it will generally be most useful set before calling uibuilder.start().

The default level is set to 1 (warn). The levels are: 0 'error', 1 'warn', 2 'info', 3 'log', 4 'debug', 5 'trace'.

Changing the log level outputs an info note to the console telling you what the level is.

The log function is also available to your own code as uibuilder.log(level, prefix, ...outputs).

PS my thumb is in a pot at the moment and will be for a few weeks so progress may be slow for a while.

18

the new module is much easier / cleaner when using a bundler.
and i also switched to Vite because parcel-bundler(v1) was having some issues ..
mostly due to my lack of knowledge of how modules work
but its good that we dont have to worry about loading the socket io client etc

nothing critical .. you comment in docs "if you haven't loaded your own." helped me get around it.
shall i open a Github issue on this so it can tracked ?

if you don't want the uibuilder default stylesheet (uib-brand.css) to be loaded if you haven't loaded your own.

yeap .. this works fine

great work as always.

19

:+1: yes please.