[node-red-contrib-uibuilder] NEW MAJOR RELEASE v5.0.0 🎁

TotallyInformation · 2 April 2022 19:09

Phew, it is finally here! The big one - and it is big, really big! I've included the release notes below.

Yes, I'm afraid it has some breaking changes and you will need to do some updates in your flows and in the file system.

If you go ahead and install and restart Node-RED, you will get some warnings like this:

2 Apr 19:36:07 - [warn] [uibuilder:socket:addNs:mybluprint] Socket.IO middleware failed to load for NS - check that uibRoot/.config/sioMiddleware.js has a valid exported fn.

That's nothing to worry about. Just go ahead and delete any of the files in ~/.node-red/uibuilder/.config (or wherever you have moved the uibRoot folder to) that you aren't actually using. If you are using any, please compare your files against the new *.js-template files that are copied over for you.

Next, refresh the Node-RED editor page if you have it open. You will now see that any deployed versions of uibuilder nodes are marked with a red triangle showing that they need attention. Go ahead and open each one and just press Done unless some errors are shown. Don't redeploy yet though. (Hopefully you don't have 16 uibuilder nodes deployed in live as I do )

Finally, you need to go into any one of your uibuilder nodes and reinstall any front-end libraries you had installed before. That's because the location of the libraries have moved. No longer in the Node-RED userDir, they are now in the uibRoot folder. While that is a pain, it means that you no longer have to worry about managing the separate list files if you want to install manually. It also means that you can install any version/tag/branch from any location recognised by npm from the library manager - great for VueJS users needing v2!

Once you've re-installed the packages (what you didn't make a note before you upgraded! Not to worry, look in ~/.node-red/uibuilder/.config/packageList.json), don't forget to also remove them from ~/.node-red/. Don't forget that you can delete the two json files in the .config folder once you've done this, they are no longer needed. You can also delete the security.js file from that folder as it is no longer used. If you delete the *.js-template files, they will just come back next time you restart node-red.

NOW you can redeploy. And you should be done - all upgraded. A couple of other things you might want to do are listed below like clearing client cookies.

Sorry to put you through that but you should note that the new ways of working are much better than previous and are opening up some massive opportunities for the future! Please see the roadmap in the tech docs for some of the amazing stuff that will hopefully be coming soon.

Changelog

WARNING: v5 is a very major release and WILL break some deployments. Please check through the changes before deploying. In most cases, opening each uibuilder node in the Node-RED Editor, clicking Done and re-deploying will be enough.

BREAKING

Clear client cookies - You may get some odd results if you don't because cookie handling as been significantly reworked.
Installation of packages for use in your front-end code has been moved from the userDir to uibRoot.

Note that only packages installed into the uibRoot folder will be recognised.

Unfortunately, this means that you will need to re-install your packages in the correct location. You should uninstall them from the userDir.

By default: userDir = ~/.node-red/, uibRoot = ~/.node-red/uibuilder/. However, both can, of course, be moved elsewhere.

uibuilder will automatically create a suitable package.json file in uibRoot. That file not only lists the installed packages but also has a custom property uibuilder that contains metadata for the uibuilder modules. Specifically, it lists all of the necessary detailed data for the installed packages.

The files <uibRoot>/.config/packageList.json and <uibRoot>/.config/masterPackageList.json are no longer used and may be deleted.

You can now install not only packages from npmjs.com but also from GitHub and even local development packages. @scopes are fully supported and versions, tags, and branches are supported for both npmjs and GitHub installs.
Removal of Security Processing - The built-in security features of uibuilder have now been removed.

This is because I was not getting any closer to a design that was robust and safe and it was having a serious impact on development.

The recommendation is that you use external security processing via a reverse proxy service (e.g. NGINX). I will be trying to provide some documentation for this.

Alternatively, you can still provide security processing by making use of the various middleware features - see the <uibRoot>/.config folder for templates.

Because of this, the security settings part of the Editor config panel has been removed. This included the use security flag, session length, JWT secret and auto-extend flag. If you have been using the vNext development branch, the Security tab has been removed.
Peer installation of VueJS and bootstrap-vue yet again removed. Since these now need to be in the uibRoot folder which
we don't necessarily know at preinstall time.

We will be looking at another alternative method. Now that template switching is even more powerful and so is
package management, it is likely that we will build something into the template installation process.

Until then, please install the vue and bootstrap-vue packages via the uibuilder library manager if you need them.
.config folder templates - In previous versions, any files in the master templates for the <uibRoot>/.config folder would be copied when Node-RED (re)starts - however, they would not overwrite existing files. From v5, the master .config templates have been renamed to end with .js-template and these will now ALWAYS overwrite whatever is in the <uibRoot>/.config folder whenever Node-RED (re)starts. That way, you always have easy access to the latest templates.

In addition, some of these templates have significant changes and you should review the new templates before going live with uibuilder v5. You can (and should) also safely delete any .config/* files you are not using.

In particular, the masterPackageList.json and packageList.json files are no longer needed or used and should be deleted.
URL cannot be "uibuilder" - As this url is now used by various services, allowing it would potentially create name clashes and hard to debug errors. If you have an endpoint named "uibuilder", please rename it to prevent problems.
Clear any old cookies - Cookie handling has changed for v5 (see the details below). New cookies are clearer and last only for the current browser tab session. You should clear out any old cookies that have been saved by your browser for each uibuilder app URL.
Minimum Node.js version supported is now v12.20. Minimum browser version remains the same and must be one that supports ES6.
New Dependency - I've finally given in and included the socket.io-client package in the dependencies. The reason for this is that the server does not correctly include the client package as expected. While it works if you are including the client using a <script src="..."> line in your index.html, it does not work if you are using web components or a build step with an import statement. So in that instance, you should use.

Note the change of path when writing your HTML:
```
<script src="../uibuilder/vendor/socket.io-client/socket.io.min.js"></script>
```
Though the old path works as well.

However, if you want to use uibuilder with web modules or with a build step and wish to import socket.io client, you must not load via a script tag. Instead, in your JavaScript or framework code:
```
import { io } from '../uibuilder/vendor/socket.io-client/socket.io.esm.min.js'
```
Better still, use the module compatible version of the uibuilderfe client uibuiderfe.mod.min.js or uibuiderfe.mod.js which will import the correct socket.io client for you.

New

New layout for the Editor panel

This is a much cleaner and clearer layout. It also blocks access to parts of the config that don't work until a newly added node has been Deployed for the first time so that its server folder has been created.

There are also some additional error and warning messages to make things clearer.
New node uib-sender - this node allows you to send a msg to any uibuilder instance's connected front-end clients.

That means that it is pretty much the same as sending a message directly into a uibuilder node.

You select the instance of uibuilder you want to use by selecting an existing uibuilder URL from the dropdown.

You can also select whether you want input messages to go straight to the output port as well.

Or, more usefully, you can allow "return messages". This allows a front-end client to send a message to node-red with some pre-defined metadata added that will route the message back to the uib-sender node. In this way, the sender node can be used as a semi-independent component.

Note that this same method can be used by ANY custom node, check out the code to see how it works. It requires the use of an external, shared event module @TotallyInformation/ti-common-event-handler. The msg metadata looks like: { _uib: {originator: <sender_node_id>}, payload: ... }. The sender node id is just that, the Node-RED node id for the sender node instance.

The uibuilderfe.js library has been updated to allow easy use of the originator property for uibuilder.send(). See below for details.

There is a new page in the Tech Docs on using the sender node.
New node uib-cache - this node allows you to cache input messages in various ways and recognises uibuilder's
cache control messages so that a client browser (re)connecting to a web page will automatically get a copy of the cached pages.
See the Tech Docs for details. An example flow is included in the uibuilder examples library.

Note that you can use this node without uibuilder if you want to.

There is a new example flow demonstrating the use of the cache node.
New Feature Msg send middleware. You can now add a custom middleware file <uibRoot>/.config/sioMsgOut.js. The exported function in it will be called every time any msg is sent from any uibuilder node to any connected client. Please see the template file for more details. This rounds out the ExpressJS and other socket.io middleware (connect and on msg receipt) and helps make up for the removal of the uibuilder security features by allowing you to create your own bespoke identity and authorisation processing.
New Feature Instance API's. You can now define your own API's to support your front-end UI. These run as part of the Node-RED server and can be called
from your UI, or indeed from anywhere with access to the Node-RED server's user endpoints.

You can add any number of *.js files to a folder <uibInstanceRoot>/api/. Each file will be loaded into the uibuilder instance and tested to make sure that it contains either a single function or an object containing functions who's property names match either an HTTP method name (get, put, etc) or the generic use.

Such functions are added to the instances router. See the Tech Docs for more information on how to use the instance API's.

Note that, because such API definitions reside in a potentially user-facing folder and may be significant security risks, their use is controlled by a flag in Node-RED's settings.js file uibuilder.instanceApiAllowed. This must be set to true for API's to be loaded.
New Feature - Added a version checker that allows uibuilder to notify users if a node instance must be updated due to a change of version.
New Feature - A default CSS style sheet has been introduced. Either include in your index.css file as @import url("./uib-styles.css");. Or in your index.html before the reference to ./index.cssas`.

Currently this contains some :root classes defining colours and a switcher that picks up whether your browser is set to light or dark themes.

It also has a number of classes that style the toast notifications if you are not using bootstrap-vue.

Add the uib class to your <body> tag in index.html to pick up the full styles for your page. If you use this, your page will switch between light and dark modes depending on your browser settings. Note that, when using bootstrap-vue, these styles are ignored.

The style sheet file may be found in the front-end/src/uib-styles.css package folder.
New Feature - HTTP Ping. The uibuilder server now has a new endpoint ../uibuilder/ping that will return a 201 status code (No Content). The client has a new function uibuilder.setPing(ms) that will call that endpoint either once (ping(0)) or every n milliseconds (ping(n)). You can use this to check whether the server is responding. But more usefully, you can use it to help keep alive a security session.

The function also fires an event that you can listen for:
```
uibuilder.setPing(3600000) // Once an hour
uibuilder.onChange('ping', function(data) {
  console.log('pinger', data)
})
```
Extended Feature Package Management - You can now install not only packages from npmjs.com but also from GitHub and even local development packages. @scopes are fully supported and versions, tags, and branches are supported for both npmjs and GitHub installs.

Note that only packages installed into the uibRoot folder will be recognised.

Also note that if you manually install a package rather than using the library manager, you will need to restart Node-RED.
Extended Feature - Added uib version to the connect msg to clients and a warning in the client console if the client version not the same as the server.
Extended Feature - Now allows socket.io options to be specified via a new property in settings.js - uibuilder.socketOptions. See the discussion here. The Tech Docs have also been updated.
Extended Feature - If using a custom ExpressJS server for uibuilder, allow different https settings (key and cert files) from Node-RED itself. Uses a new property in settings.js - uibuilder.https.
uibuilderfe library
- New Feature - Received cookies are now available as an object variable key'd on cookie name. uibuilder.get('cookies').
- New Feature - A new unique client id set by uibuilder is available as a string variable. uibuilder.get('clientId'). This changes if the page is reloaded but not if the client loses then regains a Socket.IO connection (where the socket id will change). It is passed to the client as a cookie. The client sends it to the server as a custom header but only on Socket.IO polling requests since custom headers are not available on websocket connections). It also adds it to the socket.handshake.auth.clientId property which should always be available to the server event handlers. Caution should be used if making use of this feature since it is likely to change in the future. See the updated sioMiddleware.js for an example of use. The client id is also included in the uibuilder control msgs output to port #2 on a client connect and disconnect. The ID is created using the nanoid package.
- Extended Feature - Toast notifications (notifications that overlay the UI) are now available even without VueJS and bootstrap-vue. They can be styled using the uib-toaster, uib-toast, and uib-toast-head classes when not using bootstrap-vue. Toast notifications can be set either by a standard msg from Node-RED or by calling uibuilder.setToast(msg) (where the msg matches the same format used from Node-RED). Internal uibuilder visual notifications will also use this mechanism. Notifications auto-clear after 10s (used to 5) unless otherwise controlled via the options.
  
  There is a new example flow to illustrate the use of toasts.
New Example Flows
- uib-sender - How to use the uib-sender node. A new flow tab. Based on the blank template so does not need any libraries installing.
- uib-cache - How to use the uib-cache node. A new flow tab containing two examples, one with and one without uibuilder. The uibuilder example is based on the blank template so does not need any libraries installing.
- toast-notifcations - A group containing 2 uibuilder nodes (with empty URL's) and a bunch of inputs for testing Toast notifications. One of the uibuilder nodes uses the blank template (so no libraries needed), the other uses VueJS and bootstrap-vue.
- Other example flows have been updated to remove the default URL to ensure that duplicate folders are not accidentally created on import. In addition, the MoonJS example has been removed as it was out-of-date.

Changed

Cookie handling has changed for the better. There are 3 cookies set by uibuilder: uibuilder-namespace (what SockeT.IO needs to communicate), uibuilder-client-id (see new features above), and uibuilder-webRoot (if you are using httpNodeRoot in settings.js). Each is set as a session cookie which means that if you close the window/tab showing your UI, the cookies are deleted. However, if you are being super-strict about EU and California law, you should inform your users that they exist. The cookies are limited to the exact path for the uibuilder instance they come from so there shouldn't be any cross-contamination. However, you should clear any old cookies when upgrading to uibuilder v5 from v4 or before. The cookies do not capture or share any client data and they cannot be used for tracking.

Custom headers are also added by uibuilder. These are only accessible via XHR API calls, not by uibuilderfe itself. x-powered-by (set to uibuilder), uibuilder-namespace, and uibuilder-node (the node id of the uibuilder node). In addition, uibuilder sets X-XSS-Protection to 1;mode=block and X-Content-Type-Options to nosniff for added security. You can, of course, override all of these using custom middleware.

The addition of the uibuilder-webRoot cookie should now mean that you very rarely need to pass startup parameters (except for the Vue app) to uibuilder.start() as the library should now be able to work everything out for itself.
uibuilder nodes now show the url in angle-brackets. If the url is not defined, <no url> shows. If the node has a name, this is shown before the url. e.g. My UI <myui>. If you want to have the url show on a different line to the name, add \n to the end of the name.
When adding a new uibuilder node, the url is now blank. This helps prevent accidentally creating two nodes with the same url which is confusing to recover from. As a blank url is not a valid configuration, the red triangle will show.
When copying and pasting a uibuilder node, the pasted node(s) will have their URL changed to blank to prevent nodes with duplicate url's being deployed.
Improvements to the "uibuilder details" page should make it easier to read. The data for ExpressJS Routes is much improved.
Improvements to the "instance details" page. Now includes the ExpressJS routes for that instance.
Improved uibuilder logo, many thanks to Calum Knott.
Editor panel
- Improvements to the Editor help panel. Should hopefully be clearer and includes all of the settings and custom msg properties. Now uses a tabbed interface.
- File editor now excludes .git/**, .vscode/**, node_modules/** and _*.
- When editing the configuration for a uibuilder node, if the URL is invalid or the server folder hasn't yet been created, you cannot access various parts of the panel.
- ctrl-s - in the file editor, pressing ctrl-s will save the file rather than trying to save the web page. If, like me, you have a strong muscle memory for saving using ctrl-s, this should save you a LOT of annoyances.
uibuilderfe.js client library updated to allow for the use of an originator metadata property. This facilitates routing of messages back to an alternative node instead of the main uibuilder node.

There are three ways to make use of this:
- Use the new uibuilder.setOrigin('<sender_node_id>') function. This will then route ALL messages from the client back to the specified node. This is of marginal use because the main use-case for the property is to automate routing of data to/from web components of which there are likely to be several on a web page.
- Use the new optional override parameter for the send function. uibuilder.send(msg, '<sender_node_id>'). This will send this one message back to the specified node. It will override the setOrigin. The utility uibuilder.eventSend() method (that lets you easily send a msg back to Node-RED from a DOM event) has also been updated to allow the originator parameter.
- Manually add the metadata to the node { _uib: {originator: <sender_node_id>}, payload: ... }. This is not generally recommended as it is error prone. However, if writing custom front-end components, you may want to include the origin property as an option to allow end-to-end automatic routing of messages to/from your component instances.
See the new uib-sender node details above for an example of using the originator property. That node adds the property to its received msgs before sending to your connected clients.

Note that at present, control messages from the front-end cannot be routed to a different originator node, they all go to the main uibuilder node. This will be reviewed in a future release. Let me know if you think that it is needed.
Added documentation for Socket.IO middleware and error handling.
Added socket.io-client as a dependency and use it where possible - this simplifies access to the client library, especially when using uibuilderfe as a module. It will also help people doing their own build step and wanting to bundle the socket.io client and uibuilderfe library.
Minor improvements to the .config middleware templates.
Improved logging for npm commands in library manager.
Added client IP address to client connect & client disconnect control msgs
Removed Features - all security processing has been removed from uibuilder along with the appropriate settings. Please use the ExpressJS and Socket.IO custom middleware features instead. These can be combined with external authentication and authorisation services such as a reverse proxy.

Internal and development improvements

Shared event handler implemented. This enables external nodes to send and receive data to/from uibuilder front-end clients.
Gulp implemented
- initially for composing the uibuilder.html from the contents of src/editor
- and to replace the previously manual minify step for uibuilderfe.js
- other tasks likely to be added in the future to make more efficient code and ease the release/publish process.
New eslint rulesets implemented & config restructured. Along with the .html file decomposition, this makes for a much more accurate linting process.
Massive number of minor code improvements to uibuilder.html and uibuilder.js & to the supporting libs and uibuilderfe.js thanks to the impoved linting.
Removed deprecated functions.
Remove old console.log statements used for testing and no longer required.
Even more massive restructuring of uibuilder.js.
- A lot of the core logic now moved into dedicated modules, each containing a singleton class.
- Removing the need for the node object. This meant the use of some arrow functions to be able to retain the correct context in event handlers and callbacks.
- Destructuring the big exported function into a series of smaller functions. Makes the code a lot clearer and easier to follow. Also helped identify a few bits of logic that were not quite sane or not needed at all (the result of evolutionary growth of the code).
- Using named functions throughout should make future debugging a little easier.
- npm package handling moved to a separate singleton class in package-mgt.js
Removed inputHandler function from uiblib.js. Code folded into the inputMsgHandler function in uibuilder.js which has been destructured so is small enough to have it as a single function.
Package management rewritten. Should be faster and uses async/Promise functions.
Removed uib.masterStaticDistFolder and uib.masterStaticSrcFolder, no longer required. Only a distribution folder now gets served. Source folder will never be served. This is defined in uib.masterStaticFeFolder. If the folder cannot be accessed, uibuilder throws an error.
Editor changes
- Added a version checker that allows uibuilder to notify users if a node instance must be updated due to a change of version.
- Minified the Editor panel html file (using Gulp) - should load faster now.
v3 admin API changes
- Moved v3 admin API to its own module (libs/admin-api-v3.js) and changed to be an ExpressJS router instance.
- Moved the setup from uibuilder.js to web.js
- New v3 admin API command added to list all of the deployed instances of uibuilder. Issue a GET with cmd=listinstances. This allows other nodes to get a list of all of the uibuilder instance URL's and the ID's of the nodes that create them. See the uib-sender node's html file for details.
v2 admin API changes
- Moved v2 admin API to its own module (libs/admin-api-v2.js) and changed to be an ExpressJS router instance.
- Moved the setup from uibuilder.js to web.js
web.js changes
- Both admin and user routes restructured, making use of Express.Router's to improve layout and control.
- Setup of v3 admin API moved to web.js class module (out of uibuilder.js).
- Setup of v2 admin API moved to web.js class module (out of uibuilder.js).
- New web.dumpRoutes(print=true) method added to web.js - dumpRoutes outputs a summary of all the relevant ExpressJS routes both for uib user facing web and Node-RED admin web servers. Also added individual methods: web.dumpUserRoutes(print=true), web.dumpAdminRoutes(print=true), and web.dumpInstanceRoutes(print=true, url=null) (where passing a uib url will just dump that one set of routes).
- Removed the separate serve-static npm package. This is now built into ExpressJS and not required separately.
- Removed the separate body-parser npm package. This is now built into ExpressJS and not required separately.
- Moved the user-facing API's to web.js from uibuilder.js and moved to their own Express.Router on ../uibuilder/....
- Added a new this.routers object - this helps with uibuilder live configuration documentation as it records all of the ExpressJS Routes that uibuilder adds.
Redundant security code files moved to an archive folder.

Fixed

URL validation should now work as expected for all edge-cases.
Fixed the problem that required a restart of Node-RED to switch between src and dist folder serving.
Fixed Issue #159 where sioMiddlware.js wasn't working due to the move to Socket.Io v4.
Fix issue reported in Discourse of an error in masterMiddleware when setting headings. Corrected heading syntax for ExpressJS v4.
Client connect and disconnect msgs not being sent to uibuilder control port (#2). NOTE: As of Socket.io v4, it appears as though the disconnect event is received after the connect when a client is reconnecting. You cannot rely on the order.
Fixed CORS problems after move to Socket.IO v4. (NB: CORS is defaulted to allow requests from ANY source, override with the uibuilder.socketOptions overrides available in settings.js).
A number of hard to spot bugs in uibuilder.html thanks to better linting & disaggregation into component parts
Fixed an issue when removing uibuilder nodes caused by the move to socket.io v4. Should fix the failure to remove unused uib instance root folders and fix renaming problems as well.
URL rename failed if user updates template before committing url change. This is now blocked.
File editor failed if the node hadn't been deployed yet. Blocked if instance folder hasn't yet been created.
Change degit call to turn off cache which was producing a could not find commit hash for HEAD error. See degit Issue #37. Partial fix for Issue #155.
If deleting a node that hasn't been deployed, a delete folder warning is given - add check to see if the folder actually exists before giving the error.
If using Node-RED Docker with recommended install, uib couldn't find the Socket.IO client folder to serve. Issue. Extra check and cleared warnings added.
Spurious instance folder rename when it wasn't needed.
Bad cookie handling!
Copy/paste of uib nodes detected correctly, url forced to blank and url change process surpressed - prevents several bugs
Correct template now loaded when new instance deployed - fixes copy/paste where copied node had non-default template
Fixed folder not deleted when new deploy is followed by delete of node

Experimental and partially working new features

WARNING: Consider these features experimental, some parts may not work and might even cause Node-RED to crash if used. Do not yet use on production.

NOT YET FULLY WORKING

Added configuration option to add browser/proxy caching control to all static assets - set the length of time before assets will be reloaded from the server. This may sometimes significantly improve performance in the browser. It depends on the performance of your server and the complexity of the UI.

Added on options variable for serve-static to allow control of caching & other headers. uib.staticOpts.

Some static folders are served at module level and so don't have access to instance settings. Would likely need to have different settings on global serves from instance ones. Needs more thought.

This lets you control caching of your "static" assets like JavaScript, HTML, CSS, Images and any installed front-end library resources (Vue, etc).

Note that this is not for caching the msg's coming through the node, see the caching examples in the WIKI for that.
If you use Node-RED's projects feature, restart Node-RED after changing projects otherwise uibuilder will not recognise the new root folder location.

hotNipi · 2 April 2022 20:50

Longest release post ever. For sure I'll try it out using with my preferred "no front end libraries" option. Let's see how it goes.

TotallyInformation · 2 April 2022 21:03

I've never been known for writing short documentation!

Oh, you are going to love my experiments with Web Components then

I already have 2. One that lets you load remote html (full pages and fragments) into your host page. The other takes json or a JavaScript object and HTML formats it for display on page. I'll create a repo for them at some point now that uib v5 is out.

Not sure how far I'll be able to take pure Web Components though because they have some serious limitations right now. I think that I also want to create an additional library to simplify using DOM and Shadow DOM because, like you, I can see that there is a lot you can do without the real need for a framework when working with ES2020 and beyond.

Svelte though is interesting me more and more. One of the nice features of Svelte is that there is no need for an extra library when using it unlike Vue or REACT. Everything is compiled into your bundle and so is pretty efficient. Svelte can also create Web Components too though there are, again, some limitations. That's more about Web Components than Svelte I feel. With Svelte, I've already been able to come up with a simple proof of concept that lets you dynamically load both Svelte and Web components. So having a fully dynamic, configuration-driven dashboard is clearly feasible. I also have created some proofs-of-concept for dynamic list and table components. The roadmap in the tech docs has a list of requirements I want to meet with both Web and other framework components along with a list of components I'd like to create. More to come.

I've also been playing with some ideas from the latest realms of HTML and JavaScript and there are some really cool things that the next few years will bring as the new API's bed into mainstream browsers. Check out the roadmap in the tech docs to see some of the ideas

HaroldPetersInskipp · 3 April 2022 01:02

Seems to be working great, I had a little hiccup setting the .config files (had them is two locations, forgot I was using the ones in the projects folder), but all is well now.

Steve-Mcl · 3 April 2022 07:44

Hi Julian, I find the approach taken by alpine.js quite interesting. I feel it sits somewhere between svelte and Vue. It's very light but with just enough functionality to make it useful & aid reactivity etc.

Here is a write up that tweaked my interest...

TotallyInformation · 3 April 2022 10:36

Thanks Steve. I think I did look at Alpine previously. The main problem is that if I spend time on a particular framework, it needs to be one that is not only easy to use but also popular and one that will stand the test of time. However, Alpine will, of course, work well with uibuilder and will certainly suite some people. I really hope that someone creates an example that can be added to the WIKI and perhaps even creates an external uibuilder template and example flow. It certainly looks quite nice and simple to use.

I've already made some good progress on moving some things away from VueJS - such as the built-in "Toast" feature for instance that now works with any/no framework. That uses pure DOM to get the job done. Might not look quite as pretty as Vue/Bootstrap but I don't think it looks bad. I'll be expanding on those kinds of features and will probably incorporate a pure Web Components package into uibuilder with a few, uibuilder-friendly components like the http-loader and syntax-highlight components already mentioned. As these can be used just like ordinary HTML tags, they are easy to use but they have uibuilder "smarts" built in so that they are even easier to use.

So I want and need to keep the core of uibuilder as "pure" as possible. But I still want to demonstrate how to create ever easier and lower-code solutions with uibuilder. I had hoped that Web Components would be the answer but it clearly isn't, at least for now. Maybe in a few years time if it continues to develop. So for the "higher-level" demonstrations and proofs-of-concept, I need to choose a framework that is low overhead, easy to use and mostly stays out of the way while still being well supported and actively developed.

What I like about Svelte:

Its dev server works seamlessly with uibuilder/Node-RED
Its build configuration is simple to configure for use with uibuilder
Its component model is familiar and easy to work with
It is highly compatible with Web Component standards
Its output is relatively small and tight
It doesn't require complex HTML to use
It has no need to add unfamiliar attributes to HTML
It has great support, is growing fast and importantly is liked by developers

It isn't a tool for every job by any means though. But it is a lot simpler to use than Vue or REACT (and I like Vue).

Bottom line is that uibuilder will always be framework agnostic. Now that my HTML/CSS/JavaScript skills are a lot higher than when I started this journey, I'm in a better place than ever to follow through on that approach. Also helps that these technologies continue to develop and improve.

But, I recognise that higher-level tools are also needed and I need to show the way if nobody else steps up to the mark. Somehow I need to develop those as well as uibuilder

Along the way, uibuilder will get some new web endpoints that I'm looking at that will significantly help with debugging and monitoring and will continue to get features that support data-driven web apps. It should be quite a ride

bakman2 · 3 April 2022 12:41

There is also petite-vue that removes build complexities, with disadvantages.

Alpine.js + tailwindcss are fantastic together. There are many pre-made components already available [1] [2]. Tailwind can be used in the browser directly as well with the JIT version (although postcss would be preferred in a production environment).

If you want to create something quickly that needs to look good and you know css, tailwind is a massive speedboost with quick results.

Personally, if anything needs to be build/packed I consider it a project larger than hobbiest stuff and will most likely be over-engineered (must watch ;)).

In the end: whatever works, works

TotallyInformation · 3 April 2022 14:39

Always looking for new examples to share

Yes, I must get round to trying it. Though in truth, I'm having a lot of success with plain CSS now that I've discovered and rediscovered some of the modern capabilities such as variables, :root and the browser light/dark indicators.

I've long thought that. However, Svelte is so well put together that it makes the process very easy. Now that uibuilder includes the Svelte Basic example, all you need is the template then to do an npm install then npm run watch and you are good to go. Then when you've done making changes, npm run build and that's it. And it does have the advantage that your HMTL tends to be simpler since you are breaking it up into logical components.

How true - should be the strapline for uibuilder!

HaroldPetersInskipp · 3 April 2022 17:37

I typically use a lot of statements like

node.error("msg.payload.special must be a boolean.");

in my code to debug. But I noticed if I use them in the index.js file in a uibuilder node, that they do not work as I would expect. How can I convert them for use with uibuilder?

I noticed that I can use

uibuilder.send({"payload":"msg.payload.special must be a boolean."});

but is that my only option?

TotallyInformation · 3 April 2022 20:18

Hi, node.error is a node-red function and can't be used in your front-end code as the browser doesn't understand it.

You can simply use console.error instead. However, that will show up in the browser dev tools console. You will find lots of such console output if you turn on the front-end debugging with uibuilder.debug(true) early in your index.js file.

That will send data back to Node-RED where you can view it using a debug node. In that case, you may prefer to add a specific msg.topic, something like

uibuilder.send({"topic: "uib-debug", "payload":"msg.payload.special must be a boolean."});

So you have a couple of choices.

I'm toying with the idea of a custom logging feature for the uibuilder front-end that logs back to Node-RED but is visible via a dedicated web url. That might have search/filter functions. Not entirely sure that it adds enough above what is already available though since you can already filter and search the browser console output.

HaroldPetersInskipp · 3 April 2022 21:07

I ended up using

uibuilder.send({"error":"The error message"})

sent into a switch that checks for msg.error is not empty then to a function node containing

node.error(msg.error)

This replicates the results I wanted exactly. Thank you for the help.

shrickus · 5 April 2022 03:36

Once again, your development path and mine have converged... I've been using the vNext branch for months now, but last week I started on a new project to use UIBuilder for serving up... tada... pure js/css/html web components!

My goal seemed straight-forward, although not simple: create a dir structure of web component modules like this:

src
├── index.css
├── index.html
├── index.js
├── modules
│   ├── loginView
│   │   ├── index.css
│   │   ├── index.html
│   │   └── index.js
│   └── verifyDeployment
│       ├── index.css
│       ├── index.html
│       └── index.js
├── resources
│   ├── failure.png
│   ├── favicon.ico
│   ├── info_sprite.png
│   ├── loading.png

Each directory in src/modules contains the 3 files that define the component's dom structure (index.html), the styling of that component (index.css), and the ui logic behind it (index.js).

The App is made up of the same components (dom, styling, logic) with simple <div>s used to hold each component -- for example, in this part of an accordion style installation wizard:

    <div id="login">
        <login-view data-package-name="Transport"></login-view>
    </div>

    <div id="wizard" class="container" style="display: none;">
        <form id="config-form">
            <div class="list-group">

                <div class="list-group-item py-2" data-acc-step>
                    <h4 class="mb-0" data-acc-title>Verify your Deployment
                        <span class="infoIcon" id="accVerifyOrgQ"></span>
                    </h4>
                    <div data-acc-content>
                        <div class="my-3">
                            <div class="form-group">
                                <verify-deployment></verify-deployment>
                            </div>
                        </div>
                    </div>
                </div>

                <div class="list-group-item py-2" data-acc-step>
                    <h4 class="mb-0" data-acc-title>Integration User
                        <span class="infoIcon" id="accMasterGroup"></span>
                    </h4>
                    <div data-acc-content>
                        <div class="my-3">
                            <div class="form-group">
                                <master-group></master-group>
                            </div>
                        </div>
                    </div>
                </div>
...

My hope was to be able to have uibuilder merge the individual files (easier for editing in VSCode) into something the browser could render -- without having to use webpack or another separate build step/tool. Alas, I quickly found out how little support there is actually is in generic html5 for actually making web components useful. Sigh

I also decided to migrate some of the nodejs api express endpoints (use by ajax calls to fill the web components with data) into proper node-red flows. However, with UIB v5 I'm on the fence with leaving them as express endpoints/routes (more traditional api coding), or porting them all to flow logic (easy to tweak as needed). And then there's the other possibility that I have not fully explored: using the socket.io link between uib server and the browser to pass events and data to the ported api flows without even using ajax at all. I guess the only way to truly determine which techniques work best for us is to write the same app all 3 ways... as if there was time for that!

All this is to say that you have again far exceeded what would normally be expected in a new release, by anyone with a full time day job -- bravo! If I can make some progress on the html5 only version of an app, I'd be happy to contribute it to your example library.

TotallyInformation · 5 April 2022 08:29

Ah, the age-old conundrum! I know exactly what you mean. Also further confused by HTTP/2 where you can do more socket style communications. Not yet supported by uibuilder but on the backlog.

I also hit this when I stumbled across the method for the uibuilder.setPing() function which uses modern Fetch.

What I would say though is that the two different methods have different scalability issues. The main thing to remember is that each http call has to do its own handshake unless you configure keepalive or http/2. Socket.IO only needs to do a handshake on initial connection and on reconnection after losing connection (for example if your browser puts the tab to sleep). For small to medium deployments, I doubt this makes much difference unless you have a complex session management back-end. For for large deployments, or perhaps more accurately, for deployments with very large numbers of users with a rapid turnover, this could be something to consider.

Thanks and de-nada.

In that case, I will go ahead and get the two that I've developed into their own GitHub repo and you can have a look at the style. I need to do that anyway since I may well bundle them with future versions of uibuilder.

I've a number of side projects on a slow simmer at the moment, looking at different ways to use uibuilder. One that peaked my interest was a content management system inspired in part by Docsify which I use to deliver the tech docs. This is leading to thoughts about dynamic loading of html using the http-loader component coupled possibly with a markdown converter. As you may remember, I wrote another node a while back that creates a markdown static CMS. This would be a lot more dynamic as you could combine the use of web components (and/or individually packaged Svelte/Lit components) and server-side rendering (which is in its initial form in v5).

TotallyInformation · 5 April 2022 20:34

Hi Steve, you may be interested to note that I've put together a new github repo as mentioned. It has instructions on installation though I've not yet documented the components properly. I've also started (see the README) to document the kinds of requirements needed to see a new component in the repo.

The components work with or without uibuilder but will always have additional features for use with uibuilder. As always, I'd love to get feedback and any contributions would be welcomed.

Because of the changes in uibuilder v5, installation using the uibuilder Library manager is a breeze directly from GitHub.

TotallyInformation · 6 April 2022 08:59

Oops, forgot the link!

Ashfaak · 7 April 2022 14:20

Hi Totally

I've updated to 5.0.1 and now my NR is broken.
In the Nodes Palette under node-red-contrib-uibuilder it shows

TypeError: Cannot read property 'setup' of undefined (line:286)

Node-red warning shows:

notification.warnings.undefined

I cannot see NR flows or nodes (all greyed out).

Any help would be appreciated.

Edit:
I've managed to downgrade to 4.1.4 using npm and it all works again.

TotallyInformation · 7 April 2022 22:50

What is your version of node.js? And are you using Docker?

Ashfaak · 8 April 2022 07:34

node.js v12.22.10
It's not running in docker.

TotallyInformation · 8 April 2022 08:11

OK, can you please check the node-red log from startup and see what errors you have. You might also have an npm error log.

Oh, and also please check the console of the browser dev-tools to see if there are any errors there as well.

Node.js v12.22.10 is a little marginal and I think it is about to go out of service shortly so we might need to think about getting you onto node.js v16.

Ashfaak · 8 April 2022 09:00

I've installed NR using the rpi script (not running on a pi) and updated with the same script. Never looked at nodejs version until now.

After updating 4.1.4 -> 5.0.1 and restarting:

Starting as a systemd service.
8 Apr 08:52:13 - [info]
Welcome to Node-RED
===================
8 Apr 08:52:13 - [info] Node-RED version: v2.2.2
8 Apr 08:52:13 - [info] Node.js  version: v12.22.10
8 Apr 08:52:13 - [info] Linux 4.19.0-18-cloud-amd64 x64 LE
8 Apr 08:52:14 - [info] Loading palette nodes
[uibuilder] REQUIRE SOCKET failed:: /home/admin/.node-red/node_modules/node-red-contrib-uibuilder/nodes/libs/socket.js:42
        socket.request?.headers['x-real-ip']
                       ^
SyntaxError: Unexpected token '.'
    at wrapSafe (internal/modules/cjs/loader.js:915:16)
    at Module._compile (internal/modules/cjs/loader.js:963:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (/home/admin/.node-red/node_modules/node-red-contrib-uibuilder/nodes/uibuilder.js:42:25)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
[uibuilder] REQUIRE WEB failed:: /home/admin/.node-red/node_modules/node-red-contrib-uibuilder/nodes/libs/socket.js:42
        socket.request?.headers['x-real-ip']
                       ^
SyntaxError: Unexpected token '.'
    at wrapSafe (internal/modules/cjs/loader.js:915:16)
    at Module._compile (internal/modules/cjs/loader.js:963:27)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
    at Module.load (internal/modules/cjs/loader.js:863:32)
    at Function.Module._load (internal/modules/cjs/loader.js:708:14)
    at Module.require (internal/modules/cjs/loader.js:887:19)
    at require (internal/modules/cjs/helpers.js:74:18)
    at Object.<anonymous> (/home/admin/.node-red/node_modules/node-red-contrib-uibuilder/nodes/libs/web.js:36:16)
    at Module._compile (internal/modules/cjs/loader.js:999:30)
    at Object.Module._extensions..js (internal/modules/cjs/loader.js:1027:10)
8 Apr 08:52:15 - [info] Dashboard version 3.1.6 started at /
8 Apr 08:52:16 - [warn] ------------------------------------------------------
8 Apr 08:52:16 - [warn] [node-red-contrib-uibuilder/uibuilder] TypeError: Cannot read property 'setup' of undefined (line:286)
8 Apr 08:52:16 - [warn] ------------------------------------------------------
8 Apr 08:52:16 - [info] Settings file  : /home/admin/.node-red/settings.js
8 Apr 08:52:16 - [info] Context store  : 'default' [module=localfilesystem]
8 Apr 08:52:16 - [info] User directory : /home/admin/.node-red
8 Apr 08:52:16 - [warn] Projects disabled : editorTheme.projects.enabled=false
8 Apr 08:52:16 - [warn] Flows file name not set. Generating name using hostname.
8 Apr 08:52:16 - [info] Flows file     : /home/admin/.node-red/flows_ip-172-31-17-66.json
8 Apr 08:52:16 - [info] Server now running at https://127.0.0.1:1880/admin/
8 Apr 08:52:16 - [warn]
---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.
If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.
You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------
8 Apr 08:52:16 - [info] +-----------------------------------------------------
8 Apr 08:52:16 - [info] | uibuilder v5.0.1 initialised
8 Apr 08:52:16 - [info] | root folder: /home/admin/.node-red/uibuilder
8 Apr 08:52:16 - [info] | Using Node-RED's webserver at:
8 Apr 08:52:16 - [info] |   https://0.0.0.0:1880/
8 Apr 08:52:16 - [info] | Installed packages:
8 Apr 08:52:16 - [warn] Error loading flows: TypeError: Cannot read property 'uibuilder' of undefined

Browser console:

GEThttps://example.com/menu/?s=lbh
[HTTP/1.1 404 Not Found 21ms]

Content Security Policy: The page’s settings blocked the loading of a resource at https://example.com/favicon.ico (“default-src”). resource:191:19

Topic		Replies	Views
[Update on progress] node-red-contrib-uibuilder vNext/v5 Share Your Nodes uibuilder	5	1069	17 February 2022
🌟 [Major Release] uibuilder v6.0.0 - it's here at last! Share Your Nodes uibuilder	7	548	5 March 2023
Uibuilder v2 - Here at last! Published to npm Share Your Nodes uibuilder	31	3920	22 October 2019
Uibuilder v6 beta - Ready to test Share Your Nodes uibuilder	4	457	1 December 2022
:boom: uibuilder v2: New update and near completion Share Your Nodes	3	848	26 June 2019