🌐 UIBUILDER new release v7.6.1: (Bug fix) Introducing Markweb and other enhancements

Share Your Nodes
1

Yes, at long last, a new version of UIBUILDER for your web pleasure!

Markweb is the outstanding feature of this release. A new and ultra-simple way to publish an entire website using simple Markdown files. It is a v1 release so there might be a few rough edges, I'm sure you will let me know if you find them. Also features are relatively limited in this release - this is not (yet) a full replacement for Astro or Hugo but it should be sufficient to produce simple documentation, blog, news, etc sites.

UIBUILDER now also can act as a development server that automatically updates when you make changes to your source files.

There is some branding clarification with some updated icons both in the Editor and in default web templates.

The documentation has also had some TLC applied. In particular, the navigation sidebar now has collapsible sections and the open/closed state is saved in your browser. Expect more improvements in the next release, I plan to move the page table-of-contents into the sidebar to match the layout of Markweb.

Some new documentation pages and example flows to play with as well.

Lots and lots of other updates, improvements and bug-fixes too, please check out the full changelog below.

The coding for Markweb, even with AI help, was far more complex than anticipated. It also threw up some interesting issues and challenges for some of uibuilder's libraries. Got there in the end though! :man_mage::smiley:

Have fun!
Julian.

Code commits since last release.

:pushpin: Highlights

  • In the Node-RED Editor, a popover is now shown after a UIBUILDER update. It contains highlights of the changes in the new version. It will only be shown once per version update.

  • NEW NODE markweb - Enables simple creation of dynamic web sites using Markdown files. It supports navigation menus, search, front-matter placeholders, custom templates and much more.

  • If you have set the Reload connected clients on save? option on the uibuilder node's File tab, clients will automatically reload the visible page not only if you change a file in the file Editor but also if you make changes elsewhere such as using an external editor or the uib-save node. So you now have the option of a full development server workflow.

  • Two new example flows. "Built-in Web Components" and "Easy UI Updates".

  • The various uibuilder icons now have more open names in the palette. In addition, the icons have all been updated to use customised, coloured icons. In particular, the uibuilder and markweb primary nodes now use :globe_with_meridians: and :spider_web: icons respectively which is in line with their log and debug outputs.

  • The uibuilder docs, front-end templates and markweb page template now all have different favicons to help differentiate them in the browser.

  • Default Content-Security-Policy (CSP) header updated for improved browser security. You can override it in your settings.js file if needed.

Front-end

  • uibuilder.get() and uibuilder.set() functions now support deep object paths. This allows you to get and set nested properties of reactive variables without needing to replace the entire variable. e.g. myvar.myprop, myvar.myprop.subprop or myvar[5].

  • All HTML custom elements and reactive attributes also now support nested object paths. e.g. <uib-var topic="myvar.myprop"> or <div uib-var="myvar.myprop.subprop">. This allows you to directly bind to nested properties of reactive variables without needing to replace the entire variable.

  • Updated uibuilder.onTopic() to process control messages as well as standard ones. Added so that the <uib-var topic="..."> component and uib-topic attribute can listen for control messages.

  • <uib-var> component now recognizes data-before and data-after attributes. Useful for adding units, labels, or other contextual information around the variable value without needing extra HTML elements.

  • New uib-var custom reactive HTML attribute added. Mostly of use for <meta>, <title>, <link> elementts in the <head> HMTL section.

  • If DOMPurify is used, it is now configured to allow custom web components, since uibuilder now makes more extensive use of them.

  • New stack and logStack functions. These can be useful for debugging and understanding the flow of your code especially with complex and deeply nested functions.

Documentation

  • Mermaid diagrams are now supported in UIBUILDER's documentation. With the first example being for the new markweb node's page.
  • The Roadmap has now been split into separate pages as it was totally unmanageable as a single page. You can use the links on the main Roadmap page to navigate to the different sections.

New node: markweb

Create a folder containing at least an index.md file for each (sub-)folder. Add the new node and configure the base URL and source folder.

The node will serve the markdown files as HTML pages using Single-Page Application (SPA) style navigation. The markdown is converted to HTML and inserted into a page template.

The Markdown conversion is done on Node-RED startup and re-done when a source page changes. So loading and navigation from the browser remains very fast.

It supports YAML front-matter in the markdown files. All front-matter attributes are available as placeholders in the page template and in the markdown using {{attributeName}} tags. Special instructions (directives) are also available to create navigation menus, index listings, search results, etc. These use %%...%% syntax.

CommonMark and GitHub Flavored Markdown (GFM) are supported. Syntax highlighting for code blocks is also included. Some additional extensions are also supported such as custom attributes.

An optional web component <show-meta> is also provided to display the current page's metadata (front-matter attributes). This is useful for debugging and development.

See the node documentation for full details.

Examples (Node-RED library flows)

  • NEW Built-in Web Components

    Shows how to use the uibuilder client web components: <uib-meta>, <uib-var>, <apply-template>, and <uib-control>. These are all included in the main client library and do not need to be loaded separately.

  • NEW Easy UI Updates

    Illustrates the use of the different ways to update the UI from Node-RED or your own front-end JavaScript. Includes examples of using the (new) uib-var and (existing) uib-topic custom HTML attributes, and the existing <uib-var> web component.

Documentation

  • The sidebar of documentation page links now scrolls the current page link into view.
  • The sidebar top-level entries that have children are now collapsible sections. Added because the documentaiton continues to grown.
  • The sidebar expand/collapse state for each section is remembered across page loads.
  • Each page now automatically shows status and/or since front-matter.
  • Fixed Issue #575 - Broken CSS loads.
  • Improvements to developer detailed documentation including details on the uibuilder/markweb instance setup. Should make things a lot easier if other developers want to take part in the future.
  • Mermaid diagrams are now supported. With the first example being for the new markweb node's page.

uib-cache node

  • Added several techniques to reduce resource overheads when the cache is receiving very large numbers of inputs.

uib-elements node

  • FIXED Issue 580 - Missing data could cause a Node-RED crash. Additional checks and try/catch trap added.

uibuilder node

  • NEW Function uibuilder.getCommandList(). This function returns a list of available commands that can be sent from Node-RED to the front-end client.
  • FIXED VScode link not properly updating on node url change.
  • Added a new config variable instancePath. This is the first change that will eventually allow uibuilder instances to use a different server folder than <uibRoot>/<url>.

uib-var component

  • FIXED Not updating when the given variable is zero (0). Ref.

uib-brand.css front-end styles

  • Added .visually-hidden class for elements hidden from sighted users but still accessible to screen readers. Use for skip links, form explanations, and status updates otherwise not needed for sighted users.
  • FIXED Misconfigured fieldset border.
  • Tweaks to blockquote and code styles for better appearance.
  • All z-index values changed to use CSS variables. This allows you to easily change the z-index values in your own CSS if needed.

uibuilder client library

  • NEW Function formatDate. This uses the Intl API to format dates according to locale and optional pattern. See the documentation for details. Really useful as a filter function and works great with the new markweb features.

  • NEW Functions

    • randomUUID. This function generates a random UUID (Universally Unique Identifier) using the browser's crypto.randomUUID() method if available. If not available, it falls back to a simple implementation that combines the current timestamp and a random string. This can be useful for generating unique IDs for elements, messages, or other purposes in your front-end code.

    • asyncSend. This function allows you to send a message to the server and wait for a response. It returns a promise that resolves with the response message. This is useful for request/response patterns where you need to get data from the server before proceeding. Ref.

    • stack and logStack. These functions allow you to log a stack trace to the console. stack returns the stack details, while logStack logs the details as a regular console log message. This can be useful for debugging and understanding the flow of your code especially with complex and deeply nested functions.

  • NEW Added the _receivedHRtime property to messages received from the Node-RED server. This is a high-resolution timestamp (in milliseconds) of when the message was received. It can be used to measure latency and performance. It uses the performance.now() method which provides sub-millisecond accuracy. The value is the elapsed time since page navigation started.

  • NEW Added instanceFolder to uibuilder node settings. This to bring it into line with the new markweb node, in the future, allow instance root folders to be specified anywhere.

  • NEW httpHeaders property added. This contains the HTTP headers received when the front-end client first connects to the server. This can be useful for debugging and for advanced use cases such as authentication and user tracking. Async so issues a custom event when ready. The start() function is now not called until they are ready because the headers are the most reliable way to get the namespace and Node-RED web root (stupid Firefox refuses to read the cookies!).

  • <uib-var> component now recognizes data-before and data-after attributes. These allow you to specify text to show before and after the variable value. This is useful for adding units, labels, or other contextual information around the variable value without needing extra HTML elements.

  • NEW Custom reactive HTML attribute uib-var added. Similar to the component above but much simpler processing. However, it allows you to control any HTML element.

    NOTE: Does not (yet) support any of the extended HTML attributes that <uib-var> supports such as data-before, data-after or filter.

    May be of most use on HTML elements that don't allow HMTL content.

  • uibuilder.get() and uibuilder.set() functions now support deep object paths. This allows you to get and set nested properties of reactive variables without needing to replace the entire variable. e.g. myvar.myprop, myvar.myprop.subprop or myvar[5].

  • Updated uibuilder.onTopic() to process control messages as well as standard ones. Added so that the <uib-var topic="..."> component and uib-topic attribute can listen for control messages.

  • If DOMPurify is used, it is now configured to allow custom web components, since uibuilder now makes more extensive use of them.

  • Tidied up unnecessary async processing in the DOM Mutation Observer. Giving a minor performance boost.

  • Delayed startup of DOM observing so that scripts loaded after the client library have time to set initial values.

  • 2 new managed variables added, reconnect and initialConnect. reconnect is the more useful, it contains a count of the number of times the client has reconnected to the server. initialConnect is a boolean that is true until the first successful connection to the server is made. Use if (uibuilder.get('reconnect') > 0) to trigger actions on reconnects but not on the initial connection.

  • Improved processing of uib-var custom attribute processing when using object property paths. e.g. myvar.myprop or myvar[5]. This may previously have been inconsistent.

Development changes

  • SECURITY IMPROVEMENTS

    • Added minimum package age requirements to help prevent supply chain attacks.

    • Default Content-Security-Policy (CSP) header updated for improved browser security. You can override it in your settings.js file if needed. New default is:

      default-src 'self' 'unsafe-inline' data: blob: https:; connect-src 'self' ws: wss:; img-src 'self' data: blob: https:; font-src 'self' data: https:; style-src 'self' 'unsafe-inline' data: blob: https:; script-src 'self' 'unsafe-inline' 'unsafe-eval' blob: https:;

    • syntaxHighlight function in tilib.cjs - limited the size of JSON strings to 10k characters to prevent potential denial-of-service attacks via extremely large JSON payloads.

    • admin-api-v2.cjs and admin-api-v3.cjs - ensured that parameters expected to be strings are not arrays. This prevents potential injection attacks via array parameters.

  • NEW npm script bugfix-worktree - creates a new git worktree for bugfix branches. This enables work on a bug fix in a separate directory while keeping the current dev branch work intact. Both directories can be open simultaneously without needing to stash changes or switch branches. When done with the bug fix, commit, push, create a PR, and then remove the worktree.

  • NEW Added npm workspaces under folder packages. This is to allow easier management of shared utility packages.

    • NEW workspace private package @totallyinformation/uib-md-utils - A collection of Markdown utility functions that can be shared between uibuilder's server and front-end client libraries. The package bundles its own dependencies using ESBuild.
    • NEW workspace private package @totallyinformation/uib-mf-utils - Filing System utility sub-pages. Currently only chokidar to allow extended FS watch for markweb. The package bundles its own dependencies using ESBuild.

  • NEW Control message added. msg.uibuilderCtrl = 'internal' allows internal control messages to be sent to nodes.

    Provides a hook such that receipt of a control msg can trigger a node process. Requires msg to include {uibuilderCtrl: 'internal', controlType: 'someControlType', ...}. The node can then define its own internal control message handlers in the node.internalControls object.

    WARNING: Carefully validate all inputs.

  • nodes\libs\admin-api-v3.cjs - Removed reference to node:inspector which is not used. ref.

  • Removed DEP0190 error when using node.js v24+ by removing shell:true from child_process spawn calls and replacing with an OS explicit shell command.

  • nodes/libs/web.cjs

    • Made instanceSetup more flexible by adding routeSpec and handler (function) arguments. This allows different types of routes to be added for an instance, e.g. static file serving, dynamic routing, markdown rendering, etc.
    • Also in instanceSetup, reduced the number of routes added to an insance if the node is not a uibuilder node. Allows for simpler nodes such as markweb.

  • nodes/libs/tilib.cjs

    • Added formatDateIntl function. This formats a date using the Intl API with a given format string and locale. Underscores in the format string are replaced with spaces. Formate uses standard date/time formatting tokens such as YYYY, MM, DD, HH, mm, ss, etc.

  • nodes/libs/fs.cjs

    • Added options argument to the fgSync function. Allows for exclusions and other options to be passed to the underlying fast-glob library.

  • Updated ESLINT - now includes eslint-plugin-security to catch potential security issues in the code. Also updated the ESLINT configuration to reflect the new plugin and added some additional rules for better code quality and security.

2

...and breath! WOW, that looks a lot!!

3

I know, I keep promising myself that I'll do shorter releases. But then some big idea comes into mind and I can't stop myself! :smiley:

Thankfully, as usual, there shouldn't be anything too disruptive in there so existing sites should continue to work as normal.

4

I have a tinsy problem. I use material icons which I load using https://cdn.jsdelivr.net/npm/@mdi etc.. However, I now get an error message Content Security Policy directive: "connect-src 'self' ws: wss:"

Firstly - in the documents it says that this can be overcome by editing settings.js but.. it does not say how. (or possibly I just can't see it). So would you explain - hopefully step-by-step - how to do this. As an extra - is it possible to only allow designated URIs.

Secondly - if this is an unsafe operation how would I do it in a safe way?

A note on the documentation - amazing level of detail - when in light mode the <strong></strong> elements are invisible. (MS Edge)

Finally - with the introduction of the asyncSend() function you have now made my life difficult. I currently use the uib.send() - uib.onTopic() - uib.set() - uib.propertyChanged() loop to get updated data for preloading a form, so now I have to decide if re-writing things to use asyncSend() is a better option :thinking: (and unfortunately with the built in error catch it probably is)

5

Ah, I knew I needed to but some more text in about that.

I need to do some research to work out how to fix if you aren't using a custom web server. Let me get back to you.

6

The safe way is to define which remotes you want to be able to load and for what.

Ah, I never test light mode any more! I probably should just kill it for the docs. :smiley:

Ha :smiley:

Good news - you don't need to change anything you don't want to. :wink: My advice would be not to bother changing anything not worth the effort.

7

Ah ha, but the question then is - how do I do that?

Thank you, I await your reply :laughing:

8

Unfortunately, it seems we have a problem Huston.

You should be able to override the CSP header using middleware. But it looks like it isn't doing that. :frowning:

I'm guessing this is an order-of-operations thing. It will need some more digging.

If you need an urgent fix, you can simply hack the source. You can find that in ~/.node-red/node_modules/node-red-contrib-uibuilder/nodes/libs/web.cjs.

At line 733, you have:

                    'Content-Security-Policy':
                        "default-src 'self' 'unsafe-inline' data: blob: https:; "
                        + "connect-src 'self' ws: wss:; "
                        + "img-src 'self' data: blob: https:; "
                        + "font-src 'self' data: https:; "
                        + "style-src 'self' 'unsafe-inline' data: blob: https:; "
                        + "script-src 'self' 'unsafe-inline' 'unsafe-eval' blob: https:; ",

Change that to just read:

'Content-Security-Policy': '*',

I will do some kind of fix release. Probably to allow an override parameter in settings.js.

9

I found the issue. Hopefully will release a v7.6.1 tomorrow.

Thanks for finding a bug for me :smiley:

10

Aren't you are glad you are retired :grin:

I will wait on the fix - I have just not imported the file for the time-being. It will be handy because at some point I will be using eCharts.

Well I did try that, and I am not going to do it again. Node-RED decided that the uibuilder node was no longer present. I had to delete all the uibuilder nodes, remove and reinstall uibuilder. I had a backup of all the src files - just in case, but as it happens once I replace the missing nodes, used the same URL, all the files were readopted. Wheew :sweat_smile:

11

Yep, that was a fairly early feature I made sure of. :smiley:

Of course, since uibuilder simply uses the same folder name as the url you specify for the node, you could also reconnect things manually if you got things really messed up. Stop Node-RED, manually rename the instance folder and restart. No magic involved.

12

And it is now available! Please update to v7.6.1

To override the CSP, you can rename ~/.node-red/uibuilder/.common/uibMiddleware.js-template to uibMiddleware.js.

Then add the CSP header into the provided function.

function uibMw(req, res, next) {

    console.log('[uibuilder:uibMiddleware.js] Custom ExpressJS middleware called.')
    res.setHeader('X-Custom-JK-Header', 'This is a custom header set by uibMiddleware.js')

    // Define your policy string
    let csp =
        "default-src 'self' 'unsafe-inline' data: blob: https:; "
        + "connect-src 'self' ws: wss:; "
        + "img-src 'self' data: blob: https: https://cdn.jsdelivr.net; "
        + "font-src 'self' data: https:; "
        + "style-src 'self' 'unsafe-inline' data: blob: https:; "
        + "script-src 'self' 'unsafe-inline' 'unsafe-eval' blob: https:; "
    // csp = '*' // Disable CSP - not recommended for production but can be useful for development and testing
    // Set the header
    res.setHeader('Content-Security-Policy', csp)
    
    next()

} // Do not forget to end with a call to `next()`

You may have to play with the CSP somewhat to get what you want but hopefully it is fairly clear. Each resource type has some settings that end with a ;.

I've shown an example of adding https://cdn.jsdelivr.net to the image resource allowance. Though, in theory it shouldn't be needed since https: is already in the default.

I am by no means an expert in CSP's so you might have to get some more expert help if the obvious doesn't help. Keep us updated though so we can all learn!

You asked about "safety" before. Hopefully you can see that the CSP can lock down what the browser will accept. And note that this IS a browser instruction, not a server one. That's to say that the CSP informs the browser what it should process.

Quick note on the terminology in the CSP:

  • self - the browser will process stuff it has loaded internally.
  • unsafe-inline - the browser will process directives added to HTML code such as styles for example.
  • unsafe-eval - will process JavaScript eval and similar potentially unsafe functions. This is included in the default since, when creating a generic platform library, it is sometimes impossible to avoid having to interpret text as code. The uibuilder client library does have a couple of places where it does this though I've taken care to limit the process and made it as safe as possible.
  • data: - can process data: URI's
  • blob: - can process binary blobs

You can take out things you know you won't need for additional safety.

13

Is there a beginners guid to UIBUILDER with examples etc...?
Perhaps even some video tutorials...?

thanks

14

Absolutely!

Check out the documentation which is available both inside Node-RED and on the Internet.

And my Video playlist on YouTube:

The documentation has both a Quick-start and a beginners walkthrough page. The walkthrough has a matching video.

There are also quite a few example flows in the Node-RED import library.

image
image972×739 33 KB

15

Oh, and I'll (hopefully!) be putting up a video about the new Markweb feature to YouTube very soon.

16

Many thanks, I'll take a look :+1:

17

Brilliant. And don't forget that I'm generally lurking around in the forum so feel free to fire some questions my way. :slight_smile:

18

If you don't already know, it might be worth taking the time to look at some of the examples under Import|Examples (node-red-contrib-uibuilder) in the Node-RED top bar menu. (Hamburger)

19

So, for those, like me, who have no idea what CSP is - here is an explanation from Co-pilot??
(well MS Edge anyway)

Content Security Policy (CSP) is a powerful web security feature designed to mitigate a wide range of attacks, particularly Cross-Site Scripting (XSS) and data injection attacks. It works by defining a set of rules (directives) that specify which sources of content (e.g., JavaScript, images, CSS) are allowed to load and execute on a web page. By enforcing these rules, CSP helps prevent malicious scripts or resources from being executed, even if an attacker manages to inject them into the page.

How CSP Works

CSP is delivered to the browser via the Content-Security-Policy HTTP response header or a tag in the HTML. The policy consists of directives that control specific resource types. For example:

Content-Security-Policy: default-src 'self'; script-src 'self' https://trusted.com
Copy
This policy:

Allows all resources to load only from the same origin ('self').

Permits JavaScript to load from the same origin or https://trusted.com.

When a browser encounters a resource that violates the CSP, it blocks the resource and logs a warning in the console. Optionally, violations can be reported to a specified endpoint for monitoring.

Key Features of CSP

Protection Against XSS: CSP prevents the execution of malicious scripts by: Restricting inline scripts unless explicitly allowed via nonces or hashes. Blocking scripts from untrusted sources. Disabling unsafe JavaScript functions like eval().

Resource Control: CSP directives like script-src, img-src, and style-src define trusted sources for JavaScript, images, and styles, respectively. A fallback directive, default-src, applies to all unspecified resource types.

Clickjacking Defense: The frame-ancestors directive prevents unauthorized embedding of your site in iframes, protecting against clickjacking attacks.

Upgrading Insecure Requests: The upgrade-insecure-requests directive ensures that all HTTP requests are automatically upgraded to HTTPS, reducing the risk of mixed content vulnerabilities.

So what I had to do to stop the failed on "connect-src 'self' ws: wss:; " error.

  • Stop Node-RED
  • Edit ~/.node-red/uibuilder/.config/uibMiddleware.js-template Note .config not .common :grin:
  • Edit line + "connect-src 'self' ws: wss:; " to + "connect-src 'self'; "
    Also, since the file I was loading was a font file from https://cdn.jsdelivr.net I amended the + "font-src 'self' data: https: " to + "font-src 'self' data: https: https://cdn.jsdelivr.net; " No real idea if this is helpful / more secure. Probably not needed due to already having https: :confused:
  • Rename file to uibMiddleware.js
  • Restart Node-RED.
    If you still have the uibuilder web page open you may get a load of red error messages. Ignore them and just reload the page.

That seems to have fixed the problem. It may be that ws: and wss: can be added somewhere else but for the moment my problem is fixed.

20

Hmm, OK, odd. Perhaps I misunderstood something when I set the default up. I'll check.

Anyway, thanks for sharing this, very helpful. I've made a note to add some instructions to the docs for the next release.