Uibuilder: Working towards v2 - dev3 released - notification of breaking changes

[TotallyInformation]

Trying my best to get v2 out of the door!

the dev3 release of the v2 build is now suitable for trying. As usual, please install using:

npm install TotallyInformation/node-red-contrib-uibuilder#v2 --save

Lots more bugs squished this time around, hopefully no new ones! The file editor now has new and delete buttons working. And lots more, please read the updated readme file and the v2-progress file. Issue #46 on GitHub also tracks progress.

---

To get you thinking about the release though, I thought I would share a summary of the breaking changes in v2 over v1. This seems like a lot, sorry about that but I wanted to get everything out of the way at once so that you have a stable platform to use from now on.

None of the following really affects you if you are just getting going with uibuilder, you will just find things simpler and more flexible than ever!

Summary

It boils down to a couple of things though, especially if you use standard front-end libraries. You will need to make some tweaks to your html files to pick up the vendor libraries from the right locations. You also now need to start the communications going manually so add the uibuilder.start() function as early as possible to your code (after your other libraries have finished loading).

Details

No more settings files!

Neither the old settings in <userDir>/settings.js nor the "new" ones in <uibRoot>/.settings.json are needed any more.

This greatly simplifies configuration.

Managing front-end library packages

Many front-end library packages are now auto-discovered: vue, bootstrap, bootstrap-vue, jquery, moonjs, reactjs, riot, angular, picnic, umbrellajs

Packages that are not auto-discovered must now be installed using the node's admin UI.

If you previously had a package installed and configured (via settings.js) that isn't in the list above, you will need to "reinstall" it via the admin UI. This won't actually reinstall but will make the configuration fall into line.

New URI path available ../uibuilder/common/

This is used to serve your own common resources (available to all instances of uibuilder).

It maps to the <uibRoot>/common folder which is created if it doesn't exist.

Unfortunately, you cannot yet edit files in here from the admin ui.

Node-RED's middleware no longer used

httpNodeMiddleware in <userDir>/settings.js Used by Node-RED's core http-in node.

Middleware is mostly used for security processes to be added into the web server processing (e.g. JWT).

Previously, uibuilder would make use both of the middleware defined in httpNodeMiddleware and its own middleware defined in the uibuilder section of settings.js. Neither of these are now available.

This is now replaced by ...TBC - this is one of the last parts I need to finish...

Front-end library packages only loaded once

When you load up a 3rd-party (vendor) npm package such as REACT, VueJS, etc. deployed to <userDir>/node_modules. Each is made available to your front-end code via the Node-RED web server.

This used to be done for every instance of a uibuilder node which wasn't terribly efficient if you use many of them. This is now only done once.

The main impact is a change to the URL's used by your front-end code.

Whereas you previously used something like: <link rel="stylesheet" href="./vendor/normalize.css/normalize.css">

You now need to use: <link rel="stylesheet" href="../uibuilder/vendor/normalize.css/normalize.css"> (note the double leading dots)

Script links need to change similarly.

Any link that started like ./vendor must be changed to ../uibuilder/vendor.

Using this new URL format avoids any issues with the Node-RED setting httpNodeRoot which alters the URL path for non-admin resources.

Socket.io client library URL has changed

Socket.io client URL now matches the pattern for all other vendor libraries: ../uibuilder/vendor/socket.io/socket.io.js (/vendor added).

You will need to change your index.html file and any others you have defined.

All vendor packages deployed to userDir/node_modules and made available by uibuilder are now served using the same pattern: ../uibuilder/vendor/<moduleName>/.

Minimum supported version of Node.JS

The minimum supported version of Node.JS is now v8.5

uibuilder Node.js programming makes increasing use of ES6 features available after Node.js v8.5.

Note that all front-end code including uibuilder.html (the admin ui) and uibuilderfe.js (the front-end library) using ES5 only in order to retain maximum compatibility with browsers.

Server logging/debugging

Detailed logging of uibuilder on the Node-RED server should rarely be needed any more. So all of the settings have been removed and you can delete any log files.

There are plenty of log outputs however still in the code but no interface to turn on logging any more.

Set Node-RED's logging level to debug or trace to see detailed logging.

Front-end message trace variables

Previously, it was quite hard to trace incoming/outgoing messages in your front-end code, especially the difference between control and standard messages.

The variable sentMsg (uibuilder.get('sentMsg'))now only contains a copy of the last standard message sent back to the Node-RED server. sentCtrlMsg is a new variable that contains a copy of the last control message sent.

In addition, the variable msgsSentCtrl is now actually being updated.

This has a knock-on impact to the default index.js file.

You will need to change the

Server file locations

If using Node-RED's "projects" feature, each project now gets its own uibuilder folder.

Without projects, this is located at <userDir>/uibuilder/.

With projects, it will not be located at <userDir>/projects/<projectName>/uibuilder/.

This location will now be referred to as <uibRoot>. See below for other files and folders that have been moved to <uibRoot>.

Default Front-end template code has changed

The default templates have changed from jQuery+normalize.css to VueJS+bootstrap-vue.

Vue, bootstrap and bootstrap-vue are automatically installed for you.

You may remove the jQuery and normalize.css packages if you aren't using them for anything else (removing jQuery from <userDir> does not impact the admin ui which also uses it).

To get the new code, make sure that the appropriate flag is set in advanced settings in the admin UI then delete the index.html, index.js and index.css files. Alternatively, just rename them. The new files will be copied over into the correct folder.

The uibuilder front-end no longer self-starts

In previous versions, the uibuilder front-end library (uibuilderfe.js) self-started itself.

While this made its use very simple, it did add one critical limitation. Once communications between your front-end and Node-RED started, you couldn't change the path.

This meant that it was impossible to server a uibuilder front-end page from a different server or even to use a sub-folder in the node's src code folder.

Now, you have to start the library manually using:

uibuilder.start()

But if you want to have a web page in a sub-folder, you will need to call it like this:

uibuilder.start(namespace. ioPath)

The namespace will be something like <httpNodeRoot>/<uibUrl>.

The ioPath will be something like /<httpNodeRoot>/uibuilder/vendor/socket.io

You can find these out by looking at a normal, default uibuilder page or by looking at the details admin page which is accessible from the admin ui for any uibuilder node under the "Path & Module Details" section.

To serve a uibuilder page from a different web server, in addition to the change above, simply load the socket.io client from the one provided by uibuilder. (NOTE not yet tested, sorry).

[tmichaeltx]

@TotallyInformation

But if you want to have a web page in a sub-folder, you will need to call it like this:

uibuilder.start(namespace. ioPath)

The namespace will be something like <httpNodeRoot>/<uibUrl> .

The ioPath will be something like /<httpNodeRoot>/uibuilder/vendor/socket.io

I have looked at this post, the wiki, and the comments in the default index.js:

/** REQUIRED Start uibuilder comms with Node-RED @since v2.0.0-dev3
* Pass the namespace and ioPath variables if hosting page is not in the instance root folder
* The namespace is the "url" you put in uibuilder's configuration in the Editor.
* e.g. If you get continual uibuilderfe:ioSetup: SOCKET CONNECT ERROR error messages.
* e.g. uibuilder.start('uib', '/nr/uibuilder/vendor/socket.io') // change to use your paths/names
*/

I clearly am not interpreting these directions correctly. Every variation of what I think this might mean fails to allow me to connect.

My base url is uibuilder6

My sub path is m2 thus the page URL is http://cs:1980/uibuilder6/m2/

And this is what I get:

What should I use in place of uibuilder.start() and why?

In case it matters, I trying to use bootstrap-vue. I think I am in part confused because the examples make some reference to moon and to nr. I don't think those are relevant to my case, but haven't been able to figure it out.

[TotallyInformation]

Part of the problem is that you need to know whether you are using the httpNodeRoot setting which changes the URL paths. I always leave that set to "nr" on my dev machine to make sure that I'm always testing it.

So for me, using your example, my full URL would be http://cs:1980/nr/uibuilder6. That is where the nr part comes from. If you aren't using httpNodeRoot, you don't need to be concerned about that.

The other part of the problem comes from how the Socket.IO library works. We have to know what Socket.io namespace has been created. For the default index.js, there is a whole set of code baked into the uibuilder library to work that out because, again, it can change based on things like the httpNodeRoot setting. But when you use a sub-folder, uibuilder can no longer work out what the Socket.IO namespace is and you have to tell it explicitly instead.

You can always find out these things by looking at the Detailed Information page - you can get to that from the Editor as I've included a button that will open it in a new tab. On that page, look for the "Configuration" section and uib_socketPath.

In most cases, the namespace will be /uibuilder/vendor/socket.io.

Using this feature it is even possible to use uibuilder from a completely different web server as well. So you could use that other web server to serve up your static pages for example but still use the Socket.IO communications back to Node-RED.

[tmichaeltx]

When I click the "Show Detailed Information" button in the editor I get taken to this URL:
http://cs:1980/uibindex
Which just says: Unauthorized

Any thoughts on that?

I think in another thread I saw you say that you aren't using projects, which I am. This may be part of the challenge, but that is more supposition on my part than anything else.

[TotallyInformation]

You are using authorisation on the Editor and haven't given your logged in user sufficient rights. Try giving it "*" rights.

[tmichaeltx]

I have found this in .config.json:

"mike": {
   "editor": {
    "view": {
     "view-show-grid": true,
     "view-snap-grid": true,
     "view-grid-size": "20",
     "view-node-status": true,
     "view-node-show-label": true,
     "view-show-tips": true
    }
   }

And I found this in setting.js:

// Securing Node-RED
    // -----------------
    // To password protect the Node-RED editor and admin API, the following
    // property can be used. See http://nodered.org/docs/security.html for details.
    //adminAuth: {
    //    type: "credentials",
    //    users: [{
    //        username: "admin",
    //        password: "$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN.",
    //        permissions: "*"
    //    }]
    //},
    adminAuth: {
        type: "credentials",
        users: [
            {
                username: "tmichael",
                password: "$2a$08$...rest removed to not share publicly",
                permissions: "*"
            }
        ]
    },
    // To password protect the node-defined HTTP endpoints (httpNodeRoot), or
    // the static content (httpStatic), the following properties can be used.
    // The pass field is a bcrypt hash of the password.
    // See http://nodered.org/docs/security.html#generating-the-password-hash
    //httpNodeAuth: {user:"user",pass:"$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN."},
    httpNodeAuth: {
        user: "tmichael",
        pass: "$2a$08$...rest removed to not share publicly",
 //mike addition for to try to fix permisions for uibuilder
        permissions: "*" 
 //End of mike addition for to try to fix permisions for uibuilder
    },
//httpStaticAuth: {user:"user",pass:"$2a$08$zZWtXTja0fB1pzD4sHCMyOCMYz2Z6dNbM6tl8sJogENOMcxWV9DN."},
        //mike additon for uibuilder part 2
    httpStaticAuth: {
        user: "tmichael",
        pass: "$2a$08$...rest removed to not share publicly",
        permissions: "*" 
        //End of mike additon for uibuilder part 2
    },

In the httpNodeAuth section I added permissions: "*" , and I had previously not had an entry for httpStaticAuth and so added what you see here. I also tried it without the permissions * in httpNodeAuth and httpStaticAuth sections as that seems to not be standard per the node red documentation.

Any idea what I am doing wrong in giving myself the correct permissions?

[tmichaeltx]

Having found this in a separate thread:

Uibuilder uibindex not working

Try adding the defualt permissions to settings.js: adminAuth: { type: "credentials", users: [ /* list of users */ ], default: { permissions: "read" } }

I added default: { permissions: "read" } and now it it giving me access. Ironically of course the page itself says Note that this page is only accessible to users with Node-RED admin authority. and yet the permission I added was read.

Now I will spend some time understanding all of the info that page shows and how to apply it to my situation.

[TotallyInformation]

Hi, yes you found the discussion that I was going to link for you.

As you will see from that, we never did get feedback on what I saw as a number of inconsistencies (or at least my serious lack of understanding) in the way the Node-RED permissions are applied.

There was also an issue but it self-closed after inactivity.

I'm still not happy with the way that it works but at least there is a solution.

[tmichaeltx]

Having now fixed the permissions part to get to http://cs:1980/uibindex, I have used the information to get the following info:

So as you said since I am not using httpNodeRoot , I don't need to be concerned about that.
Then also on that page I have this:

So based on all of that I thought the correct entry was this:

        uibuilder.start('uibuilder6', '/uibuilder/vendor/socket.io')

I also tried

        uibuilder.start('uibuilder6/m2', '/uibuilder/vendor/socket.io')

and

        uibuilder.start('uibuilder6', '../uibuilder/vendor/socket.io')

and

        uibuilder.start('uibuilder6/m2', '../uibuilder/vendor/socket.io')

All of them failed to connect. Next suggestion for what I am missing?
I am also trying it under uibuilder7/m2 and I am seeing this doing inspection in Edge:

The URL I pointed to seems like a clue to what is wrong.

This is what it looks like in the working uibuilder 7

[TotallyInformation]

What that is telling us is what we already knew. That you are loading the socket.io client but that the client is not managing to connect to the namespace.

Just did some testing on my dev machine.

I have a uibuilder instance with a url of uib. Here is the layout of the files:

And in that file uib/src/test/index.js I have the following start command in the mounted function:

uibuilder.start('/uib', '/uibuilder/vendor/socket.io')

As you can see, I steered you right on the 2nd parameter but not on the first. That needs to start with a leading slash. Apologies, I got my parameters mixed up. It is the first parameter that is the namespace and the second is the path to the socket.io client.

The namespace is the url with a leading slash. The easiest way to discover that is to open a standard root index.html, open your browser's developer console. Then do uibuilder.get('ioNamespace') on the console. But of course, you weren't to know that and you've made me realise that somehow I'd missed properly documenting the settings.

For the next release, I've updated the documentation and the default template so that the parameters are correctly explained.

[tmichaeltx]

Thanks for the info from you test. I thought I had tried every variation myself, but sure enough adding that slash to create uibuilder.start('/uibuilder7', '/uibuilder/vendor/socket.io') was all that it took to get it working.