Options for simplifying custom node runtime code?

Yes, this is the relevant gulp config:

/** Combine the parts of ti-template.html
 * @param {Function} cb callback
 */
function buildTiTemplate(cb) {
    try {
        src('src/ti-template/main.html')
            .pipe(include())
            .pipe(once())
            .pipe(rename('customNode.html'))
            .pipe(htmlmin({ collapseWhitespace: true, removeComments: true, minifyJS: true }))
            .pipe(dest('nodes/ti-template'))
    } catch (e) {
        console.error('buildTiTemplate failed', e)
    }

    cb()
}

/** Combine the parts of ti-dummy.html
 * @param {Function} cb callback
 */
function buildTiDummy(cb) {
    try {
        src('src/ti-dummy/main.html')
            .pipe(include())
            .pipe(once())
            .pipe(rename('customNode.html'))
            .pipe(htmlmin({ collapseWhitespace: true, removeComments: true, minifyJS: true }))
            .pipe(dest('nodes/ti-dummy'))
    } catch (e) {
        console.error('buildTiDummy failed', e)
    }

    cb()
}

//#endregion ---- ---- ----

/**
 * Watch for changes during development, rebuild as needed
 * @param {Function} cb callback
 */
function watchme(cb) {
    // Re-pack uibuilderfe if it changes
    watch('src/ti-template/*', buildTiTemplate)
    watch('src/ti-dummy/*', buildTiDummy)

    cb()
}

exports.default = watchme
exports.watch   = watchme

The src folder looks like this:
image

main.html is the template for the Editor html:

<link type="text/css" rel="stylesheet" href="./resources/@totallyinformation/node-red-testbed/ti-common.css" media="all">
<script src="./resources/@totallyinformation/node-red-testbed/ti-common.js"></script>
<script src="./resources/@totallyinformation/node-red-testbed/template.js"></script>

<script type="text/html" data-template-name="ti-template">
<!--=include panel.html -->
</script>

<script type="text/html" data-help-name="ti-template">
<!--=include help.html -->
</script>

Unfortunately, you can't load the help script externally as Node-RED does not recognise it if you do. So the build function simply replaces the <!--=include help.html --> sections with the source files. The htmlmin extension makes the output a bit more compact.

As you can see, I also have not only the Editor JS but also a common Editor library being loaded from resources. This lets me standardise some code and settings across nodes withing the package. And there is a common CSS style library that helps standardise som custom colours, widths & padding, tooltips and emoji.

The common js library adds standard debugging - turns it on by default if you are running on localhost, gives nicer tooltips. Adds an extra property to my own node-types addType which records how the nodes were added. This lets you know whether they were just loaded when the Editor page was loaded, added by a copy/paste or import (important for some nodes that might need to blank certain fields in that case), or were newly added to a flow. Rather more is done in the UIBUIILDER common library, you can check it out here: node-red-contrib-uibuilder/resources/ti-common.js at v6.9.0 Ā· TotallyInformation/node-red-contrib-uibuilder Ā· GitHub.

2 Likes

As someone who went through the hype of everything should be a class and isn't this encapsulation wonderful, I don't see it as the hammer that solves all problems. Also not a criticism more of an observation and the parallels I want to show are that some languages are better left as they fulfil a certain purpose.

Agree and most frameworks do that with templates that get generated by some rake or make command. I find this approach great for textual, what I'd like to see is a visual equivalent solution - its remaining in the same paradigm that is important for me. Hence I don't like the hybrid model.

Three? I thought two: the UI definition had the registering node JS code - what do you see as the third thing?

Do you mean the localised helper message text thingy - because that I outsourced into a separate file in the locales directory (when I generate my boilerplate). Hence I have four files for each node.

I was thinking about this and I think I would break complexity down into four sub-complexities:

  • structural complexity whereby there are four files per node with a mix of html, js and json. Complexity here is to know for what each is there for?
  • architectural complexity in knowing that one set of files is for the frontend, one set for backend and one is shared across both (i.e. the locales json). That's the complexity you get when you have a server-client architecture.
  • coding complexity in the sense that there are two different APIs in the frontend and backend, i.e. RED.nodes.node("...") v. RED.nodes.getNode("...") for example. Or knowing that RED._ is not underscore.js but i18n - data-i18n but I use RED._(....) to get the localisation translated. Of course, _ has become a pseudo-default for i18n but for beginners this isn't clear.
  • visual complexity and that is using Node-RED in the first place. This is complexity that every new Node-RED user faces. Here also is the complexity of doing meta-programming, that a flow can be used to describe a flow or a node or a blog page.

All of these (and probably more) play a role in a Node-RED users day-to-day life. That is the complexity I see so therefore I find it difficult to think in terms of classes to simplify these complexities.

I'm thinking of added that to my nodedev package as a template that gets inserted into the html - the html file committed to GitHub would include both but in my Node-RED flow for the node it would be separated out into a package file and a template. And this is using Node-RED as a CI server yet its still the code for the node. Only the flow has the "editable" form of code, GitHub has a representation that is generated before being pushed to GItHub.

Yes but as I said, the only common dominator is having Node-RED installed. So having a solution that is based on Node-RED would be more inclusive than one where I have to install a terminal, a gruntfile and whatever else to get some templates thrown out with which I can create a node. Remember: a package.json, README, LICENSE and ideally a CHANGELOG are all required to publish a package with a node.

Here I disagree. I think that Node-RED can do anything that a textual code can because it has code. It's not no-code but low-code. I think having a hybrid solution is not as productive as having either a textual only solution or a visual only solution. Context switching comes with a price.

Of course, many will disagree but that's fine. If we were having this discussion talking about textual and punch card programming, I think we'd all laugh. One day, the same thing will occur and people will look back at this discussion and laugh at us all for thinking that a hybrid solution would be good.

Imagining the future is the hard part, in experience it, it becomes the present.

Just a note on this: I was a big big keyboard-only person and did a lot to avoid using the mouse. Today I'm constantly on the mouse and only occasionally on the keyboard (like now ;)). I'm still a big fan of keyboard-only or mouse-only but what changed was my use of a big old trackpad on my keyboard - it brought the mouse to the keyboard instead of me bring my hand to the mouse.

It's these small things that can make visual programming more productive: if your mouse is twenty centimetres from your hand, the visual coding experience using NR isn't as much fun!

Sorry one final point: I was thinking (perhaps I'm overthinking) about the classes and flow based programming. Node-RED (i.e. FBP) separates the data from the functionality. It moves the data and the data gets worked on by various points in the conveyer belt (hence think assembly line) as data gets moved through. This does not happen in a OO paradigm - the data travels with it functionality. For me that's another reason to be careful when bring the OO paradigm to Node-RED. It goes against what Node-RED is about. Soon then there might be the question way the msg object isn't a class ... and then we could subclass the msg object and pass around special messages ...

We know you like visual all the way down. :grinning: But it isn't for all of us, there is room for both.

In my deconstructed source, there are actually 6 to be totally accurate.

But for the node's html file there are 4 sections: The master page, the JavaScript, the Edit panel HTML "script" and the help panel HTML "script". I now put the JS code into a separate file in the resources folder leaving me with 3 remaining files to build into 1, the master, the panel and the help.

I'll be honest here and say that getting my head around the locales folder is on my backlog but I've never quite got round to it. What does your HTML file look like if the help is in the locales folder?

I would mostly agree with that. Except that I'm not known for writing simple nodes! :mage:

So for me and anyone writing a complex node, there is added complexity, but this hits hardest in the runtime where the most complex code will generally be. The main uibuilder node now splits code into 11 libraries, each performing a specific set of tasks such as real-time socket.io comms, web server routing, package management, admin API's, no-code to HTML conversion and so on.

And, of course, you need to understand the different time-periods and contexts that your node has to deal with. From Node-RED startup, Node initialisation, instance initialisation through to inbound message receipt.

I could never have got as far with uibuilder if I hadn't, over the years, split out these different functional areas into their own modules. Also, being prompted a few years ago (when someone in the forum was having issues with apparent memory resource issues when copying this to node in the instance function) to find a much (to my mind) better way of structuring the runtime code helped me find several classes of bugs and accelerated the splitting of monolithic code into modules.

I get that uibuilder is hardly average though. And probably not everyone might need/want to find further simplifications of node runtime code. But that is what this thread is about. The rest is true, but worthy of its own (set of) threads.

Well, that is life as a programmer I guess. And on steroids for JavaScript programmers since there are usually so many different ways of doing things.

All I can say, is that I've found the class approach really helpful to me creating much more disciplined and comprehendible code.

I've really noticed an acceleration in development since restructuring this way. And bugs are much easier to find and eliminate.

Classes are not the only way to achieve this of course. But they are working for me anyway. Maybe its just a phase, but hey, it works. :slight_smile:

Yup, but most of that is pretty simple. And the LICENSE file should be totally standard anyway and GitHub will create most things for you. More if you are creating lots of nodes because it will be worth setting up a template repo and then GitHub really will create a ton of stuff for you.

Well, we can agree to have different opinions I'm sure.

What I will say is that my latest Deepscan code check tells me that the UIBUILDER package contains 11,361 lines of monitored code. That ignores the thousands of lines of documentation as well of course. Not entirely sure how many functions that encompasses. Thousands certainly. And each has accompanying in-line dev documentation. I can't even begin to comprehend what that would look like expressed visually. I think there is a tool somewhere that will turn a package into architectural diagrams, if anyone finds it, I'll try to run it just for fun.

I know from many decades of experience that diagrams have a complexity limit, once you exceed that, they stop being useful. Visual programming is the same. So to get anywhere near being able to reproduce this kind of complexity, you would certainly also need to have methods to break down your programmes into smaller chunks just as I have with the uibuilder modules.

I'd also be interested in seeing the memory utilisation for building something this complex visually.

Don't get me wrong, I know it could be done if someone puts there mind to it. And I also really do understand the desire, but like anything in tech, just because you can, doesn't mean you should (or that you want to). I won't be but I will certainly continue to follow your innovative ideas.

This really isn't an issue. The clear separation between DATA and APPLICATION remains just as COMMUNICATIONS is the 3rd leg of the equation. These are standard paradigms in IT and clearly understood.

Applications have a great deal of runtime information within them but nobody mistakes class data or function data for that matter for actual DATA (well, nobody will certainly be an over-estimation I'm sure, but still). A class is merely a collection of functions and local variables acting as a repeatable template. No different in that sense to when you create a node.js module that exports a collection of functions and variables. That is effectively also a class in the loosest sense - just one that can only have a single instance in your application.

And, as has already been pointed out, in JS, classes are a special case of prototypes which are core to programming with JS. They just happen to trade off a little flexibility with a lot of comprehension.


Anyway, we are now a very long way of the original topic I'm afraid. We are no longer covering how to simplify the coding of node runtimes.

Of course, each to their own. You see a more efficient solution in a hybrid model, I see a more efficient solution working visually. Somewhere in-between lies a possible solution. Everyone else will come up with other solutions!

For example - it's just the data-help-name template in the locales/en-US/ directory.

Discipline is the key - it doesn't much matter which concepts or language you use if you are a disciplined programmer with an eye for detail.

In all seriousness, I would also point out that the current solution is a great solution for what it represents: frontend events, backend events, UI definition and locales all in only four files which are clearly delineated across responsibilities and easily extendable. All because it shows great discipline and much thinking out.

2 Likes

Oh my! I would never have thought of that! OK, next release of UIBUILDER will certainly make that change & that will then be enough to ditch the build process because each node's html file will only have the references to the external script and css along with the actual panel HTML.

Just in case anyone else is following this. In my node packages, I tend to put my node definitions in a sub-folder of nodes, each has the same file name:

nodes/
  my-node/
    customNode.html
    customNode.js
  ...

In this case, the locale folder needs to be in the sub-folder and not in the nodes folder. And the name of the help file has to match the file name NOT the node name.

nodes/
  my-node/
    customNode.html
    customNode.js
    locales/
      en-US/
        customNode.html
  ...

I believe that en-US is the default locale for Node-RED and so this will be used for any non-explicitly defined locales.

2 Likes

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.