Options for simplifying custom node runtime code?

Me either :rofl:

3 Likes

Oh dear, I knew it was too good to be true.

So progressed enough to get the node into the palette. But when added to a flow and re-deployed, I get this:

[error] [ti-class:8c682abd3ef797bd] TypeError: nodeTypeConstructor is not a constructor

Which feels as though it is probably a related issue. Can't find that error in the code.

Mmm, see L171 :thinking:

EDIT: arrow functions can NOT be used as constructors.

I think that the prototype needs to be something other than itself. It needs to be a Node but I don't know that a Node prototype is accessible to custom nodes.

Maybe @knolleary knows?

Hopefully i am not misunderstanding what you are discussing, but maybe you are trying to construct your node as a class? I will give the answer presuming that, feeling a bit sick and my eyes are going in different directions :nauseated_face:

A long while back, i had asked nick about exporting it as a Node class so that we could inherit from it. I think he said that he had considered it at one point, and that was about as far as the conversation went.

current working method as this is how my nodes are built:

'use strict';

module.exports = RED => {
  const {
    nodes: { createNode, registerType },
  } = RED;

  class MyCustomNode {
    constructor(config) {
      createNode(this, config);
    }
  }

  registerType('mycustomnode', MyCustomNode);

};

if the base class of Node existed and was made accessible, then we would change a small part to our class and constructor. The Node class would have to be attached such as RED.Node

const {
    Node,
    nodes: { registerType },
  } = RED;

class MyCustomNode extends Node {
    constructor(config) {
      super(config);
    }
  }
2 Likes

Yes, that is exactly what I am hoping to do.

I wonder whether the concepts of classes/objects/methods make any sense in a functional/event driven language? I ask this purely in the sense of language design.

Javascript is fundamental a functional language and an object oriented language is about passing objects, I.e., data bound to functionality.

I remember php being extended with classes in response to the success of Java. Class were simply a kludge.

Javascript is completely different to php and everything is an object but objects aren't implemented as they would be in a OO-first type of language such as Java (see prototype and binding 'this' everywhere).

I really don't see the advantage of having classes in something like NR where nodes (and stateless for that matter) are the basic unit of computation. Classes introduce state and that makes no real sense in an stateless, event driven environment. For me!

You should investigate more to understand about the prototype language and that classes are just an easy to use syntactic sugar. By adding methods to a class, in the background they are added like the old way:

MyClass.prototype.mySpecialMethod = function (val) {
  // do something with this
  this.whatever;
};

Your way of functional programming would mean that every object created would have its own copy of each of the functions, which increases memory usage. With prototyping, each instance of an object all share the methods defined on the prototype. Less duplication. Less memory.

We must be efficient with other people's resources if they are willing to use our code.

1 Like

I had a go on using classes myself, but dropped it, because it was complicating more things than actually creating a benefit.

Mostly due to the way nodes are currently created, registered, and also due to their life-cycle management.

E.g. there is no way to tell the runtime, that the node is completely initialized when doing some async stuff, as there's no callback for that. Due to the fact that it's treated as a constructor function, you can't return anything (like a promise) either.

So I think this is something that has to be supported first-class (pun intended :nerd_face:) by the runtime itself.

Like an alternate way of registering new nodes. It would need to get rid of the exported module function (module.exports = function(RED) {}). (On second thought, no.)
Plus you could offer first class async support and get rid of the event handlers (on message/on close).

Maybe something similar to the API for classes provided by the NodeJS stream package.

Here are some ideas in code, don't kill me for that: :see_no_evil:


// provided by node-red

class AbstractNode {

    constructor() {
    }

    // replaces the constructor function
    async init(RED) {
        // implemented by client
    }

    // replaces the on('close') event
    async close(RED) {
        // implemented by client
    }

    // replaces the on('input') event
    // called by the runtime
    async onInput(msg) {
    }

    // node.send() 
    async sendMessage(msg) {
    }
}


// implemented by user

class MyNode extends AbstractNode {

    async init(RED) {
    }

    async onInput(msg) {
        this.sendMessage(msg);
    }
}

module.exports = function(RED) {
    RED.nodes.registerClass('my-node-class', MyNode);
}

1 Like

Not really any different to many other languages. Most of which also have classes.

Don't think that PHP is a good example of a tier 1 language. :grinning: It was/is the web's equivalent of BASIC. It was very useful for its time but the world has moved on.

Very few reasonably complex compute tasks have no state at all in my view. And event-based doesn't remove the need for some state.

Rightly or wrongly, classes in JavaScript for me provide some advantages.

  • Encapsulation - To me, classes make code boundaries and encapsulation much easier to comprehend and work with. Though JS classes aren't yet fully enabled in this area (taken a long time to get truly private variables and methods), they provide a good framework. Kevin's response not withstanding, yes I know that this can be done with prototypes, but OMG they are hard to follow and understand and very easy to mess up. Classes provide a clear grammatical framework.

  • Extendability - Again, I know you can do this with prototypes but again, class extendibility and the super() function make the whole thing a lot easier to comprehend.

Thanks for sharing that. Seems like I'm probably wasting my time then. :sigh:

Not at all. Couple of points to add though. The RED object needs to be available throughout the class and so should be available to the class. Assigning RED to a global variable in my own nodes has made a massive simplification of everything. At the moment, the createNode "magically" adds RED which is a horrible, hidden way that is confusing for anyone trying to get their heads around what happens when creating a node. (It also confuses the heck out of code linters).

In my view, the node author should not be forced to call register...., this is a background task that the master class could do and should do in the init function (I called it start in my example, but the same thing). Also, I think that registerClass is superfluous since you might as well stick with the JS standard of creating a new instance. An instance of the (extended) class should self-register the node type in the init. That way, an author creates a node in a more JS-like way, instantiate an instance of the class and export that instance would be the preferred approach I would think?


Apologies, some of the above possibly seems like a criticism of the original node-red programming. Well, I suppose it is, but I fully understand that both JS and people's knowledge and skills change over time. So please don't take this as a personal criticism if you have ever been involved in creating that core code. It is certainly not meant that way.

3 Likes

I'm not quite sure what you mean here.

The node module is expected to export a function that will get called when the module is loaded by the runtime and the RED object is passed to that function, providing the module a handle to the runtime that is loading it. There is no magically adding the RED object - it is explicitly passed to the module's exported function.

This is entirely unrelated to the call to createNode.

Every single OO language does this under the hood. But not every OO language expects me to bind 'this' and use crappy syntax to create a class.

Javascript simply does not treat objects (those derived from classes) and classes as first-class citizens - the syntax sugar is confusing, aka a kludge.

Well, I don't think that it can't be done, but resorting to ugly hacks to get it working doesn't seem like a clean solution either. I'd rather have a clean API in the runtime to allow for that. :slightly_smiling_face:

Also, it's not a criticism of how it's implemented at the moment. It's probably been that way since the early days of Node-RED, classes weren't there back in the day, and it's hard to change such an integral part of a system. I've been down that road many times, as systems and designs evolve over time. :sweat_smile:

I simply wanted to point out some ideas to improve the node registration and its life cycle management.

2 Likes

These are the two main arguments for classes in every language because those are the main concepts behind OO theory.

For me, JS is not a tier-1 language when it comes to the implementation of OO concepts.

I think pointing fingers and claiming things that aren't true does not help. PHP is arguably the most widely used language on the web if one considers in what WordPress is implemented. Hint: it's PHP, the BASIC of the web. Perhaps that's why so many use it: it's simple. Unlike Node-RED. Perhaps there are some learnings that PHP can offer Node-RED?

Adding OO concepts to creating Nodes won't make NR any simpler - IMHO.

Yes to something like AI generated code so that the developers definitely do not need to think.

Exactly the point. And having a simple exported class instead of double-wrapped functions that passes the RED context is much cleaner and loosely coupled in my opinion. That's what makes it hard to get around at the moment.

You'd pass everything you need either via constructor or it's life-cycle methods.

Am I the only who thinks that the problems lies not with having a class for abstracting or encapsulating the creation of a node?

The problem IMHO is the concept whereby I write code for the frontend (the .html file with some js) and some code I write for the backend (the .js file). And then if I get advanced, I have to create a .json and a second .html for the localisation.

Four files for one node. Now I need to remember that the JS code in the .html file is browser code and therefore I have to remember that Node-RED has a different API in the browser than on the backend, so that in the .js, I have to use a completely different Node-RED API.

But now I have a class to help me. How does that help me to abstract away all that complexity?

I think that is the fundamental complexity that every developer of Node-RED nodes face. Perhaps I lack the imagination but how does having a class help me with these fundamentally architectural design decisions of Node-RED?

Edit: I think a lot would be done if there were five files per node: it would be great if the .html defining the frontend UI would be split into a dedicated .js file that is included in the .html file (or vice versa). I believe the underlying issue in this class discussion is the problem of mixing JS code and HTML code in one file (reminds me a bit of PHP TBH) - having a class to encapsulate this would move the JS code out (perhaps).

Agreed this is the crux of the issue. As I see it, itā€™s solved by simply getting used to it, through a build step that produces the files, or through first-class support. So, how do we decide which one to go for?

Getting used to it is fine for companies creating one or two custom nodes, or those who spend their time with many instances of NodeRED instead of nodes themselves. IMHO, I think the core maintainers / FlowFuse have no impetus or desire to complicate the system for the sake of providing first class support. Adding backwards compatibility, new tests, etc is going to be hard. That doesnā€™t mean we here shouldnā€™t advocate for it though because NR is open-source and the right design would improve the developer experience for everyone.

Of note is that the core development page and outgoing links donā€™t appear to have been updated for a number of years.

Eventually, we will have some good documentation on the internals of Node-RED

Emphasis mine. Iā€™m going to delve into the design notes later, but is it fair to say that we need (to create) more documentation in order modify node registration, or at least communicate it here? Have I plainly missed some documents in my cursory look?

I think Iā€™m after a sequence diagram of the node registration process so that I can see where the changes may be needed to improve the authoring and it could be that I end up creating one. All time-permitting of courseā€¦

Also, I donā€™t currently use a build step for nodes, so developing one is attractive to me as an interim solution. Something that consumes a JS module, then produces the HTML and JS currently needed by the runtime. Does anyone here have an existing one they can share?

I think this is defeatist and shows a lack of imagination. Here I don't mean you, I mean the statement is defeatist. Think of current world politics and we see how far it (the sentiment) has brought us.

Why do they only create one or two? Is it because that's all the nodes they need or is it that the process is so complex and they can't be bothered?

Nodes have first-world support - the creation of nodes is complicated but central to Node-RED. I would advocate for backward compatibility before fundamentally changing the architecture of node creation and breaking existing code.

I would go for a templating system that generates files for nodes and adds code as necessary ... oh wait, I've already built that with nodedev. Folks would use this but only if it worked outside of Node-RED. The fundamental problem with the nodedev package is that it all happens in Node-RED and it's code is generated in Node-RED as flows. Folks don't understand that because they live in a hybrid world of textual coding for node and node development, and visual coding, i.e. flow construction in Node-RED.

I really wonder why this isn't a concept that is talked about: how to get people using Node-RED and only Node-RED to do they coding? No need for a text editor. No need for a textual based version control, etc.

Remember: no one is using punch cards, so why is textual coding the end of the line?

I spent many years developing stuff with PHP so I'm all too familiar with it. I understand it is widely used. One of the reasons for it being that the developers made it super easy to attach it to web servers (something that the Python devs really screwed up when the web was getting going and why Python is generally not the back-end development tool of choice for most web developers). I also spent many years developing in different versions of BASIC. So again, not a criticism, merely and observation.

We all have our opinions! That's what makes things interesting :rofl:

I am certainly open to other ideas, that's why I created this thread really - to see if other ideas for simplifying the creation of nodes could be surfaced.

The class approach came to mind because I find myself gravitating back to classes again now that they have become more useful and integrated in Node.js and because they seem to offer an obvious and easily understood extension mechanism which is, on the face of it, ideal for the use case of having a load of boilerplate code in the master class and then extending it with just what you need for your node.

But surely there are other ways? The key thing for me is that node authors should never have to write reams of recurring boilerplate simply to get a node's runtime working.

Nope. :grinning: See above.

Ah, well, in my view, that is another issue with creating nodes. But I certainly agree, that too could be simplified. Unsurprisingly, I have my own views on the process of creating the Editor code as well.

It is worse. Because the html file is actually 3 separate things with different rules for each. And because the html file lives in the same folder as the runtime, you cannot have different linting rules.

That's why I now put the code for the Editor in a separate JS file in the resources/ folder and load it via the html file and I use a build process to build the html from 3 other files (a master template, the panel HTML and the help HTML). While this is more complex to set up initially, using a gulp watch process means that the build is fully automatic. Actually, now that I use the resources folder for the JS, I probably don't really need the build process any more and might ditch it if I write a new node package.

It, of course, does not help you with that complexity. What it helps with is the complexity of writing the runtime. And I should note here that, for a really simple node, the runtime code isn't too hard though the standard example could really do with a refresh.

But, once your node starts to get more complex, there are a number of complex things you need to get your head around. Such as what happens when in the different lifecycles of a node, its instances and the messages each instance handles. It is this that my ideas were targeting.

True, my thoughts in this thread are around how to simplify the runtime code complexity for node authors. The other issues are equally valid but probably worthy of their own threads?

You can already do this and I'm doing it in the UIBUILDER nodes now.

But I agree with the rest. Splitting the 2 HTML sections to separate files would be good.

Yes, but it is messy and based on Gulp. Happy to share?

:rofl:

To be fair though, I think some of us find that solution rather complex and only worth using if you need to create a LOT of nodes. Most of us only produce a fairly limited number (as that's all we have time for), making that SIMPLER would probably free people up to do more.

Possibly because visual programming cannot do everything that text-based coding can do. And even if it could, it is sometimes a lot more complex than doing things in text. That Node-RED provides a hybrid approach is one of its strengths, not a weakness.

1 Like

I thought the same as I wrote it but for those users, the whole point of NodeRED over another solution is not to write code, so without analytics we can only assume.

I've been meaning to try nodedev, but didn't get around to it just yet. One for tinkering with in the future.

I suspect it's this? GitHub - TotallyInformation/Node-RED-Testbed: A blank canvas to test out ideas for Node-RED custom nodes - I'll take look, but if you think you can eliminate it by using the resources link, then I'd be curious to see that too. I'll have ago my self this weekend

2 Likes