Refactoring a node with only one file to multiple files

novski · 1 August 2023 16:40

I found a bug in a node and tried to find the code.
But the file was 2600 lines big, and I had a hard time finding the passage.
Then I thought I could create a PR refactored to multiple files and started to work it out.
I created 3 files each for a generation of api.
My workflow for testing is to use npm pack and upload over the palette Manager.
But even if the nodes get successfully installed after a node-red-restart (I'm on a rpi4) the nodes don't show up in the sidebar and the palette manager tells me MODULE_NOT_FOUND.
My version is number is 10.13.6

I can install the untouched 10.13.5 version built with npm pack the same way.

After installation over palette manager without restart
[warn] [node-red-contrib-xy] Error: Cannot find module 'gen1'
after a restart, it works fine again.

My first question is, how can I get more insight of what I have done wrong?
Second question is, it expected to need to restart node-red after deleting a module?

Further, I was wondering about how to add multiple nodes correctly. The doc only covers one node. It would be nice to provide a further step and integrate a second node as well.
It's unclear to me if I better declare a second node in package.json or register a second node in the same file.

Like this with two nodes in packet.json:

packet.json

{
    "name" : "node-red-contrib-example-cases",
    ...
    "node-red" : {
        "nodes": {
            "lower-case": "lower-case.js",
            "uper-case": "uper-case.js"
        }
    }
}

lowercase.js

module.exports = function(RED) {
    function LowerCaseNode(config) {
        RED.nodes.createNode(this,config);
        var node = this;
        node.on('input', function(msg) {
            msg.payload = msg.payload.toLowerCase();
            node.send(msg);
        });
    }
    RED.nodes.registerType("lower-case",LowerCaseNode);
}

upercase.js

module.exports = function(RED) {
    function UperCaseNode(config) {
        RED.nodes.createNode(this,config);
        var node = this;
        node.on('input', function(msg) {
            msg.payload = msg.payload.toUperCase();
            node.send(msg);
        });
    }
    RED.nodes.registerType("Uper-case",UperCaseNode);
}

Or like this with two registers in one file:

{
    "name" : "node-red-contrib-example-cases",
    ...
    "node-red" : {
        "nodes": {
            "cases": "cases.js"
        }
    }
}

cases.js

module.exports = function(RED) {
    function LowerCaseNode(config) {
        RED.nodes.createNode(this,config);
        var node = this;
        node.on('input', function(msg) {
            msg.payload = msg.payload.toLowerCase();
            node.send(msg);
        });
    }
    function UperCaseNode(config) {
        RED.nodes.createNode(this,config);
        var node = this;
        node.on('input', function(msg) {
            msg.payload = msg.payload.toUperCase();
            node.send(msg);
        });
    }
    RED.nodes.registerType("Uper-case",UperCaseNode);
    RED.nodes.registerType("lower-case",LowerCaseNode);
}

What is to be preferred?
Thanks a lot.

TotallyInformation · 2 August 2023 16:47

I believe you can do either. Personally I like to keep my nodes in separate files but some people do put multiple nodes in single files if they are simple ones. More important is to think about how you want to manage the nodes in the future.

This is probably your issue. A more normal workflow is to dev and test on the same device. You can "install" your dev node with

cd ~/.node-red
npm install /path/to/local/folder/holding/your/node

When you make changes, you have to restart node-red but don't need to do anything else if you work this way. Then, when you've finished and tested, you can push the changes to GitHub (it is recommended to create a new tag for a new release) and publish to npm.

novski · 7 August 2023 19:52

I have done some refactoring and got stuck on a part I can't understand.
My VS Code IDE tells me that the definition is not defined, but other functions get found.
Really strange. Any hints?
Bildschirmfoto 2023-08-07 um 21.49.09

TotallyInformation · 7 August 2023 20:13

That is because you've used a traditional JavaScript function for the second point. Change the line that defines the function to use the modern setInterval( () => { ..... } ) form instead which retains the this context. Or bind this as the correct context to the setInterval. But that is more complex.

novski · 9 August 2023 06:31

I tried to change all those parts to your suggested modern typewise. But unfortunately, I still get errors in my module and my IDE does not help to show me the unresolved links to function names.
Can i somehow make my IDE to understand node. and imports from (RED).. I assume that would make my work a lot easier…

I read a lot about the different cases of functions in JS and was amused about the multiple different solutions.
Still, one style I didn't find until now. It's the one used in my print screen above:

start: function(node,types){...}

What kind of function is that, and why is it written with : between the function name and function descriptor?
Is that related to the prior encapsulation in exports?

module.exports = {
    start: function (){}
}

Where can I learn more about it?

TotallyInformation · 9 August 2023 10:50

You can add some level of intelligence to VScode. Though I would say that I don't think that the Node-RED typings have been updated for quite a while. Here is a snippet from the uibuilder node package:

  "devDependencies": {
    "@types/express": "^4.17.13",
    "@types/jquery": "^3.5.8",
    "@types/node": "^14.18.29",
    "@types/node-red": "^1.1.1",
    "eslint": "^8.31.0",
    "eslint-config-standard": "^17.0.0",
    "eslint-plugin-es": "^4.1.0",
    "eslint-plugin-html": "^7.1.0",
    "eslint-plugin-import": "^2.26.0",
    "eslint-plugin-jsdoc": "^46.4.2",
    "eslint-plugin-n": "^16.0.1",
    "eslint-plugin-promise": "^6.0.0",
    "eslint-plugin-sonarjs": "^0",
    "gulp": "^4.0.2",
    "gulp-clean-css": "^4.3.0",
    "gulp-debug": "^4.0.0",
    "gulp-esbuild": "^0.11.1",
    "gulp-htmlmin": "^5.0.1",
    "gulp-include": "^2.4.1",
    "gulp-json-editor": "^2.5.6",
    "gulp-once": "^2.1.1",
    "gulp-prompt": "^1.2.0",
    "gulp-rename": "^2.0.0",
    "gulp-replace": "^1.1.3",
    "gulp-uglify": "^3.0.2"
  },

It is the types that give you some VSC intelligence. The eslint stuff gives you a check of both obvious errors and code formatting.

One of the frustrating things about developing nodes is that the Editor HTML file has to be in the same location as the runtime js file - since they really need different eslint settings. That's one of the reasons I no longer directly develop the Editor HTML file. I use a separate src folder with the 3 separate parts of the html file and use a gulp watch to automatically build the actual html file. A bit overkill for simple nodes but much easier to work on.

In regard to the node-red typings, I actually use my own custom typedefs.js file where I define all of the main object structures I am using. I use that to define the base node-red node definition that I can reuse and then I supplement it with each nodes specific definitions.

Here is an extract:

/** Node-RED runtimeSettings - See settings.js for static settings.
 * @typedef {object} runtimeSettings Static and Dynamic settings for Node-RED runtime
 *
 * @property {string} uiPort The port used by Node-RED (default=1880)
 * @property {string} uiHost The host IP used by Node-RED (default=0.0.0.0)
 * @property {string} userDir The userDir folder
 * @property {string} httpNodeRoot Optional base URL. All user url's will be under this. Default empty string.
 * @property {object} FunctionGlobalContext Add values, Functions, packages to the Global context variable store.
 * @property {Function} mqttReconnectTime : [Getter/Setter],
 * @property {Function} serialReconnectTime : [Getter/Setter],
 * @property {Function} debugMaxLength : [Getter/Setter],
 * @property {Function} debugUseColors : [Getter/Setter],
 * @property {string} flowFile : [Getter/Setter],
 * @property {Function} flowFilePretty : [Getter/Setter],
 * @property {string} credentialSecret : [Getter/Setter],
 * @property {string} httpAdminRoot : [Getter/Setter],
 * @property {string} httpStatic : [Getter/Setter],
 * @property {Function} adminAuth : [Getter/Setter],
 * @property {Function} httpNodeMiddleware : [Getter/Setter],
 * @property {Function} httpAdminMiddleware : [Getter/Setter],
 * @property {Function} httpServerOptions : [Getter/Setter],
 * @property {Function} webSocketNodeVerifyClient : [Getter/Setter],
 * @property {Function} exportGlobalContextKeys : [Getter/Setter],
 * @property {Function} contextStorage : [Getter/Setter],
 * @property {Function} editorTheme : [Getter/Setter],
 * @property {string} settingsFile : [Getter/Setter],
 * @property {string} httpRoot : [Getter/Setter],
 * @property {Function} disableEditor : [Getter/Setter],
 * @property {Function} httpAdminAuth : [Getter/Setter],
 * @property {Function} httpNodeAuth : [Getter/Setter],
 * @property {object|Function} [https] If present, https will be used for ExpressJS servers.
 *
 * @property {object} [uibuilder] Optional uibuilder specific Node-RED settings
 * @property {number} [uibuilder.port] Port number if uib is using its own ExpressJS instance
 * @property {string} [uibuilder.uibRoot] Folder name that will hold all uib runtime and instance folders
 * @property {('http'|'https')} [uibuilder.customType] Connection type - only if using custom ExpressJS instance
 * @property {object|Function} [uibuilder.https] Override https server settings (key/cert) - if not specified, uses main NR https prop
 * @property {object} [uibuilder.serverOptions] Optional ExpressJS server options for uib custom server
 * @property {object} [uibuilder.socketOptions] Override Socket.IO options if desired. See https://socket.io/docs/v4/server-options/
 * @property {boolean} [uibuilder.instanceApiAllowed] Allow instance-level custom API's to be loaded. Could be a security issue so it is controlled in settings.js
 *
 * @property {string} coreNodesDir Folder containing Node-RED core nodes
 * @property {string} version Node-RED version
 *
 * @property {object} logging Controls the type and amount of logging output
 * @property {object} logging.console Controls output levels and types to the console log
 * @property {string} logging.console.level What level of output? (fatal, error, warn, info, debug, trace)
 * @property {boolean} logging.console.metrics Should metrics also be shown?
 * @property {boolean} logging.console.audit Should audit also be shown?
 *
 * @property {Function} get Get dynamic settings. NB: entries in settings.js are read-only and shouldn't be read using RED.settings.get, that is only for settings that can change in-flight.
 * @property {Function} set Set dynamic settings
 * @property {Function} delete .
 * @property {Function} available .
 *
 * @property {Function} registerNodeSettings : [Function: registerNodeSettings],
 * @property {Function} exportNodeSettings : [Function: exportNodeSettings],
 * @property {Function} enableNodeSettings : [Function: enableNodeSettings],
 * @property {Function} disableNodeSettings : [Function: disableNodeSettings],
 *
 * @property {Function} getUserSettings : [Function: getUserSettings],
 * @property {Function} setUserSettings : [Function: setUserSettings],
 */

/** Node-RED runtimeLogging
 * @typedef {object} runtimeLogging Logging. Levels that are output to the Node-RED log are controlled by the logging.console.level setting in settings.js
 * @property {Function} fatal Lvel 0. Lowest level, things that have broken Node-RED only.
 * @property {Function} error Level 1. Copy is sent to Editor debug panel as well as error log.
 * @property {Function} warn Level 2.
 * @property {Function} info Level 3.
 * @property {Function} debug Level 4.
 * @property {Function} trace Level 5. Very verbose output. Should tell the operator everything that is going on.
 * @property {Function} metric Log metrics (timings)
 * @property {Function} audit Audit log
 * @property {Function} addHandler Adds a log handler
 * @property {Function} removeHandler Removes a log handler
 */

/** Node-RED runtimeNodes
 * @typedef {object} runtimeNodes Gives access to other active nodes in the flows.
 * @property {Function} registerType Register a new type of node to Node-RED.
 * @property {Function} createNode Create a node instance (called from within registerType Function).
 * @property {Function} getNode Get a reference to another node instance in the current flows. Can then access its properties.
 * @property {Function} eachNode .
 * @property {Function} addCredentials .
 * @property {Function} getCredentials .
 * @property {Function} deleteCredentials .
 */

/** runtimeRED
 * @typedef {object} runtimeRED The core Node-RED runtime object
 * @property {expressApp} httpAdmin Reference to the ExpressJS app for Node-RED Admin including the Editor
 * @property {expressApp} httpNode Reference to the ExpressJS app for Node-RED user-facing nodes including http-in/-out and Dashboard
 * @property {object} server Node.js http(s) Server object
 * @property {runtimeLogging} log Logging.
 * @property {runtimeNodes} nodes Gives access to other active nodes in the flows.
 * @property {runtimeSettings} settings Static and Dynamic settings for Node-RED runtime
 *
 * @property {Function} version Get the Node-RED version
 * @property {Function} require : [Function: requireModule],
 * @property {Function} comms : { publish: [Function: publish] },
 * @property {Function} library : { register: [Function: register] },
 * @property {Function} auth : { needsPermission: [Function: needsPermission] },
 *
 * @property {object} events Event handler object
 * @property {Function} events.on Event Listener Function. Types: 'nodes-started', 'nodes-stopped'
 * @property {Function} events.once .
 * @property {Function} events.addListener .
 *
 * @property {object} hooks .
 * @property {Function} hooks.has .
 * @property {Function} hooks.clear .
 * @property {Function} hooks.add .
 * @property {Function} hooks.remove .
 * @property {Function} hooks.trigger .
 *
 * @property {object} util .
 * @property {Function} util.encodeobject : [Function: encodeobject],
 * @property {Function} util.ensurestring : [Function: ensurestring],
 * @property {Function} util.ensureBuffer : [Function: ensureBuffer],
 * @property {Function} util.cloneMessage : [Function: cloneMessage],
 * @property {Function} util.compareobjects : [Function: compareobjects],
 * @property {Function} util.generateId : [Function: generateId],
 * @property {Function} util.getMessageProperty : [Function: getMessageProperty],
 * @property {Function} util.setMessageProperty : [Function: setMessageProperty],
 * @property {Function} util.getobjectProperty : [Function: getobjectProperty],
 * @property {Function} util.setobjectProperty : [Function: setobjectProperty],
 * @property {Function} util.evaluateNodeProperty : [Function: evaluateNodeProperty],
 * @property {Function} util.normalisePropertyExpression : [Function: normalisePropertyExpression],
 * @property {Function} util.normaliseNodeTypeName : [Function: normaliseNodeTypeName],
 * @property {Function} util.prepareJSONataExpression : [Function: prepareJSONataExpression],
 * @property {Function} util.evaluateJSONataExpression : [Function: evaluateJSONataExpression],
 * @property {Function} util.parseContextStore : [Function: parseContextStore]
 */

/** runtimeNode
 * @typedef {object} runtimeNode Local copy of the node instance config + other info
 * @property {Function} send Send a Node-RED msg to an output port
 * @property {Function} done Dummy done Function for pre-Node-RED 1.0 servers
 * @property {Function} context get/set context data. Also .flow and .global contexts
 * @property {Function} on Event listeners for the node instance ('input', 'close')
 * @property {Function} removeListener Event handling
 * @property {Function} log General log output, Does not show in the Editor's debug panel
 * @property {Function} warn Warning log output, also logs to the Editor's debug panel
 * @property {Function} error Error log output, also logs to the Editor's debug panel
 * @property {Function} trace Trace level log output
 * @property {Function} debug Debug level log output
 * @property {Function} status Show a status message under the node in the Editor
 * @property {object=} credentials Optional secured credentials
 * @property {object=} name Internal.
 * @property {object=} id Internal. uid of node instance.
 * @property {object=} type Internal. Type of node instance.
 * @property {object=} z Internal. uid of ???
 * @property {[Array<string>]=} wires Internal. Array of Array of strings. The wires attached to this node instance (uid's)
 */

/** runtimeNodeConfig
 * @typedef {object} runtimeNodeConfig Configuration of node instance. Will also have Editor panel's defined variables as properties.
 * @property {object=} id Internal. uid of node instance.
 * @property {object=} type Internal. Type of node instance.
 * @property {object=} x Internal
 * @property {object=} y Internal
 * @property {object=} z Internal
 * @property {object=} wires Internal. The wires attached to this node instance (uid's)
 */

module.exports = {}

novski · 10 August 2023 04:57

Thanks for those snippets. I'll work out a solution for me in the next days.

Do you have me a resource where I can learn more about this : type of functions below?

novski:

start: function(){...}

What kind of function is that, and why is it written with : between the function name and function descriptor?
Is that related to the prior encapsulation in exports?
module.exports = {
    start: function (){}
}
Where can I learn more about it?

Steve-Mcl · 10 August 2023 08:12

This is plain JavaScript object where a property is a function instead of a string or number or object!

Just like the abs function in the Math object is

chrome_KIEUOPjt7W

for example:

const MyMath = {
   myAbs: function (num) {
      // remove the minus and return value
   }
}
MyMath.myAbs(-1)

chrome_qQnoRsHedg

novski · 12 August 2023 12:33

Thanks a lot for that explanation.
So it's called a "method"?
Just found that here: javascript - Method vs Functions, and other questions - Stack Overflow

I still have big troubles debuging.
I now get ReferenceError: node is not defined
But i don't know from where.
I was able to trace it down to this line but it does not seam to reach the following function named getInitializer1
But i don't get any line number where that ReferenceError happens nor a better debug message what node is ment to be.
Any thaughts?

This is my view:
I am in Terminal ssh in the NR machine log and showing the console.log line and the node.warn-ings.

TotallyInformation · 12 August 2023 12:41

A function is normally described as a method when it is part of a class. JavaScript is slightly odd though in that you can create objects containing data and methods even though the object isn't strictly a class. Class objects are a more recent addition to JavaScript.

It is best to paste logs as text in code tags so that people can read it more easily.

The messages have a node id listed. If you click on that, it takes you to the node that produced the error.

novski · 3 September 2023 18:59

Is there no way to get the line where it breaks?
Can I somehow alter the logging to show lines?

ReferenceError: node is not defined is a really bad error message because almost every line of m code has the term node in it…

I really struggle with how to find where it breaks…

I managed to connect VS Code per ssh to develop on the device itself.
Is there any way maybe to define a breakpoint and step through the code?

TotallyInformation · 3 September 2023 20:32

Not always, it is a consequence of the way that your module is included in the Node-RED modules.

It is usually pretty easy to find though. Knowing the node in your flow tells you the underlying Node module. Then you can simply search in your code for something referencing node. Most likely searching for node. (a dot at the end) will find what you need.

You have either referenced an object that doesn't exist at all or doesn't yet exist.

In node-red custom nodes, it is common to have a line const node = this in the function that defines the node. In my nodes, I don't ever do that because it can be confusing. Instead, the function given to the RED.nodes.registerType function as its callback is called nodeInstance (as that is indeed what that actually does - creates an instance of your custom node in Node-RED). And in the nodeInstance function, I always refer to this. I only ever use something called node in functions called from nodeInstance where this is passed as an argument and becomes node in the function. Here is an example:

function nodeInstance(config) {
    // ....

    /** Create the node instance - `this` can only be referenced AFTER here
     * @param {uibNode} this _
     */
    RED.nodes.createNode(this, config)

    // ....

    web.instanceSetup(this)
    
     // ....
}

And in web.instanceSetup:

/** *️⃣ Setup the web resources for a specific uibuilder instance
 * @param {uibNode} node Reference to the uibuilder node instance
 */
instanceSetup(node) {
    // ....
}

So make sure of 2 things. Firstly that you don't try to reference this before the line RED.nodes.createNode(this, config)

And secondly, don't reference node until it has been created.

novski · 4 September 2023 19:06

Thanks for that. I read it about 3 times and still don't catch the meaning of node really.
I then changed all node. to this. except my print statements, which I left with node.warn.
Then I immediately got a Reference Error for node.
When I change that node.warn to this.warn I get the print before I get the Reference Error.
But that is after the createNode statement:

function ShellyGen1Node(config) {
        RED.nodes.createNode(this, config);
        this.warn('this is the start');
...

while in the same file little bit deeper i have this:

RED.nodes.registerType("shelly-gen1", ShellyGen1Node, {
        credentials: {
            username: { type: "text" },
            password: { type: "password" },
        }
    });

my package.json starts that (I beleve...):

"nodes": {
            "shelly-gen1": "shelly/shelly-gen1.js",

Why do I have to use this when the RED.nodes.createNode(this, config); is already read before?

TotallyInformation · 4 September 2023 20:22

RED.nodes.createNode actually populates this, if you try to access a this object before you've called that, I believe you will get an error. That's what I remember from my testing anyway when I was looking for the best structures to use for custom nodes.

The object node is merely a convenience. The reason a lot of people use const node = this immediately after RED.nodes.createNode is that, in JavaScript, this is a context easily changed simply by dipping into another function.

So if you did something like:

function ShellyGen1Node(config) {
        function myFunc() {
            console.log(this)
        }

        RED.nodes.createNode(this, config);
        this.warn('this is the start');

        myFunc()
...

You wouldn't get get the result you expect.

But if you do this:

function myFunc(node) {
    console.log(node)
}
function ShellyGen1Node(config) {

        RED.nodes.createNode(this, config);
        this.warn('this is the start');

        myFunc(this)
...

You will get the expected result.

Things get even more complex when passing functions as callback arguments to another function. And this is where the idea of assigning this to node (it could be any name, not just node, that is only a convenience) comes from:

function ShellyGen1Node(config) {

        RED.nodes.createNode(this, config);
        this.warn('this is the start');

        /* This function needs a callback function as an argument */
        myFunc( function myFunc() {
            console.log(this)
        } )

...

That won't work because when JavaScript compiles the callback function, the "context" of this changes. But this would work:

function ShellyGen1Node(config) {

        RED.nodes.createNode(this, config);
        this.warn('this is the start');

        const node = this

        /* This function needs a callback function as an argument */
        myFunc( function myFunc() {
            console.log(node)
        } )

...

And this would also work because arrow functions do NOT change the this context - yup confusing right!

function ShellyGen1Node(config) {

        RED.nodes.createNode(this, config);
        this.warn('this is the start');

        /* This function needs a callback function as an argument */
        myFunc( () => {
            console.log(this)
        } )

...

Quite some time ago, I created myself a test set of custom nodes to try out various things. This node may (or may not!) help you understand the order of things in your custom node's runtime code:

github.com

TotallyInformation/node-red-contrib-jktesting/blob/master/nodes/jktest2.js

/** A simple template for defining custom nodes
 * Destructured to make for easier and more consistent logic.
 */
'use strict'

/** --- Type Defs ---
 * @typedef {import('../typedefs.js').runtimeRED} runtimeRED
 * @typedef {import('../typedefs.js').runtimeNodeConfig} runtimeNodeConfig
 * @typedef {import('../typedefs.js').runtimeNode} runtimeNode
 * typedef {import('../typedefs.js').myNode} myNode
 */

//#region ----- Module level variables ---- //

/** Main (module) variables - acts as a configuration object
 *  that can easily be passed around.
 */
const mod = {
    /** @type {runtimeRED} Reference to the master RED instance */
    RED: undefined,

This file has been truncated. show original

It is roughly the template I use for all new custom nodes as, to me at least, it follows a more logical pattern that the standard template from the Node-RED documentation. The Node-RED one is fine for simple nodes but I don't think it helps understand the order of things and some of the subtleties that become important in more complex nodes. My uibuilder node is thousands of lines long now and has been refactored multiple times over the years.

system · 3 November 2023 20:23

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Multiple Nodes In One Module/Package? General	25	1910	24 August 2020
How create new nodes? Developing Nodes	10	288	20 June 2023
A code error in a node in my palette Developing Nodes	9	393	20 March 2021
Very Basic Question [Solved] Developing Nodes	7	317	18 July 2020
Lost Node-Red node AGAIN General	19	4657	25 September 2018

Refactoring a node with only one file to multiple files

Related topics