Accessing external libraries

[bwims]

Hello!

One for the JS programmers...

I can gain access to an external library using the setup tab of the function node and this works well in the case where access is gained either by a "require (module)" or an "import (module)"

Where I get unstuck is in the situation where the declaration is of the form:

import { parseDomain, ParseResultType } from "parse-domain";

with the following exports

export parseDomain(hostname: string | typeof NO_HOSTNAME, options?: ParseDomainOptions): ParseResult

and

export ParseResultType

An object that holds all possible ParseResult type values:

const ParseResultType = { Invalid: "INVALID", Ip: "IP", Reserved: "RESERVED", NotListed: "NOT_LISTED", Listed: "LISTED", };

Is it possible to access these routines using the Setup tab of the function node?

Thanks!

B.

[msillano]

Developing a project involving many flows, I needed to create function libraries, function arrays, and singleton objects (with data and methods), shared and accessible from each node/flow.

The 'standard' solution (import etc.) doesn't excite me.
I found a solution that works best for me: it uses singleton objects, and this is a good replacement for shared libraries, having the following advantages, also keeping the distribution issues in mind:

• Easy to implement.
• Easy to debug.
• Visible in node-red 'Context data' debug pad.
• No extra files: all in JSON import/export.
• No config file editing, auto-installation.

An implementation can be seen here: ‘Meteo Utils’ functions library.

In short, place in 'On Start' of some function node, in your flow or in one flow of your project:

global.set("a_sing_obj", {});  // only data, empty in this example
context.global.a_sing_obj = global.get("a_sing_obj");

Then add methods, like:

context.global.a_sing_obj.one_method= function(....){
                // more code, you can use 'this.' here
                 }

done: then you can do calls like:

context.global.a_sing_obj.one_method(...)

in any node and in any flow or subflow.

[marcus-j-davies]

@bwims

Sorry, not sure I follow - have just come back from a loonnngggg walk - so might not be in top gear

BUT...

In the setup tab you just import as-is parse-domain
then you can call any exported method/property/enum, such as parseDomain

so, in a function node

const Result = parsedomain.parseDomain("www.some.example.co.uk")
if(Result.type == parsedomain.ParseResultType.Listed){
  //do Something
}

Here are the possible values.

export enum ParseResultType {
  Invalid = "INVALID",
  Ip = "IP",
  Reserved = "RESERVED",
  NotListed = "NOT_LISTED",
  Listed = "LISTED",
}

[TotallyInformation]

@bwims

import

Import works only with ES Modules and they cannot be required.

To get an ES Module into a CommonJS module (which is all of Node-RED's modules and libraries), you have to use a dynamic import

const { parseDomain, ParseResultType } = import("parse-domain")

However, dynamic imports are async which means that your javascript has to take that into account.

The whole thing is an absolute pigs-ear and the most ridiculous part of "modern" JavaScript.

To be honest, your best bet is probably to use something like esbuild to turn the ES Module into a CommonJS one - if it can.

[bwims]

Thanks - I did try that, but, probably because of the next answer, there is a mismatch. I got this error message:

"TypeError: Cannot read properties of undefined (reading 'parseDomain')"

[bwims]

This looks hopeful! I presume that goes into settings.js and use "await" when calling the function?

As for converting, I did see this, but couldn't figure how I could use it in Node Red (being a newby!)
ESM

Thanks!

[marcus-j-davies]

This is where the CommonJS, ES and all that horrific sounding jazz evades me.
I should really know to be honest, given I use typescript these days

[bwims]

@TotallyInformation

const { parseDomain, ParseResultType } = import("parse-domain")

didn't work in settings.js ...

I tried it in a function and got the following crash, so luck so far!

7 Apr 17:28:26 - [error] TypeError [ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING]: A dynamic import callback was not specified.
    at new NodeError (node:internal/errors:372:5)
    at importModuleDynamicallyCallback (node:internal/process/esm_loader:39:9)
    at Function node:11896326bed805e9 [function 11]:2:42
    at Function node:11896326bed805e9 [function 11]:22:3
    at Script.runInContext (node:vm:139:12)
    at processMessage (C:\Users\williams_b\AppData\Roaming\npm\node_modules\node-red\node_modules\@node-red\nodes\core\function\10-function.js:401:33)
    at FunctionNode._inputCallback (C:\Users\williams_b\AppData\Roaming\npm\node_modules\node-red\node_modules\@node-red\nodes\core\function\10-function.js:335:17)
    at C:\Users\williams_b\AppData\Roaming\npm\node_modules\node-red\node_modules\@node-red\runtime\lib\nodes\Node.js:210:26
    at Object.trigger (C:\Users\williams_b\AppData\Roaming\npm\node_modules\node-red\node_modules\@node-red\util\lib\hooks.js:166:13)
    at FunctionNode.Node._emitInput (C:\Users\williams_b\AppData\Roaming\npm\node_modules\node-red\node_modules\@node-red\runtime\lib\nodes\Node.js:202:11)

[bwims]

I think the routines that I am trying to use are in typescript... at least they have file types as .ts

I went to github to have a look and noticed that.

[marcus-j-davies]

Typescript (.ts) is precompiled source code.
what you see in Github is the "source code" - later (during publish/release) it is compile to JS files.

The main use for Typescript is the clue in its name Type. - it allows (during development) type safety, better error checking, its more for the developer than the end user - the End User will usually only deal with the resulting JS files.

I use Typescript during development, then compile to JS before releasing

[TotallyInformation]

@bwims

As for converting, I did see this, but couldn't figure how I could use it in Node Red (being a newby!)
ESM

That's 4 years out of date and has many unanswered issues sadly.

@marcus-j-davies

This is where the CommonJS, ES and all that horrific sounding jazz evades me.

Basically, I cannot find any way to realistically migrate from CommonJS (const x = require('x')) to ESM in any meaningful way for something as complex as Node-RED. ES Modules can consume CommonJS ones easily enough but the reverse is not true.

I've no idea how Node-RED will be able to transition to ESM and I can't find a sensible way to create an ESM based node or even to incorporate an ESM only module such as the latest execa module.

While you can use a dynamic import on an ES Module, it requires you to effectively rewrite much of your node as async from the top down - just to accommodate a single ES Module.

@marcus-j-davies

I use Typescript during development, then compile to JS before releasing

And just make sure that you direct the typescript compiler to output CommonJS modules and then you won't have an issue.

[marcus-j-davies]

@TotallyInformation

And just make sure that you direct the typescript compiler to output CommonJS modules and then you won't have an issue.

Oh! Of course

Snippet of tsconfig

{
	"compilerOptions": {
		"module": "commonjs",
		"target": "ES2020",
         ....
    }
}

[TotallyInformation]

@marcus-j-davies

"target": "ES2020",

For reference, ES2020 maps roughly onto node.js v14 LTS. ES2021 maps roughly onto Node 16LTS.

[marcus-j-davies]

Yup!

The core driver i use (I'm sure you know - ZWave JS) its min version is Node 14, hence why I target.. that target , to allow greater compatibility

[TotallyInformation]

Same here. But not for much longer because I think v14 LTS is near end of life. The next major version of uibuilder (some way off as I'm busy on smaller updates at present) will target v16 as one of its potentially breaking changes for example.

The next major release of Node-RED will also likely target v16. It too currently targets v14.

[bwims]

Thanks for all the help folks! It's been an education! At least I didn't miss anything simple!

B.