Some thoughts for people who like `function` nodes and coding

General
1

Hey folks.

I know it has been sort of mentioned before in this tread

But I think this is a bit of a different perspective on it.

I am still very new to all this and some stuff I am seeing/learning these days is still WAY ABOVE me and my understanding.

I have also seen this link/video which is very interesting too.

And I am about to go against all it says. But why not? :wink:

If you are new to it all, you don't understand the big picture and I know it is easy to fall into habits which can (or not) be bad for you in the future.

BTW, I am NO past experience with GIT backups and the like. All too new for me.

So, you are writing a function node to do something.

WHERE DO YOU START??

Now, before I get to the STRUCTURE of the code, how about a few ideas to help you track the version of the code?

A HUGE thing that I am starting to do when writing code is to add a Header with the date/time when it was written.

eg:

//  2025 11 15 1450

Year, month, date and time.
It isn't Rocket science but gee, it goes a long way to keeping track of the code.

Yes, my flows are littered with backups of function nodes as I am developing them.
So what? It isn't illegal.

You write the code with this Header and copy it to a spare part of the flow - backup.
Test it. It works. Great, improve it.
When you've done the improvements, edit the header with the new time and repeat.

The times don't have to be THAT ACCURATE, but they help you see which is which.

Why so many backups? I hear you ask.

Well - my take on it - you don't always know where you are going with the code.
You are TRYING things. Seeing if they work.
And you may be several layers down and find this path is a dead end.

You need to go back a few steps to get rid of the stuff you added when you went down this path.
Back to a Good working copy then start again.
Of course you need to delete all the dead end path nodes. Helps reduce the clutter.

Then you start a new line of thought.
And it may happen multiple times.

When all is working - FINALLY - you can delete all the old copies.
But WHEN you delete them depends on how confident (and good your testing is) at that point in time.

A lot of things may not have been considered and bite you later.
So keeping some older versions for a few .... days/weeks/...... may be a good idea.

Try to group things.
There's no exact method I'm going to say, but maybe this way:

All functions together at the top
All configurable constants / definitions next
Then the main execution flow
Then outputs / status / return

It has been said that for every line of code there are FIVE lines of comments to accompany it.
I'm not going to quote exact numbers. But I do agree that a well laid out structure is helpful.

Break up the blocks with nice lines like:

==================================================
// New function block description here

As an example.

And maybe even add comments at the top which describe what the block is doing and a brief walk through of how it works.

Although the clip above mentions how DEEP your tests should be (maximum)
Yes, try to keep your conditional testing from being too deep.

Cut things down and have a lot of shallower testing happen before the code so if you want to change things it is at these shallower depths you are changing things.

I know it can be hard to see/understand all this - and believe me, I am still coming to terms with it - it really can/does help.

Hope this is helpful to at least one person.
(Though more would be nicer.)

All the best.
Happy coding.

Disclaimer.
A lot of this is what I've learnt from getting ChatGPT to help me and me getting it to optimise the code and stuff like that.

Compared to the mess/dog's breakfast I had, the new code is so much more compact and clearer to read.

So I think it is only fair I share what I've learnt with others and Pay it forward.

2

Oh, just remembered some other stuff to help too.

I've been caught a few times with variable names.

If they are not going to be used further downstream, convert them all to either upper or lower case when parse.

Eg:

// Iterate through the message properties and convert all keys to lowercase
Object.keys(msg).forEach(key => {
    // Convert key to lowercase and update the message
    msg[key.toLowerCase()] = msg[key];
    // Delete the old key (in original case) to avoid duplicates
    if (key !== key.toLowerCase()) {
        delete msg[key];
    }
});

Because now and then your finger may not clear the <shift> key quick enough and you don't see the wrong name.
It causes heaps of headaches weeks/months later.

3

This is a good discipline for sure. But while you are at it, if you think you will need to explain to yourself, or some future person, what it is about, expand that header with who wrote it and WHY.

Have a look at https://jsdoc.app for some ideas around standardising JavaScript self-documentation.

One thing I would add is not to forget that Node-RED is great at prototyping. I will often start by using the standard (and a few custom) nodes but then when I understand how I want the flow to actually work, if it has a lot of nodes in it and I've more to do, I may choose to condense those nodes into a function node now that I know how the processing should go. This is sometimes much easier than puzzling over JavaScript code logic. But sometimes I will go the other way. And sometimes I will start and finish either way!

That's kind of the point really, there isn't a right and wrong way to do things. Well, there kind of is, if it works well enough, that is probably the right way, at least for now. :smiley:

I guess this is a case of how experienced you are writing JavaScript and other languages. To me, I always (in JavaScript) use camelCase for variables and properties, just because I've become so used to it and I do it without thinking. For class names, I use PascalCase.

The key thing is to be consistent always. If you are working with others, then you all need to work to the same consistency (and in that case, you need to document it as well).

When you work on code in something like VSCode, one of the nice things is being able to define your standard "linting" to help maintain consistency. I REALLY wish we could use eslint in function nodes. Another advantage to working in VSCode is the ability to integrate AI and provide a copilot-instructions.md file which describes to the AI how you want it to behave. These can be quite long and also become useful guides for humans as well.

Thanks for sharing your thoughts to the community, these ideas are always welcome.

1 Like
4

Learn git and you won't do such things. It's like doing version control by creating files with dates, i.e. code-version-1.js then code-version-2.js - great and the perfect solution if you don't know how to use git (or any other versioning software - svn, cvs, rcs, mercurial,...). But once you do, you can easily get the dates of your changes by doing git log - no more extra stuff in source code files. (even cooler git whatchanged to see what changed a file and when for its entire lifetime).

Exactly! NR is great for copy & paste - create a function node, write some code. Want to change that code? Copy and paste the node and make the changes. Keep each function node as you copy and paste and you have your version control - visually :wink: Create a group to put all the old versions (i.e. function node) and e'Voila, you're doing it right! I say this in all seriousness because that's very much doable in NR, I would never do this using textual programming but copying nodes and keeping them around is super simple in NR.

But because I come from git and proper versioning, I created flowhub.org to do exactly that - visually version control my visual flow code. I don't want collection of old code cluttering my flows, so my old code is versioned off into a git repository via a button inside NR - done.

I would go the opposite direction - I would create my own node for specific functionality.

Or perhaps one step on from what you're saying: from those function nodes, I would definitely go to creating my own nodes to cover the functionality. Hence my nodedev collection of nodes so that I can create those nodes inside of NR. The difficult (or rather the hurdle) is the overhead of creating the boiler-plate code for new nodes. So it's simpler to stick with the function nodes. This I find a mistake.

The nodedev nodes are designed to generate the boiler plate and allows me to focus on pushing functionality into my own nodes. It's taking the subflow approach to the logical conclusion of generating node packages inside NR.

Anyway, it's good that folks are thinking about these things and how to program in an organised manner but always remember: you're not the first - there are a bunch of learnings from others who came before us. That's why there tools for source versioning, that's why there are style guides for how code should be formatted, that's why projects have certain directory structures, that's why makefiles exist ... etc

1 Like
5

I don't really see how that can help when using vanilla function nodes and editing in the Editor. For more complex code, you probably don't want to just rely on git control of the flow file, too many variables. AFAIK, there is currently no way to apply git to a function node - something to think about for the future.

Of course, that is another option. However, one that is not open to everyone and even I wouldn't quickly decide to do that given time constraints.

Indeed. I have some templates for new nodes but it is still quite a bit of effort. I really should create a generator - another future project. :face_with_raised_eyebrow:

6

Neither is ten finger typing, one can do it just as well with two. But once one has learnt to type with ten fingers, one doesn't go back to two. So it is when one learns a new skill. Git is a skill. You learn, you understand, and you might use the principles (managing change) in another context.

My point was to learn different skills so that you can make use of those skills. I learnt to do structure version management using tools, I would not go back to putting dates into source codes because I have tools that do that for me. I can still do that if it is necessary, and perhaps NR is a case in point where it is necessary to go back to putting dates into source code.

But then the question becomes how can I improve my workflow so that I don't have manage dates and instead its done automatically:

RED.events.on("nodes:change", (node) => {
        var dtStamp = (new Date()).toISOString();

        if (node._def && node._def.defaults) {
            if (node._def.defaults.hasOwnProperty("createdAt") && node.createdAt === undefined) {
                node.createdAt = dtStamp;
            }

            if (node._def.defaults.hasOwnProperty("updatedAt") && (node.changed || node.updatedAt === undefined)) {
                node.updatedAt = dtStamp;
            }
        }
    });

I added that bit of code to update dates on my nodes (that have createdAt and/or updatedAt) to maintain information about when I change things. (Aside: one of the first thing we were taught about CVS was to use $Id$ at the top of files, that would be replaced with the current version. Some my emacs templates for new files still has that :smiley: - If I used that today ...)

7

Most of my function nodes are far too simplistic to require more formal documentation though they do often have lots of explanatory comments. I generally don't bother with date stamps though (1). For actual nodes, those are, of course semantically versioned for the most part, though a few modules may have date strings for versioning since they may change a lot more rapidly than the package - mostly things like my CSS files. Of course, git and GitHub do maintain the formal commits but version strings can still be useful - especially if you want/need runtime version checks.

And yes, I do have some automated created/updated strings - mostly in individual documentation (markdown) files, if nothing else, these are also used to generate runtime visible copywrite statements. Mostly I use a custom shortcut in VScode. It probably wouldn't be hard to create an Editor plugin though that could automatically create headers with author and created/updated dates. Not something I would use enough to make it worth creating myself but I can see that in more commercial settings, that might be a really good time-saver.

  1. I should point out that I very rarely ever go back to earlier versions so this isn't really an issue for me. I may, however, sometimes leave commented out code in place until I'm sure that the new code works. :smiley:
8

Where to start? Start simple with what you enjoy the most. I started with simple coding similar to what you do in function nodes. Much later I learned git the hard way. However my background is from university when I studied programming and a decade worth of professional experience. I still remember how beginner unfriendly git was, I had already learned subversion(svn), but git is now universal. The thing about function nodes I enjoy is the freedom and "universal" way of coding. You don't need to know much about node red to understand the contents of a function node if you already know js. And if you already know js, your literal understanding reaches far outside node red. It is widespread, while arbitrary no-code node red nodes are not. To each their own. Enjoy the freedom to choose :smiley:

But if you want to go further, and take the steps of the pros, then you'd look into things like git (version control). Ideally there would be more support for test environment too, perhaps in the future.

That manual parsing of keys you added... I feel your pain, but your solution doesn't scale well. You'd need to repeat that everywhere? Perhaps typescript function node can help. That's the built-in way to ensure correct variable names. Unfortunately there's no other ts options or support in node red. It is a lifesaver for typos like that.

9

Replace git with Node-RED and only one of those statements is true. It's strange how git has taken over version control - it's just shows how well Linus Torvalds knew his target audience. (But he did model git very heavily on Mercurial - so he had a good starting point.)

It's incredible how one forgets that git has a greater market audience than NR - so which is harder to understand and use? Or are the problems that NR solve not relevant? And here I mean that NR isn't being used by the broader developer community to create simple prototypes to test stuff. It is completely capable of doing that (with function node, NR is Turing complete) yet it's not been done broadly.

That might well be changing. The argument that visual programming isn't for everyone no longer counts with n8n just doing a X-million seed round. So that excuse no longer counts.

My point here is that if you understand how to use NR than obviously a certain level of complexity isn't a problem, so why would git be a problem? But saying something is or isn't hard is also related to oneself - I find dancing hard, so I don't it. But I wouldn't tell you to not go dancing just because I find it hard.

P.S. Please don't use my arguments to present MS Word as the best word processing software and something that should be learnt!

1 Like
10

keys?

Do you mean variable names?

Yeah, but Way back when I did them one way.
As I progressed, it changed. (don't know if for the better or worse. But change they did)

As it is OLD CODE and I haven't touched it since it was written, the exact names used gets lost.
(Ok, no excuse. I wrote the code, etc etc etc.)

But sometimes subtle things catch me.

I'm sharing the stuff through which I've been hoping someone may find it helpful.

11

Even better argument for typescript. It never forgets the variable names, even for old code :wink:

12

I'll need to look into it.

But I'm not sure of the cost/benefit ratio.

13

Like a compiled language, TS has benefits for large-scale development with large teams doing complex things. Typically, that is not what Node-RED is being used for. The benefit of staying with native JavaScript is that you don't have to learn a language variant (personally, I find all the extra stuff crammed into the code to be far more a distraction than a help) and you don't have to do a build-step every time you change something.

I guess Node-RED's deploy process could take care of a build but it would certainly add yet more complexity to the already very complex Node-RED platform.

TS is not the only way to deal with variable/property name and type consistency. Monaco is perfectly capable of remembering and helping and can be further steered using JSDoc notations where needed.

Adding the ability to add Monaco extensions would be a massive help since you could then add things like eslint which is massively helpful as your code gets more complex.

But lets not forget that, for most people using Node-RED, this is a bit of a non-issue I suspect, because they are mostly using visual coding and not JavaScript. And even those of us using JavaScript in Node-RED, for the most part, we are writing relatively short, sharp bits of code that, with the help of some simple in-line documentation, is easily followed without the need for other tools.

If you do need something complex, I would strongly recommend not using a function node anyway, you would be far better off writing a node.js module that exports the required functions and that you include either by using mycode: require('./mymodule.js'), in the globals section of settings.js or that you include using the Setup tab in your function node. That lets you use the full power of node.js and an IDE such as VSCode with all of its powerful extensions.

1 Like
14

Could you elaborate on doing this then please?

It sounds like the better way to go (IMO)

15

Unfortunately, I think it may need some changes to core. Unless someone else has worked it out like @Steve-Mcl or @GogoVega ?

16

Never noticed the build step in practice (or "transpiling" or whatever). The process of translating ts to js. Of course it is there, but when you work on web for example, it's as easy as Ctrl+S and you see the result in browser immediately. Haven't tried ts function node in node red much, but I imagine it is as fast or faster than normal function node. Whether you like it or not is matter of personal taste. I imagine most people with backend experience of the classic strongly typed OOP languages would feel much faster at home with ts than js. At least I did.

The problem I have of making modules is it is more repos, more things to set up, and development is suddenly moved out of node red. I enjoy working in node red, it's easy :smiley: So ideally, for the more complicates or reusable bits, I would love to have a custom javascript library in node red, exactly like for example Thingsboard has. Another downside is I guess you need to restart node red to pick up changes from external modules? I have grown to enjoy updates during runtime.

17

I have several. :smiley: They are not hard to do and there are several options. I have multiple modules and plugins in UIBUILDER for example. This gives me all sorts of useful shared code, data and styling. For the Editor, runtime and even in function nodes. You just need to be a little careful about naming conventions so you don't clash with anything.

Yes, and plugins. However, not too hard to set up a watcher that will auto-restart for you. Probably not such a good idea for production use though!

Also, if you use a published package to host your modules, you can potentially have that as a node-red package. Some of those can now be upgraded in the Editor without a restart.

Another approach would be to have compute and/or data handling modules in either a separate instance of Node-RED or in a simple node.js microservice acting as an API server. That way, you don't have to restart your production node-red, only an external service.

Oh, a further thought. It is, in fact, possible to live reload a node.js module. So one idea might be a custom node that manages external modules and has a reload function built in. That would be a fun little project.

18

Sorry, I've kind of lost track. What is the purpose of this discussion?

1 Like
19

is a good article that describes transpiling.

20

It might have drifted slightly over time! Not like us, I know! :wink:

The bit you responded to was about how to enable Monaco extensions for use in Node-RED. In particular, the use of ESLINT to help people who want to do more complex JavaScript in a function node.

Some research suggests that the ESLINT vscode extension will not work directly in Monaco. However, it should be possible to achieve something similar using a browser-based web worker - or at least so Leo AI tells me. :slight_smile:

But then I started to think about some alternative approaches to providing more complex custom compute functions to Node-RED.

Interested in your thoughts on the matter.