Brittle flows or is it the visual paradigm?

Core Development
1

Hi There,

Over at the Node v5 beta 6 thread, @Steve-Mcl and I got into a discussion on the linkcall function call. I am opposed because of two main reasons:

a) it breaks the do-one-thing-and-one-thing-only Unix principle (basically the link call node is for doing linkcalls and a function node is used to execute JS code) and

b) the visual contract is broken that data flow is clearly shown in a visual manner (using a hidden calls inside a function node to pass data through a flow breaks that if there isn't a visual que to indicate that a function node is passing data around).

Steve talked about his flow being less brittle by having this added possibility inside the function node. The example he gave was this image:

image
image1537×1094 147 KB

That made me think: I don't see that as particularly "brittle" or confusing, in fact, it seems to me to be a normal "complex" flow in NR. The core comparison are the two groups on the left side (see below).

As a counter example:

Screenshot 2026-05-05 at 10.35.27
Screenshot 2026-05-05 at 10.35.271485×700 147 KB

That's a typical flow in my setup. That flow handles content storage (video, audio, image) and delivery for my mind map. Content storage is triggered by dragging and dropping urls into my mind map. The flow also handles interaction for audio transcription (via Whisper) and AI reformatting of texts included in my mind map. The AI and whisper stuff uses two instances of NR and therefore need to handle async messages passing to those instances (via http requests). Not shown are the AI and Whisper flows on those instances.

Premise 1: I believe that there is a certain level of visual complexity is simply inherent to NR and a visual development approach.

For me this complexity is simple part of using NR. I understand wanting to reduce the complexity by having a linkcall inside a function call but that goes against the ideas of having a visual approach in the first place.

For me, this is like saying "well I have too many characters in my C code, I want to use classes and objects to reduce that" - i.e., well then use C++. Secondly, one will use a lot of characters when coding C: it is a given since it's a low-level textual programming language with few high level abstractions. No C program that does anything will have zero characters and with each character complexity raises.

The good thing is, that C programming language is standardised and there is not arguing about what is and what isn't C. Even if its purely arbitrary (what is part of C and what isn't) it defines clear boundaries and folks can move on and build on top of C without worrying about functionality going missing or changing.

NR isn't standardised and the node granularity isn't fixed however I thought that a few guiding principles did exist. Such as NR is visual first and all data flow should be visual and that nodes perform just one task and one task only but that task really well.

Where is the line drawn between an visual approach to solving a problem and a textual solution? C is a standardised programming language, NR isn't and therefore this question will continue to arise for NR.

Premise 2: Better tooling would make complexity more understandable.

When I don't understand my own flows, I turn on message tracing to watch messages pass through my flows in real time. Visually I can then see where data is flowing and in the debug panel, I can see how data is altered as it flows - without deploying the flow.

Secondly, if I make changes and I forget what I changed, I use a visual comparison to visually show me what has changed in the editor - without deploying the flow.

Thirdly, the link call node doesn't have a visual representation of what it calls, i.e., no dotted grey lines. So I created a highlighter to for that uses green dots.

Fourth, visual refactoring - straightening lines, adding junctions to have straight lines. Reflecting on complexity and making it simpler to understand is also a "tool" that I use. Textual coding has many rules for refactoring (DRY, KISS, code smells, etc). There are no rules for visual refactoring.

I don't believe that removing nodes and replacing them with textual code will fix complexity in the long run. Of course everyone has their own level of visual complexity that they can understand, so pushing that to a textual complexity makes sense.

Here the flow using linkcall and not using linkcall:

Screenshot 2026-05-05 at 11.07.33
Screenshot 2026-05-05 at 11.07.33827×645 48.5 KB

For me, the bottom is clearer - the non-linkcall version. Why? Because I see that other bits of the flow are being called. In the top version I can read that the function node is doing the same. By reading, I mean I have to read the source code of the function node - the documentation may or may not be up to date. This limits the number of folks who can understand this flow - those that can code NodeJS have a clear advantage in this case.

The logical complexity of the flow hasn't changed, what has changed is that extra complexity is hidden in the function node, so that there is less visual complexity.

The intention of the post is not to discuss the virtues of the linkcall functionality, it is to point out that better tooling can help understand complexity and secondly, that NR isn't standardised - which is a shame since these kind of "discussions" will continue to arise as there are not standardised lines in the sand.

I made an attempt to standardise node functionality (granularity), creating a repository with visual unit tests to ensure that the inbuilt NR nodes do what they promise to do. I did that for my Erlang-Red project so that I could be assured that functionality was compatible. Again, a kind of effort to standardise node functionality so that NR becomes a pseudo-visual programming standard.

2

The function node is great because it allows to create/simplify the role of multiple nodes.

There are two types of users: those who decompose all the logic using basic nodes and those who mix basic nodes with function nodes. There's no right or wrong way to use NR; they're simply two ways to solve a task. Using only basic nodes is more visual but makes the flow more cluttered. It's a matter of personal preference.

3

Very well said and that's because there is no standard way of doing things. So in the long term, folks are all going to do their own thing. Hence my comment on standardising what NR does otherwise NR will drift into a niche and ever shrinking space. Because folks don't really understand how to use NR in the "right" way or for that matter, how to use NR in the first place.

Hence my comment on C (the programming language) being standardised. I think your comment is correct but it's just a summation of the current situation. If that's what is desired, then so be it.

Personally I don't mind because my livelihood doesn't depend on NR and I can just as well use any other visual tool such as n8n that seem to have better and clearer use cases that folks actually want.

4

I won't personally be calling links from inside a function node, precisely because it goes against the visual nature of Node-red.

However, there are already several ways to hive off processing, either internally in NR or via the exec and related nodes.
This aligns very well with the Unix principle that processes can easily be assembled into pipelines, and it's the reason that I first adopted Node-red.

If people find it more convenient to write a function rather than a switch node to only pass messages where payload > 0, that's fine by me.
Similarly, it's not unreasonable if people want to hide entire flows inside a function node rather than seeing all those nodes and wires. Let them do so.

5

You are pushing very hard on this topic. While there is, of course, some truth - there are folk like that. For many of us, Node-RED is a convenient platform for rapid development.

Function nodes are essential (for us) to reduce the visual noise of complex processes.

There will and should never be "one true way" to use Node-RED. That has never been its focus and if it were, it would be far less useful to an awful lot of people.

Before link nodes came along, a lot of use were abusing MQTT to disaggregate message flows, now we don't need to, but there are still plenty of occasions where we are happy to bundle some complex logic into a function node, add some comments to remind ourselves of what we did and call it a day.

The new proposal simply provides another link between that approach and reuse of visual flows.

6

Must admit, I am on the @gregorius , @jbudd side of this in that I would be very unlikely to ever use it, as to me it does break the visual contract. But then again I dont classify myself as a power user either in that I’m not running a business or have any mission critical apps running personally.

So I’m not against letting it happen, but would like some form of indication on the node so that if I am sent a flow that does use it can be seen easily.

7

Sorry let me rephrase this so that folks stop referring to the function node:

Standardisation and guiding principles are about preventing conversations such as these.

I literally don’t care about the opinions on the function node functionality - do whatever you want, I’ve given up on NR.

This discussion won’t happen if there were golden guidelines for what nodes should be responsible for (i.e. standardisation of functionality).

Good programming languages have standardised APIs - heck, even NodeJS has consortium to do that. What does NR have? Nothing.

This will limit and distract the further development of NR. But thats fine since nobody really uses NR for anything seriously.

That’s what I tried to say but I understand that unless you happen to be an enterprise architect, standards aren’t important.

The irony is that NR only exists because IoT equipment heavily uses standards to define how to access and control devices, yet NR is lacking documentation on what exactly each node should be responsible for.

Which is fine.

8

And that's where we are at odds I'm afraid. Because Node-RED is NOT a programming language. It is a platform for delivering dynamic apps.

Not that Node-RED doesn't need some standard approaches, it does, of course.

OK, I think it is probably time to call a halt to this discussion since it is going into mud-slinging now which isn't helpful. What you have stated is demonstrably untrue. There are lots of very large organisations that use and even rely on Node-RED - Siemens and ATOS for example are two off the top of my head.

9

Finally.

So lets discuss this - how could this be done? What are the steps for doing this? Any ideas?

I've heard of equal number of companies leaving NR behind. So I'm not particular convinced.

My experiences don't overlap with yours, where is the mud-slinging there? Simple different experiences because companies have different expectations of NR.

As the marketing says:

Its low-code nature makes it accessible to users of any background, whether for home automation, industrial control systems or anything in between.

So for what can you use NR? Well anything in between - which is quite accurate because that's what is its best use case: transporting and transforming data between data sources and data sinks.

But lets face it, there are plenty of SaaS solution for doing exactly that. There are also plenty of SaaS solutions for generate pages with gauges on them. So why should NR be any better? What is the NRs USP?

10

Why do you keep responding since you stated:

11

The same reason why we still have Churches - hope is eternal.

12

To me, the highest priority would be for people to actively contribute to Node-RED documentation.

If more people were actively contributing to it, it would help enormously towards filling in the currently often unwritten details about the use of Node-RED. Node-RED's API documentation, for example, is somewhat lacking.

Good documentation is hard, as I am fully aware from the 280 pages of UIBUILDER documentation, and most people don't want to focus on it.

We will have to somewhat disagree as to the nature of Node-RED - its flexibility will always make it harder for some things such as the things you've outlined, but much easier in other ways as I and Steve have mentioned. As has been said repeatedly now, these represent different and totally valid use-cases.

If there are areas that need better documenting, lets focus on getting those improved since that is more easily agreed upon.

There are, but none like Node-RED. And most of them cost $$$ for their more involved use-cases.

Now that is a decent question. Hard to answer easily (largely because of its flexibility) which is why it is worth trying to do. But should that be in a separate thread I wonder?

A few thoughts on USP:

  • Graph-based (in the math sense, not the charting sense) visual compute service.
  • Unique in style (other graph/flow-based tools are either n8n/Unreal Engine style or Microsoft PowerFlow style)
  • JavaScript (node.js) based - the language of the web, only 1 language to learn for custom nodes as well as for function nodes and web UI's.
  • Able to blend ETL workloads with data-driven presentation.

There's more but maybe that's a start?

13

To get back to the original topic: the philosophy of NR is to be able to perform complex tasks using a visual interface and configurable nodes. Each node contains internal logic that breaks down (reduces) the task to be performed. This makes the user's work simpler.

Since functionality is attached to the function node, it's natural to discuss its usefulness.

Every node has a task to accomplish, and the function node is no exception. But the difference is that it can perform several tasks in one, which simplifies the workflow. As I mentioned in my previous message, there's another, more visual way to accomplish the same task: using basic nodes, either alone or in combination with Subflow or link nodes.

Just because the function node performs several tasks in one doesn't mean it loses its visual appeal. For example, I use a function node to define a context while modifying the value. Visually I could use several nodes to do it but from my point of view (and my experience) it doesn't make sense to break down this problem because it is not complex.

14

Documentation is secondary, I mean standardisation - I mean MUST BE ... SHOULD BE.... etc something like an RFC - clearly defining functionality of each and every node so that folks can rely on them being exactly that - now and into the future.

AS EXAMPLE:

"The switch node WILL define the flow of a message based SOLELY on attributes of the message." - something like that. So that using a global/flow variable would be not be allowed and hence maintain the stateless of nodes.

Hence the POSIX standard that defines how the Unix system works? Because Unix is so flexible. The basis of which (the POSIX standard) allowed Linux to develop in parallel to existing proprietary Unix system but be 100% (see wikipedia article) partially conform at the same time.

Or are you claiming that NR is more flexible than Unix? No. So it is completely possible for NR to become the standard implementation for a standardised visual programming environment, something like a Flow Based Programming standard that is implemented by NR and which can be implemented by others in parallel but a certain level of compatibility is maintained.

Of course this would mean that nodes are defined and not changed but because there is a plugin system with 5900+ nodes, folks can develop their own variations that others can install if desired. The core set of nodes would remain constant - unless the standard were updated.

And yes that would also include the function node which, ironically, is already limited, i.e. I can't actually access the NR runtime (or the NodeJS engine) in the backend because Nick limited what the function node can do. So there we have a start of a standardisation: "The FUNCTION node SHALL only do what Nick says it WILL do." :wink:

15

NodeJS is yes the language of the web but no, not the language of data - that seems to be Python. So in fact, creating ETLs with NR isn't such a good idea unless you speak of ETL orchestration. Don't send large data packets through NR - clone message becomes prohibitive. Now if there was a standard for visual FBP programming, then there could be a Python-RED that could be used for ETL but whose flows could also as easily be ported to Node-RED.

Unfortunately most corporates that build ETL pipelines consider python and AirFlow to do that. NR doesn't seem to be considered or is considered until the point when large data chunks are passed through it.

Unique in style but how is that a USP? Yes it's unique but is that a selling point? In fact UIs are meant to be standardised or why do all websites look the same, why do all word processing software look the same, etc. What you really want is to become that UI for flow based programming, so that others have to follow your UI because folks are used to that UI for flow based programming.

That's what MS did with Word and hence LibreOffice looks exactly like Word and co. (including the bugs). Word became the word processing UI that everyone knows and expects.

So in fact, creating a standard and being the first to do that, could be beneficial for NR. Being the go-to tool for FBP but also allowing others to inter-operate in a visual manner.

17

I think you are missing one important aspect of this.

The demo flow i supplied works with the node.linkcall (the /ai1 flow) - it does not work with visual flows (the /ai2 flow is an an outline only)

The brittle part is how I solved /ai2 flow.

If you were to load import the flow, you would see there is there is a working Agent on /ai1 (sans the AI/LLM - its just a langgraph with random tool choices). As I stated several times, in this instance (and others I have encountered), I had to do non-visual hacks (don't wanna give too many clues how I did it) to make things work due to the state machine that is imposed by the npm package (I have no choice here). That (amongst many requests to support reusable logic) was what led me to raise the whole topic and write support in core.

I really am truly interested to see how someone (other than myself) would solve this (the /ai2 flow) using standard wiring and without context hopping

19

Sorry guys - I just realised this thread was locked.

I reopened it in case anyone wants to show me how the demo flow I posted can be done via standard wiring.

If no responses in a day or two, feel free to re-lock

20

Dave, do you ever store logic (functions) in context and call them from other functions (so as to be DRY)?