(Async) multiple input for nodes*

[Robert0]

Yes, this works.
No, it is not what you think it is.

I went on to make an asynchronous subflow, which in reality is a upgrade to the grouping system. Doing so I accidentally allowed myself to mimic existing nodes, but with more than 1 input (or output).

I am not trying to spark a debate about a topic brought up this often. I just realised many people were asking or wondering about a node with multiple inputs.

The majority of posts I have read regarding multi inputs were about asynchronous inputs. This you can eassily solve using some merge, split and message properties. Now if we mix that idea with some links and illusions, then that gives us an async subflow with multiple inputs.

The plugin/node allows me to create any type of custom node with multiple in- and outputs. I just happen set the appearance of this node as a function node. Although in reality it's nothing more than native Node-RED nodes being smashed into together. This is why your flow remains working after deleting the plugin.

So what happens?

It's actually not really a custom node as much as it is not a multiple input node. There are some illusions going on. Which was my goal from the start; make it work without changing the philosophy and core of Node-Red.

Step 1

You start by making your set up in a flow. This can be whatever you like, however in this case I take the Bacon and Cheese example.

Each function has its specific code, example:

msg.payload = "bacon";
return msg;

Step 2

Create or drag your setup into a group. Right click the group and create a localflow.

Step 3

The group gets replaced by this localflow node.

Double click the node and click 'Edit localflow'.

Step 4

Edit the localflow to your desire. Add your in- and outputs and go back to your original flow.

Step 5

Connect your localflow node to the nodes you want.

Step 6

Change the appearance and name of your localflow node to your likings.

Done

You can see this as a seperate flow that lives inside your main flow. The in- and outputs are nothing but masked links that help sell the illusion of a multi input node. This is the main reason your flows remain functional even without the plugin, which helps you mask reality.

Additional things

• You can copy paste the localflow node to create multiple versions. Unlike a subflow, it's not an instance and changing one localflow does not change your copies.
• You can create unlimited localflows in a flow. However currently you cannot nest localflows inside localflows due to complex problems (for now).
• There is no input/output limit.
• Change input and outpunt names on the localflow node and inside the localflow.
• Your explorer will handle your localflow as a group:
  
  

What I am working on

• Solving edge cases and visual glitches. It happens that the facade is lifted during specific changes in your editor. None of these effect the functionality, however to remove the illusion of having a multi input node.
• Nesting localflows in localflows
• Saving localflows as a template, so you cansave them or drag them out of your pallete sidemenu on the left.
• Adding synced templates, where 1 change in your template changes them in all other templates with sync enabled.

Why?

This makes it possible to package complex logic into a single reusable node, keeping the main workspace clean and easy to understand. In practice, it is similar to creating a separate flow and connecting every input and output with link nodes. You can also see it as a group that is coated as a node. The difference with another flow is that localflow keeps everything inside the same flow, can be copied as a single unit, and behaves visually like a normal node.

I am sure there are many more use cases for localflow than the ones I have mentioned, and everyone will probably find their own way to use it. I mainly used multiple inputs as an example because it is a topic that comes up frequently in Node-RED discussions.

Your thoughts?

Let me know your thoughts on this proof-of-concept. Is it what you would've expected from a plugin attempt? Is it different? Would you use it, and why (not)?

I understand anyone disliking this as it might give the wrong impression of what is and what is not considered the idea behind Node-RED. It might even be confusing for anyone using it, that might think it's synchronous.

My thought on this matter is that despite staying true to Node-RED native features and philosophy, it does what many people discussed or requested in posts. This might satisfy their needs and since it is a optional plugin one can install, there is no downside for those that are against it.

Help me

If you enjoy my attempt at creating this abomination, feel free to message me if you would like to help test the plugin. I want to explore as many possible situations as I can before I can confidently say it works properly.

The main purpose of the Localflow project is not simply to add multiple inputs to a custom node. Its real purpose is to let you create a local flow inside your existing flow.

~ Stay positive please.

[jbudd]

I think it's a very interesting proposal, suggesting a recursive ability for Node-red.

Node-red is a visual programming environment.
You can't tell exactly what this part of a flow does, but you can see how the data flows through it.

It gets harder to comprehend a flow as it gets bigger and the wires become spaghetti. Then it can be useful to group nodes so they visually form a block with a common action.

And the group can occupy less space if the node names are not shown, but the data flow is still clear.

Sometimes it would be nice to shrink the group further using your proposal, though I don't like that you made it resemble a function node.

Perhaps clicking once on it would popup an expanded view, double clicking would open the editor for that entity, whatever structure it really has - group? subflow? linked flow?

There are of course other techniques to reduce visual clutter (and to reuse code): link calls, subflows, rewriting an entire flow as a single complex function node.
Recently we have seen a proposal to reduce spaghetti by making long wires invisible.
Go too far down that path and there is no longer any point in using a visual programming language, better to use a more traditional language.

[Robert0]

Hi, I agree with what you said. It was a challenge to define this local flow like thing.

Will it be a group, a modified subflow? Its own actual flow but just represented in another flow?

I ended up making it a reference to how group works, as you mentioned this is used to clear things up.

One of my initial attempt was when double clicking this node, it would popup the group flow and blur all outside the group. You were still able to move nodes in and out of that group and when you were done, it would collapse again.

There were some implication that caused me to go through this route. However in fact, the group you make never dissapears and remains as a hidden group in your flow. This localnode is just linking its in and outputs to the special input and output nodes in the group.

The fact it looks like a function node, was for the sake of the example. I mentioned before that you could customize the node if you wish, making it look like other nodes. In reality you should keep the standard localnode appearance.
I just decided to allow it to be modified like all nodes.

I will have another look at the group popup view, rather than a seperate tab. However i did notice, it felt better if you were able to keep a tab open for that grouped node for quick lookups, checks, etc.

[gregorius]

I once wrote an article comparing the visual representation used in various visual programming settings. There are reasons for multiple versus single input nodes and these are very much valid.

Adding multiple inputs might reduce visual clutter but it increases the semantic clutter of the underlying code - so the clutter is only being moved deeper. IMHO, no real big win.

The reason why Node-RED only has a single input is synchronisation - how to define that two or more events arrived at the same time? Obviously you want to handle one event arriving, two events arriving and only one event arriving but on the other input port - what is the time frame for defining simultaneous arrival?

Other visual approaches allow multiple inputs - why? Because they use state: each wire has a state and that has value at every and any point in time. Therefore nodes sample values and us these to compute their output value - execution of nodes is triggered by a clock.

In Node-RED, execution of nodes is triggered by events arriving on their input port. There is no synchronisation clock that ticks over and ensures that nodes are triggered to compute their values.

I think of Node-RED links not as wires but as pipes along which something might or might not be flowing. Wires always have a value, wires carry electricity and are either on or off - but have state at all times.

Hence I don't see how two or more inputs can either work within the existing execution model of Node-RED and secondly what exactly (besides reducing the visual clutter) is the benefit of two or more inputs is.

@Robert0

This you can eassily solve using some merge, split and message properties.

Sorry this is hand waving. Solve this first in the context of Node-RED and then solve the visual issues. It's harder than one thinks because of the underlying assumptions made by Node-RED and it's execution model.

If you want to use auto merging "when ten messages have arrived then execute" - well I can do that today using the join node in front of the node to execute. "when ten messages have arrived on port two execute" - well I can do that today by having separate streams of messages in parallel, i.e. simulating a second port but passing in all messages (once collected) on the same port and setting a flag on the message to indicate that it's port two messages.

@Robert0

The majority of posts I have read regarding multi inputs were about asynchronous inputs.

It's a pity you didn't provide a reading list, then we could all be on the same page and then also better understand your thinking and motivation.

It is also, with such a broad sweeping statement such as "lets have two or more input ports", that experience with Node-RED, problem being solved, approach to problem solving, background before Node-RED, all those things have an influence on how one understands the original statement "2 or more ins are good!".

I accept the single input dogma and get on with solving the problem I'm facing. I've not had a moment of going "Oh I wish I had two inputs" because I understand underlying "why" and know that it's a fundamental decision and something that - probably - won't be changed in the near future. Or something that would take a lot of effort to implement for, as I say above, little benefit because the clutter is being moved, not removed.

[knolleary]

There have been many discussions historically around this type of thing. I'm wary of seeing a third way of doing things emerge, rather than looking at improving what's already there.

It feels like you are reinventing subflows but with two differences:

• Subflows can only have one input
• Subflows are templates - edit the template, you edit all instances

For this latter thought, your work is more like the group feature; they are entirely independent sets of nodes - copy and paste gets you a separate copy you can edit without modifying others. But groups today don't 'collapse' down.

Have you considered working with us in the project to propose enhancements to what's already there?

[Robert0]

Hi,

Thank you for that article. I was aware of how exactly Node-RED works and why they have selected this approach. I am also familiar with many other frameworks, such as Vykon Niagara and Siemens AF. I wanted to avoid any discussions as enough has been said throughout this forum.

It is my mistake for not being clear on what side I am and what type of plugin I was making.

Apologies

EDIT: Some posts I tried to find back:

Subflow multiple inputs support

I has viewed the forum why-dont-node-red-nodes-allow-multiple-inputs. And I have some different perspectives about multiple inputs in subflow node. In an example like the one below,multiple inputs will be more convenient. [20240909-144956]

Multiple Inputs and Multiple Outputs

Good morning everyone, I am using Node-Red with OPCUA to connect to a controller and display the values in their own gauge to the dashboard. I was wondering if there may be a better way to do this. My thinking is that it is not necessary to create an OPC UA Client node for each OPCUA Node that I want to read. I also wonder if this is creating a separate connection for each node that I want to read. Can someone please advise if my thinking is correct and if there would be some way to use only…

Reading multiple items with node-red-contrib-opcua

I am including several production lines into my data-gathering program. They use OPC-UA to offer me the information. Single parameter readings are not a problem anymore: I send the nodeid for the parameter I want to read, and I get a message with that single parameter. This is fine. However, we have 3 production lines (times 3) and we're talking about reading hundreds of parameters. Having a full message for each one will be an incredible overhead, so I want to use the read multiple function. T…

[Robert0]

Hi, you are right.

My original plan was to recreate a subflow that was not a single instance and with multiple entry points. Again to be clear, not for synchronous inputs, but for multiple gateways to a single subflow.

The group part is a base where i started to easily select and convert a group of nodes into this local flow (collapsing), rather than creating my own selection, detection and explorer visualisation methods.

Note that as mentioned before, what I have made is not an invention nor is it anything new.
Node-RED and its users already do what I do. They already use nodes and groups with multiple in- and outputs. I only visually changed what it looks like and how you interact with it.

If you delete the plugin, you end up with this:

In some cases, people put these groups in another flow or keep it as a group in the same flow, but without the link nodes.

Plugin related, I see this as a enchancement of what is already there. However it was not my intention to request this to be added to the core. As you might have seen the opinions of this are divided, rightfully so.

[gregorius]

So visually, this is what you're talking about:

Creating a group of nodes where the group construct has inputs which are routed to specific nodes within the group?

So giving groups the ability to have inputs would be an alternative?

EDIT:

which then opens interesting possibilities for nesting and doom-scrolling into goups (imagine the sub-groups here being minised and each group having inputs):

would become when scrolling out:

[gregorius]

@Robert0

This is what happens.

Ok, well then it's a good idea.

:-b but do it with groups instead of plugins!

EDIT: using groups clearly indicates that multiple inputs are something non-node'ian, i.e. nodes don't implement multiple inputs but groups can.

[Robert0]

Let me rewrite my message, as you added context to yours.

So yes, nesting is possible. However as I mentioned in my to do. Nesting caused some naming issues and display issues. I am working on it. I needed to find a way to also avoid having the local flow being placed inside of itself, which crashed node red ofcourse.

The goal of this, is to replace the image of the group. Its all node-red native what i use.
I decided to cherry pick features from subflows: (toolbar, flow in a flow), group: (selection and usually definition).

I am trying to now, template the local flow. So you can share, store, easy drag and drop predefined action/functions into your flow. A black box one can say (except you can edit it if needed).

What @knolleary said already. Subflows are templates and a single edit changes all.
My goal was to make this optional. A checkbox that either syncs it to a certain instance in your template folder or a copy of that template that has no attachments to the original.

Besides a group, we are not far from looking like a subflow either. As i mentioned in the post, its a mix of both. Cherry picking the "best" of each.

I enjoy making subflows for repeating complex tasks I want seperated from my regular node flows. What i also want is a subflow that can be grabbed from the template pallete, but adjusted per flow if needed.

The multi input is not a change to the node-red core. Its just a different visualisation of making it easy to drag and drop this new type of template, link the wires, link your output, give it a name and slap a price tag on it.

EDIT:

Hence my first paragraph:
"Yes, this works.
No, it is not what you think it is.

I went on to make an asynchronous subflow, which in reality is a upgrade to the grouping system. Doing so I accidentally allowed myself to mimic existing nodes, but with more than 1 input (or output)."

Its not what you think.

[gregorius]

@Robert0

I went on to make an asynchronous subflow, which in reality is a upgrade to the grouping system.

I think you should be aware that end users are going to be - potentially - confused between what you're doing, what already exists with subflows and how groups are used. It's great to create something new and learning stuff but Nick is quite right when he says that taking the existing and extending that out might be a simpler first step.

But each to the their own and it's still a great learning for you. I would personally rather just have groups with input ports that get visually routed to the nodes they contain. Seems like something "relatively" simple to implement and visually everything is there (in my screenshots I'm just using junction nodes on the edge of the group).

[Robert0]

I agree, users can get confused and mistake it for being a synchronous node. However I must say for a plugin a user downloads and for the group nodes they self design; I think they will manage to avoid using multiple inputs into a single other function. In the localflow editor, it looks like any other flow. You are right with your design that you see the flow go into certain nodes. However in the localflow editor, you also see the same behaviour. It should not be more confusing right?

Its when you give someone a premade localflow and they use it, they might misunderstand how it actually works. But then again, for a optional plugin you can search for in the pallete manager it should be fine.

Regarding how I represent this idea, you are right. You can have many methods and ways of displaying the same idea. Ive struggled some days to get something I was happy with.

Yet different enough to be actually useful for myself and others.

If you wish, feel free to download it from the pallete manager and give it a go. Im open for any changes that either improves it or visually makes more sense.

[TotallyInformation]

This does feel that you've perhaps explained things from your own perspective. In doing so, I think you've made things more complex and confusing for others. Something I find myself doing all too often.

The concept of a collapsible group is something that has been raised before and something that I would absolutely welcome as a core feature.

Reading through the thread, isn't that really the main thrust of your idea?

With the addition of having these groupings available as templates for re-use. And actually, it is that part that I'm not personally so convinced about. Not that it isn't a useful idea, just that it seems to clash somewhat with the existing features of Node-RED and the way it works currently.

If we were to get collapsing groups, I think that would be really helpful. Though it would need some visual clue. Having something that simply looks like a node but allows multiple inputs because it is really a group would be very confusing and not a good UX at all. But, with some visual indication to set it apart, it would be super useful and a sensible extension to Node-RED's existing features.

For the template part, personally, I would be against that. At least for now. If people want it, they can already mostly replicate it with a sub-flow and copy/paste I think.

It would be really good if you could take Nick up on his suggestion of a discussion with the core devs to see what could sensibly be adopted for future releases.