Add opt-in property that allows specifying the types of nodes that a given node can connect to

Core Development Feature Requests
1

I have a use case where certain node must not be connected to other nodes.

Given a set of nodes types A, B, C

node A can only send its output to node B
node B can only send its output to node C
node C can only send its output to node A

Examples of valid flows

A -> B -> C
B -> C -> A
C -> A -> B
A -> B
B -> C
C -> A

Examples of Invalid Flows
B -> A -> B
C-> B

Feature Specs

  • The client-side of a node must have a new property that will instruct the editor to not let it be connected to types that are not listed. This property can receive function that resolves to a boolean, or an array of node types. When using a function, the destination node properties are available to the context of this function, and this refers to the node that the user is pushing the wire from. When the resolved value is true, the connection is allowed. For example:
RED.nodes.registerType("my-type", {
    category: "foo",
    inputs: 1,
    outputs: 1,
    connectsTo: function(destination) {
        // NOTE: destination has information about the node that I'm about to connect to
       // NOTE: "this" is also available and it refers to the node's own context
       return destination.type === "foo"
    },
    ...
})

or

RED.nodes.registerType("my-type", {
    category: "foo",
    inputs: 1,
    outputs: 1,
    connectsTo: ["my-type-2"],
    ...
})

  • connectsTo is an opt-in feature. This means that nothing that is currently working will stop working.

  • Developer is responsible for declaring all nodes, such as debug, when using this feature. If he fails to do it, consumers won't be able to attach a debug node to the output of the origin node, unless they disable the "Connection Checks" for that node type, as described below.

  • This feature can be disabled per node type in the Editor. This way consumers can also opt-in when combining these nodes with their custom nodes. To disable this feature, users can "Double Click" on any node in the canvas of that particular type, and uncheck a prop called "Disable Connection Checks".

  • When the User drags the wire to a node that is not allowed to receive a connection from the origin node, the editor will display an "x" where the connection would have happened as a means to show the User that the connection isn't allowed. Additionally, while dragging a wire, similarly as the "catch" node, only the nodes that can receive a connection from another node must be highlighted to minimize the number of tries.

  • Additionaly, in the server, when the flow is about to be started, a validation will run to verify all connectsTo that are enabled (remember, the User can disable connectsTo for a particular type in the editor). If an error is found, the editor is notified, and the flow won't be started. All erros messages are sent to the editor as follows

The following error(s) were found: 
- Node (id/name) of type A is not allowed to connect to Node (id/name) of type C
- Node (id/name) of type B is not allowed to connect to Node (id/name) of type A
  • All node types that had its connectsTo disabled in the editor have to be serialized and available in the exported flows.json.

Another Reason for this feature

Type checking messages using JSON Schemas, or even Typescript, isn't enough to validate a node can process a certain message because 2 distinct node types can ouput messages with the same Schema/Type, but with different contextual data that won't be understanded by the node that is going to process the message. For example, the following messages from Node types A and B have the same schema:

output from A

{
 "source" : "bar"
}

output from B

{
 "source" : "#0103213913"
}

Both schemas are the same, but Node C can only process the output of B, based on the context of the data.

Use Case

I'm using Node-RED flows to describe flows using AWS resources, like SQS, SNS, Lambda, S3, EventBridge Triggers. Because the SQS can't send messages to my SNS node, I must not allow Users to do such connection in the Editor.

Valid flow

http-in -> SNS -> SQS -> Lambda -> S3 -> function -> http-out

Invalid flow

http-in -> SQS -> SNS -> Lambda -> S3 -> function -> http-out

I don't need this feature to make this work, but with it I would be able to provide a better UX for developers.

2

Does the editor, when drawing a wire, know what type of node it started from or what type it's currently hovering over?

Are these nodes A B & C custom nodes? Can they be coded so they

  • Append msg._nodetype to outbound messages
  • Give a runtime error if msg._nodetype is not aceptable?
3

Nodes A, B, C are all custom nodes.

This validation can run in two moments

  1. editor (client): node-red client, the editor, will not allow connections based on the new connectsTo attribute
  2. starting a flow (server): node-red server will validate if all connections are valid before starting a flow. When it finds invalid connections, based on connectsTo, it will throw an exception and not start the flow. It must collect ALL wrong connections because as a User I want to know all errors with a single deploy action. An "error" message will appear in the editor saying
The following error(s) were found: 
- Node (id/name) of type A is not allowed to connect to Node (id/name) of type C
- Node (id/name) of type B is not allowed to connect to Node (id/name) of type A
4

That could be terribly inefficient. Really probably not worth doing as a core task for what is, after all, a rare edge-case.

However, you could easily create a common library used by all your nodes that has a function to run the checks as node-red starts up.

I don't believe that nodes "know" when they are wired - not sure if there is an existing event, I don't believe so.

However, you should be able to check already when doing a deploy.

But remember, one of the guiding principals of Node-RED is that nodes should not care about upstream and downstream connections. So it would seem unlikely that this would be adopted into core. But nothing stopping you from doing your own checks in your own nodes.

5

This reminds me of typing discussions - is it better to have strong typing (e.g. C,Go,Typescript) or loose typing (e.g. Ruby, Javascript, Lisp).

I personally like the fact that Node-RED doesn't do any checks on what is connected with something else.

Having these checks will lead to a rabbit warren if UI questions: how does a user now that A cannot be connected to C? What is the error feedback in the Editor? How does a user know that there are these limitations? How about nodes from other node packages, how can I define that my nodes cannot to be connected to some other nodes from another node package?

It just seems like a completely different editor having these checks.

6

Why is it inneficient if it only runs JIT, during restarts? Could you explain the "why(s)"?

I also think that the client-side part of a node can't know which nodes it is about to be connected to. But I don't think it is hard to add a new hook method called in the origin node, and only once, when the User started to drag the wire.

I don't see why this feature request would make anything that is currently working stop working. If nodes don't provide a connectsTo, nothing should change. It is an "opt in" feature.

7

I think I answered your questions:

8

I updated my post with my usecase to make it clearer

9

@gregorius

I added an explanation of why I think typechecking using either ts or json schema isn't enough for determining which nodes can be connected together. Take a look and tell me what you think about it.

10

What if you have hundreds of nodes and a limited set of resources. Why should we force Node-RED core to wade through all the nodes with all of the possible connections just to catch a marginal edge-case? Inefficient for most of us.

Also, what about link nodes? Would your process have to deal with those as well? And sub-flows?

The runtime execution of Node-RED should be as efficient as possible in order to cope with a future expected separation between the editor and the runtime.

If you are allowing none-Editor changes to the flows, it is your alternative editor's responsibility to validate that changes are valid.

So no, I really would not want to see something like this added to the runtime startup of Node-RED.

Have you checked the core code? Do you know how complex this might be. What other impacts it might have?

And it goes against some of the current design principals. Not that those couldn't be changed if the core devs agree but such changes should not be undertaken lightly.

And again, this is a rare edge-case. Mostly nodes should not and do not care at all about their upstream/downstream neighbours. They care about the data they are given and there is a single exception for outputs regarding forced cloning of a message if 2 or more wires are connected to an output. That's it.

I didn't suggest it would. What I'm saying is that it is a lot of change, potentially away from current design principals, for what is a rare edge-case.

Particularly, as has already been stated, you could do a lot if not all of this yourself in your own nodes using existing features. Not quite the way you've outlined but in a way that prevented deployments from succeeding if connections between your custom nodes we incorrect. This would also completely avoid the need for a startup process having to re-assess the entire set of flows. It would also avoid the need to change the core for an edge-case.

That isn't what you said previously since you are suggesting that, at Node-RED startup, a review of flows would have to happen.

11

you don't understand what I meant, I meant:

  1. Does a node have a type? Yes, a switch node has the type 'switch' (for example)
  2. Is checking whether type A can connect to type B typechecking? Yes it is since it's a check based on node type.
  3. Does having a connection declined because of type imply type checking? Yes it does.

Hence my metaphor between typing versus non-typing programming languages.

It's the concept that I meant not its implementation in javascript v. typescript.

And that's why I dislike this feature, because I dislike type checking - especially within the context of high level flow based programming. This feature just adds complexity that makes starting out with Node-RED for new users even harder.

The only benefit I see is that there is no benefit, just more confusion and complexity.

You can also think of this in the context of shell and using pipes. I don't want my bash shell telling me that command-a | command-b doesn't work because command-b doesn't like command-a - wtf!

If you really want this, then build yourself a node package that does this (at least in the frontend). There are plenty of events generated by the Editor upon which you can build. Even accessing the catch overlay is possible. In addition, it is incredibly simple to extend the existing definition of nodes using extra callbacks - just as you've shown above. These are all accessible via the _def of a node (in the frontend).

12

Mostly agree with you on this @TotallyInformation!

Just recently I implemented a feature that I though I would never use - it was an edge case but I thought "well might as well build this while here". Anyway, turned out that a "edge case" became a useful feature in another context that I hadn't realised.

Therefore I won't use "its an edge case" as an argument per-se for not doing something. It now might seem to be a and edge-case but who knows what happens tomorrow.

But still, I would not endorse this feature - simply because it adds complexity and confusion for limited benefit.

13

You can do this check with a plugin or code that runs in the editor.

There is no reason in the core nodes to prohibit a node from connecting to another.

1 Like
14

It is a very simple iteration that would happen right before starting the flow. The worst case scenario is if all node types of your flow have connectsTo enabled, which would be extremely rare like you said. This validation would be as follows, using pseudo code

for all nodes in this flow
  type does not have connectsTo 
     continue
  type connectsTo is disabled
     continue
  validate connections

Core nodes would not change because they don't, or will ever have, connectsTo configured for them.

This validation won't not run at runtime. It happens only once, right before starting a flow.

I'm providing a palette of Nodes to my customer, and they can use as they want. This proposal is to find a way to avoid wrong connections.

Yes, I have checked the core code a few times and I know it is complex. I said this could be easy because this is an additional opt-in validation. My line of thought can be summarized as follows

BEFORE:
validation-1 -> validation 2 -> start flow

AFTER:
validate-connections -> validation-1 -> validation ->2 -> start flow

I still don't understand why it goes against design principles. Can you say which principles this breaks and why?

How can I ensure users won't connect Node A to Node C while providing a good user experience?

15

Again another "there is no reason for" but without saying a why.

I gave a use case, as a user and developer, and you ignore. How is you evolving a product without user feedback? It is annoying...

Can you say "this can't be done because there is another better way that our brains can do". Here is how you can achieve your use case?

16

First:

And:

The annoyance you mention is that no one here put up their hand and said "hey let me build that for you for free" - you have a clear path if you want this, either take that path or don't. QED.

17

Just my 2c.

I would hate for Node RED to stop me from being able to do what I want in the editor in terms of message flow/connections

if A node stopped me connecting it to another node - Its a node that won't be in my palette for long

Example:
A 3rd Party Debug/Logging Node, can't be used if this was implemented by a Node.
This is just 1 example, I'm sure there are more situations that would infuriate me

Agree with these fully!

18

Fine I build it myself and won't shere too.

19

Do you know how many nodes I have that are private? It's definitely more than none but less than ten!

But yes, it's definitely up to you. The worst thing that can happen is that you learn how the editor works in more depth, you then share your prototype and suddenly people are excited and it gets integrated into NR 6.x!

EDIT: It just dawned on me, what you're describing is something like Blender node edtior which is very type oriented and each port is coloured coded so it's obvious which types ports have i.e. its typing on port level not node level. But the Node-RED isn't Blender, NR doesn't make assumptions on types. NR doesn't have multiple inputs, it doesn't have state (Blender node editor is basically a state machine with a clock) and it doesn't handle graphics.

20

There is no enforcing anywhere in the list of specs I added in the post.