Strictly typed message payload across nodes

Core Development Feature Requests
1

(i know there was a similar request , but it was shut down with "its impossible" rather quickly)

1 allow "consumer" nodes to specify a json schema for incoming payloads
2 allow porducer nodes to specify a schema
2.1 use "json2dts" to create schemas from actually emitted events to fill in for those that dont
3 strictly type all transformation nodes to create similar schemas
4. validate schemas of connected nodes against each other as much as possible

  1. add autocompletion to msg.payload wherever possible
    (fields with msg.payload, the monaco editor,...)

  2. indicate possible mismatches in the ui

2

"all"? I don't think that could ever work? As an option maybe.

3

well whereever possible and applicable (for the nodes provided by nodered) - i think something like

  • "function" should generate an output schema (and be able to auto complete based on connections)
  • "filter" should be able to autocomplete msg.payload fields
  • "inject" with json should provide a schema based on that json

things like that
(this feature request is mainly about creating the framework, which nodes and to which extent they are typed in the end can be a work in progress )

i think it could also be a possible alternative to create a "type from data" node that explicitly creates a schema based on the payloads it has been provided which can then be used as an explicit "opt in" for typed payloads

4

At what overhead? This would have to be optional I would think? Would the "cost" of producing it really always be justified? Also, I assume that the schema would have to be passed with each message (since there is generally no other link between nodes) which again could be a very significant overhead, especially for high message rates.

"should"? Again, same issues.

I am not entirely certain but I believe there already is such a node.

Please don't get me wrong, I'm not dismissing your idea or denying that there are some genuine use-cases. I am just asking you to think about the wider consequences and how they would be mitigated.

5

im not gonna say that might not also be a vlaid otpion depending on the use case , but my initial idea was to just generate that schema when there are changes in the ui
nodered would only have to trigger a revalidation for all connnected nodes based on the schemas

6

:thinking:

Well, if nothing else, that would require some significant internal core change since the strategic assumption at present is that nodes do not need to know about each other.

What you are suggesting I think is some mechanism that lets one node communicate its schema to another and not via a msg? If so, that would need to work regardless of how many intermediate nodes there were in the flow. That is the elegance of the message passing approach because it doesn't matter how many nodes there are as long as they continue to pass the message along.

7

well yes .
my suggestion would target the core moreso than the nodes
nodes can declare type information and the core will validate them against each other
so the ndoes wouldnt have to know about which nodes they are connected to for this validation step

for passing along the type info of previous notes it would be enough to accumulate the schemas of all input connections and only pass along the final schema

8

here a few more outlines:

i collected a way to transform between schema types the past few days (for my own version)

the general idea is that nodes can specify a JsonSchema

the core then converts it

  • into a dts for monaco editor types
  • into a zod schema for runtime validation

then every time a node changes:
the core creates a union dts with the output dts of all input connections
=> homeautomation/server/src/services/generic-node/generic-type.utils.ts at master · jonnytest1/homeautomation · GitHub

and recomputes the nodes output schema
it then repeats for all connected nodes

if a node has an input schema specified it will also get compared against that

9

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.