Need advice on node-red node design

[Steve-Mcl]

Hi all. I'm after some advice on developing nodes. Firstly I have read the guidelines
however most of the info is unfinished.

I am creating some nodes that are essentially PLC Read and PLC Write nodes. These nodes share a configuration node. Their operations are asynchronous (as they send a read/write request to the PLC then upon receiving a reply, send a message to the next node). Fairly straightforward right?

So, while developing, I have many thoughts about how best to handle the likes of timeouts, read errors, disconnections etc. How should dynamic requests be made via input message, should I retain the original msg and send this along with my data??? Many thoughts, difficult to put into easily answerable questions I suspect. I'll try tho...

Best means of accepting input...

• Should I use topic and payload msg.topic = "D5" /*address*/; msg.payload = 5 /*count*/;. This seems ideal however it's limiting. What if I need to include data or a timeout parameter.
• should I use an object for payload to pass in the parameters? E.g. msg.payload = {address:"D5", count:10}
• should I leave the original payload and request the input msg has distinct properties e.g. msg.address="D5"; msg.count = 10; or an object added to msg?

What to send in output...

Is it preferred that an asynchronous node outputs only to payload? Should all properties of my nodes asynchronous response (e.g. .success, .data, . responseTime, .errorCode) be all contaied in one object? In payload?

To send or not to send...

My first version would simply not call node.send(msg) if there was a timeout, or bad response from PLC etc however, this means the next node has no opportunity to act on a bad response.
So my second version sends the results and additional properties would indicate a bad response e.g. msg.quality = "BAD"; this then led me to think, should I encapsulate all of my response into the payload? Is there a preferred convention?

Other thoughts include...

When should my node raise an error. If the PLC response is bad? Times out? At first this seemed ideal however the catch node would be going nuts if the user is polling at say 0.1sec while the PLC is turned off / disconnected.

Should a node that is essentially asynchronous store the original msg (and all the properties a user might have tagged on to it) then send this original msg with my nodes results appended? The user may have added some info to the msg for use after my node has done its job!

Thanks in advance for any advice or thoughts on this topic.

Steve.

[TotallyInformation]

@Steve-Mcl

These nodes share a configuration node

I have an example set of nodes that might be helpful. Recently updated to include a configuration node example. They are heavily documented.

GitHub - TotallyInformation/node-red-contrib-jktesting: Test nodes for...

@Steve-Mcl

asynchronous

This is normal and easily handled in Node.JS (which underpins Node-RED).

@Steve-Mcl

Should I use topic and payload msg.topic = "D5" /*address*/; msg.payload = 5 /*count*/; . This seems ideal however it's limiting. What if I need to include data or a timeout parameter.

msg.topic and msg.payload are useful default properties as most people will expect them to exist and many nodes will automatically process them or pass them through.

In reality, topic is just a tag, a nice way to identify the source of the msg. It comes, of course, from the MQTT world. Using topic means that it is often easier to forward your msgs direct to MQTT. Even if you don't use it, please do forward it from input to output. If you set it in your node processing, it can be useful to take the input msg topic if your node doesn't need to set it for some reason.

If you need complex data such as you mention, you will need to decide the best approach. Often, I would choose to make the output payload contain everything. However, if some of the data is really "metadata" - e.g. data about the data - then it may be better to keep that off the payload. It really comes down to how you expect the data to be processed downstream. Ideally though, the core data should - in my opinion - be on the msg.payload.

@Steve-Mcl

should I use an object for payload to pass in the parameters? E.g. msg.payload = {address:"D5", count:10}
should I leave the original payload and request the input msg has distinct properties e.g. msg.address="D5"; msg.count = 10; or an object added to msg

Yes, if you need to allow for dynamic settings/configuration, that is the best way. Not the only way though. Again, you might consider keeping some/all configuration data on a different msg property - perhaps if it is incidental. Alternatively, if some of your input should also end up in the output, it may be sensible to keep that on the input payload but if some shouldn't be passed through, you might consider putting that on some other property.

@Steve-Mcl

Is it preferred that an asynchronous node outputs only to payload?

As above - core output information is generally anticipated on the msg.payload but it isn't mandated.

@Steve-Mcl

Should all properties of my nodes asynchronous response (e.g. .success, .data, . responseTime, .errorCode) be all contaied in one object? In payload?

As already mentioned, any data not core to the downstream processing may be better on a different property. For example, if your output msg.payload will always contain something valid even if the request fails (maybe then an empty string?) , you could probably relegate a success flag to a different property.

I would generally want to put any error data onto a separate property like msg.err. Though if the purpose of the node was to output a single string msg, I'd consider putting the error message text onto the msg.payload as well.

@Steve-Mcl

My first version would simply not call node.send(msg) if there was a timeout, or bad response from PLC etc however, this means the next node has no opportunity to act on a bad response.
So my second version sends the results and additional properties would indicate a bad response e.g. msg.quality = "BAD"; this then led me to think, should I encapsulate all of my response into the payload? Is there a preferred convention?

It will really depend on how you will trap and process errors. Node-RED has a standard for dealing with errors but you might not want that (or you might). If you want your flows to continue regardless of errors, that will influence your design. For example, you might want error outputs to go to a second output, keeping the first output just for valid data. That can be helpful if you need to be able to re-trigger your node on error as it simplifies the processing. There is no absolute answer here.

@Steve-Mcl

When should my node raise an error. If the PLC response is bad? Times out? At first this seemed ideal however the catch node would be going nuts if the user is polling at say 0.1sec while the PLC is turned off / disconnected.

Nodes that trigger continuous streams of actual Node errors are a nightmare. Really, I'd try to restrict that kind of error processing for when things go really wrong - e.g. something you didn't expect at all or something that requires urgent, manual intervention. Having "normal" error information coming out of a second port is a nice compromise and you can put a throttle on the output if you need to.

@Steve-Mcl

Should a node that is essentially asynchronous store the original msg (and all the properties a user might have tagged on to it) then send this original msg with my nodes results appended? The user may have added some info to the msg for use after my node has done its job!

Only you can really decide that. Many nodes don't keep input data other than topic. Some replace the msg.payload but keep everything else. Keeping input data intact is helpful if you are processing flows that come from http or websockets as there is additional information in the msg that you will need.

If the input msg.payload is relevant to downstream processing then you will need to retain it of course - though if it is only marginally useful, you could move it to a different property.

There is no absolute right or wrong in Node-RED, the docs contain useful guidelines but most things are open in order to maintain the flexibility of the platform.

[knolleary]

There's lots to pick through here, and I'm not in a position to respond fully this weekend. But a couple immediate comments.

@TotallyInformation

It will really depend on how you will trap and process errors. Node-RED has a standard for dealing with errors but you might not want that (or you might). If you want your flows to continue regardless of errors, that will influence your design. For example, you might want error outputs to go to a second output, keeping the first output just for valid data. That can be helpful if you need to be able to re-trigger your node on error as it simplifies the processing. There is no absolute answer here

Node-RED does have a standard way for nodes to report errors and you'd have to have a very particular requirement to not use it. Reporting errors on a second output from the node is certainly not a recommended pattern given the standard mechanism available.

@TotallyInformation

Many nodes don't keep input data other than topic.

And many nodes are doing it wrong. Nodes must preserve the msg properties it receives when it outputs a message unless they have a very good reason not to do so.

If you don't preserve message properties, it will be impossible to use you nodes with many pairs of nodes that require certain properties to remain on messages - for example split/join require msg.parts. httpin/response require msg.req and msg.res.

[Steve-Mcl]

@TotallyInformation, @knolleary,
Thank you very much for taking the time to reply. Lots of good info to process.

I promise to review and propose a summary of all thoughts and comments when I get a little time.

If anyone else has any proposals or thoughts on these matters, please pitch in... The more the merrier (and the more, the closer we'll get to an ideal)

Hopefully, we can end up with some useful guidelines (or at least some no-no's) for others to implement (or avoid).

Cheers, Steve.

[BartButenaers]

@Steve-Mcl

should I retain the original msg

Hey Steve,

Nowadays I 'try' to keep as much as possible the content of the original input message. Like the others already say, you might loose very important information otherwise...

But most nodes will need to put their result (data) somewhere in the output message. In the beginning I hardcoded payload in all my nodes, but then users had two problems:

• Sometimes the original payload contained important information, which would be lost now.
• Moreover they didn't always want to the result to be stored in the payload field, because e.g. the next node expects his input in an other message field (e.g. msg.url). To solve that, extra Switch nodes were required to move the data to the required message field (e.g. from msg.payload to msg.url), resulting in messy flows.

For those reasons I nowadays add a TypedInput field to my config screen, containing only type 'msg'. For example my node-red-contrib-interval-length node, the user can choose themselves in which output message field the interval length should be stored:

Similar I add TypedInput fields in my config screens to let the user choose from which input message field the input data has to be read.

Bart

[tilleul]

@BartButenaers

Nowadays I 'try' to keep as much as possible the content of the original input message .

I second that and I do the same ! In fact even some "official" nodes should be rewritten to respect that rule ... Payload is sacred, you don't touch it, you pass it along !

[Steve-Mcl]

@BartButenaers do you mean "input properties of msg are sacred, you don't touch it, you pass it along !"
As most nodes do pass results in payload?

[knolleary]

@tilleul

some "official" nodes should be rewritten

Which nodes did you have in mind?

[dceejay]

Payload is not sacred. It is the most useful part of the processed data. It may or may not be useful to maintain the input values as well. You can always copy the inputs to another property to save them if you wish, but we aren’t going to do this by default

[BartButenaers]

@Steve-Mcl

@BartButenaers do you mean " input properties of msg are sacred, you don't touch it, you pass it along !"
As most nodes do pass results in payload?

@Steve-Mcl Like the others already explained: sometimes the input data is indeed sacred, but sometimes it is not. It might even contain complete junk . That is something you cannot decide as a node developer, but every user of your node will have to make that decision for his specific situation.

But you should at least give your users the opportunity to choose whether they want to override the original data (e.g. msg.payload), or to keep it (and put the new data in msg.anotherfield). In that case the TypedInput field is very useful (and can be added with very few code).

And the default value of the TypedInput can be e.g. be msg.payload, to give your users a hint ...