Serialport input options when multiple characters define the beginning and end of a message

General
1

I would like to have the Serial In node break up messages or merge messages so that each message ends with three hex bytes (0x10, 0x04, 0x**) the last one being a wildcard byte.

I would also like to have each message begin with a sequence of 4 hex bytes (0x10, 0x02, 0x10, 0x02)

I see in the config that I can use a single hex value, is there a method to be able to use a combination of hex values? Ideally I'm thinking the config should look as follows but I don't know how to implement it

Edit Serial In Node > Edit Serial Port Node
Input:
Optionally wait for a start character of:
0x10, 0x02, 0x10, 0x02

Split input on the character:
0x10, 0x04, 0x**

Any help is appreciated.

2

This functionality doesn't come included in the serial in node. You should yet be able to build it on your own, with a single function node (or some other standard nodes) processing the data that comes in via serial in node.

3

Is there a time gap between messages? Often this sort of problem can be solved by using the timeout option and then processing the message in a function node to trim it or check for validity etc.

4

You can get relatively close using the split node in Stream Mode.
But using a wildcard will be difficult I think

I think using a time mechanism as @Colin suggests is easier if possible.

[{"id":"0782e146be5fa081","type":"split","z":"2d7bf6e3.84c97a","name":"Split","splt":"[16,4]","spltType":"bin","arraySplt":"1","arraySpltType":"len","stream":true,"addname":"","x":1275,"y":465,"wires":[["e994bc8c4cab35f1"]]},{"id":"f03b3c5953430b24","type":"debug","z":"2d7bf6e3.84c97a","name":"debug 2","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1670,"y":465,"wires":[]},{"id":"e994bc8c4cab35f1","type":"function","z":"2d7bf6e3.84c97a","name":"Append to buffer","func":"const Complete = Buffer.concat([msg.payload, Buffer.from(msg.parts.ch)]);\nmsg.payload = Complete\nreturn msg","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1460,"y":465,"wires":[["f03b3c5953430b24"]]},{"id":"df30c9a57e575572","type":"group","z":"2d7bf6e3.84c97a","name":"Serial Message 1","style":{"label":true},"nodes":["0bfab46620bf633c","92629647ee0da0a9"],"x":829,"y":404,"w":352,"h":122},{"id":"0bfab46620bf633c","type":"inject","z":"2d7bf6e3.84c97a","g":"df30c9a57e575572","name":"Byte Collection 1 (ends with 0x10)","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[1,2,3,16]","payloadType":"bin","x":1015,"y":445,"wires":[["0782e146be5fa081"]]},{"id":"92629647ee0da0a9","type":"inject","z":"2d7bf6e3.84c97a","g":"df30c9a57e575572","name":"Byte Collection 2 (starts with 0x04)","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[4,1,2,3]","payloadType":"bin","x":1015,"y":485,"wires":[["0782e146be5fa081"]]},{"id":"269fb9aed58f32a6","type":"group","z":"2d7bf6e3.84c97a","name":"Serial Message 2","style":{"label":true},"nodes":["a15eae8c11cab119","a92bb40e5503ebb9"],"x":829,"y":544,"w":352,"h":122},{"id":"a15eae8c11cab119","type":"inject","z":"2d7bf6e3.84c97a","g":"269fb9aed58f32a6","name":"Byte Collection 3 (ends with 0x10)","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[1,2,3,16]","payloadType":"bin","x":1015,"y":585,"wires":[["0782e146be5fa081"]]},{"id":"a92bb40e5503ebb9","type":"inject","z":"2d7bf6e3.84c97a","g":"269fb9aed58f32a6","name":"Byte Collection 4 (starts with 0x04)","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[4,1,2,3]","payloadType":"bin","x":1015,"y":625,"wires":[["0782e146be5fa081"]]}]

The inject Nodes are just emulating the chunked packets, and is delivering 2 complete payloads
its not perfect - but worth investigating the split node in stream mode

Screenshot 2023-12-06 at 17.27.13
Screenshot 2023-12-06 at 17.27.131389Ă—568 63.4 KB

3 Likes
5

this worked beautifully, I'm so glad I asked, I wouldn't have considered the split node in stream mode, thank you so much!

2 Likes
6

Cool -- you learn something new every day!

I'm having the same issue, where the serial data messages are delimited by two bytes at the start (0x10 0x02) and a different two bytes at the end (0x10 0x03). So if the serial in node waits for the start byte 0x10 and splits on the last byte 0x03 I am seeing partial msg buffers.

I'd like to have the choice (on both the serial in and split nodes) to set start and/or end byte arrays -- does anyone else think this would be a good candidate for a PR?

In the past there has been hesitation to add features like this, since the node format inside the flow file would be extended to support an array of bytes...

1 Like
7

Perhaps another new useful feature would be to use a regex to define the split -- instead of a character (or even an array of bytes)?

I'd be willing to work on these new features, if the core developers are interested.
__
Steve

1 Like
8

Happy to consider a PR

9

Ok, is been a few years since I added code to the core nodes...

Is the process still to clone thedev branch and push changes up from there?

10

Yeah. New features should branch off dev branch

Ps, only skimmed the topic but are you proposing to permit the serial node to split on more than 1 character (as well as regex?)

I remember having issues some time ago needing to split on \r\n would be good if this becomes possible but I'm not sure how strings of non printable chars should be represented in the input tho. Perhaps a typedInput with the usual suspects (CR, LF, CRLF, COMMA, TAB, etc) presented as a drop list and then a freeform input?

11

Serial port is not a core node. It’s in node-red-nodes package so no dev branch. Just go for it :slight_smile:

12

can't we handle non-printable chars as hex ? 0x00,0x02 etc, ?

13

Sure it is doable but not super intuitive/friendly? Was sounding out a way of making the split option a bit more approachable? (I'm easy but, you know, in the spirit of low/no code n all :angel: )

14

Oh wait, are you saying "can't we currently split on multiples using 0x02,0x03"?

15

The current serialport node editor UI only allows a single split character, according to the docs:

The node can optionally wait until it matches a pre-defined character.

The serial data reception can either

  • wait for a "split" character (default \n). Also accepts escaped shortcut (e.g. \n) or hex notation (0x0d).

However, the split node in streaming mode allows for defining more than 1 character -- using the "buffer editor" UI:

image

As Dave suggests, I thought I could transplant this typed input (from split) into serialport as a simple way to allow splitting on either a string (e.g. \r\n), a buffer (e.g. [16,3] or the hex [0x10,0x03]), or potentially a regex (e.g. \w+ to split on whitespace).

2 Likes
16

My 5 cents worth: I think the SERIAL IN/OUT node config would benefit if INPUT and OUTPUT parameters worked in the same way:

If you can specify more than 1 char to be appended to the output, it seems intuitive you should be able to split an input based on multiple chars as well....after all, the UI actually lets you enter multiple chars for the Split (though only the last char is used).

*- Such symmetry is always nice.
*- Multiple Split chars makes the node more flexible - I mean, it's a very common instance where devices use a "\r\n" as their end-of-message delimiter. An improved Serial Node could handle this without the need of external Split Node.

Hope these improvements are considered.

17

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.