Problem converting Xport Ethernet output to string

engrjayze · 22 October 2019 05:14

Hi All,

This is my first thread. May I kindly ask for support on below concern.

Data source: Xport (RS232C to Ethernet)

I cannot fully convert the buffer data from tcp -> I always get some boxes and ??.
Can you help me how to properly convert the 19byte data detailed in page 10 of the manual.

Link to manual:

Thank you.

Capture3

dceejay · 22 October 2019 09:33

I would look to use the node-red-contrib-binary node - that can be specified to slice apart data like this.
It does use a bit of a weird syntax to specify it - but isn't too bad..
If you can supply some example messages in text format rather than images it would easier to help. Or at least expand out the buffer so we can see all 19 values properly.

On second thoughts - looking at that spec - it's just ascii encoded byes in there so treat it as a string and just slice it using substr

[{"id":"4d6bcd45.bf3474","type":"function","z":"5051cb93.cf0d34","name":"","func":"msg.payload = msg.payload.toString(\"binary\");\nvar id = msg.payload.substr(4,10);\nvar data = msg.payload.substr(14,2);\nreturn {payload:{id:id, data:data}};","outputs":1,"noerr":0,"x":250,"y":920,"wires":[["39620aaf.a23836"]]},{"id":"39620aaf.a23836","type":"debug","z":"5051cb93.cf0d34","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":450,"y":940,"wires":[]}]

of course this doesn't check the crc is valid or start and stop byes are in place etc... so you may wish to add error checking

engrjayze · 23 October 2019 01:49

Thanks for the reply.
Sorry but I do not know how to start on my first post so I will briefly discuss the system:

I have 14 transmitters with unique ID (10 digit) connected to torque wrenches. These were paired to 1 receiver.
Example ID: 0101005493
Every time a torque wrench tightens a bolt - it will hit a limit switch and it will send it's own TX ID + some data to a receiver (using 2.4 GHz).
The receiver acknowledges this data and outputs 19bytes of data to ETHERNET (Lantronix Xport RS232 to TCP)
The Receiver and Raspberry PI were connected to the same Router.
Each torque wrench will tighten 1,2 or 3 bolts every minute based on work spec (repeat 2000 times per day)

Inside Node-Red:

TCP : 10.0.0.20:10001, stream of buffer
FUNCTION
msg.payload = msg.payload.toString('utf8');
return msg;
SWITCH: Each ID will move to 14 separate CHANGE NODES
If msg.payload contains 0101005493 -> 1
CHANGE: Change ID to simple string
Change 5493 to BCONV
From here it goes to several nodes to count and compare to spec.
If count is under or over it will send STOP signal to a MQTT topic to stop the conveyor.

Problem: Some transmitters send data but some counts are missing or not registered in node-red resulting to frequent conveyor stop.

I would like to verify that all data sent by receiver to node-red are processed properly.
I am focusing on the program (signal and hardware issues are being check separately).
The transmitter ID are being read for the purpose of incrementing the count but other data are boxes or question marks.
These remaining data are important to decode what error the transmitters sent.
Given the speed and qty daily - I do not know how to log properly.
When count spec was set to OFF to disable a tool - error appear in debug due to the subtraction in one of my functions. I don’t know if this affects the program as well.
Kindly look at the attached code for your other inputs - I would like to optimize but I'm still a beginner. I just learned how to make flows a month ago.

Capture.PNG874×636 43.5 KB

engrjayze · 23 October 2019 02:22

Link to the flow: https://pastebin.com/KQAxaH8r

engrjayze · 23 October 2019 03:09

Here is the debug data direct from TCP:

10/23/2019, 11:03:17 AM[node: c7703f1.6be75c]
msg : Object

object

topic: ""

payload: buffer[2][raw]

0: 0xff

1: 0xff

_session: object

type: "tcp"

id: "3f5f5744.5b1118"

_msgid: "b5928d83.2c53a"

10/23/2019, 11:03:17 AM[node: c7703f1.6be75c]
msg : Object

object

topic: ""

payload: buffer[17][raw]

[0 … 9]

0: 0xff

1: 0x2

2: 0x30

3: 0x31

4: 0x30

5: 0x31

6: 0x30

7: 0x30

8: 0x35

9: 0x34

[10 … 16]

10: 0x39

11: 0x33

12: 0x30

13: 0x31

14: 0x3

15: 0x30

16: 0x39

_session: object

type: "tcp"

id: "3f5f5744.5b1118"

_msgid: "3c1cbbcb.2fb244"

engrjayze · 23 October 2019 03:12

Here is the debug data after Convert to String function:

10/23/2019, 11:03:17 AM[node: be06ac3c.0b332]
msg : Object

object

topic: ""

payload: "��"

_session: object

type: "tcp"

id: "3f5f5744.5b1118"

_msgid: "b5928d83.2c53a"

10/23/2019, 11:03:17 AM[node: be06ac3c.0b332]
msg : Object

object

topic: ""

payload: "�01010054930109"

_session: object

type: "tcp"

id: "3f5f5744.5b1118"

_msgid: "3c1cbbcb.2fb244"

Steve-Mcl · 23 October 2019 07:59

Please wrap logs/code/etc in three backticks e.g.

```

var example = "putting code here"
return "looks much nicer and doesnt get messed up by the forum"

```

dceejay · 23 October 2019 09:45

Did you try the code I appended above ?
Here is a newer version... attach it direct to the tcp in node.
The function now has two outputs... - the second of which goes to a file - edit it as necessary - which will then log all the inputs - and you can analyse in whatever way you need to show what does - and doesn't arrive...

[{"id":"97e5d0fd.5dcae","type":"function","z":"5051cb93.cf0d34","name":"","func":"msg.payload = msg.payload.toString(\"binary\");\nvar id = msg.payload.substr(4,10);\nvar data = msg.payload.substr(14,2);\nvar ts = new Date.now();\nvar p = ts+\",\"+id+\",\"+data;\nreturn [{payload:{ts:ts, id:id, data:data}},{payload:p}];","outputs":2,"noerr":0,"x":430,"y":500,"wires":[["6dd8afff.36a51"],["921108e7.0d7788"]]},{"id":"6dd8afff.36a51","type":"debug","z":"5051cb93.cf0d34","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":630,"y":480,"wires":[]},{"id":"921108e7.0d7788","type":"file","z":"5051cb93.cf0d34","name":"","filename":"/tmp/my.log","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":630,"y":520,"wires":[[]]}]

engrjayze · 24 October 2019 03:24

Just tried the newer code you posted but it returned an error: TypeError: Date.now is not a constructor

I removed reference to TS in the function node and I get below output:

msg.payload : Object
object
id: ""
data: ""

msg.payload : Object
object
id: "1010054930"
data: "1"

Still have some boxes...but the logfile is working.

During actual use - stream of data into the switch node is very fast:

I set to "stop at 1st match" to prevent extra load..is this ok?
Recreate message sequences is also OFF
My worry is that because of this continuous stream - some data is being cut or omitted resulting to incorrect count.

Thanks

engrjayze · 25 October 2019 00:40

I think I may have found the reason why some counts are not processed.

Yesterday, I wired 4 transmitters to send signal at the same time. Then I logged to text as shown below.

Activity: Send 4 signals simultaneously for 10 tries
Expected total: 40
Actual: 38

Log direct from TCP node = 40 Transmitter IDs were received

ÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ01010056A60176ÿÿÿ0101005494010Eÿÿÿ0101005494010Eÿÿÿ01010056A60176ÿÿÿ01010056A70177ÿÿÿ01010056A50175ÿÿÿ01010056A60176ÿÿÿ0101005494010Eÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ01010056A70177ÿÿÿ0101005494010Eÿÿÿ01010056A60176ÿÿÿ01010056A50175ÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ01010056A60176ÿÿÿ0101005494010Eÿÿÿ01010056A60176ÿÿÿ01010056A70177ÿÿÿ0101005494010Eÿÿÿ01010056A50175ÿÿÿ01010056A60176ÿÿÿ0101005494010Eÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ01010056A60176ÿÿÿ0101005494010Eÿÿÿ01010056A60176ÿÿÿ0101005494010Eÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ01010056A60176ÿÿÿ01010056A50175ÿÿÿ01010056A70177ÿÿÿ0101005494010E

Log direct from Convert to String Function: = 40 Transmitter IDs were received

��
�01010056A50175���01010056A70177���
01010056A60176���0101005494010E
�
��0101005494010E���01010056A60176���
01010056A70177���01010056A50175
��
�01010056A60176���0101005494010E���
01010056A50175���01010056A70177
��
�01010056A70177���01
01005494010E
���01010056A60176���01010056A50175
��
�01010056A50175���01010056A70177���
01010056A60176���0101005494010E
��
�01010056A60176���01010056A70177���
0101005494010E���01010056A501
75
��
�01010056A60176���0101005494010E���
01010056A50175���01010056A70177
��
�01010056A50175���01010056A70177���0
1010056A60176���010100549401
0E
�
��01010056A60176���0101005494010E��
�01010056A50175���01010056A70177
�
��01010056A60176���01010056A501
75���01010056A70177���0101005494010E

Node-red UI: only 38 were registered

In the debug window - 4 messages were received but transmitter ID were not in every one.
Some are being joined in 1 message.

I think the SWITCH node outputs are too many or it does not completely process all received ID which results to missing counts.

dceejay · 25 October 2019 08:11

Are you also capturing anything coming out of the the otherwise (the last) output of the switch - in case you are not handling all codes correctly ?

Topic		Replies	Views
Help with converting string to hex array General	6	793	12 September 2022
Binary Buffer to Strings while using serial port General	3	575	19 May 2021
TCP outputs a string instead of byte values General	3	2247	24 January 2019
TCP parsing String General	12	1456	14 March 2021
String type difference General	11	497	9 March 2021

Problem converting Xport Ethernet output to string

Related topics