Help understanding buffer parser output

ThingsTinkerer · 23 January 2025 08:55

Got a specific example which got me confused.

Configuration:

Swap: none
Output: key/value
Type: int16 (be)
Length: 1
Offset: 0
No mask
Calculation: 1

Input msg.payload is [35800,0].
Output: -29736

How is output determined here?

First value in input (35800) translates to 2 bytes 10001011'11011000. 2nd value translates to 00000000'00000000. Output (-29736) translates to 11111111'11111111'10001011'11011000. So the first value is pushed right to byte nr 3 and 4. First 2 bytes are all 1s?

Is the int16 (be) supposed to match each individual value in the array (2 x 16bit values)?

Example flow:

[
    {
        "id": "d0de03fa646462c8",
        "type": "inject",
        "z": "5ed12f1487755951",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "x": 320,
        "y": 240,
        "wires": [
            [
                "665e84c517835998"
            ]
        ]
    },
    {
        "id": "e32552b274e94be4",
        "type": "buffer-parser",
        "z": "5ed12f1487755951",
        "name": "",
        "data": "payload",
        "dataType": "msg",
        "specification": "spec",
        "specificationType": "ui",
        "items": [
            {
                "type": "int16be",
                "name": "item1",
                "offset": 0,
                "length": 1,
                "offsetbit": 0,
                "scale": "1",
                "mask": ""
            }
        ],
        "swap1": "",
        "swap2": "",
        "swap3": "",
        "swap1Type": "swap",
        "swap2Type": "swap",
        "swap3Type": "swap",
        "msgProperty": "payload",
        "msgPropertyType": "str",
        "resultType": "keyvalue",
        "resultTypeType": "return",
        "multipleResult": false,
        "fanOutMultipleResult": false,
        "setTopic": true,
        "outputs": 1,
        "x": 650,
        "y": 240,
        "wires": [
            [
                "d62e00ae1b7685f0"
            ]
        ]
    },
    {
        "id": "d62e00ae1b7685f0",
        "type": "debug",
        "z": "5ed12f1487755951",
        "name": "debug 3",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 820,
        "y": 240,
        "wires": []
    },
    {
        "id": "665e84c517835998",
        "type": "function",
        "z": "5ed12f1487755951",
        "name": "function 2",
        "func": "msg.payload = [35800, 0];\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 480,
        "y": 240,
        "wires": [
            [
                "e32552b274e94be4"
            ]
        ]
    }
]

Colin · 23 January 2025 09:14

You have said it is int16, so I think that means it is only looking at the first 16 bits.

[Edit] I should have said 'You have said it is int16, with a length of 1, so it is ...`
What value are you expecting to get out?

ThingsTinkerer · 23 January 2025 09:45

According to documentation it should be int32. Which means each of the 2 values in the array should be int16?

Jayeshpitale1 · 23 January 2025 09:52

Did you get the solution

ThingsTinkerer · 23 January 2025 13:52

Sorry I don't have full overview of this problem. Ended up brute-forcing which option to choose. Seems like documentation was insufficient particularly around byte swaps.

Colin · 23 January 2025 14:21

If the values in the array represent an int32 then specify the type as int32, and make sure the buffer going in represents 32 bits.

Steve-Mcl · 23 January 2025 15:51

Output is determined by the rows you configure.

Your data going in is clearly an array of 2 numbers and because the first is > 32768 they are likely UINT16 values. Buffer parser can worth with that. If they were something else, then you can use buffer-maker to form a correctly dimensioned buffer that you then feed into a parser node.

To get -29736 you must have set int16 which is correct (as a signed 16 bit value can not be higher than 32768

As you have said "According to documentation it should be int32" then you need to set the type to be int32. However whether that is BE (big endian) or LE (little endian) depends on the source data. Also, many docs are not clear whether they mean signed or unsigned. this can affect the produced value.

It is far easier to figure out if you provide the expected output value (or even a ball park figure)

for example - which of these look closest to the value you expect?

How Buffer Parser works:

items...

name A name to identify the resulting data. This can also be used as the topic when returning multiple values.

type The type that output data should be changed to (see Allowable types below)

offset The offset value denotes which byte to start reading data from. This is regardless of data source being a buffer or an integer array e.g. when data source is an integer array, specifying 3 means item source data will start from the middle of the 2nd WORD

length The quantity of items to be returned. e.g. 6 bools or 12 floats or 34 int32s. NOTE: setting length to -1 will attempt to read all bytes from offset to the end. NOTE: If source data does not have enough bytes, the operation will fail.

offsetbit The first bit to return when specifying type bool. e.g. requesting offsetbit 6 and a length of 3 will return byte0-bit6, byte0-bit7 and byte1-bit0

mask An optional mask to apply to the final data value. Enter a decimal mask e.g 65535 or a hexadecimal mask e.g. 0xffff

scale A multiplier or a simple Scale Equation to apply to the final value (after the mask). Supported Scaling Equations are...

ThingsTinkerer · 23 January 2025 20:51

Thanks all for your sharing your thoughts, made a lot of sense.

BambosD · 22 February 2025 15:03

Hello everyone,

i'm posting here, since i have searched for a buffer-parser thread and this thread is still open.

There is a codesys PLC (modbus TCP Server) with "real" type values , which is float32 (not swapped) (IEEE754 single precision). With other modbus client apps i can read successfully the values as float32, not swapped.

What will be the correct setup for this kind of values ?

i have used duplicated output to decode both endians but none of them seem correct.
The correct value confirmed by other modbus tcp clients apps is:

Please assist. any suggestions appreciated.

Steve-Mcl · 22 February 2025 15:40

Can you provide a copy of the data going into the parser node?

BambosD · 22 February 2025 15:45

Hello Mr. Steve,

i have a debug node and get the values as 16bit integers (direct from modbus read node)

polling : msg.payload : array[20]
[ 3321, 16691, 0, 0, 0, 49152, 0, 17593, 32768, 50210 … ]
22/02/2025, 17:42:12[node: debug 1](http://localhost:1880/#)

Steve-Mcl · 22 February 2025 15:48

Instead, please add debug (set to output complete message) to both outputs of the modbus nodes and capture the full msg using the copy value button.

canned "how to"

There’s a great page in the docs (Working with messages : Node-RED) that will explain how to use the debug panel to find the right path/value for any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

BX00Cy7yHi

BambosD · 22 February 2025 15:56

Both outputs look the same, except the second one it has json object { data: }

Upper modbus output :

{"topic":"polling","payload":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,52429,16620,0,0],"responseBuffer":{"data":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,52429,16620,0,0],"buffer":[102,102,65,52,0,0,0,0,0,0,192,0,0,0,67,185,128,0,196,34,0,0,65,200,0,0,0,0,0,0,0,0,204,205,64,236,0,0,0,0]},"input":{"topic":"polling","from":"1550-20","payload":{"unitid":"1","fc":3,"address":"1550","quantity":"20","messageId":"67b9f31ce2eef850569c1123"},"queueLengthByUnitId":{"unitId":1,"queueLength":0},"queueUnitId":1,"unitId":1},"sendingNodeId":"c1d11d5d9a56ee3b","_msgid":"3d1ce1ea77cab608"}

Lower modbus output:

{"topic":"polling","payload":{"data":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,26214,16654,0,0],"buffer":[102,102,65,52,0,0,0,0,0,0,192,0,0,0,67,185,128,0,196,34,0,0,65,200,0,0,0,0,0,0,0,0,102,102,65,14,0,0,0,0]},"values":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,26214,16654,0,0],"input":{"topic":"polling","from":"1550-20","payload":{"unitid":"1","fc":3,"address":"1550","quantity":"20","messageId":"67b9f2d6e2eef850569c10de"},"queueLengthByUnitId":{"unitId":1,"queueLength":0},"queueUnitId":1,"unitId":1},"sendingNodeId":"c1d11d5d9a56ee3b","_msgid":"6f20dddb07cf573f"}

Admin Edit: wrap data in triple back ticks ```

Steve-Mcl · 22 February 2025 16:16

Here you go. I've seen this before. you need to swap 32 then swap 16 (or use the little endian with swap16)

My preference is to use the buffer from the lower output.

[{"id":"adbff919ad985082","type":"inject","z":"72c6b31a07855fae","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":350,"y":220,"wires":[["f7cc027fb44ef904"]]},{"id":"f7cc027fb44ef904","type":"function","z":"72c6b31a07855fae","name":"fake modbus","func":"const m1 = {\"topic\":\"polling\",\"payload\":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,52429,16620,0,0],\"responseBuffer\":{\"data\":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,52429,16620,0,0],\"buffer\":Buffer.from([102,102,65,52,0,0,0,0,0,0,192,0,0,0,67,185,128,0,196,34,0,0,65,200,0,0,0,0,0,0,0,0,204,205,64,236,0,0,0,0])},\"input\":{\"topic\":\"polling\",\"from\":\"1550-20\",\"payload\":{\"unitid\":\"1\",\"fc\":3,\"address\":\"1550\",\"quantity\":\"20\",\"messageId\":\"67b9f31ce2eef850569c1123\"},\"queueLengthByUnitId\":{\"unitId\":1,\"queueLength\":0},\"queueUnitId\":1,\"unitId\":1},\"sendingNodeId\":\"c1d11d5d9a56ee3b\",\"_msgid\":\"3d1ce1ea77cab608\"}\nconst m2 = {\"topic\":\"polling\",\"payload\":{\"data\":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,26214,16654,0,0],\"buffer\":Buffer.from([102,102,65,52,0,0,0,0,0,0,192,0,0,0,67,185,128,0,196,34,0,0,65,200,0,0,0,0,0,0,0,0,102,102,65,14,0,0,0,0])},\"values\":[26214,16692,0,0,0,49152,0,17337,32768,50210,0,16840,0,0,0,0,26214,16654,0,0],\"input\":{\"topic\":\"polling\",\"from\":\"1550-20\",\"payload\":{\"unitid\":\"1\",\"fc\":3,\"address\":\"1550\",\"quantity\":\"20\",\"messageId\":\"67b9f2d6e2eef850569c10de\"},\"queueLengthByUnitId\":{\"unitId\":1,\"queueLength\":0},\"queueUnitId\":1,\"unitId\":1},\"sendingNodeId\":\"c1d11d5d9a56ee3b\",\"_msgid\":\"6f20dddb07cf573f\"}\n\nreturn [m1, m2];","outputs":2,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":540,"y":220,"wires":[["24a177523e686fc5"],["df3b9425125609c2"]]},{"id":"df3b9425125609c2","type":"buffer-parser","z":"72c6b31a07855fae","name":"","data":"payload.buffer","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"floatbe","name":"swap32/swap16/float/be","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""}],"swap1":"swap32","swap2":"swap16","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"value","resultTypeType":"return","multipleResult":true,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":750,"y":260,"wires":[["9be544b69361eee6"]]},{"id":"9be544b69361eee6","type":"debug","z":"72c6b31a07855fae","name":"debug 18","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":920,"y":260,"wires":[]},{"id":"24a177523e686fc5","type":"buffer-parser","z":"72c6b31a07855fae","name":"","data":"payload","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"floatle","name":"swap16/float/le","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""}],"swap1":"swap16","swap2":"","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"value","resultTypeType":"return","multipleResult":true,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":750,"y":200,"wires":[["ec91a2df8d135580"]]},{"id":"ec91a2df8d135580","type":"debug","z":"72c6b31a07855fae","name":"debug 17","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":920,"y":200,"wires":[]}]

BambosD · 22 February 2025 16:30

Thank you Mr. Steve,

This works for me, but i can't really understand why we need to make this swap settings, since the modbus values coming as 16bit integers are correct from the modbus read node.
i guess this is how the buffer - parser is made, but i suggest this to be added as decode option because is very important and widely used nowadays (real type values of codesys PLC's)

something else for the mask: what i should enter to limit this many decimals number 11.274999..., to just 11.275 ? I check the documentation and says something for binary mask and hex mask with example 65535, but this removes all the decimals.

Thanks again for any suggestions and your prompt response.

Steve-Mcl · 22 February 2025 16:37

What is "correct"?

The order of bytes are specified by the device manufacturer.

Many other devices do not pack thier data in the same way.

The buffer parser is doing nothing other than unpacking the data in the order supplied. If it needs a 16 bit swap it is because of the order data was delivered.

You dont. That is how data resolution is lost. You would ALWAYS store the highest precision data possible (those fractions add up over time). However, I can understand the desire to make the value "pretty" for display (on a dashboard etc) - that is where you would then deliberately remove precision. You can do that with a change node JSONata or a function node.

Change node:

Steve-Mcl · 22 February 2025 16:44

This is your data plugged into the node Buffer

> buf = Buffer.from([102,102,65,52,0,0,0,0])
<Buffer 66 66 41 34 00 00 00 00>
> buf.readFloatBE()
2.7183676910492487e+23
> buf.readFloatLE()
1.8011778024629166e-7
> buf.swap16()
<Buffer 66 66 34 41 00 00 00 00>
> buf.readFloatBE()
2.7177705137386594e+23
> buf.readFloatLE()
11.274999618530273
> buf.swap32()
<Buffer 41 34 66 66 00 00 00 00>
> buf.readFloatBE()
11.274999618530273
> buf.readFloatLE()
2.7177705137386594e+23

^ See how your data is interpreted by the built in buffer of NodeJS ^

BambosD · 22 February 2025 16:51

Thank you Mr. Steve,

i agree, is for dashboard.

i did that with jsonata expression and looks great, i start to really like node-red.

Topic		Replies	Views
Simple (should be) buffer parser question General	3	339	24 April 2022
Question about Buffer Parser and Hex General	2	383	15 February 2022
Understanding Node-red-contrib-buffer-parser input values General	3	178	1 December 2024
I have a question about buffers General	5	341	6 September 2021
Processing int16 values General	43	1861	15 February 2021

Help understanding buffer parser output

items...

canned "how to"

Upper modbus output :

Lower modbus output:

Related topics