[UPDATED] node-red-contrib-buffer-parser - V3.1.2

[Steve-Mcl]

@dceejay

But loving the fan out. I won't mention port labels just yet

No, please do. I'm interested in your thoughts - can we pass the name through or something else?

[dceejay]

Port labels - yes I was going to suggest exactly that - pass the name through by default when in fan out mode.

outputLabels: function(index) {
    return "my port number "+index;
}

Worst case look at the change node for an example that also uses an editable list. - https://github.com/node-red/node-red/blob/master/packages/node_modules/%40node-red/nodes/core/function/10-switch.html#L103

When in other modes it could just reflect the current out mode selected - values, objects etc.

[Steve-Mcl]

Now in flows library - V3.1.1

Update includes...

• Fan Out
  • Send the parsed items out of their own ports
    
    
• Scaling Equations to simplify things like bit shifting ...
  • 
  • << (e.g. <<2 would left shift the parsed value 2 places
  • >> (e.g. >>2 would right shift the parsed value 2 places
  • >>> (e.g. >>>2 would zero-fill right shift the parsed value 2 places (returns a 32bit unsigned value)
  • + (e.g. +10 would add 10 to the parsed value
  • - (e.g. -10 would deduct 10 from the parsed value
  • / (e.g. /10 would divide the parsed value by 10
  • * (e.g. *10 would multiply the parsed value by 10
  • ** (e.g. **2 would raise the parsed value to the power of 2
  • ^ (e.g. ^0xf0 would XOR the parsed value with 0xf0
  • == (e.g. ==10 would result in true if the parsed value was equal to 10
  • != (e.g. !=10 would result in false if the parsed value was equal to 10
  • !! (e.g. !! would result in true if the parsed value was 1 (same as !!1 == true )
  • > (e.g. >10 would result in true if the parsed value was greater than 10
  • < (e.g. <10 would result in true if the parsed value was less than 10
• Dynamic Port labels
• Binary and Octal masks possible
• updated icons

[mikeS7]

@Steve-Mcl,
there is an issue with fan output, there are additional outputs after pressing cancel and selecting the node.
Please take a look to the recording:

Should I open an issue on Github?

[Steve-Mcl]

@mikeS7

Should I open an issue on Github

Yes please.

Could you also take a look in the Devtools console, is there an error or something?

[Steve-Mcl]

Update (hopefully the last for a while) - V3.1.2

1. improve scaler operand checking
2. tidy up code
3. improve fan out pin descriptions
4. add missing scaling equation operator validation for == & !=
5. Improve port labels
6. fix issue with output pins changing "on cancel"
7. add tooltips for checkboxes
8. clean up console logging

[cymplecy]

Hi Steve
I want to take a buffer (audio) and have Node-RED reduce each sample volume.

So I thought - split it - process each byte - join - job done

But I need each byte converting to a number to then use the range node to reduce the values down

I've tried a few things but can't seem to just convert the buffer[1] payload after the split to a number

Will your node to that?

[cymplecy]

Cancel this (althought still would be nice to know just for intellectual sake as to whether it can convert a buffer length 1 to a number)

Much better/easier (following DM prompt by Steve) to control volume another way

I've found using

amixer sset Master xx%

in an exec node

sorts me out on my Pi

[ericplan]

Thanks for the buffer-parser. Saving me lots of work. Any ideas on implementing IEEE 754? IEEE 754 - Wikipedia

This standard is used in the new Finder 7M Modbus kWh-meters and other modbus devices. For now I use the buffer-parser to combine two 16-bit Modbus registers and to split it up in an int8 and a 24-bit hex part. I have a working function node to combine this to the correct value.

The buffer-parser is a simple way to reduce the Modbus polling and correctly naming the output.

[Steve-Mcl]

Hi @ericplan

Could you provide a demo flow with your data (provided by an inject node) → buffer parser → you function - i will take a look.

Thanks.

[ericplan]

Pfff, that's lightning fast.... Thanks.
Screenshot and code below. This is for two Modbus registers, but I would like to read 8 adjacent registers in a row. For instance this gives the voltage for L1, L2 ,L3 and average voltage.

[{"id":"a02f30f382ba30f2","type":"modbus-flex-getter","z":"ea65f3bc16fcb67b","name":"Finder 7M.38","showStatusActivities":true,"showErrors":false,"logIOActivities":false,"server":"1c1c9200198a90e2","useIOFile":false,"ioFile":"","useIOForPayload":false,"emptyMsgOnFail":true,"keepMsgProperties":false,"x":610,"y":240,"wires":[["74cca90e4d1b3c7c"],[]]},{"id":"db7517ec6182ec1b","type":"inject","z":"ea65f3bc16fcb67b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":260,"y":240,"wires":[["ccc104c47ce77917"]]},{"id":"ccc104c47ce77917","type":"function","z":"ea65f3bc16fcb67b","name":"","func":"msg.payload = { \n    value: msg.payload, \n    'fc': 4, \n    'unitid': 44, \n    'address': 107, \n    'quantity': 2\n} \n\nreturn msg","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":405,"y":240,"wires":[["a02f30f382ba30f2"]],"l":false},{"id":"23a715852d7ef0ad","type":"debug","z":"ea65f3bc16fcb67b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1190,"y":240,"wires":[]},{"id":"74cca90e4d1b3c7c","type":"buffer-parser","z":"ea65f3bc16fcb67b","name":"","data":"payload","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"int8","name":"Exponent","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"hex","name":"Mantisse","offset":1,"length":3,"offsetbit":0,"scale":"1","mask":""}],"swap1":"","swap2":"","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"keyvalue","resultTypeType":"return","multipleResult":false,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":810,"y":240,"wires":[["5b175f1c730b805c"]]},{"id":"5b175f1c730b805c","type":"function","z":"ea65f3bc16fcb67b","name":"","func":"msg.payload = parseInt(\"0x\"+msg.payload.Mantisse) * Math.pow(10, msg.payload.Exponent);\nmsg.payload = msg.payload.toFixed(1) ;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1000,"y":240,"wires":[["23a715852d7ef0ad"]]},{"id":"1c0e2f9620e6ce43","type":"comment","z":"ea65f3bc16fcb67b","name":"Finder T5 datatype","info":"For two Modbus 16-bit registers.\n\nT5 \tUnsigned Measurement (32 bit)\nbits # 31..24 \tDecade Exponent(Signed 8 bit)\nbits # 23..00 \tBinary Unsigned Value (24 bit)\nExample: 123456*10-3 stored as FD01 E240(16)\t","x":250,"y":180,"wires":[]},{"id":"1c1c9200198a90e2","type":"modbus-client","name":"dongle 2","clienttype":"serial","bufferCommands":true,"stateLogEnabled":false,"queueLogEnabled":false,"tcpHost":"127.0.0.1","tcpPort":"502","tcpType":"DEFAULT","serialPort":"/dev/ttyFinder","serialType":"RTU-BUFFERD","serialBaudrate":"19200","serialDatabits":"8","serialStopbits":"1","serialParity":"even","serialConnectionDelay":"100","serialAsciiResponseStartDelimiter":"0x3A","unit_id":"1","commandDelay":"1","clientTimeout":"1000","reconnectOnTimeout":true,"reconnectTimeout":"2000","parallelUnitIdsAllowed":true}]

[Steve-Mcl]

I do not have your device that's why i said

@Steve-Mcl

Could you provide a demo flow with your data (provided by an inject node) → buffer parser → you function

To be specific...

• Set the function to read all of the registers you are interested in
• Add a debug node after the modbus node
• Capture the data from the debug output using the "Copy Value" button on the debug view
• Add an inject node & set the payload to JSON and paste the copied array of data into the inject
• Export the flow & paste into reply along side what output values you expect these to become e.g.
  • "the sample data provided should produce 240.12, 139.98, 242.711, 241.54"

[ericplan]

Sorry for that.

Here is the corrected flow. After some more reading I doubt my formula is correct.

It should be something like this:

N.n = (-1)S 2 e’-127 (1.f )

where S is the sign bit, e’ is the first part of the exponent and f is the decimal fraction placed next to 1. Internally the exponent is 8 bits in length and the stored fraction is 23 bits long.

A round to nearest method is applied to the calculated value of floating point.

(taken from https://www.kdk-argentina.com/distribuidas/finder/assets/comunication-protocol-_modbus_7e_64_68_78_86--series-7e---finder-(en-inlglés).pdf paragraph 2.3)

[{"id":"23a715852d7ef0ad","type":"debug","z":"ea65f3bc16fcb67b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1190,"y":240,"wires":[]},{"id":"74cca90e4d1b3c7c","type":"buffer-parser","z":"ea65f3bc16fcb67b","name":"","data":"payload","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"int8","name":"Exponent","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"hex","name":"Mantisse","offset":1,"length":3,"offsetbit":0,"scale":"1","mask":""}],"swap1":"","swap2":"","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"keyvalue","resultTypeType":"return","multipleResult":false,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":810,"y":240,"wires":[["5b175f1c730b805c"]]},{"id":"5b175f1c730b805c","type":"function","z":"ea65f3bc16fcb67b","name":"","func":"msg.payload = parseInt(\"0x\"+msg.payload.Mantisse) * Math.pow(10, msg.payload.Exponent);\nmsg.payload = msg.payload.toFixed(1) ;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1000,"y":240,"wires":[["23a715852d7ef0ad"]]},{"id":"368dbe9539dace9b","type":"inject","z":"ea65f3bc16fcb67b","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[65280,2294]","payloadType":"json","x":530,"y":240,"wires":[["74cca90e4d1b3c7c"]]}]

[Steve-Mcl]

So, the manual you linked to shows an example...

This is a 32 bit IEEE 754 float (1 bit sign, 8 bit exponent)

Buffer Parser floatBE and floatLE operators are standard Node JS Buffer functions readFloatBE``readFloatBE. From the nodejs buffer docs...

Reads a 32-bit, big-endian float from buf at the specified offset .

As JavaScript numbers are 64 bit double precision (1 bit sign, 11 bit exponent) I decided to test the example from the manual in a demo and it seems the Buffer floatBE function is IEEE 754 Single Precision compliant.

In short...

• forget all your maths & use the built in floatBE
• access the correct registers (see image below) from your device as those values are not an 32 bit IEEE 754 float in the expected range of 200.x ~ 240.x as I would expect a voltage to be.
  
  

PROOF...

Demo flow

[{"id":"549c70dbd6110e8e","type":"debug","z":"7741f79c9d2c194c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":2250,"y":300,"wires":[]},{"id":"ea36666785169780","type":"buffer-parser","z":"7741f79c9d2c194c","name":"","data":"payload","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"int8","name":"Exponent","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"hex","name":"Mantisse","offset":1,"length":3,"offsetbit":0,"scale":"1","mask":""},{"type":"floatbe","name":"floatbe","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""}],"swap1":"","swap2":"","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"keyvalue","resultTypeType":"return","multipleResult":false,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":2070,"y":240,"wires":[["eb9da536cf8d4e3e"]]},{"id":"eb9da536cf8d4e3e","type":"function","z":"7741f79c9d2c194c","name":"","func":"var your_calculated_float = parseInt(\"0x\"+msg.payload.Mantisse) * Math.pow(10, msg.payload.Exponent);\nmsg.payload.your_calculated_float = your_calculated_float.toFixed(1) ;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":2260,"y":240,"wires":[["549c70dbd6110e8e"]]},{"id":"35a365278b18ede9","type":"inject","z":"7741f79c9d2c194c","name":"Data you provided [65280,2294]","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"YOUR_DATA","payload":"[65280,2294]","payloadType":"json","x":1790,"y":180,"wires":[["ea36666785169780"]]},{"id":"1b0de2ced3f8ab4a","type":"inject","z":"7741f79c9d2c194c","name":"Manual Sample  45AACC0","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"BUFFER_HEXADECIMAL","payload":"[\"0x45\",\"0xAA\",\"0xCC\",\"0x0\"]","payloadType":"bin","x":1770,"y":240,"wires":[["ea36666785169780"]]},{"id":"94ca50e6b111f8a3","type":"inject","z":"7741f79c9d2c194c","name":"","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"BINARY_DATA","x":1675,"y":300,"wires":[["408005c6bb275e2e"]],"l":false},{"id":"408005c6bb275e2e","type":"function","z":"7741f79c9d2c194c","name":"binary example from manual","func":"msg.payload = [\n    0b0100010110101010,\n    0b1100110000000000\n]\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1840,"y":300,"wires":[["ea36666785169780"]]}]

[ericplan]

Thanks for the reply. I messed things up. But you pointed me to the right direction. I didn't post the 7m.38 manual, because it is a lot about other things, but I do it now. It's a sixty page pdf with about twenty pages about the modbus coding.

manual7M38_EN.pdf

It wasn't clear to me that in this manual that they use different registers to get the same data but in different formats. And due to the order on register number, the T5 option came first. So register 30107-30108 gives U-L1 in T5 format and on page register 32500-32501 gives the same value in T_float.

BTW, this is the T5 dataformat:

T5 Unsigned Measurement (32 bit)
bits # 31..24 Decade Exponent(Signed 8 bit)
bits # 23..00 Binary Unsigned Value (24 bit)
Example: 123456*10-3 stored as FD01 E240(16)

For now I am able to use the buffer parser with a float (be) and my problem is solved. Thanks again.

[JohPePe]

Hi Steve!

Using your mode-red-contrib-buffer-parser version 3.2.2

I think there is a bug in scaling,

Without scaling / scale =1 I get 65534
If I enter ‘-10’ into scale, I get -655340

If I enter +10, I get 655340

Looks to me it performs multiplication instead of subtraction and addition.

If you could look at this it would be great.

[dceejay]

To me "scale" usually means multiplication... for addition I would expect the word "offset"

[Steve-Mcl]

@JohPePe

Looks to me it performs multiplication instead of subtraction and addition.

Yes, you are correct. This is a documentation issue. Because the scaler is a "VERY BASIC" interpreter it doesnt know that you mean "add 10" because it thinks you want to "Scale by +10". In essence +10 is a number, + 10 is an equation

Example...

[{"id":"45b74d7b34dc1bc5","type":"buffer-parser","z":"90cd7ca5230acd50","name":"","data":"payload","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"uint16be","name":"raw","offset":0,"length":1,"offsetbit":0,"scale":"0","mask":""},{"type":"uint16be","name":"-10","offset":0,"length":1,"offsetbit":0,"scale":"-10","mask":""},{"type":"uint16be","name":"+10","offset":0,"length":1,"offsetbit":0,"scale":"+10","mask":""},{"type":"uint16be","name":"- 10","offset":0,"length":1,"offsetbit":0,"scale":"+ 10","mask":""},{"type":"uint16be","name":"+ 10","offset":0,"length":1,"offsetbit":0,"scale":"+ 10","mask":""}],"swap1":"","swap2":"","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"keyvalue","resultTypeType":"return","multipleResult":false,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":2010,"y":140,"wires":[["272d57267655e1f1"]]},{"id":"43d530deec0c9618","type":"inject","z":"90cd7ca5230acd50","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[65534]","payloadType":"json","x":1980,"y":80,"wires":[["45b74d7b34dc1bc5"]]},{"id":"272d57267655e1f1","type":"debug","z":"90cd7ca5230acd50","name":"debug 151","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":2020,"y":200,"wires":[]}]

Issue raised: Update scaling documentation to indicate a space is require for -10 and +10 · Issue #14 · Steve-Mcl/node-red-contrib-buffer-parser · GitHub

[JohPePe]

Hi Steve!

Thanks, problem solved!

Merry Christmas!

[lukasMega]

Hello, thank you for great parsing lib!
I have a question, is possible to define a bit more complex scale expression?
I need to apply: >>4 and then -5. I tried >>4 -5 but it's not working. It is possible to achieve it?
Or outside your parser?

I quickly took a look into source code but seems that your regex should support multiple scale expressions.