[UPDATED] node-red-contrib-buffer-parser - V3.1.2

[Steve-Mcl]

Hi all so, first release of node-red-contrib-buffer-parser

The name doesnt do it justice so I will post snippets from the readme below - to try and explain it.

Hope its of use to people.

About

A dynamic Node-RED node to convert values in a buffer or integer array into the many different data type(s). Supports Big/Little Endian, BCD, byte swapping and much more.

A picture is worth a thousand words

Summary of functionality

• Setup a specification and convert multiple parts of an array or buffer to...
  • int, int8, byte,
  • int16, int16le, int16be, uint16, uint16le, uint16be,
  • int32, int32le, int32be, uint32, uint32le, uint32be,
  • bigint64, bigint64be, bigint64le, biguint64, biguint64be, biguint64le,
  • float, floatle, floatbe, double, doublele, doublebe,
  • 8bit, 16bit, 16bitle, 16bitbe, bool,
  • bcd, bcdle, bcdbe,
  • string, ascii, utf8, utf16le, ucs2, latin1, binary
• Specification is dynamic & can be sent in as a msg/flow/global property - permitting fully dynamic setup (e.g. via a dashboard)
• The specification format permits random access (e.g. no need for any skips when accessing only first and last elements)
• You can specify the same offset many times to convert the same piece of data several times
• The data can be byte swapped one or more times. 16, 32 or 64 bit swaps are possible. The byte swaps are done prior to any data conversions like LE or BE functions (sometimes it is necessary to do multiple swaps)
• The output can be sent in any msg property. e.g. you can send results out in msg.my.nested.property . This has the advantage of leaving the original payload in tact.
• Input data can come from any msg property (not limited to msg.payload)
• Input data can be a 16bit array (common plc data format) simplifying working with PLC type data arrays
• Output results can be multiple messages as topic and payload
  • ideal for taking PLC data and sending it directly to MQTT
• Output results can be a single msg style output
  • ideal for converting multiple different data elements into one object to pass on to perhaps a template node for preparing a SQL or HTML statement using {{mustache}} formatting
  • additionally, output results can be 1 of 4 styles...
    • "value" : the parsed values are sent in an array
    • "object" : the parsed values are sent as named objects with the value set .value and other contextual properties included (like the item specification)
    • "array" : the parsed values are sent as objects in an array, with each object containing a .value property and other contextual properties included (like the item specification)
    • "buffer" : this mode simply returns a buffer (no item processing)
• Built in help

[ghayne]

@Steve-Mcl

The name doesnt do it justice

Cough and

[Steve-Mcl]

In fairness, Its the best of a bad bunch.

The name I preferred was node-red-contrib-array-or-buffer-to-any-of-bcd-int-int16-int32-bigint64-float-with-multibyteswapping-and-big-or-little-endian-capabilities as it was more descriptive

[ghayne]

@Steve-Mcl

The name I preferred was node-red-contrib-array-or-buffer-to-any-of-bcd-int-int16-int32-bigint64-float-with-multibyteswapping-and-big-or-little-endian-capabilities as it was more descriptive

Easy to find though!

[TotallyInformation]

Nice, could be useful for making sense of the crazy stuff that comes out of SNMP.

[Steve-Mcl]

Hi all, V2.0.0 now in flows library

This adds a full UI to simplify setting up the node

Hopefully this makes it a bit more friendly.

[cymplecy]

I've no idea about this node but if I could just offer some feedback on your UI?:

data - I'd use core node standard label ... Property

The return fields just look wrong to me
I wouldn't have the arrow at all
return - I'd use Send
Should the node give a choice of output property on the message? - not seen that before
If it was removed then there wouldn't be visual confusion over the return to and return fields

Pure social comment in passing

[Steve-Mcl]

@cymplecy

if I could just offer some feedback on your UI?:

Of course, it's welcomed.
Before I go forward though, I'd like to point out, all elements on this UI use standard node-red elements & classes (label, typedinput etc)

@cymplecy

Should the node give a choice of output property on the message? - not seen that before

100%

I hate having to use a change node after a node to move the payload to another property.
Definitely won't be removed. There are plenty of nodes do this for that very reason. Also, it leaves the current payload in tact if you set the "return to" to something different to the incoming payload - often I need the original payload down stream.

@cymplecy

data - I'd use core node standard label ... Property

I don't understand? The data is a standard label. No different to any of the core nodes.

@cymplecy

The return fields just look wrong to me
I wouldn't have the arrow at all
return - I'd use Send

Yeah I struggled picking a reasonable icon for return.
Is that fa-send you refer to?

Also, the controls are node-red typedInputs & I am not certain how to have no icon on the left (I support I could set the typedInput "type" text to a space?)

@cymplecy

Pure social comment in passing

No, I appreciate feedback Simon.

My 2c...
The labels are hard to choose as the core standard CSS classes form-row defaults label width to 30% if your label is too descriptive, it wraps the text and the UI looks terrible so labels are kept short (but not very descriptive). So while not ideal, I do full help in the side bar for every item in the UI (not that many folk read the help )

[dceejay]

@Steve-Mcl

@cymplecy

Should the node give a choice of output property on the message? - not seen that before

100%

I hate having to use a change node after a node to move the payload to another property.
Definitely won't be removed. There are plenty of nodes do this for that very reason.

Well - none of the core nodes do this. (Some do allow selecting which property to operate on.) - and the label is Property.

I also prefer send rather than return as you are sending the message onwards not returning from a call.

You can of course override the width of the form-width css in your specific edit config so the labels don't wrap (within limits of course) - as long as you do it consistently in your ui.

[Steve-Mcl]

@dceejay

Well - none of the core nodes do this

Imho, I wish they all would (where sensible). Since the introduction of the typedInput, things have improved (with regards to reducing the need for change nodes before and after) but not all nodes employ them unfortunately.

@dceejay

and the label is Property.

Is it standard that input data be labelled "Property"? I honestly hadn't noticed. Taken on board. Thanks for clarifying.

@dceejay

I also prefer send rather than return

No problem. I do try to stick to convention (mostly). Will review.

@dceejay

You can of course override the width of the form-width css in your specific edit config so the labels don't wrap

Yeah but again, trying to be uniform Dave.

[Steve-Mcl]

@dceejay @cymplecy

What do you guys think of these tweaks to the UI?...

I tried to stick with what was most common in the core nodes...

(PS: I knew I'd seen <-- Return somewhere before)

[dceejay]

So yes - I prefer just "Output" to "Output what" (not sure that is even good English)...
Also to my mind the order reads wrongly... I would read it like a sentence as -
Output .... something
to a property called .....
with these options

I have no idea what the Swap row is about - even having read the info - but I guess/hope it defaults to do nothing. Then the info does say .

[cymplecy]

Was just about to reply exactly the same as Dave s first sentence.

Much nicer looking

[Steve-Mcl]

@dceejay

I would read it like a sentence as -
Output .... something
to a property called .....
with these options

I understand and agree however the options affect the available "Output what" options

@dceejay

I have no idea what the Swap row is about - even having read the info - but I guess/hope it defaults to do nothing

It does default to "none". Its byte swapping of data with incompatible endianess (a common requirement with modbus / profinet etc). Perhaps it would be instantly more clear as "Byte swap"?

Good spot on the help info "2st"

Maybe this is closer / better?

[seth350]

The PLC gods are pleased with this contribution.

@Steve-Mcl, you are a hero to many.

[Steve-Mcl]

Cheers @seth350

I often need to get consistent data & therefore arrange logic in the PLC so that items are contiguous - ready for collection in one poll (and therefore consistent with each other). Then I have the need to convert the 1st word into 16 control bits, the next 1 word into an uint16 identifier, the next 2 words into an unsigned byte reversed 32bit timestamp value etc etc.

This node was borne of frustration

[dceejay]

Ok I’m getting there. Order ok if options change what comes next.
So why 3 byte swaps ?

[Steve-Mcl]

@dceejay

So why 3 byte swaps

I should note in the help, the UI permits 0, 1, 2 or 3 consecutive swaps. I can't post an example right now but sometimes i have needed to swap 32 then swap 16 in order to make sense of mixed bit and word data. The third swap is just in case

Thanks everyone for the feedback - much appreciated.