Parsing binary encoded data from decimal array

General
1

I'm not too sure how to start with this problem, so looking for some direction.
I'm dealing with a Huawei Luna2000 battery, pulling data from it via Modbus and sending some of it on to MQTT

I receive an array of numbers (U16) that represent sets of start times, end times, days of the week on which the interval applies, & a charge/discharge mode flag.
The first number in the array contains the number of declared periods.
The times are easy to deal with but the mode and days of the week are where I struggle.

The mode flag is in the first 8 bits of the U16 binary number, either 1 or 0
Days of the week are coded into the second 8 bits, bytes 0 -7, 0=inactive, 1=active.

Example data from a Debug node:
[5,901,1260,383,720,840,65,720,900,62,1260,1439,383,0,480,383,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]

Decoding this:
5 = number of time periods
901 = minutes since midnight, = 15:01
1260 = 21:00
383 = 0000000101111111 reading from the right we get All days Active (01111111) and mode is discharge (00000001)

If the flag/days was eg 65 we'd get 0000000001000001 meaning Sat & Sun only, and mode = Charge

So I'm after ideas how best to tackle decoding this flag/days data into something I can more easily operate with.
I've looked at a couple of binary-decimal library contribs but not sure which is best suited, or if it might be simpler to use a function node (although my Javascript skills are only slightly above non-existent).

Ultimately I'm looking to be able to query what the current state is.
Check if current time is in between a start/end time and then check if the day is flagged Active, and finally determine if it's a Charge or Discharge mode. Potentially I'll run a flow that checks this periodically and posts a result to MQTT.
I could either poll the Modbus very infrequently to get the current settings then do all the parsing on that stored data. Or I could poll Modbus whenever I want to update the current state. The schedule is only updated maybe monthly or less once it's setup.

It would be nice if I could find a way to read the current state direct from a battery Modbus register, but this doesn't seem to exist.

TIA

2

I am not happy with myself for saying it, but......

(It is amazing what Chatgpt can do.)

You don't need to log in. Just go to the site.

I would believe if you said you were working with Node-red and received this message (as posted) and gave the description as you did.

You would be a long way to getting what you want.

Basically everything from the I receive an array....... down to If the flag/days was ......
You may need more, but to me that's a start.

It will repeat back what you are asking - for checking - and then offer you a solution.

Hope that helps.

3

Hi,

maybe this helps,
my flow to get data out of sun2000 and luna2000,
got it from somewhere here from the forum,

flows(2).json (92.6 KB)

1 Like
4

Thank you both.

RedTom's Huawei flow will take me a little while to digest, but it's interesting they grab a big block of Modbus registers then just use a few of them. I'll think about that. I had a look at buffer-parser but couldn't work out how to get it to do what I wanted. Probably my lack of comprehension I suspect.
At first look through I'm not sure it helps me with the TOU schedules, naybe it's buried deeper than I've looked so far.

Trying_to_learn's suggestion of ChatGPT has provided a couple of solutions.
Good suggestion, but it feels odd to turn to a Forum of humans only to be directed to an automaton alternative. Brave New World.

5

Yeah, I know that feeling.

Yes, we are here to help, but some things are just too....... weird that as hard as you try (and we are good at understanding/reading between the lines) I have to admit that sometimes it is easier to ask and maybe have to go through some iterations to get to the final version.

Just out of curiosity..
How much of your original post did you enter to get the results that were usable?

6

I just copied in as you suggested, "I receive ..." to "If the flag/days...". With a first sentence stating "I am working in nodered"

The suggestions included a Function Node:

let arr = msg.payload;  // your input array
let periods = [];
let numPeriods = arr[0];

for (let i = 0; i < numPeriods; i++) {
    let start = arr[1 + i*3];
    let end   = arr[2 + i*3];
    let flag  = arr[3 + i*3];

    let mode = (flag & 0xFF) === 1 ? "discharge" : "charge";
    let daysMask = flag >> 8;

    // Decode days into array of names
    let daysOfWeek = ["Sun","Mon","Tue","Wed","Thu","Fri","Sat"];
    let activeDays = daysOfWeek.filter((d, idx) => (daysMask >> idx) & 1);

    // Format time back to HH:MM
    function toHHMM(minutes) {
        let h = Math.floor(minutes/60);
        let m = minutes % 60;
        return `${h.toString().padStart(2,"0")}:${m.toString().padStart(2,"0")}`;
    }

    periods.push({
        start: toHHMM(start),
        end: toHHMM(end),
        mode: mode,
        days: activeDays
    });
}

msg.payload = periods;
return msg;

An alternative JSONata for Change node:

(
  $arr := $;
  $numPeriods := $arr[0];
  $days := ["Sun","Mon","Tue","Wed","Thu","Fri","Sat"];

  $toHHMM := function($minutes) {
    $string($floor($minutes div 60)).padStart(2,"0") & ":" &
    $string($minutes mod 60).padStart(2,"0")
  };

  $map([1..$numPeriods], function($i) {
    $start := $arr[1 + ($i-1)*3];
    $end   := $arr[2 + ($i-1)*3];
    $flag  := $arr[3 + ($i-1)*3];

    $mode := ($flag and 255) = 1 ? "discharge" : "charge";
    $daysMask := $flag div 256;

    {
      "start": $toHHMM($start),
      "end": $toHHMM($end),
      "mode": $mode,
      "days": $filter($days, function($d, $idx) { ($daysMask shr $idx) and 1 })
    }
  })
)

And then some options to format it all into tables.

I also asked if contrib-buffer-parser could do this and it explained that could not decode bitmasks, which is where I'd got stuck with it.

Now to see if the code it spat out works (unlikely in my past experience with AI coding on microcontrollers) and where the little bugs are, if any.
When I used AI for microcontroller coding recently I found Claude gave nicer, more useful code than ChatGPT. Maybe I'll ask Claude as an experiment.

1 Like
7

This is the better way to do it.
Here is an article I wrote that might help you understand what & why

Search this forum for buffer-parser there are lots of posts that take users from zero to hero: Search results for '@steve-mcl buffer-parser order:latest' - Node-RED Forum

Or post a demo flow (using injects/functions to simulate your data) and explain how it needs to be processed (including real, final values helps too)

8

I'm not saying ChatGPT is THE best.... Gotta try more than one.

It may help if you tell it things like:
add comments when you are specifying what you want.
As strange as it sounds, it kind of helps you understand the code.

Also - IMO - don't have ending ;
Use const as much as possible.

(There are a lot of things you can specify but they are above my skill set just now)

9

Thanks Steve.
I've followed those links and learnt a bit for sure.
The Modbus registers I'm interested in are pretty scattered amongst 150 available registers, usually 2 or 3 that I'm interested then 10 or so I'm not, then another couple. There's one gap of about 70 registers, so maybe I make two blocks either side of that gap and do two reads.
I'll play around and see what looks sensible.

Next step is get my head around using buffer-parser.

I put my original question into Claude AI which returned a function that output good results almost immediately and had clearer array structure than ChatGPT's, so I'm tweaking that one to get just what I need. A bit more complex than ChatGPT's but "it just works".

After several reiterations of ChatGPT's Change function JSONata trying to resolve errors, it finally advised that it couldn't be done with NR JSONata.

1 Like
10

Would you mind elaborating on this bit?

But...
Glad Claude did it for you.

:wink:

11

ChatGPT initially offered two solutions, first a function/JavaScript then a Change node using JSONata. I asked for both out of interest.

The function needed several iterations to get in a form that didn't throw NR errors due to syntax. The array it produced was ok but no naming to the array sections (not sure that's the correct term) so not quite as user friendly.

I tried the Change node but after several rounds back & forth with ChatGPT it finally said that actually it wasn't possible with NR's specific JSONata.
Didn't really surprise me.

Claude's function only needed swapping of the bit mask bytes & it worked perfectly.

12

Ok, thanks.

I was just wondering what you meant.

I know I have to go through several iterations of the code before I get a good one.
But such is life.

So Who is this Claude?

13

Claude is Anthropic's AI.
Their ethics & social responsibilty is equally as questionable as OpenAI's.
Formed largely by ex-OpenAI employees.

I don't really like supporting any of them until they get forced to pay for the IP they harvest. And let's not get into the responsibility of open sourcing LLM's with no control over where they go & what they do!

14

I don't log in to ChatGPT myself.

(The url for Claude?)
(You can't use him for free can you? -- aka: without logging in.)

15

Chatgpt is free even if you do login. The advantage of logging in is that you can go back to previous threads and continue them. At least I presume that is not available if you don't login.

16

Yes, but EVERYTHING you type is kept FOREVER.

I'm not sure I like that idea.

Though I have recently seen good reasons to log in.

You can get PDF files created, which could be handy.
Still getting splinters on it though.

17

Claude.ai

You have to give an email & receive a link. I usually use disposable email addresses.

1 Like
18

(That's a good idea.)

Silly me didn't think of that.

:slight_smile:

19

(Why am I seeing notifications his thread is updated, yet I'm not seeing any new posts?)

20

One way to do it is to use node-red-contrib-buffer-parser.
https://flows.nodered.org/node/node-red-contrib-buffer-parser

Like in this test flow i made

[{"id":"46e3cba39cab3df2","type":"inject","z":"c3f5769dd0078883","name":"Create Buffer","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[5,901,1260,383,720,840,65,720,900,62,1260,1439,383,0,480,383,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]","payloadType":"json","x":1230,"y":180,"wires":[["22c4c0f0e8a9cabb"]]},{"id":"5438741cf860918d","type":"debug","z":"c3f5769dd0078883","name":"debug Buffer","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1370,"y":240,"wires":},{"id":"39a478c6b7834d66","type":"buffer-parser","z":"c3f5769dd0078883","name":"","data":"payload","dataType":"msg","specification":"spec","specificationType":"ui","items":[{"type":"uint16be","name":"periodCount","offset":0,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"uint16be","name":"minSinceMidnight","offset":2,"length":1,"offsetbit":0,"scale":"1.66667","mask":""},{"type":"uint16be","name":"minSinceMidnightHHMM","offset":4,"length":1,"offsetbit":0,"scale":"1.66667","mask":""},{"type":"8bit","name":"days","offset":7,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"8bit","name":"mode","offset":6,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"bool","name":"exModeBit0","offset":6,"length":1,"offsetbit":0,"scale":"1","mask":""},{"type":"bool","name":"exModeBit1","offset":6,"length":1,"offsetbit":1,"scale":"1","mask":""},{"type":"bool","name":"exDayBit2","offset":7,"length":1,"offsetbit":2,"scale":"1","mask":""}],"swap1":"","swap2":"","swap3":"","swap1Type":"swap","swap2Type":"swap","swap3Type":"swap","msgProperty":"payload","msgPropertyType":"str","resultType":"keyvalue","resultTypeType":"return","multipleResult":false,"fanOutMultipleResult":false,"setTopic":true,"outputs":1,"x":1530,"y":180,"wires":[["ae92d1ae271a3d21"]]},{"id":"ae92d1ae271a3d21","type":"debug","z":"c3f5769dd0078883","name":"debug Parsed","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1620,"y":240,"wires":},{"id":"22c4c0f0e8a9cabb","type":"junction","z":"c3f5769dd0078883","x":1340,"y":180,"wires":[["5438741cf860918d","39a478c6b7834d66"]]}]
1 Like