Node serial port, how to perform crc check

General
1

I am using the node 'serial in' from node-red-node-serial-port, to read data from the smart meter.
This node is connected to a function node to process the data coming from the smart meter.
It works well, but sometimes I get corrupted telegrams. I am trying to figure out how to do a crc-check on the data, since there seems to be no option for this in the configuration in the serial-port node.

Can anyone help me with this issue?

afbeelding
afbeelding500Ă—684 9.78 KB

This is the output from the serial-in node:

afbeelding
afbeelding453Ă—744 9.11 KB

2

I'm not a pro, but to do a CRC check, the source of the data must provide one, and for you to do your own CRC check to ensure it matches with what was provided.

So if the original data does not carry a CRC - you have no way to determine if its altered/currupted.

I could be wrong here.

EDIT
The serial port node, will not have a place to do CRC - as it needs to understand the data structure, to be able to do so (and for the source to provide one)

3

The value after the exclamation mark is the crc value. So in this case 6FEC.

4

Ok that makes things easier. Do you know the format of the CRC (8, 16, 32) etc?
There are node libraries, that you can use to calculate the CRC of a value.

Using such library, you can generate the CRC and check it with the provided CRC.

5

Assuming you know how to extract that CRC at the end of the data, you can likely use this solution to compute and verify it.

1 Like
6

Nice!

I was looking for a library to use as an example - you saved me having to do it. :pray:

7

@Steve-Mcl and @marcus-j-davies I first tried getting the telegram through the crc calculator. I tried all different ways, but no outcome returned a match. This telegram I tried,

/Ene5\T211 ESMR 5.0



1-3:0.2.8(50)

0-0:1.0.0(231028193754S)

0-0:96.1.1(4530303632303030303130313639363232)

1-0:1.8.1(000545.093*kWh)

1-0:1.8.2(000237.780*kWh)

1-0:2.8.1(001126.383*kWh)

1-0:2.8.2(002106.647*kWh)

0-0:96.14.0(0001)

1-0:1.7.0(02.354*kW)

1-0:2.7.0(00.000*kW)

0-0:96.7.21(00008)

0-0:96.7.9(00004)

1-0:99.97.0(1)(0-0:96.7.19)(220816003006S)(0000000380*s)

1-0:32.32.0(00001)

1-0:52.32.0(00001)

1-0:72.32.0(00002)

1-0:32.36.0(00000)

1-0:52.36.0(00000)

1-0:72.36.0(00000)

0-0:96.13.0()

1-0:32.7.0(225.0*V)

1-0:52.7.0(227.0*V)

1-0:72.7.0(224.0*V)

1-0:31.7.0(001*A)

1-0:51.7.0(000*A)

1-0:71.7.0(008*A)

1-0:21.7.0(00.301*kW)

1-0:41.7.0(00.133*kW)

1-0:61.7.0(01.920*kW)

1-0:22.7.0(00.000*kW)

1-0:42.7.0(00.000*kW)

1-0:62.7.0(00.000*kW)

0-1:24.1.0(003)

0-1:96.1.0(4730303733303034313539343039353232)

0-1:24.2.1(231028193500S)(00019.612*m3)

!668D

I guess it must be crc16. What part of the telegram is crc calculated to get the good result?

I just found out, that the type of CRC algorithm = CRC-16-IBM. This algorithm is not in the CRC calculator and also not present in easy-CRC!

8

I have just put this together - it uses easy-crc

9

@marcus-j-davies @Steve-Mcl

That seems to be from "/" until "!" included. See also my previous remarks about the calculator.

10

Since it is a 4 digit WORD, I believe it will be a 32bit CRC. It could be a well known or a completely made up CRC.

11

@Steve-Mcl @marcus-j-davies I found a python programm that seems to do the trick.

def calc_crc_telegram(telegram):
x = 0
y = 0
crc = 0
#print "Lengte telegram",len(telegram)
while (x < len(telegram)):
crc = crc ^ ord(telegram[x])
x = x + 1
y = 0
while (y < 8):
if ((crc & 1) != 0):
crc = (crc >> 1)
crc = crc ^ (int("0xA001",16))
else:
crc = crc >> 1
y = y + 1
#print "CRC calculated :", crc
return crc

I just need to translate it to javascript. :smiley:

12

Shamelessly pulled from a converter :angel: (untested)

function calc_crc_telegram(telegram) {
    let x = 0;
    let y = 0;
    let crc = 0;

    while (x < telegram.length) {
        crc = crc ^ telegram.charCodeAt(x);
        x = x + 1;
        y = 0;
        while (y < 8) {
            if ((crc & 1) !== 0) {
                crc = crc >> 1;
                crc = crc ^ parseInt("0xA001", 16);
            } else {
                crc = crc >> 1;
            }
            y = y + 1;
        }
    }

    return crc;
}
13

@marcus-j-davies Woow great, I was struggling already.

Will test it in one hour or so, have to do some urgent things.

14

@marcus-j-davies @Steve-Mcl Yep it is working.

I made a test function like this,

function calc_crc_telegram(telegram) {
    // This function calculates a CRC with algorithm CRC-16-IBM,
    // that is used in Dutch smart meters.
    let x = 0;
    let y = 0;
    let crc = 0;

    while (x < telegram.length) {
        crc = crc ^ telegram.charCodeAt(x);
        x = x + 1;
        y = 0;
        while (y < 8) {
            if ((crc & 1) !== 0) {
                crc = crc >> 1;
                crc = crc ^ parseInt("0xA001", 16);
            } else {
                crc = crc >> 1;
            }
            y = y + 1;
        }
    }

    return crc;
}

let telegram = msg.payload;
let str = telegram.split("!");

let telStr = str[0] + "!";
let crcStr = str[1];

let crc = calc_crc_telegram(telStr)
let h_crc = crc.toString(16).toUpperCase().padStart(4, '0');

msg = {telegram: str[0], crc: str[1], crc_calc: h_crc}

I wil integrate this in my project, hopefully I will then get rid of corrupted messages that appear now and then.I will tell you the result.

15

@marcus-j-davies @Steve-Mcl

As I said, I integrated the solution to my project, and it worked well. About 30% of the messages was corrupt. By now I also know the reason of the corrupt messages. Before I had a 4-fold hub between the smart meter and the pi. Later I removed that. After the removal the corrupt messages started coming in. So I placed back the hub and the corrupt messages were gone. My conclusion is that this hub improves the signal coming from the smart meter. I have zero corrupt messages now.

2 Likes
16

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.