How to remove unreasonable numbers from moving on

Cyprus125 · 10 January 2022 01:33

Good day.

so i have a esp8266 with a DS18B20 (temp sensor). the sensor is outside and it's set up to transmit the temp every 30 seconds. (side thing to trouble shoot the sensor, i'll deal with when more parts arrive)

it's transmitting the temp fine, but randomly it transmits a -127. and this is missing with graphana and my nice charts.

in node red when i take the mqtt message and before i send it to the database is there a way to compare the last number with the new number and if it's greater then 10% don't pass it.

or if input = "-127" do not transmit it?

[
    {
        "id": "6430e60673a0017d",
        "type": "tab",
        "label": "Flow 1",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "812badc5d14cbe03",
        "type": "mqtt in",
        "z": "6430e60673a0017d",
        "name": "",
        "topic": "OUT",
        "qos": "0",
        "datatype": "json",
        "broker": "a04cd6a0.80a1d8",
        "nl": false,
        "rap": true,
        "rh": 0,
        "inputs": 0,
        "x": 510,
        "y": 240,
        "wires": [
            [
                "6e49b7a471bd51ba",
                "e8130ce8519450cf",
                "64101053f9ccb449"
            ]
        ]
    },
    {
        "id": "6e49b7a471bd51ba",
        "type": "debug",
        "z": "6430e60673a0017d",
        "name": "",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 730,
        "y": 200,
        "wires": []
    },
    {
        "id": "e8130ce8519450cf",
        "type": "influxdb out",
        "z": "6430e60673a0017d",
        "influxdb": "47ad9374b37cea08",
        "name": "OutSide",
        "measurement": "Outside",
        "precision": "",
        "retentionPolicy": "",
        "database": "database",
        "precisionV18FluxV20": "ms",
        "retentionPolicyV18Flux": "",
        "org": "organisation",
        "bucket": "bucket",
        "x": 720,
        "y": 240,
        "wires": []
    },
    {
        "id": "64101053f9ccb449",
        "type": "ui_gauge",
        "z": "6430e60673a0017d",
        "name": "",
        "group": "732cf2d4.e7e11c",
        "order": 1,
        "width": 4,
        "height": 4,
        "gtype": "gage",
        "title": "Out Side Temp",
        "label": "C",
        "format": "{{value}}",
        "min": "-10",
        "max": "50",
        "colors": [
            "#0065ff",
            "#00ff0c",
            "#ff0000"
        ],
        "seg1": "17",
        "seg2": "23",
        "className": "",
        "x": 740,
        "y": 280,
        "wires": []
    },
    {
        "id": "a04cd6a0.80a1d8",
        "type": "mqtt-broker",
        "name": "",
        "broker": "192.168.1.15",
        "port": "1883",
        "clientid": "Server",
        "usetls": false,
        "protocolVersion": "4",
        "keepalive": "60",
        "cleansession": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthPayload": "",
        "birthMsg": {},
        "closeTopic": "",
        "closeQos": "0",
        "closePayload": "",
        "closeMsg": {},
        "willTopic": "",
        "willQos": "0",
        "willPayload": "",
        "willMsg": {},
        "sessionExpiry": ""
    },
    {
        "id": "47ad9374b37cea08",
        "type": "influxdb",
        "hostname": "127.0.0.1",
        "port": "8086",
        "protocol": "http",
        "database": "sensors",
        "name": "Sensors",
        "usetls": false,
        "tls": "",
        "influxdbVersion": "1.x",
        "url": "http://localhost:8086",
        "rejectUnauthorized": true
    },
    {
        "id": "732cf2d4.e7e11c",
        "type": "ui_group",
        "name": "Home",
        "tab": "bcc983c6.f12668",
        "order": 1,
        "disp": true,
        "width": "8",
        "collapse": true
    },
    {
        "id": "bcc983c6.f12668",
        "type": "ui_tab",
        "name": "Home",
        "icon": "dashboard",
        "order": 1,
        "disabled": false,
        "hidden": false
    }
]

Trying_to_learn · 10 January 2022 01:40

You (probably) need to use the filter (or rbe node)

Example flow below.

[{"id":"581f105619104477","type":"rbe","z":"437e4108076e5928","name":"","func":"narrowbandEq","gap":"50","start":"1","inout":"out","septopics":true,"property":"payload","topi":"topic","x":430,"y":830,"wires":[["efa5c2e3feaba7eb"]]},{"id":"28ea8176f333dba5","type":"inject","z":"437e4108076e5928","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"-127","payloadType":"num","x":240,"y":790,"wires":[["581f105619104477"]]},{"id":"9f9494fc438016cd","type":"inject","z":"437e4108076e5928","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"20","payloadType":"num","x":240,"y":830,"wires":[["581f105619104477"]]},{"id":"0dd225f37914946e","type":"inject","z":"437e4108076e5928","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"11","payloadType":"num","x":240,"y":870,"wires":[["581f105619104477"]]},{"id":"3f2c03f6c720f8e8","type":"inject","z":"437e4108076e5928","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"40","payloadType":"num","x":240,"y":910,"wires":[["581f105619104477"]]},{"id":"efa5c2e3feaba7eb","type":"debug","z":"437e4108076e5928","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":610,"y":830,"wires":[]}]

This is an example and you need to change the value in the filter node.

Screenshot from 2022-01-10 12-36-36

(How to get this flow)
Over on the right side of the code (grey area) there is a small icon (if you move the mouse there) that copies it to the clip board.
Click.

Go to node-red. Top right burger menu and import.
Paste code there and OK.

Cyprus125 · 10 January 2022 01:46

ok i got it in the filter i can set the max or min i want to have change and it's going to do some comparing and stuff. this is perfect i'll play with this

Trying_to_learn · 10 January 2022 01:54

Well, so long as the changes aren't more than 127......

Just put 100 in where I have 50.

I say 100 because I tried 120 and for some reason it let the -127 through.

Sorry, I am near the bottom of the ladder of wisdom.
But that node should be the one you need.

craigcurtin · 10 January 2022 01:57

-127 is the error code for a bad read from the sensor - you could just put a switch node and if the value is < (insert reasonable value here) - throw it away i.e. have the switch part with a rule like

msg.payload > -100 then continue on to your grafana node - otherwise and leave that output blank

Craig

Trying_to_learn · 10 January 2022 01:58

Thanks. I didn't think of that.

Trying_to_learn · 10 January 2022 02:03

I'd go with what Craig said. A lot easier.

Cyprus125 · 10 January 2022 02:08

i understood your method better. i'm using it like a band pass filter. i only expect to see from -30 to 30.

i will do more research in his method too, cause now i know what to look for in the forums i'm actually finding a bit of the same issues.

Trying_to_learn · 10 January 2022 02:12

As Craig said though: the -127 is a specific value sent if the sensor isn't working.

So you could just put a switch node in like this:

and hook up to the second (lower) output.

Then any bad reading are filtered (blocked?) there.

Cyprus125 · 10 January 2022 02:18

ok i see when you add stuff to the switch you get more points to pull from.

Trying_to_learn · 10 January 2022 02:33

Yeah, not exactly intuitive....

But you don't use the first output as that is for values where the payload is -127 and not wanted.

Colin · 10 January 2022 07:55

If you configure the switch the other way round, as suggested earlier, so that it sends to output 1 if not equal to -127 then you don't need to configure a second output at all. -127 will be blocked and anything else will be passed.

Trying_to_learn · 10 January 2022 08:05

That's how I think.

I sometimes have problems with double negative things.

The switch passes only matches. Comparing it to a not value sometimes does my head in.

As there is no harm in it....
and somewhere I saw a link to the problems with optimising things too soon.

zenofmud · 10 January 2022 10:18

What software are you using on the esp8265? Why not check the reading there and if it is -127 ignore it and take another reading. That way you don’t have to worry about it in Node-RED

Colin · 10 January 2022 11:06

I am not sure why there would be a problem with the logic of:
Pass message to output 1 if it is not -127 and ignore anything else
versus:
Pass message to output 1 if it is -127
Otherwise pass message to output 2
Remember to use o/p 1 to connect to next node.

However I absolutely agree that above all else one should always use the solution one finds easiest to get ones head around. That way it is most likely to work first time and will be the easiest to understand in a years time when you come back to the flow for some reason. That is why I larger keep clear of JSONata and complex regular expressions. For me they take too much brain power to get to grips with.

Cyprus125 · 10 January 2022 12:19

the arduino IDE? i'm playing with a tutorial from https://randomnerdtutorials.com.

i don't want to modify that code. i know the sensor is not the best (as mentioned in the start i'm waiting for a replacement.) adding a node in node red was my idea of a temp fix while i wait for new sensors to come in the mail.

for everyone else. thanks for the help again. i'm so glad i can come to these forums and get assistance even with the blending of crap in my setup.

shrickus · 10 January 2022 13:51

I see that this switch is comparing msg.payload to the number -127
but your test inject node passes the string "-127" (without the quotes, of course)

I'm not sure what happens with that comparison, but I would expect it to work (if it actually uses the JS == operator as the pulldown implies). But perhaps this is why some of your tests are not doing what you think it should?

zenofmud · 10 January 2022 13:59

There are lots of tutorials there

Trying_to_learn · 10 January 2022 19:57

Colin,

Just now my brain isn't worth a pinch of......

So clear thinking is conceptual for me (just now)

Having a few bad months.

Trying_to_learn · 10 January 2022 19:59

Oops.

Thanks.

But I hope the idea was conveyed.

Topic		Replies	Views
Remove / smoothen unrealistic item values which randomly appear General	5	338	7 June 2022
Switch Node \| Beginner Question General	3	173	28 November 2022
MQTT - DS18B20 Sensors and Rules Question General	19	242	3 September 2023
Sorting positive and negative digits General	7	1163	2 July 2021
Accessing Temperature property in payload (DS18B20 - Tasmota - MQTT) General	9	1990	13 May 2021

How to remove unreasonable numbers from moving on

Related topics