The best way to notice lack of message?

General
1

Hi there

What will you propose to inform user if the message with known topic won't show for some time?

I'm using rpi-ds18b20 node to periodically read all my DS' sensors. The network is quite big (I know, 1w was not designed for that) so I've decreased a bit pull-up resistor and generally all is working fine. However, I would like to be informed (and display it on my floorplan) that data from particular DS is missing for some time (I'm reading every minute, so I would guess 3 minutes for 3 readings sounds reasonable to dismiss one-time glitches).

My flow is currently similar to: inject [every 1'] -> rpi-ds18b20 -> function: map ID to meaningful topic -> mqtt and then mqtt -> join [key/value to store all data] -> ui_template. The problem with such scenario is that join node (as described in my previous thread) is updating only new data, so when one of my DS will die or RPi won't be able to read it, it will be lost - the last valid value will be displayed forever.

I was thinking of using trigger node but that node is not aware of missing data, so that's wrong direction.

This sounds like it should be a popular issue but I wasn't able to track similar problem.
Thanks

2

Can you link to which DS18B20 node you are using ?

The one i just looked up lets you specify an ID to read and returns a temperature and timestamp - although i have not used it i would assume this would give you a Zero temperature if nothing returned ?

Craig

3

Yes, this one. It's the only one that let's you not specify ID but read all available sensors and just publish one message per DS and I'm using that feature. It's nice as adding extra sensor does not require to edit the flow at all if you just want to see it on the graph (and not being associated with particular mqtt topic).

Having separate input with selected ID for every DS didn't sound so good for me.

4

Maybe you could keep an array of objects in a context variable with the IDs and timestamps and compare all timestamps with current time, then if any timestamp is too old then you know you are missing data and act on it?

5

Here's a flow that that you you could adapt to your situation:

[
    {
        "id": "4269891d.0c1e78",
        "type": "inject",
        "z": "143ac8d4.56de37",
        "name": "",
        "topic": "test1",
        "payload": "",
        "payloadType": "date",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 340,
        "y": 580,
        "wires": [
            [
                "40fd5fbc.0fd28"
            ]
        ]
    },
    {
        "id": "40fd5fbc.0fd28",
        "type": "function",
        "z": "143ac8d4.56de37",
        "name": "topics update frequency test",
        "func": "var topics = context.get('topics') || []\nvar found = 0\nvar threshold = 10000\n\nfor (var i = 0; i < topics.length; i++){\n    if (msg.topic == topics[i].topic){\n        topics[i].timestamp = msg.payload\n        found = 1\n    }\n    if (Date.now() - topics[i].timestamp > threshold){\n        node.warn(topics[i].topic  + \" hasn't been updated recently\" )\n    }\n}\nif (found === 0){\n    topics.push({\"topic\":msg.topic, \"timestamp\":msg.payload})\n}\ncontext.set('topics',topics)\nmsg.payload = topics\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "x": 582,
        "y": 581,
        "wires": [
            [
                "ab7a8e37.5ac69"
            ]
        ]
    },
    {
        "id": "f6fbcb8a.e05f68",
        "type": "inject",
        "z": "143ac8d4.56de37",
        "name": "",
        "topic": "test2",
        "payload": "",
        "payloadType": "date",
        "repeat": "1",
        "crontab": "",
        "once": true,
        "onceDelay": 0.1,
        "x": 350,
        "y": 620,
        "wires": [
            [
                "40fd5fbc.0fd28"
            ]
        ]
    },
    {
        "id": "ab7a8e37.5ac69",
        "type": "debug",
        "z": "143ac8d4.56de37",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "x": 790,
        "y": 580,
        "wires": []
    },
    {
        "id": "a8f08606.0b4f88",
        "type": "inject",
        "z": "143ac8d4.56de37",
        "name": "",
        "topic": "test3",
        "payload": "",
        "payloadType": "date",
        "repeat": "1",
        "crontab": "",
        "once": true,
        "onceDelay": 0.1,
        "x": 350,
        "y": 660,
        "wires": [
            [
                "40fd5fbc.0fd28"
            ]
        ]
    },
    {
        "id": "6991b8e8.d86aa8",
        "type": "comment",
        "z": "143ac8d4.56de37",
        "name": "Press test1 and wait 10 seconds, a warning will appear in debug",
        "info": "",
        "x": 530,
        "y": 520,
        "wires": []
    }
]
1 Like
6

Not understanding sorry - you must already be keeping a table somewhere that matches the ID of the sensor with a name/location/identifier ?

Why not use that table/list to just loop through and make a call to each individual sensor ?

How long is your 1Wire run ? If you are doing just a generic call for all sensors on the cable it must take a reasonable amount of time where your flow is blocked until they all respond ?

I personally try and limit it to no more than 5 sensors on an individual run (i run mine from Arduino and ESP8266)

Craig

7

The trigger node can do what you want. You set it to output nothing on first input, delay for longer than your read value loop time, extend delay on new input, and output something on second output.

That way it will keep not sending anything until it doesn't see a reading, then it will send a message.

1 Like
8

hugobox, thanks. Looks like I might try to write similar code.

craigcurtin, it looks much cleaner to have simple 2..3 node flow and just add one line in func node for new sensor rather than separate node per sensor. I'll rethink this approach now when I need to catch lack of data.
MCP23017 node must be one per input and I have it ~12 - it does not look clean, hence I was happy that there was a node for DS that can grab all at once.

dceejay, but again, such trigger node would need to be added per sensor in approach suggested by craigcurtin - in my current flow I can't think of working solution.

Thanks for your input. I'll work on my flow and see how it goes.

9

If you set the trigger to handle each msg.topic independently and arrange your different sensors to have different topics then a single node could handle them all.

1 Like
10

Will check that today, thanks for the hint.

11

That is a great way to handle watchdogs. Even more elegant than what I've been doing with the node-red-contrib-mytimeout.

12

We try to please

13

You inevitably do :slight_smile:

14

I use this technic a lot I must admit,

I have a trigger node like you described to monitor many of my services of various kind. The trigger node also triggers a second trigger node if other measures like restarting the service fails, leading to a complete reboot (of a dedicated Pi in my case)

In the example below, I am monitoring that the connection to our heat pump is up and functioning as expected

image
image.png777Ă—375 25.7 KB

3 Likes
15

I must say that I'm ashamed a bit for loosing faith in trigger node. Thanks dceejay, indeed, simply adding just one node (with handle each msg.topic independently) to the current flow did the trick. All is working as expected.
Thanks!

16

welcome back

17

Hi krambriw,

I am also looking for monitoring my ESP-sensors and your example seems to be fine. Can you please share your code with us, because I have no idea of how to set the trigger-node. Thanks.

18

Just set the trigger node to send nothing immediately then whatever you want after the timeout (a failure message for example), with Extend Delay ticked so that once it receives a message it will never send anything, unless it times out.
Set up a simple flow with an inject, a trigger, and a debug node and play with the settings so you can understand how it works.

19

With ESP sensors - if they are using MQTT - you can also use the LWT (Last Will and Testament) feature of MQTT to get the broker to report a missing sensor to you.

20

Thank you!

@ Colin: yes, I did it and I understand now.
@ dceejay: I am using MQTT, but never heard of LWT. Since my platform usually is ESPEasy, I have to find out where and how I can set it.