Filter by change or x min

I want to reduce data flow. From a sensor I get a value every 10s. In case there's no change I don't want to save this value (RBE-Node) except every 5 min (something like "still alive")... Otherwise it'll look in the dashboard as there have been no values which might be interpreted as error...

Any ideas? Currently I'm using the RBE-filter...

follow the filter/RBE node with a trigger node set to resend existing msg - then resend every 5 minutes...

So after the RBE a trigger...
But won't the trigger resend EVERY msg? Meaning also the messages, that have different values compared to the message before?
Maybe I haven't described in detail... As long as the values vary, I want to have the values every 10 seconds. Only if the value is the same than before (i.e. PV power during night), then I want to have a value (although it's the same value) every 5 min...

You can use a trigger node to reset the filter if no message has emerged from it for 5 minutes.
The next input message will then pass through the filter even if it's the same value.


[{"id":"9bf344f8c13cc042","type":"rbe","z":"d2183b1d5cb6d0b1","name":"RBE","func":"rbe","gap":"","start":"","inout":"out","septopics":true,"property":"payload","topi":"topic","x":530,"y":120,"wires":[["e20b1c5b5a7bf5eb","b01cda49dd383cf5"]]},{"id":"e20b1c5b5a7bf5eb","type":"trigger","z":"d2183b1d5cb6d0b1","name":"Wait 5 min","op1":"","op2":"","op1type":"nul","op2type":"payl","duration":"5","extend":true,"overrideDelay":false,"units":"min","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":430,"y":180,"wires":[["2f243365e5746f7d"]]},{"id":"2f243365e5746f7d","type":"change","z":"d2183b1d5cb6d0b1","name":"set msg.reset","rules":[{"t":"set","p":"reset","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":600,"y":180,"wires":[["9bf344f8c13cc042"]]}]
1 Like

(Just passing by and noticed this)

Ok, so you are wanting

But that means - say - you get a value (say 20) and for the next 5 minutes it has been 20.
The dashboard will be showing the 20.

You allow the latest value of 20 to go through to keep it alive.
But nothing will change and you won't see a change.

So - forgive me - but I don't understand what you are wanting to do.

I'm asking because I am kind of like you with a lot of my dashboards and this is still an ongoing problem with my stuff too.

An option is to show a heart beat signal every so often.

Also check if the connection is lost (sorry, I can't elaborate on that any more than that as I am not sure where all the things are) that the existing 20 will be replaced with either a 0 or CONNECTION LOST kind of message.
(That was a big stumbling block for me for a wile in the early days)

You could have a second trigger node which sends "No input received" or whatever after a bit longer delay than the first trigger eg 6 minutes.

There are good and bad reasons to use the RBE node.

Stopping duplicate data is one case.

If I may:

Case in point - and this is going to be a bit of a long read.
But I think it is relevant to what is happening here.

Way back when I used to do a PING to detect machines on my network.
Fixed IP addresses.

The PING would be sent out every so often and I would get either false or a value of the ping time.
If it was false the machine was either OFF or not connected.
And so if it was a value the device would be online.

So the node to detect that device would look for false and if that was the case the machine was off-line/had a problem or (with a value) was online and would adjust the indication on the dashboard.

This worked for years and I kind of became comfortable with it as it worked.
But then....... as with all good ideas: it was no longer practical.

As I added more devices I had to edit the ping list for any new devices. Not the end of the world, but was slow and painful.

nmap -sP <ip range>/mask was SO MUCH easier, as it did a whole network scan and gave me ALL connected devices.
WAY easier than the PING method.
But there was a trade off.
If a device wasn't online (or what ever) I wouldn't get a false for that IP address.

That meant I had to use something like what @jbudd suggested.
It wasn't thatcut and dry but it was something I hadn't considered and so I had to make some code to account for that.

The important thing I found I needed to consider:

Do I get BOTH a I'm alive signal as well as a I am dead signal?

If you only get an alive signal, you need to add some code to detect if the messages stop arriving.
Otherwise that isn't a problem.

The code to do this MUST go BEFORE the RBE part or you will never see the messages are not being received.

I would strongly suggest you make the timeout AT LEAST TWICE the time the messages arrive.
Unless it is CRITICAL! But then the time between messages would be much lower. But there are traps with being too tight on the timing as a missed message may set off alarms for no reason other than a missed message.

I usually set it to THREE times the message frequency before saying it has failed.

Hope that helps in some way.

No, it resends only the last message it received. dceejay's suggestion will do exactly what you have asked for.

[{"id":"bc41b91b8c855ddb","type":"inject","z":"bdd7be38.d3b55","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"1","payloadType":"num","x":230,"y":5380,"wires":[["4b52a6bc5817b27e"]]},{"id":"e2e69a86f310dd4f","type":"inject","z":"bdd7be38.d3b55","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"2","payloadType":"num","x":230,"y":5440,"wires":[["4b52a6bc5817b27e"]]},{"id":"4b52a6bc5817b27e","type":"rbe","z":"bdd7be38.d3b55","name":"Block duplicates","func":"rbe","gap":"","start":"","inout":"out","septopics":true,"property":"payload","topi":"topic","x":410,"y":5400,"wires":[["51fc458dd49e1f38"]]},{"id":"51fc458dd49e1f38","type":"trigger","z":"bdd7be38.d3b55","name":"","op1":"","op2":"0","op1type":"pay","op2type":"str","duration":"-5","extend":false,"overrideDelay":false,"units":"s","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":600,"y":5400,"wires":[["668298de4e1a94be"]]},{"id":"668298de4e1a94be","type":"debug","z":"bdd7be38.d3b55","name":"debug 2514","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":770,"y":5400,"wires":[]}]
1 Like

Thank you @Colin... I was just about to post my own example (that is exactly that... even down to the 5s for test purposes :wink: )

Debug 2514 :exploding_head:

If you're not sending the 20 any more and the value will go to 100... then you'll see a line from 20 to 100 over 6 hours. But in reality 5:59 hours the value was 20 and then went to 100...

Hey, that's a lot of reaction to my question... Thank you very much. I will test in the evening.

I personally don't like the idea of resending an old message because it falsifies the info on the dashboard.

If the sensor data stops arriving, @dceejay's solution will continue to send the last message for ever, or at least until you redeploy the flow.

My approach will only ever pass fresh messages as they arrive from the sensor.
If dots are getting added to the graph you can be confident that the sensor data is still arriving.

Of course my post responding to @Trying_to_learn, a warning when sensor data fails, is applicable to either solution.

Depends on the chart you are using and the interpolation... on Dashboard v1 - ui-chart node you can select step interpolation which will step up to your 100 value only when it arrives... so yeah... depends on your chart as well :slight_smile:

That's a good idea. Just checked it and yes, Grafana also has this options...
Thank you very much for the idea!

I imported a flow from here which bumped it up to a ridiculous value. I think node-red uses the current highest value as its base for new ones. What we need is a simple flow to renumber all the debug nodes in a flow to get it back down again. Well thats what I need anyway

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.