Calculating water flow rate from water meter

General
1

Hello,

I want to publish the current flowrate of my water meter to MQTT.
The watermeter gives a pulse every 0,5L.

I have following data from my Push Server:

timestamp=1538162426119 value=5.4418313576462
timestamp=1538162427740 value=1110.4256631709
timestamp=1538162429598 value=968.78363832078
timestamp=1538162431256 value=1085.6453558504
timestamp=1538162433150 value=950.36958817318
timestamp=1538162434838 value=1066.3507109005

The timestamp is a timestamp in milliseconds.
The value is the flowrate in Liters/hour.

When I divide the value by 60, then I get the flowrate in liters/minute, which is suitable for me.
1110,4256631709/60=18,5L/min

For validation of the data:
When I subtract two consecutive timestamps from each other, then I get the time between two pulses of the water meter in milliseconds.
1538162427740-1538162426119=1621ms

When I calculate the flow rate for this difference time I get 0.5 liters, which corresponds exactly to the characteristic of the water clock, whichs means it works correctly.
1621/1000/60/60*1110,4256631709=0,5L=correct

Now I have two problems:

  1. I need to calculate the difference time between two consecutive timestamps.

  2. When the water stops flowing, there will be no more timestamps, causing the flow rate to stop updating. So I had the idea to create the flowrate by myself, which means, when the old period between two timestamps was smaller than 2 seconds, the flowrate should be set to 0 after 3 seconds. When the old period between two timestamps was bigger than 2 seconds, but smaller than 15 seconds, the flowrate should be set to 0 after the old period plus 5 seconds. And when the old period between two timestamps was bigger than 15, the flowrate should be set to 0 after the old period plus 15 seconds.

Any Idea how to solve the problems?

Thank you and best regards,
Chris

2

OK,

Break it down into sensible chunks

  1. Calculate time difference between two timestamps - basic function node
  2. output goes into a swithc node which then evaluates your various conditions < 2 > 15 etc etc and has outputs from there for each of the cases you are interested in

Once you have all that working you could look at a DSM node to do it all - but that will give you a sore head if you start there - so get it done simply and (linearlly) first and then come back and revisit as a DSM if you desire elegance later

Craig

3

I think I've seen a contrib node that can do that...

node-red-contrib-rate 1.1.0

A node-red node to calculate the rate of change for a message property.

npm install node-red-contrib-rate

A Node Red node for calculating the rate of change of a message property.

Rates of change are calculated based on the message topic, so multiple topics can be passed through the node to calculate the individual rates of change for each topic.

Rate Period specifies the number of timestamp units the rate of change is calculated for. For example, if the timestamp is in millseconds, Rate Period of 1000 would calculate the rate of change per second.

4

Hello Steve,

that's a nice idea.
I installed the node, but currently I cannot use it, because of the way I made the Output of the node before.

Here is the function node before:

var uuidMap = {
'xxxxxxxx-1111-xxxx-xxxx-xxxxxxxxxxxx':{topic: 'Haus/Wasserverbrauch/Rate'},
};

// Parse JSON
var myJsonObj = [];
var myJsonObj = JSON.parse(msg.payload);

// Get UUID, timestamp and value
var myUuid = myJsonObj.data.uuid;
var myTimestamp = myJsonObj.data.tuples[0][0];
var myValue = myJsonObj.data.tuples[0][1];

if (uuidMap[myUuid] !== undefined) {
// Create output payload
var myOutput = {};
myOutput.topic = uuidMap[myUuid]['topic'];
myOutput.payload = "timestamp=" + (myTimestamp) + " value=" + myValue;
return myOutput;
}

And this has a Output like this:
"timestamp=1538162426119 value=5.4418313576462"
(I think it is a string)

But the rate node needs a JSON object I guess?!
This means the last couple of line of the code of the function node must be changed to create a JSON Object rather than a string.

Can you help me?

Thank you and best regards,
Chris

5

If you could get an accumulated usage count or the pulses themselves into node red I think the whole thing would be simpler and more accurate.

6

Why? Your question says that you want the flow rate which you have, every value tells you the flow rate at a specific point in time. If you aren't receiving any values, the flow rate is zero.

So what is it that you are actually trying to calculate? The total volume of water used in a given period? An averaged flow rate for a given period?

Your second "problem" stems from how you are thinking about things I suspect. If we can find out what you really want to get out at the end, we will likely find an easier way.

7

Hmm, not really a good idea.
I'm running a volkszaehler.org smart metering Installation with a database.
It has also a push server from the logging system and this gives me all data from currently nearly 70 sensorvalues (temperature, humidity, pressure, fan speed, power and water).
I'm using node-RED to Forward it to an MQTT Broker.
And then I'm using different displays and apps to have a live view on specific selected data.

Maybe I did not explain it in detail. I give it another try:
Of course I have a flowrate.
But this flowrate also exists when the flow has stopped. This is the problem.
When the water is flowing I get data every 0,5L has passed the water meter, because then it sends an impulse.
For every impulse my datalogger sends a timestamp and the flowrate which it has calculated.
But when you stop the flow, no new impulse is there and no new update of the flowrate comes.
This means the last flowrate is still valid. But of course it is not, because the flow has stopped.

But I have the last flow rate and I know by what time the next impulse would have to come.
If they do not come, the flowrate must be lower and maybe it is zero.

If I would only wait the expected time, and the hard switch to zero, then you would not get a good reading when you close the valve slowly. The value would toggle between a decreasing value and zero.

To avoid this, my idea is to wait a little bit longer than the expected time.
It would be nice, if the flowrate is then calculated by the passed time without an impulse.

Here are some data:
timestamp value dt dt value
ms l/h ms s L/message l/min
1538332311091 694,71246622926 2591 2,591 0,500 11,58
1538332314045 609,34326337170 2954 2,954 0,500 10,16
1538332316727 671,14093959732 2682 2,682 0,500 11,19
1538332319790 587,65915768854 3063 3,063 0,500 9,79
1538332323168 532,85968028419 3378 3,378 0,500 8,88
1538332329899 267,41940276333 6731 6,731 0,500 4,46

After the last data, the flowrate of 267,4194 l/hour will still exists until the next impulse.
But the next impulse will maybe come several hours later.
This means this makes no sense.

But the time is still running.
So I know the last impulselength was 6731 milliseconds.
267,4194[l/hour] / 3600000[milliseconds/hour] * 6731[milliseconds] = 0,5l
And now the other way around:
0,5[l] * 3600000[milliseconds/hour] / 6731[milliseconds] = 267,4194[l/hour]

With the milliseconds running, you can now calculate new flow rates (for example every second, or every 2 or 5 seconds).
The flowrate will decrease because no new impulse is coming.
And after a while the value will become zero.
Or we can cut it after some time or unter a specific threshold to zero.

Is it now more clearly?

Thank you and best regards,
Chris

8

This looks like an "artificial" problem. You could assume an arbitrary time to the next (=last) pulse (eg 20s) and smooth the flow rate to zero. But without a pulse, it's not a real information, just eye candy.

My system only shows the water requirement (liters per day/week etc) and tries to detect leaks / open faucets...

9

Maybe.
But I think it looks nice.
And the reason why I want to have it like described is easy: I want it :slight_smile:
And it is something where one can learn something.

10

What you can do, if no meter pulse arrives first, is to 'pretend' that a pulse has just arrived when your timer fires, so that the flow rate is the volume corresponding to one pulse divided by the time since the last true meter pulse. That gives, in fact, the maximum possible flow rate consistent with not having seen a pulse before the timer fires. On subsequent timer firings, (assuming still no meter pulses) the flow rate will decrease toward zero.

11

But without a limit, it will never reach zero...reminds me of the old windows explorer progress bar coming to 100%:slight_smile:

btw: What about the start of the measurement? Now the flow rate jumps from zero to (eg)14l/min? That would not look good...

12

I'm sure careful use of the trigger node would help here. Set it to output nothing on first input - then wait timeout for whatever period you deem necessary to indicate no flow (5 secs ?) then output 0 - with extend set on so any new inputs stop the second output happening. You may also want to adda tmestamp at that point as well before feeding it back into the main flow.

13

Hello dceejay,

Could you please set up a trigger node like you described?
I was not able to "Set it to output nothing on first input".
I'm very very new to node-RED with experience of only some few days.

Thanks and best regards,
Chris

14

Screenshot-2018-10-02-16%3A11%3A10

15

Okay, now I understand.
Thank you, I think this is a brilliant idea.
I build a sequence to test it:

[{"id":"852f9f8b.753fc8","type":"change","z":"aac14499.e7214","name":"Convert l/h to l/min","rules":[{"t":"set","p":"payload","pt":"msg","to":"(payload/60)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":370,"y":1620,"wires":[["3b130fe1.1cd2a","c885fe5e.9d9ca"]]},{"id":"2978a318.54ae3c","type":"inject","z":"aac14499.e7214","name":"","topic":"","payload":"2564.12554788252","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":1540,"wires":[["852f9f8b.753fc8"]]},{"id":"1542bbc4.5f6d5c","type":"inject","z":"aac14499.e7214","name":"","topic":"","payload":"1641.1554525145","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":1580,"wires":[["852f9f8b.753fc8"]]},{"id":"15ec6ebd.7428b9","type":"inject","z":"aac14499.e7214","name":"","topic":"","payload":"636.71736823488","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":1620,"wires":[["852f9f8b.753fc8"]]},{"id":"ecbab304.018918","type":"inject","z":"aac14499.e7214","name":"","topic":"","payload":"241.569841254452","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":1660,"wires":[["852f9f8b.753fc8"]]},{"id":"35de990.e6e9668","type":"inject","z":"aac14499.e7214","name":"","topic":"","payload":"53.256478125564","payloadType":"num","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":130,"y":1700,"wires":[["852f9f8b.753fc8"]]},{"id":"3b130fe1.1cd2a","type":"debug","z":"aac14499.e7214","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":570,"y":1540,"wires":[]},{"id":"c885fe5e.9d9ca","type":"switch","z":"aac14499.e7214","name":"If flowrate is","property":"payload","propertyType":"msg","rules":[{"t":"gte","v":"30","vt":"num"},{"t":"btwn","v":"15","vt":"num","v2":"30","v2t":"num"},{"t":"btwn","v":"5","vt":"num","v2":"15","v2t":"num"},{"t":"btwn","v":"3","vt":"num","v2":"5","v2t":"num"},{"t":"btwn","v":"2","vt":"num","v2":"3","v2t":"num"},{"t":"btwn","v":"1","vt":"num","v2":"2","v2t":"num"},{"t":"lte","v":"1","vt":"str"}],"checkall":"true","repair":false,"outputs":7,"x":570,"y":1620,"wires":[["15a2cf0d.142c01"],["e59ed7e4.d3c218"],["96c13357.dbaed8"],["9025ed60.a78868"],["ded45f29.9186b8"],["f707cd0.01e9a3"],["76a9065c.f2bf3"]]},{"id":"15a2cf0d.142c01","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"2","extend":true,"units":"s","reset":"","bytopic":"all","name":">=30 then send payload, wait 2 seconds for timer reset, otherwise send \"0\"","x":970,"y":1500,"wires":[["86fab700.28f94"]]},{"id":"e59ed7e4.d3c218","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"4","extend":true,"units":"s","reset":"","bytopic":"all","name":"15-30 then send payload, wait 4 seconds for timer reset, otherwise send \"0\"","x":970,"y":1540,"wires":[["86fab700.28f94"]]},{"id":"96c13357.dbaed8","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"8","extend":true,"units":"s","reset":"","bytopic":"all","name":"5-15 then send payload, wait 8 seconds for timer reset, otherwise send \"0\"","x":970,"y":1580,"wires":[["86fab700.28f94"]]},{"id":"86fab700.28f94","type":"debug","z":"aac14499.e7214","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1370,"y":1620,"wires":[]},{"id":"9025ed60.a78868","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"12","extend":true,"units":"s","reset":"","bytopic":"all","name":"3-5 then send payload, wait 12 seconds for timer reset, otherwise send \"0\"","x":970,"y":1620,"wires":[["86fab700.28f94"]]},{"id":"ded45f29.9186b8","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"17","extend":true,"units":"s","reset":"","bytopic":"all","name":"2-3 then send payload, wait 17 seconds for timer reset, otherwise send \"0\"","x":970,"y":1660,"wires":[["86fab700.28f94"]]},{"id":"f707cd0.01e9a3","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"32","extend":true,"units":"s","reset":"","bytopic":"all","name":"1-2 then send payload, wait 32 seconds for timer reset, otherwise send \"0\"","x":970,"y":1700,"wires":[["86fab700.28f94"]]},{"id":"76a9065c.f2bf3","type":"trigger","z":"aac14499.e7214","op1":"","op2":"0","op1type":"pay","op2type":"num","duration":"60","extend":true,"units":"s","reset":"","bytopic":"all","name":"<=1 then send payload, wait 60 seconds for timer reset, otherwise send \"0\"","x":970,"y":1740,"wires":[["86fab700.28f94"]]},{"id":"15439a28.dcea5e","type":"comment","z":"aac14499.e7214","name":"Setting Flowrate to Zero when no new impulse comes","info":"","x":240,"y":1500,"wires":[]}]

It seems to work, but I found one issue.
If one clicks on different inject nodes without waiting for another timer has finished, then it sends Zeros where in reality there is no zero flowrate.
This happens in reality when the flowrate is changing.
Any idea to block or bypass this behaviour?

Thanks and best regards,
Chris

16

I'm guessing this is when you move from one rate to another so pulses stop arriving at the "old" output and then it times out and send a zero. Must admit I've not really got my head round the logic of all the switching you do... any reason why the trigger can't just be on the bottom flow (ie > 60s) ?

17

Yes, you are right, it occurs when the rate changes.
Of course it is possible to just use one Trigger. But then it is always the slowest Option. By seperating it to different timers the value will be updated as fast as possible.

18

Hi Chris,

I can see that this discussion has advanced a bit, but let me share my solution to a similar issue. I am monitoring my ESP devices that send out MQTT message every given seconds. I wanted to tell if a device appears to have disconnected from the networks (dies, turned off, lost wifi, etc.).

I redirect the incoming MQTT information to the below function node:

var devicename = "yourdevice";

var temp = global.get(devicename+"_wifi");
var current = new Date();
if (temp!== undefined && temp!==null) {
msg.payload = current.getTime() - temp;
global.set(devicename+"_wifi",current.getTime());
} else {
msg.payload = "";
global.set(devicename+"_wifi",current.getTime());
}

// Update the status with current timestamp
var now = new Date();
var yyyy = now.getFullYear();
var mm = now.getMonth() < 9 ? "0" + (now.getMonth() + 1) : (now.getMonth() + 1); // getMonth() is zero-based
var dd = now.getDate() < 10 ? "0" + now.getDate() : now.getDate();
var hh = now.getHours() < 10 ? "0" + now.getHours() : now.getHours();
var mmm = now.getMinutes() < 10 ? "0" + now.getMinutes() : now.getMinutes();
var ss = now.getSeconds() < 10 ? "0" + now.getSeconds() : now.getSeconds();
node.status({fill:"blue",shape:"ring",text:"Last update: "+dd + "." + mm + "." + yyyy + " " + hh + ":" + mmm + ":" + ss});

msg.formattedtime = dd + "." + mm + "." + yyyy + " " + hh + ":" + mmm + ":" + ss;

return msg;

This saves the timestamp of when the last message was received in global.

Next I create an inject, which triggers every second. This goes into a second function node:

var devicename = "yourdevice";

var temp = global.get(devicename+"_wifi");
var current = new Date();
msg.payload = "No data";
msg.warning = false;

if (temp!==undefined) {
current = current - temp;
current = Math.floor(current/1000);
var minute = Math.floor(current/60);
var hour = Math.floor(minute/60);
var day = Math.floor(hour/24);
if (current>246060) {
msg.payload = "Last update " + day + " days, " + hour%24 + " hours, " + minute%60 + " minutes, " + current%60 + " seconds ago";
} else if (current>60*60) {
msg.payload = "Last update " + hour%24 + " hours, " + minute%60 + " minutes, " + current%60 + " seconds ago";
} else if (current>60) {
msg.payload = "Last update " + minute%60 + " minutes, " + current%60 + " seconds ago";
} else {
msg.payload = "Last update " + current%60 + " seconds ago";
}

    if (current>300) {
        // this means we did not get a message for 300 seconds. 
    }

}

node.status({fill:"blue",shape:"ring",text:msg.payload});

return msg;

Here I am calculating the number of second passed since the last update. If this is above a certain threshold (for me it is 300 seconds) you can do your own logic. For example generate a message with zero flow and return that in msg which looks like the data sent by your sensor.

I hope you get the idea. You can even combine the two code into one function node, so it is normally getting the MQTT data and also once every second from the inject node. You can specify a unique topic in the inject node, so you will know that the msg you are receiving in from your sensor (store timestamp and pass the message along), or from the 1 second inject (calculate elapsed time since last reading and generate a zero reading).

Hope you can use the above concept.

Cheers,
Csongor

19

Hello Csongor,

Thanks a lot for your contribution.
I appreciate that very much and have tried to understand what you have written, but I have to say that I am a beginner and that I do not understand exactly and I could not apply it to my application. Maybe you can put a complete example as sequence or flow here, so that I can understand it better?

Thank you and best regards,
Chris

20

If you want to monitor somethingand put checks on it, both check if data is changing and if it is arriving at all.

I have several devices sending data: water, solar, power and heatpump.
For instance a webcam looking at the mechanical water-meter on a raspberry, another converting rs485 and modbus to mqtt
I both monitor if I get data regularly (i.e. the monitor system is working) and the data is changing (i.,e. the webcam is aimed correctly) I should get data every 60 seconds, if more than 180 seconds, the system is down.
If no water have been used for 10 hours, something is wrong with the camera.