Nodes suggestion for timed rolling average and desynchronised sum

General
21

@Colin,

Yes I do have the machine status, so I know at all time if it's running or stopped. But I don't really see how to integrate that in the low pass filter, as stated by Seth, my intervals are variable, so I don't see how to integrate such thing in the low pass filter.
I am recording all the machine stoppages so I sum the running and stopped time of each machine over 1h, over a day ...

@seth350
So what nodes are you using for your simplified version ? I tried the smooth node, it's simple but this also did not work well with machine stops.

22

I use a node called interval. It can be downloaded from the pallet manager.

It tracks the amount of time between input signals. I have it setup so that it measures the time between a ā€œcycle completeā€ signal from the machine, it outputs the interval to a smooth node that averages the last 20 values.

I can get you specifics in the morning.

23

What do you want the displayed value to be when it is stopped?
What do you want it to display as it starts up?

24

Hi Colin,

When it stopped, the last 10min average should slowly decrease, until it reaches 0 if the machine is stopped for 10min or more. (And be at 50% after 5min)
At starts up, if it's a fresh machine start, then it starts from 0, if it's after a stop, then it should restart from whenever the average has slowly dropped.

But again, I am not looking at an exact number close to 1 UPH from the real number, but something close enough to give the real trend and be a real indicator.

25

Ah ok, I see what you want now. I have done this for machine efficiency but it was a percentage not actual UPH.

The idea is to take your actual production and divide by the target. You will need the machine target cycle time to do this.
Set it up so that your target is zero at start of shift. Then setup a timer with an interval of the machines target cycle time. Enable the timer at shift start time. Every time the timer elapses, increase the target count by one.
This simulates a perfect running machine and should be producing a part at the interval you set. You let it count up unless the operator is on break/lunch.
Then count the number of actual cycles and divide it by the target.
If the machine stops off schedule, the percentage will also decrease over time. Depending on how quickly your cycle times are.

Acutal UPH / Target UPH

As long as they are running correctly, the machine should be running pretty close to 100%.

1 Like
26

Well here is a function node that will give you a true average of all the samples over a time period. I think something will still have to be done for the stopped time, but have a look at this while I think about how to cope with that.

// determines the average of all payload values passed in 
// over the specified time range
const range = 20 * 60 * 1000;   // window time millisecs
let buffer = context.get('buffer') || [];
let total = context.get('total') || 0;   // the accumulated total so far

let now = new Date();
let value = Number(msg.payload);
// remove any samples that are too old
while (buffer[0] && buffer[0].timestamp < now - range) {
    // remove oldest sample from array and total
    node.warn(`removing oldest ${buffer[0].timestamp}`);
    total -= buffer[0].value;
    buffer.shift();
}
// add the new sample to the end
buffer.push({timestamp: now, value: value});
total += value;

context.set('buffer', buffer);
context.set('total', total);

msg.payload = total/buffer.length;
node.warn(`length: ${buffer.length}, total: ${total}, average: ${msg.payload}`);
return msg;
2 Likes
27

How about this. It will inject zeros regularly whilst the machine is stopped so the average will run down. Set the interval in the function node to the typical interval you get data from the machine when it is running, and adjust the variable stoppedString to the value you get from your machine when it is stopped. Obviously feed that in the front and pass the output on to the Average node so it gets real data when the machine is running and zeros when it is stopped.

{"id":"1c1db2e0.04f2dd","type":"debug","z":"2ae8b25a.bffaa6","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":527,"y":223,"wires":[]},{"id":"b1a6bb58.e1eda","type":"function","z":"2ae8b25a.bffaa6","name":"Zero Injector","func":"// Injects zeros while the machine is stopped\nlet interval = 10 * 1000;       // interval at which to inject\nlet stoppedString = \"Stopped\";  // expected string in payload when machine is stopped\n\nlet timerId = context.get(\"timerId\") || 0;\n// stop injection if currently active\nif (timerId) {\n    clearTimeout(timerId);\n    timerId = null;\n    context.set(\"timerId\", timerId);\n}\nif (msg.payload == stoppedString) {\n    // machine stopped so start injection\n    timerId = setInterval(function() {\n        node.send({payload: 0})\n    }, interval);\n    context.set(\"timerId\", timerId);\n}","outputs":1,"noerr":0,"x":349,"y":222,"wires":[["1c1db2e0.04f2dd"]]},{"id":"8a9acbd.f17ffb8","type":"inject","z":"2ae8b25a.bffaa6","name":"","topic":"","payload":"Running","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":141,"y":215,"wires":[["b1a6bb58.e1eda"]]},{"id":"47914bb7.661b9c","type":"inject","z":"2ae8b25a.bffaa6","name":"","topic":"","payload":"Stopped","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":262,"wires":[["b1a6bb58.e1eda"]]}]
2 Likes
28

Hi Seth,

My machines run 24/7, 363 days per year.
I do not have a proper reset opportunity, but for my production counters, I have a clock sending hourly, and daily a reset signal to some variables, so I could make use of this daily signal I guess.

Your indicator is somehow not an average, but I will consider adding that as an extra indicator in my dashboard, it's a good way to track machine performances as well.
Thanks for that :+1:

1 Like
29

@Colin :open_mouth::star_struck::open_mouth:oh man !

I was not expecting anyone to do the job for me, I was trying to get insight to find a method that will use less resources on my IOT2000 for the average since I also have many more functions to be implemented later on, and I find it to be a good habit to try to use the least resourceful method to do the job. (Call me lazy :smirk:)

Thank you so much Colin, you're too kind, I will give it a try later on today or latest tomorrow.
I love the second idea and will definitely spend time to fine tune this one, that will make a perfect combination.:+1::+1::+1:

30

@Colin
I have tested the function and it works perfectly. The inject zero part part also works brilliantly.
Again thank you so much :sunglasses:.

1 Like
31

As a matter of interest, why not just use the smooth function? Granted it does not do it over time, but number of inputs. I am using it to remove jitter from a temperature gauge.

32

@Colin Hi Colin. Came across your post with your custom 10 minute RC filter and had a question about it. I want to smooth a temperature sensor for use in a hysteresis thermostat flow. The hysteresis will be only 0.2 or 0.3C so I'd like to make sure sudden changes are smoothed out.

However, because the sensor is a zigbee device, it only sends a new value every hour or when there is a significant change. Do you have any advice how I can best implement an RC filter? My initial idea is to simply send the current temperature every minute to a filter so that I can bypass the low sensor update frequency.

33

What is it and are you using zigbee2mqtt? You may be able to adjust the interval and the reporting delta.

34

It's an Aqara zigbee temperature and humidity sensor. Connected via a Conbee II running Deconz. Ideally I would not like to speed up the update interval, for battery purposes. I don't even know if that's possible.

I'm using Node Red so I was thinking I can simply use a inject node on - for example - 1 minute timings combined with a Deconz node that gets the actual measurement and send that into the RC filter. Then the frequency is fixed and the filter should work?

35

I believe that is actually the Xiaomi WSDCGQ11LM (Xiaomi WSDCGQ11LM control via MQTT | Zigbee2MQTT) which is supported by Zigbee2MQTT.

However if you cannot use zigbee2MQTT then that doesn't help. I use Sonoff SNZB-02 which work well. I have configured mine to report every 5 mins or when the temperature changes by more than 0.15 degrees. I have several of these running for over a year now and the batteries are still going.

You would have to install zigbee2mqtt and see whether yours can be configured like that in order to see whether it would be worth the change over. In node-red use the usual MQTT nodes to pickup the data from the sensors.

I will have a look at the filter a bit later, though obviously that will only be a crude solution compared to getting better data from the sensor.

36

If you have to use a filter then insert a Trigger node in the flow, set to Send the initial message, and to Resend it every minute. Then each time you get a new value it will pass it on and then repeat that value each minute, which the RC filter will then smooth.

I don't see why this will improve your control however. What do you fear will happen if you feed the unfiltered value into the hysteresis flow?

37

I have just ordered a Sonoff SNZB-02 to test alongside my Aqara sensor. I should be able to change the max report interval and temperature change interval from Deconz. Considering that I want to run my heat pump off of this sensor, the reliability should be good. Maybe even use two sensors in case one drops out (battery or signal).

Regarding your last question, why I think I need a filter. That is a good one, I guess I'm afraid of sudden reading spikes that might quickly jump the hysteresis band. For the heat pump I intend to run it for long periods of time with a hysteresis of just 0.2C. The filter might dampen out some of the sensor jumps.

38

Do you mean a spike up then a quick return to normal?

39

Yes exactly. It should be a very slow and gradual process and I feel that such a filter might help with that. Not based on hard data ofcourse!

In the meantime I have implemented resending the measurement each minute for 1, 2 and 3 hour filters and the results look very good. The 2 and 3 hour filters do introduce quite a bit of delay so I think I will use a filter with a smaller time period and try it out.

40

Thanks Colin, I used this to create a Flow that calculates a Running time weighted average plus max and min for a user defined period. you can find it in
Generate a time Weighted Average plus Max and Min for user defined period (flow) - Node-RED (nodered.org)