Also is there something wrong with the sensor or measuring circuit? Should there really be that much noise on it? looking at the chart there is over 1.5 units peak to peak of noise, with that amount of noise a PB of 2 will be too tight. If you are convinced that the noise is inherent in the system and cannot be improved then perhaps you do need a filter. Feed the 1 sample/minute values into a function node containing the code below, which will give you a 5 minute RC filter and then every tenth sample feed that into the PID. Put the filtered value on the chart. That should smooth out some of the noise.
// Applies an RC low pass filter to incoming payload values
var tc = 300*1000; // time constant in milliseconds
var lastValue = context.get('lastValue');
if (typeof lastValue == "undefined") lastValue = msg.payload;
var lastTime = context.get('lastTime') || null;
var now = new Date();
var currentValue = msg.payload;
if (lastTime === null) {
// first time through
newValue = currentValue;
} else {
var dt = now - lastTime;
var newValue;
if (dt > 0) {
var dtotc = dt / tc;
newValue = lastValue * (1 - dtotc) + currentValue * dtotc;
} else {
// no time has elapsed leave output the same as last time
newValue = lastValue;
}
}
context.set('lastValue', newValue);
context.set('lastTime', now);
msg.payload = newValue;
return msg;
No, I've tried 2 different capacitive sensors and they are both have similar levels of noise. The measuring circuit consists of a direct connection into the ADC, so not a lot to create problems.
I've added your 5 minute RC filter, and increased the PB back up to 4, as well as enlarging the chart data points, so it's now easier to compare the Moisture & PD chart (3 x 10 minute datapoints per 30 minute grid line.)
Today has been a really good test for the PID, because it's been sunny all day, 30 deg C outdoor temp, and the greenhouse soil temp at one stage went up to 39 deg C - So extreme conditions!
At the moment, a PID of 1, is scaled up to 320 ml of water, and I'm wondering if I need to increase that amount to help flatten the dip in the chart, as more water would be proportionally delivered earlier.
As can be seen, the maximum delivered today was 230 ml (at 12.15pm). Or can the PID become a little more responsive, and rise to it's maximum value earlier?
Either way, a good result today - the PID did well!
Proportional band = 4
Integral time = 10000
Derivative time = 0
Initial integral = 0
Max sample interval = 1200
Derivative smoothing factor = 4
As I suggested earlier, smooth the moisture measuring to get a better averaged curve, the spikes earlier made everything "too nervous", now it looks good and correctly responsive
Changing the PB to 2 certainly made a difference, and although the temperatures were not as extreme today, the PID delivered greater amounts of water at an earlier stage, ensuring that the moisture level only dropped by 0.6% below the setpoint, and also only exceeded the setpoint by 0.5% all day.
Proportional band = 2
Integral time = 10000
Derivative time = 0
Initial integral = 0
Max sample interval = 1200
Derivative smoothing factor = 4
No environmental issues, I'm assuming it's as a result of 2 things;
The water enters the growing medium at a point 350mm from the sensor and takes time to soak through and influence the sensor. After 10 minutes, maybe the water hasn't fully influenced the sensor, and the backlog arrives later.
Also, after the moisture level passed through the setpoint at 12:47, a further 330ml of water is delivered before the moisture level again falls below the setpoint at 16:00, keeping it high.
In that case the PB is now too narrow, as it is putting in too much water so it oscillates up and down as you see. I think you will have to put it back up to three or four to stop the oscillation. However leave it as it is for the moment for another day, to see if you see the same thing again. It is very difficult to tune it in the environment you have. It would be much better if it were possible to run it in an approximately constant environment in order to separate the what the control loop is doing to the moisture from what the environment is doing to the moisture. I realise that may not be possible however.
OK, but looking back at the above screenshots when the PB was 4, I still had that oscillation there too.
In perspective, it is performing very well! despite the hostile operating conditions, and better than I expected.
As an aside... the consistent soil humidity has resulted in a enormous crop, the best so far, so well worth the effort and your support. Thanks.
If you compare the injection graph of the two you will see that with PB=4 the rate increased and then died away more or less exponentially. With PB=4 it increased, went back to nothing, and increased again, making a damped sine wave shape, not a gently dropping exponential.
However, I have just noticed that you have dramatically changed the vertical scale on the second one which makes it difficult to properly compare them. Can you put the same scale in as the first one (on both variable) and post it again?
I wonder whether it would be possible to improve the control by mixing in a bit of ambient temperature to the loop in order to get a bit of advance warning of the fact that water will be required when it gets hot. It might be useful to include the temperature on the chart so we can consider that possibility.
It is worth noting that the reason that it is possible in this case to benefit from smoothing is that the noise on the signal is true noise, not a variation in moisture level. This was shown by increasing the sensor sample interval by ten times and the noise level stayed approximately the same. Thus it was possible to filter the much faster samples without adversely impacting the loop control.
@Paul-Reed I am still interested in why the noise level is so high. Which sensor are you using and how are you interfacing it to the pi? How long is the cable to the sensor?
The three leads are ground, +5V and analogue out, which is directly fed into the ESP's ADC.
The leads are less than 1 metre.
The scale is set dynamically by the chart node, however I can fix the scale (60 - 64) if you would prefer. The charts are a rolling 12 hrs, so I can't change it retrospectively, but I'll post again with a new chart & fixed scale.
If I add temperature to the moisture chart, the scaling would be such that you wouldn't be able to distinguish the moisture levels, but I am plotting temperature which I can add to the screenshot.
I thought the charts looked as if they come from Grafana, in which case you can scroll and zoom to get whatever time range you want, in particular you can change the scaling and replot the old data. Also if using Grafana then you can put the temperature on the other Y axis.
Have you tried connecting the ADC to a battery to check that the ADC is not noisy?
Assuming we normally are around 62 % RH the "spikes" are on average just "jumping" around 0.2 to 0.3%. That isn't much I think (0.3-0.4% of the reading), it is most likely normal for such sensors with it's tolerans. I think, to overcome, even more smoothing would be needed. To avoid lagging behind, maybe the smoothing should not be a standard averaging (simple moving average) but one with weighting where later readings are given more weight (an exponential moving average it is called).
Also another thing came to my mind; we are assuming one single set point (62%). What if we instead could have two, like an upper/lower limit? Having 62.5 as upper and 61.5 as lower as example. When the moisture level drops and crosses the upper from above, we could already start adding water and when it raises and crosses the lower from below, watering is stopped. I was just thinking about this since the "moisturizing" process is rather slow
@Paul-Reed I had assumed that the chart was showing the full range of the sensor. Is the full range 0 to 100 and you are seeing noise of +-1 in which case that is not bad?
