@Colin as discussed in another thread, I would be grateful if you could advise me how best to tune the node-red-contrib-pid node.
My settings are;
Proportional band = 5
Integral time = 3600
Derivative time = 1800
Initial integral = 0
Max sample interval = 600
Derivative smoothing factor = 4
The PID is monitoring soil humidity, and it's output is scaled & operates a water pump which proportionally hydrates the soil. The aim is to keep it near to 62%, and I've attached a screenshot below, which shows the setpoint of 62% (yellow), the soil humidity (red) and the scaled PID output is the bottom (orange).
The PID output is scaled 1:266, so the maximum output is 266ml.
What I've noticed is that even when the setpoint has been reached, a drop in the humidity can produce a disproportionate effect. For example, see about 5.30am, where the humidity sensor recorded a drop of about 1.5% and responded with over 220mls of water, despite the humidity still being in excess of 1% above the setpoint.
This resulted in the humidity rising even further away from the setpoint.
It's unfortunate that the sensor readings deviate as they do, but to date I've used a smooth node, to smooth out the deviations, and it worked pretty good - except that there was a delay in response.
I did start tuning using your tuning blog post, but I've probably made umpteen changes since...
The injections of output when the value is above teh setpoint are caused by the Derivative term. Even in theory the derivative should not be more than 1/4 of the Integral so you have much too much (I think you might have misread the tuning blog about derivative), but with such a noisy sensor derivative is never going to work. Also most applications don't need it anyway. Set it to zero and see how it goes then.
Since you don't mention where the soil is that you are reading (i.e. indoors in a climate controled location or outdoors under the open sky) I have to ask the question - could the humidity in the air be effecting your readings? Humidity can change dramatically when a front blows threw.
Note: I know nothing about PID but the question poped into my head.
Also, your soil humidity sensor seems to be a bit too sensitive I think. And it is behaving a bit strange, as example around 22:00, you can see in the chart that the humidity suddenly drops and then increases even if no water has been added. I think you need to smooth the readings during a longer period.
How did you smooth the signal? I have a similar situation but for a light intensity sensor where I read the values every minute but have 120 readings set in the Smooth node. That gives a nice smoothed curve. If I have 12 readings, it looks a bit more like yours.
For the humidity, does it change that fast that you couldn't try to add more readings to the Smooth node to make the curve smoother?
Before trying to smooth it lets try to understand what is going on. Looking at the chart it seems you are only polling the sensor about every 10 minutes or so. Is that correct? What does the chart look like if you poll it once a minute?
The sensor is fitted inside a greenhouse, and monitoring humidity inside a grow bag.
The temperature does affect the amount of water take up, and I suppose humidity in the greenhouse will affect the rate which plants lose moisture.
It's probably the best that you can hope for from Capacitive humidity sensors, but they are at least more consistent than 'resistance' moisture sensors.
I have used a smooth node to average out the input until now, but want to see what can be achieved without smoothing.
Yes, that's correct. The ESP is hardcoded to take a humidity reading every 10 minutes, pass that reading via MQTT to node-red, which calculates how much water is needed. Node-red, again via MQTT then tells the ESP to run the pump for, say 60 seconds.
The above sequence takes a matter of milliseconds.
The ESP knows to stop the pump after that time is expired.
The ESP also has a fallback mode to run autonomously if communication fails with node-red.
I did trial different intervals, but found 10 minutes was probably optimum, because I need to allow time for the dispensed water to soak through the soil, before the next reading is obtained.
If I ran the sequence every minute, it would flood (or should I say "it DID flood!")
That's looking much better Colin. There is a slight overshoot after the humidity moves above the setpoint, but it seems tighter controlled than it was previously.
Proportional band = 5
Integral time = 3600
Derivative time = 0
Initial integral = 0
Max sample interval = 600
Derivative smoothing factor = 4
Ok. The way the pid works is that the output is supposed to be a rate of supply of energy (or in your case water) to the process. Because your system actually applies a determined amount of water each time then if you ran it at one minute then each injection would have to be 1/10 of the amount you are currently using for a given PID output. So at the end of 10 mins the total injected would be the same as you have at the moment. But I suppose it should work as you have it. However, you need to increase Max Sample Interval so it is larger than the expected interval. As it is you have it the same as the actual interval so it may think the max has been infringed. Set it to 1200 so it will only be hit in failure conditions.
The thing that worries me about your graph is that it was injecting water at a fair rate for four hours and it didn't seem to make any difference to the moisture level, then the level went up. Do you understand that?
I see you have the setpoint at 2, what range of moisture would you consider to be acceptable control?
It makes sense actually, because of the sunny & high temperatures we are currently having, and this is in a greenhouse, the plants are drinking more water as moisture evaporates from their leaves etc. It's known as 'transpiration'.
Plotting temperature against the water injection rate reinforces this.
I could easily scale up the amount of water delivered, but even under these extreme weather conditions the PID is only letting the humidity fall about 1% (and even then it's only administrating 100ml out of a PID range of 266ml).
The moisture readings between 'too dry' and 'too wet' is fairly narrow, so I was hoping for 1.5% to 2% up or down ideally. Got to say though, this is more about learning node-red, C++ and project design/build than a mission critical build.
I could probably buy tomatoes, cuc's & chilies cheaper than what I spend on growing them!!!
If you assume the noise on the signal is noise rather than actual moisture level then you are already achieving that.
Back on the sample interval, could you get sensor readings every minute, but only act on every tenth one? That way you could determine whether the noise is actually noise or is real. If it proves to be noise then filtering the signal might then work.
I assume that you mean take an average of the 10 x 1 minute readings, and use that to determine the amount of water needed?
If so, the problem that I see with that approach is that if it takes approx 10 minutes from when irrigation takes place, to it soaking through the soil, and influencing the moisture sensor. So that means that most of the 10 readings obtained will not have been influenced by the previous irrigation session 10 minutes earlier.
I'm assuming therefore that the result would be that 'the average' would be falsely drier than the 'actual moisture level'.
No, not initially, for the moment I meant to plot the 1 minute readings but only use the tenth one. That way the control will be as it is at the moment but the chart will tell us more about what is going on with the sensor. That should help to determine the best way to improve the situation.
Setting the Max Sample Interval to 1200 has further improved the look of the chart, as it's rising and falling in a smoother manner now. (below screenshot).
However, it is still delivering water for a while after the moisture level cuts through the setpoint (about 14:20), which I didn't expect.
I've now also changed the readings to every minute, and just using the tenth reading to feed into the PID. Once I get some data, I'll post a screenshot of the results.
Proportional band = 5
Integral time = 3600
Derivative time = 0
Initial integral = 0
Max sample interval = 1200
Derivative smoothing factor = 4
I think you have the integral at too low a value (ie too powerful). All the time the process is too low then the integral pushes the injection up, the result it overshoots. Looking at the chart it looks as if it has a period of many hours, though I suspect that is not actually an oscillation but is as a result of external factors. For the moment at least I suggest pushing the integral up to a high value until we understand more about how the process works. I suggest set it 10000 for the moment.
The integral is now set at 10000, and I'm displaying moisture levels at 1 minute intervals, although the PID is only receiving the 10th reading.
I've also changed the graph scale so that the individual data points can be clearly seen.
It's interesting that the 'spikes' are mainly dropping in value, whilst the peak chart values seem a lot more consistent.
Proportional band = 5
Integral time = 10000
Derivative time = 0
Initial integral = 0
Max sample interval = 1200
Derivative smoothing factor = 4
I don't think it is the spikes keeping it up there. The pid algorithm is doing its best to bring the process down, by not putting any water in, but the value isn't dropping, so not much it can do really. However in a graph you posted on 7th it ran for hours with injection rate at around 70% before the value started to rise. All due to the current environment I assume. It might be worth bringing the prop band down to 2 and see what happens.
That data wasn't in the graph posted earlier. The chart doesn't make sense to me. Are you sure you have set the derivative to 0? Those look like derivative induced signals. Since something odd seems to be going on it might be best to go back to just charting the values going into the PID so we can be sure of what we are looking at.
