I've been working on a visualization project for a hydronic heating system and ran into a few glitches with charts. I'm posting temperature data via an MQTT broker, to a 3 line chart, with updates every 5 seconds. The chart is configured for an 8 hour span. Node-red, the MQTT broker, and a python script to read PLC run on a rpi 3-b. The node-red development console and dashboard are on my laptop. All seemed to be working fine, until several hours passed and browser (I was using Firefox), started to act up. CPU usage on the rpi was negligible, Yet my laptop was starting to warm up, with task manger reporting 80+ % GPU rendering peaks on 5 second intervals. I changing the PLC polling to a 10s period, which was better, but was now hanging up after 2 hours instead of 1. Eventually the charts would not load on Firefox. I tried Chrome, Chromium and Edge, they worked much better than Firefox, but the CPU usage was still a problem.
Since the graphs were not reaching the 8 hour span as configured, realized that they must have reached the 1000 point limit and the incoming data would need to be processed to eliminate redundant points. I did this by comparing the value of a new incoming point with that of the last point posted to the chart, the difference between the new point and the previous point would have to be greater than a set value for it to be posted. This scheme has solved my problems with CPU/GPU usage and the chart spans the full 8 hour period.
Here's code for the function block node upstream of the chart:
// Node-Red charts require a separate message for each series
var msg1 = {};
var msg2 = {};
var msg3 = {};
const TEMP_DIF = 0.25;
const TEMP_DIF_SR = 0.1
// tempout_mem will hold a copy of 'nLoopTempOut'/100
// initialise tempout_mem to 0 if it doesn't exist already
var tempout_mem = context.get('tempout_mem')||0;
var tempret_mem = context.get('tempret_mem')||0;
var tempset_mem = context.get('tempset_mem')||0;
var count = context.get('count')||0;
count += 1;
// read loop out and divide by 100 to get deg C
var temp_out = msg.payload[0].Heat_Loop.nLoopTempOut / 100.0;
var temp_ret = msg.payload[0].Heat_Loop.nLoopTempReturn / 100.0;
var temp_set = msg.payload[0].Heat_Loop.nLoopTempSetpoint / 100.0;
// calculate difference between this and the last reading
var tempout_dif = Math.abs(temp_out - tempout_mem);
var tempret_dif = Math.abs(temp_ret - tempret_mem);
var tempset_dif = Math.abs(temp_set - tempset_mem)
// post points to chart only if there is a temperature change of 0.25 C
// since the last point posted
var post_tempout = (tempout_dif >= TEMP_DIF);
var post_tempret = (tempret_dif >= TEMP_DIF_SR );
var post_tempset = (tempset_dif >= TEMP_DIF_SR );
if (count == 1){
post_tempout = true;
post_tempret = true;
post_tempset = true;
}
// Ensure setpoint is occasionaly updated
if (count > 120) {
post_tempout = true;
post_tempret = true;
post_tempset = true;
count = 2;
}
if (post_tempout){
msg2.payload = temp_out;
msg2.topic = "Output";
// store the loop output temperature to context memory
context.set('tempout_mem',temp_out);
}
if (post_tempset){
msg1.payload = temp_set;
msg1.topic = "Setpoint";
// store the loop output temperature to context memory
context.set('tempset_mem',temp_set);
}
if (post_tempret){
msg3.payload = temp_ret
msg3.topic = "Return";
context.set('tempret_mem',temp_ret);
}
context.set('count',count);
return [msg3, msg2, msg1];
Updating a chart every 5sec is 12 points per minute. 720 points per hour, 5760 points over 8h. I doubt there were enough pixels on your monitor to even display this amount of data points so completely OTT.
This is something often overlooked. Put a chart on dashboard that is roughly 400px wide then hammer it with data and wonder why the dashboard is slow.
Blockquote
Updating a chart every 5sec is 12 points per minute. 720 points per hour, 5760 points over 8h. I doubt there were enough pixels on your monitor to even display this amount of data points so completely OTT.
Yes, for sure. However I thought that the implied 1000 point limit (in the chart setup) would avoid this issue.
I was also under this delusion.
My Node-RED Dashboard has been crashing and restarting for a few months now, took me too long to track it down to some 1 hour graphs I had scattered over a few pages (6 of them).
Changed them all (reluctantly) to 15 minutes and the dash no longer disconnect/reconnects 4-5 times a minute like it was.
I am not sure how to use your function block code in my case, but yeah, clearly, setting the number of points in the graph node is very unclear to me how it works in conjunction with the data moving through the flow.
Hi @thebaldgeek,
The first thing I would look at is the rate that you update your charts; the number of points is directly related to the update rate. In my case a MQTT is triggering an update to the filter function node, which in turn updates the chart series only if the point data has changed signicantly. If you are triggering from a comm event, such as MQTT and can't, or don't want to, reduce the comm update rate then consider saving the comm data to global context memory. Then you can use inject nodes, on which you can configure the trigger rate, to post data to your charts
Presently I'm not having any hangup issues running node-red on a rpi-3b and the dashboard on a Chromium browser under Ubuntu 20.04 on a mini-itx AMD 5350. If your dashboard is on an rpi model < 4 you can expect problems.
Here's a snap of my current flow:
The function 'Chart Series' has code per my original posting .
Most of the activty happens when a new MQTT message arrives and this is not an ideal situation. I'll be changing this so that the MQTT data is posted to global context memory on message arrival and use timed inject nodes to trigger the update of charts and gauges etc in a sequention fashion. This should significantly reduce CPU loading.
Thanks for the reply.
I am now even more confused (grin).
I did not change the rate the messages are fed to the 6 graphs, that's rather hard for me to do since the data comes from aircraft flying around. Sometimes they fly often, sometimes not.
I just changed the duration of the graphs from 1 hour to 15 minutes.
Running Node-RED on a 12 core i7 with 32gb RAM and M2 SSD so should not have any issues on that aspect.
I think but I haven't fully tested it, that if you enter a value for max points then it will work as expected. If you leave it empty then it doesn't impose a limit, even though it shows what might be assumed is a default value of 1000.
I think but I haven't fully tested it, that if you enter a value for max points then it will work as expected. If you leave it empty then it doesn't impose a limit, even though it shows what might be assumed is a default value of 1000.
It appears to me that the maximum point count refers to the sum of the points from every series. Using the default setting of 1000 my graph was truncating to 5 0or 6 hours, after setting it to 2000 it covers a full hours and has been working fine for over a week now. Thanks to the point change filtering.
Here's the setup:
Well I am not seeing that. I have configured a chart to update every 5 secs and with a point limit of 24 it stops at 2 minutes, whether I have 1 or 2 lines, it makes no difference.
If you feed the output from the chart into a debug node you can see how many points there are in each series.
