Firstly, appreciate any and all help - first time to post here - I have a Chart Node that is ingesting data every ~10 seconds from an MQTT Source. The Chart Node itself is set to reset daily using the 1 Day setting and has an inject node being sent to it every day at 12:01AM in order to clear it at the beginning of the day for new points to plot.
This Chart Node is coupled with a file-out node that allows me to re-import the chart say if I was to deploy a change, this works really really well. I also have a trigger setup that saves the daily chart just before midnight hits in order to have a backup of the chart for that day, this also has been working really well, until just the past few nights, I have noticed I only have data from 8PM to Midnight - as if the chart completely cleared itself before it's scheduled time. I have a feeling this is due to the x axis "last" setting that is set for 1 day - my suspicion is that the chart is registering it's first point at an undesired time which would have the memory clear at a time that isn't intended. In the meantime as a test I have went ahead and adjusted this from the 1 day to a week, and I will observe what happens, but my fear with that is I may have just shifted the problem to happen at some other point if it is registering the empty array itself as a new point just like all the other points relayed to the chart, with the chart just clearing whenever it believes it reaches the last X-Axis threshold.
So my question is, is there a way to turn off the built in X-Axis setting that is responsible for this?
I would rather clear the Node manually at a desired time each night with the inject nodes to preserve the chart state as a standstill in time as opposed to having the chart clear itself.
Again, thank you in advance to any one with feedback. I really would like to see this feature work dependably, and for the most part, it has, up until recently where I am noticing the timing be off with the chart clearing.
Sorry for no example flow, but its really just a basic chart node leading to a file node, and another little setup of a file copy happening of that file at the end of the night, really basic.
If, as an experiment, you set it to five minutes you should see that the built in setting does not clear it every 5 minutes but starts scrolling off the left hand side instead, so you always have the last 5 minutes on screen. So with it set for 24 hours then unless you manually clear it there should always be the last 24 hours on screen.
I may be that something is crashing in a way that prevents your data being saved.
If you are running on Linux system such as a Pi then look back in /var/log/syslog to see if anything happened at the time the chart reset. Also, in a terminal, you can run top and it will tell you how long the system has been up so you should see if the machine reset for some reason.
If you are running on Windows then presumably you can see this information somewhere else.
@Colin thank you for the response on this, it's good to have the extra detail in regard to the older points just sliding off the display as opposed to clearing it.
Since having created this post I have done some extra work in figuring out the reason for the clearing of my chart and found some interesting things to hone in on that could be creating the issue.
The most helpful of troubleshooting steps I took has been to start actually logging the data itself that is being sent to the chart and sending that to a rolling file with the timestamp + data - within there I found one instance where I had the timestamp but an empty value for data... so I have dug into the other end that is actually publishing the data and put in additional checks there and also now logging the full payload of the inbound data to see what that looks like at the point of its arrival to try and determine why it would be empty in the first place.
Needless to say, the more I dug, the more I learned that this likely isn't an issue with the chart node to begin with. It's much more likely that very very infrequently I am sending something to the chart that it just can't plot, and that forces it to clear and start over.
At least, that is what I believe to be the case from the new results I have found.
Both this community and the Node-Red Subreddit have been super helpful, I just wanted to say again how appreciative I am of your insights posted here
@Colin I only send it the array to clear at the 00:01 mark for the beginning of the day, this clearing issue I am experiencing is happening at random times - as for if the node is restarting - I have on tracing for the logging and I don't see any restarts + I have both a production and development instance running and see this chart clearing behavior on both at the same times - the two instances are not mirrors of one another either, production is on a Windows Server 2019 Server and the Dev instance is on a RHEL Node - one thing I have seen though recently in the tracing log is a random:
comms.open
comms.close
I have not been able to match the timestamp of those events though to the chart clearing, so I am not sure if that is even related.
Are you aware of what the comms.open and close are eluding to? I have searched but didn't find anything meaningful yet.
No, are they node-red related and if so do they have a node id with them?
Did you say you are logging everything being sent to the chart? If so then are you logging it right at the chart so you see your daily reset in there? What do you see at the clear time or haven't you got that yet? All rather strange.
@Colin The comms.open and comms.close appear to be Node-Red related from what I can tell, a snippet here of what is seen: (This is just after the connected to broker message in the log).
Also for the second part, yes, I have as of today began logging the full raw data being passed on the MQTT Topic I am subscribing to in order to investigate the full payload. The logging I setup just the other day was for the payload I was yanking out of the raw payload using a simple change node that is using "Set" to get just the value I needed sent to the desired charts. That logging was just doing epoch + data .... and what I was seeing was what I would expect having data on each line with timestamp and the data, and then just a lonely entry with the timestamp and no data, it was comma separated so it was just:
epoch,
So yeah, since it was blank that really kind of riddles me, what I was able to see though from the output of the chart (as I also have that going to a file) was just before that point entry the full chart in the log, and the one after that point entry was just a completely new chart with only the newest points.
Really hoping that the storing of the raw message will uncover what is throwing this all off.
On the publisher side of things we already added a check just today to ensure the data being sent is of the correct length, just as another check just in case a payload didn't have every payload within it for the change node to be able to get its payload from... so I guess time will tell, it really happens at no specific time, so I am going to give it a bit and see what I see in these extra log files.
With Trace logging you will get all sorts of irrelevant stuff that only means anything if you know the underlying code.
Are you logging what you are sending to the chart? I would have thought that was the most important initially. You can do that easily using a debug node set to display in the log, then it will be sent to syslog (on linux, don't know about other systems) and the node-red log. Make sure you send all wires going to the chart to the debug node too.
@Colin - I know it's been some time since this post, but yes, I was logging the data being sent to the chart, and after even further digging, I finally found that somehow a random, unexpected was being sent to the topic I was watching for data. It wasn't until I decided to put in a log of the raw payloads that I was able to catch the random one offs to determine the issue. The logging I had setup prior was for the payloads after all of my functions had completed so it wasn't appearing as - instead it was just appearing as blank... so that really threw a stick into the spokes so to speak.
Now that I determined that I am getting empty payloads at random though I have invented a length check prior to the sending along of the message, and I have not seen this issue recur since then thankfully
Appreciate your time and thought on this though, this is an awesome community thanks to individuals like yourself.
(This is just after the connected to broker message in the log).