Chart only showing recent data, not specified timeline

General
1

I'm trying to track boiler temps and fuel consumption in 2 separate charts.
The charts are receiving data correctly, but only showing last 12 hours of data.
1 chart is spec'd for 3 weeks, the other for 10 weeks.

The flow has been running untouched for a week now.

Any ideas?

image
image1301Ă—616 55.4 KB

image

The flow itself:

[
    {
        "id": "cf99d9b968649254",
        "type": "tab",
        "label": "Flow 2",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "96c57bbad94268b7",
        "type": "ds18b20",
        "z": "cf99d9b968649254",
        "name": "",
        "sensorid": "28-3c01f09679bc",
        "timer": ".2",
        "x": 460,
        "y": 520,
        "wires": [
            [
                "97b12ed9628a1fb1"
            ]
        ]
    },
    {
        "id": "3523cce606609b76",
        "type": "ds18b20",
        "z": "cf99d9b968649254",
        "name": "",
        "sensorid": "28-3c01f0960ce5",
        "timer": ".2",
        "x": 460,
        "y": 480,
        "wires": [
            [
                "881f5d1eaa471f36"
            ]
        ]
    },
    {
        "id": "67edeec7228c1c24",
        "type": "rpi-srf",
        "z": "cf99d9b968649254",
        "name": "",
        "topic": "OilTank",
        "pulse": "1",
        "pins": "8,10",
        "precision": "0",
        "x": 430,
        "y": 180,
        "wires": [
            [
                "812f34c53baf6475",
                "832790bb42c36356"
            ]
        ]
    },
    {
        "id": "8dd0db19e3148f53",
        "type": "rpi-gpio out",
        "z": "cf99d9b968649254",
        "name": "Temp Sense pwr",
        "pin": "18",
        "set": "",
        "level": "0",
        "freq": "",
        "out": "out",
        "bcm": true,
        "x": 1190,
        "y": 400,
        "wires": []
    },
    {
        "id": "8e8d0b02cb4b3ba0",
        "type": "inject",
        "z": "cf99d9b968649254",
        "name": "",
        "props": [
            {
                "p": "payload"
            }
        ],
        "repeat": "600",
        "crontab": "",
        "once": true,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "0",
        "payloadType": "num",
        "x": 470,
        "y": 400,
        "wires": [
            [
                "0990bc8ed006147e"
            ]
        ]
    },
    {
        "id": "838179a3c1938dc7",
        "type": "inject",
        "z": "cf99d9b968649254",
        "name": "",
        "props": [
            {
                "p": "payload"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "1",
        "payloadType": "num",
        "x": 770,
        "y": 360,
        "wires": [
            [
                "8dd0db19e3148f53"
            ]
        ]
    },
    {
        "id": "0990bc8ed006147e",
        "type": "trigger",
        "z": "cf99d9b968649254",
        "name": "auto reset temp probe",
        "op1": "0",
        "op2": "1",
        "op1type": "num",
        "op2type": "num",
        "duration": "5",
        "extend": false,
        "overrideDelay": false,
        "units": "s",
        "reset": "",
        "bytopic": "all",
        "topic": "topic",
        "outputs": 1,
        "x": 720,
        "y": 400,
        "wires": [
            [
                "8dd0db19e3148f53"
            ]
        ]
    },
    {
        "id": "cd06a98df0430901",
        "type": "debug",
        "z": "cf99d9b968649254",
        "name": "debug 42",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 1320,
        "y": 540,
        "wires": []
    },
    {
        "id": "bc8ddda9f64b7cea",
        "type": "debug",
        "z": "cf99d9b968649254",
        "name": "debug 43",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 1320,
        "y": 580,
        "wires": []
    },
    {
        "id": "881f5d1eaa471f36",
        "type": "change",
        "z": "cf99d9b968649254",
        "name": "Return",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "return",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 630,
        "y": 480,
        "wires": [
            [
                "e43f012806d6aa38"
            ]
        ]
    },
    {
        "id": "97b12ed9628a1fb1",
        "type": "change",
        "z": "cf99d9b968649254",
        "name": "Supply",
        "rules": [
            {
                "t": "set",
                "p": "topic",
                "pt": "msg",
                "to": "supply",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 630,
        "y": 520,
        "wires": [
            [
                "058ce02babd8fb2e"
            ]
        ]
    },
    {
        "id": "f28bde6391184792",
        "type": "function",
        "z": "cf99d9b968649254",
        "name": "function 8",
        "func": "\nvar rawTemp = msg.payload;\nvar convertedTemp = rawTemp * 9 / 5 + 32;\n\nmsg.payload = convertedTemp;\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 960,
        "y": 500,
        "wires": [
            [
                "2c34d8dcaaf3a5f3"
            ]
        ]
    },
    {
        "id": "2c34d8dcaaf3a5f3",
        "type": "switch",
        "z": "cf99d9b968649254",
        "name": "",
        "property": "topic",
        "propertyType": "msg",
        "rules": [
            {
                "t": "eq",
                "v": "supply",
                "vt": "str"
            },
            {
                "t": "eq",
                "v": "return",
                "vt": "str"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 2,
        "x": 1110,
        "y": 500,
        "wires": [
            [
                "cd06a98df0430901",
                "956780e0223bb069"
            ],
            [
                "bc8ddda9f64b7cea",
                "956780e0223bb069"
            ]
        ]
    },
    {
        "id": "812f34c53baf6475",
        "type": "debug",
        "z": "cf99d9b968649254",
        "name": "debug 44",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 800,
        "y": 120,
        "wires": []
    },
    {
        "id": "fed3b12a99eff03f",
        "type": "function",
        "z": "cf99d9b968649254",
        "name": "Convert to inch",
        "func": "var cm = Number(msg.payload);\n\nvar toInch = cm / 2.54;\n\nmsg.payload = Math.round((10 * toInch) / 10);\n\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 900,
        "y": 180,
        "wires": [
            [
                "1c0a59e9ce0603a6",
                "f66a0cf108042a5e"
            ]
        ]
    },
    {
        "id": "1c0a59e9ce0603a6",
        "type": "debug",
        "z": "cf99d9b968649254",
        "name": "debug 45",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 1080,
        "y": 120,
        "wires": []
    },
    {
        "id": "fd0517cdfc3d1d33",
        "type": "smooth",
        "z": "cf99d9b968649254",
        "name": "",
        "property": "payload",
        "action": "mean",
        "count": "40",
        "round": "",
        "mult": "single",
        "reduce": false,
        "x": 740,
        "y": 180,
        "wires": [
            [
                "fed3b12a99eff03f"
            ]
        ]
    },
    {
        "id": "77108806fe40f3a6",
        "type": "ui_chart",
        "z": "cf99d9b968649254",
        "name": "",
        "group": "951f7cbe395b9312",
        "order": 0,
        "width": "12",
        "height": "10",
        "label": "Boiler op",
        "chartType": "line",
        "legend": "true",
        "xformat": "HH:mm:ss",
        "interpolate": "linear",
        "nodata": "",
        "dot": false,
        "ymin": "",
        "ymax": "",
        "removeOlder": "4",
        "removeOlderPoints": "",
        "removeOlderUnit": "604800",
        "cutout": 0,
        "useOneColor": false,
        "useUTC": false,
        "colors": [
            "#1f77b4",
            "#aec7e8",
            "#ff7f0e",
            "#2ca02c",
            "#98df8a",
            "#d62728",
            "#ff9896",
            "#9467bd",
            "#c5b0d5"
        ],
        "outputs": 1,
        "useDifferentColor": false,
        "className": "",
        "x": 1800,
        "y": 500,
        "wires": [
            []
        ]
    },
    {
        "id": "956780e0223bb069",
        "type": "rbe",
        "z": "cf99d9b968649254",
        "name": "",
        "func": "rbe",
        "gap": "",
        "start": "",
        "inout": "out",
        "septopics": true,
        "property": "payload",
        "topi": "topic",
        "x": 1310,
        "y": 500,
        "wires": [
            [
                "77108806fe40f3a6"
            ]
        ]
    },
    {
        "id": "4f17d8f2c31dd886",
        "type": "ui_chart",
        "z": "cf99d9b968649254",
        "name": "",
        "group": "ec53d7082368f1b2",
        "order": 0,
        "width": "12",
        "height": "10",
        "label": "Fuel oil",
        "chartType": "line",
        "legend": "true",
        "xformat": "HH:mm:ss",
        "interpolate": "linear",
        "nodata": "",
        "dot": false,
        "ymin": "0",
        "ymax": "275",
        "removeOlder": "10",
        "removeOlderPoints": "",
        "removeOlderUnit": "604800",
        "cutout": 0,
        "useOneColor": false,
        "useUTC": false,
        "colors": [
            "#1f77b4",
            "#aec7e8",
            "#ff7f0e",
            "#2ca02c",
            "#98df8a",
            "#d62728",
            "#ff9896",
            "#9467bd",
            "#c5b0d5"
        ],
        "outputs": 1,
        "useDifferentColor": false,
        "className": "",
        "x": 1800,
        "y": 180,
        "wires": [
            []
        ]
    },
    {
        "id": "f66a0cf108042a5e",
        "type": "delay",
        "z": "cf99d9b968649254",
        "name": "",
        "pauseType": "rate",
        "timeout": "5",
        "timeoutUnits": "seconds",
        "rate": "1",
        "nbRateUnits": "1",
        "rateUnits": "hour",
        "randomFirst": "1",
        "randomLast": "5",
        "randomUnits": "seconds",
        "drop": true,
        "allowrate": false,
        "outputs": 2,
        "x": 1110,
        "y": 180,
        "wires": [
            [
                "4f17d8f2c31dd886",
                "ff8623157d6237dc"
            ],
            []
        ]
    },
    {
        "id": "058ce02babd8fb2e",
        "type": "rbe",
        "z": "cf99d9b968649254",
        "name": "",
        "func": "narrowbandEq",
        "gap": "5%",
        "start": "",
        "inout": "in",
        "septopics": true,
        "property": "payload",
        "topi": "topic",
        "x": 790,
        "y": 520,
        "wires": [
            [
                "f28bde6391184792"
            ]
        ]
    },
    {
        "id": "e43f012806d6aa38",
        "type": "rbe",
        "z": "cf99d9b968649254",
        "name": "",
        "func": "narrowbandEq",
        "gap": "5%",
        "start": "",
        "inout": "in",
        "septopics": true,
        "property": "payload",
        "topi": "topic",
        "x": 790,
        "y": 480,
        "wires": [
            [
                "f28bde6391184792"
            ]
        ]
    },
    {
        "id": "ff8623157d6237dc",
        "type": "debug",
        "z": "cf99d9b968649254",
        "name": "debug 46",
        "active": false,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 1360,
        "y": 220,
        "wires": []
    },
    {
        "id": "1f5737f18b97fc3f",
        "type": "inject",
        "z": "cf99d9b968649254",
        "name": "Clear",
        "props": [
            {
                "p": "payload"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "[]",
        "payloadType": "jsonata",
        "x": 1630,
        "y": 260,
        "wires": [
            [
                "5a4d9f70340c192d"
            ]
        ]
    },
    {
        "id": "5a4d9f70340c192d",
        "type": "function",
        "z": "cf99d9b968649254",
        "name": "function 9",
        "func": "\nmsg.payload = [];\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 1640,
        "y": 220,
        "wires": [
            [
                "4f17d8f2c31dd886"
            ]
        ]
    },
    {
        "id": "ef07a3f983856c12",
        "type": "inject",
        "z": "cf99d9b968649254",
        "d": true,
        "name": "Clear",
        "props": [
            {
                "p": "payload"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "[]",
        "payloadType": "jsonata",
        "x": 1630,
        "y": 580,
        "wires": [
            [
                "5f50878ca1b45e27"
            ]
        ]
    },
    {
        "id": "5f50878ca1b45e27",
        "type": "function",
        "z": "cf99d9b968649254",
        "name": "function 10",
        "func": "\nmsg.payload = [];\n\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 1640,
        "y": 540,
        "wires": [
            [
                "77108806fe40f3a6"
            ]
        ]
    },
    {
        "id": "832790bb42c36356",
        "type": "rbe",
        "z": "cf99d9b968649254",
        "name": "",
        "func": "narrowbandEq",
        "gap": "5%",
        "start": "",
        "inout": "in",
        "septopics": true,
        "property": "payload",
        "topi": "topic",
        "x": 590,
        "y": 180,
        "wires": [
            [
                "fd0517cdfc3d1d33"
            ]
        ]
    },
    {
        "id": "951f7cbe395b9312",
        "type": "ui_group",
        "name": "Boiler",
        "tab": "b72753f9e30b11b7",
        "order": 3,
        "disp": true,
        "width": "12",
        "collapse": false,
        "className": ""
    },
    {
        "id": "ec53d7082368f1b2",
        "type": "ui_group",
        "name": "Fuel oil",
        "tab": "b72753f9e30b11b7",
        "order": 3,
        "disp": true,
        "width": "12",
        "collapse": false,
        "className": ""
    },
    {
        "id": "b72753f9e30b11b7",
        "type": "ui_tab",
        "name": "History",
        "icon": "dashboard",
        "disabled": false,
        "hidden": false
    }
]
2

I just noticed that two other charts I'm using in the same flow are also doing the same thing.
I'd set the other two up months ago to track a UPS battery and incoming solar power over the summer. Just noticed now that they're only showing 12 hours as well.

3

Is it the 1000 point limit that is doing it?

Alternatively, did node red restart 12 hours ago?

4

No, it appears to have been up for the past 6 days.
I didn't think the 1000 point would limit it, but could be for the boiler and the electric charts.

I have the oil chart limited to 1 sample per hour, so that should have only received 12 msg inputs in the period it's showing.

5

Are you using the latest version of node-red-dashboard?

If so then stop node red and start it again in a command window and paste the full output here please. Copy/paste please, not screenshot.

6

Will do when I can access again. I expect it's a recent version, the SD card failed a month ago and I had to rebuild it.

7

Did you bin the card and use a new good quality one from a reputable supplier?

8

I think it was a Samsung Pro. The Pi is on a UPS with NUT to shut it down on risk, but it randomly went corrupt.
I ended up going with a USB stick instead.

9

I updated the dashboard to the current version last week, and now each chart only shows 2 hours of history...
What the heck is going on?

image
image1295Ă—638 45.4 KB

10

I have setup a test with 10 week span and will leave it running overnight...

11

Well it doesn't seem to be anything simple as my test is now up to 12 hours and still going.
Can you import this simple flow into one our your failing systems and let it run and see what happens?

[{"id":"e86414f123c88c3f","type":"inject","z":"c29243ff.d97cf8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"60","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":960,"wires":[["d2c4782be0bcab75"]]},{"id":"e125951638c64a69","type":"ui_chart","z":"c29243ff.d97cf8","name":"","group":"52089abe.75c6bc","order":10,"width":0,"height":0,"label":"chart","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":"10","removeOlderPoints":"","removeOlderUnit":"604800","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"useDifferentColor":false,"className":"","x":710,"y":960,"wires":[[]]},{"id":"d2c4782be0bcab75","type":"range","z":"c29243ff.d97cf8","minin":"0","maxin":"3600000","minout":"0","maxout":"1","action":"scale","round":false,"property":"payload","name":"scale to show hours","x":500,"y":960,"wires":[["e125951638c64a69"]]},{"id":"e78ea592110ead9f","type":"inject","z":"c29243ff.d97cf8","name":"Clear","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[]","payloadType":"json","x":430,"y":1040,"wires":[["e125951638c64a69"]]},{"id":"52089abe.75c6bc","type":"ui_group","z":"c29243ff.d97cf8","name":"Controls","tab":"1e1f90f4.8c66d7","order":1,"disp":false,"width":"6"},{"id":"1e1f90f4.8c66d7","type":"ui_tab","z":"c29243ff.d97cf8","name":"Controls","icon":"dashboard","order":1}]

What is happening with the flows that were only showing 2 hours? Do they continue to show always the last two hours with samples disappearing of the left as new ones are added to the right?

12

Out of curiosity, I see you have an inject node sending a J: expression of [] to a function node that sets msg.payload to []

msg.payload = [];

return msg;

and I have to ask 'Why?'
and 'Why use the jsonata code in the inject and not just set msg.payload to a JSON []?

13

The OP once spoke to @E1cid perhaps :wink:

1 Like
14

Are you sure node-red is not restarting?

I don't use the ds18b20 node, but assuming that it is node-red-contrib-ds18b20 then you appear to be sampling five times/minute. If that is the case then a graph 3 weeks long, with two lines on it, would contain 300,000 points if I am not mistaken, which might well be enough to crash node-red. Are you running the browser on the pi? If so then it might well crash that too.

To check, open a command window and run
journalctl -b -u nodered | grep Welcome
That will show you each time node-red restarted since the pi was last rebooted. To see back before the last reboot, leave out the -b

15

I did, it doesn't work to clear the chart.

16

Does it clear the chart in the flow I posted?

17

Well son of a ....

Nov 28 13:28:20 ******** Node-RED[420]: Welcome to Node-RED
Nov 28 15:28:17 ******** Node-RED[9525]: Welcome to Node-RED
Nov 29 12:24:52 ******** Node-RED[59231]: Welcome to Node-RED
Nov 29 21:45:50 ******** Node-RED[73857]: Welcome to Node-RED
Dec 04 10:40:37 ******** Node-RED[267426]: Welcome to Node-RED
Dec 05 00:28:54 ******** Node-RED[295650]: Welcome to Node-RED

It's supposed to email me upon restart, but it has not been, so I was sure it wasn't crashing. But that appears to be the problem.

I have the samples limited to 1 per minute into the chart, so it should be about 40k in the chart after 2 weeks.
I'm not running the browser on the pi.

18

I haven't tried it yet, but now that you shared the crash-log (in a way), it seems that's the root of the chart reset problem.

19

I'm going to scrap and rebuild the system from the ground up, using a micro PC for NR and MQTT, and Wifi MCU's at the sensors.

Thanks for the help.

20

There should not be any need to do that, at least find out why it is crashing. Run
journalctl -u nodered --since "2023-12-05 00:25:00"
and it should show you the bit leading up to the last restart. It will pause at the end of each page, hit space to carry on.