Problem parsing flow variables to write to influx DB

Hello All,
As my name implies I am new to coding with Node Red and am trying to use it to track high and low temps from my home weather station. I want to write the data to the InfluxDB measurement (table) one per day so I am using flow variables to track it throughout the day and then once the day part of the date changes to the next day I am wanting to then write the line out to the database with (at that time) yesterday's date, high, and low.

My problem though is the change node that should be parsing my data and saving it to the msg.payload is instead causing the payload to be empty. Another change node that I included called "Parse Weather Data" works just fine. Even if I put that code in the Parse High/Low Data change node.

So I assume it is something with the flow variables in the change node but can someone help me figure this out please?

[
    {
        "id": "28d075bbece12065",
        "type": "tab",
        "label": "Flow 1",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "05315d1976ea1b82",
        "type": "switch",
        "z": "28d075bbece12065",
        "name": "New Day?",
        "property": "newday",
        "propertyType": "flow",
        "rules": [
            {
                "t": "neq",
                "v": "oldday",
                "vt": "flow"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 1,
        "x": 690,
        "y": 360,
        "wires": [
            [
                "fe95dc69b2fd6119"
            ]
        ]
    },
    {
        "id": "fe95dc69b2fd6119",
        "type": "change",
        "z": "28d075bbece12065",
        "name": "Parse High/Low Data",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "[\t{\t    \"Date\": flow.get(\"olddate\"),\t    \"High\": flow.get(\"high\"),\t    \"Low\": flow.get(\"low\")\t}\t]",
                "tot": "jsonata"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 880,
        "y": 360,
        "wires": [
            []
        ]
    },
    {
        "id": "a80697e34517090a",
        "type": "function",
        "z": "28d075bbece12065",
        "name": "High & Low",
        "func": "//var oldday = 0;\n\nvar high = flow.get(\"high\")||0;\nvar low = flow.get(\"low\")||0;\nvar olddate = flow.get(\"olddate\")||0;\nvar oldday = flow.get(\"oldday\")||0;\nvar newdate = flow.get(\"newdate\"||0);\nvar newday = flow.get(\"newday\") || 0;\n\nnode.warn(\"newday \" + newday);\n\nif (oldday != 0) {\n    node.warn(\"Not First Run\");\n    newdate = new Date();\n    flow.set(\"newdate\", newdate);\n    newday = newdate.getDate();\n    //flow.set(\"newday\", newday);\n    flow.set(\"newday\", 24);\n    if (high < msg.payload[0].lastData.tempf) {\n        high = msg.payload[0].lastData.tempf;\n        flow.set(\"high\", high);\n    }\n    if (low > msg.payload[0].lastData.tempf) {\n        low = msg.payload[0].lastData.tempf;\n        flow.set(\"low\", low);\n    }\n}\nif (oldday==0)\n{\n    node.warn(\"First Run\");\n    olddate = new Date();\n    flow.set(\"olddate\", olddate);\n    oldday = olddate.getDate();\n    flow.set(\"oldday\", oldday);\n    high = msg.payload[0].lastData.tempf;\n    flow.set(\"high\", high);\n    low = msg.payload[0].lastData.tempf;\n    flow.set(\"low\", low);\n}\n\nnode.warn(\"high \" + high);\nnode.warn(\"low \" + low);\nnode.warn(\"olddate \" + olddate);\nnode.warn(\"oldday \" + oldday);\nnode.warn(\"newdate \" + newdate);\nnode.warn(\"newday \" + newday);\nreturn msg;",
        "outputs": 1,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 510,
        "y": 360,
        "wires": [
            [
                "05315d1976ea1b82"
            ]
        ]
    },
    {
        "id": "847100e6ad63c29a",
        "type": "change",
        "z": "28d075bbece12065",
        "name": "Parse Weather Data",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "[\t{\t\"OutsideTemp\": payload[0].lastData.tempf,\t\"Humidity\": payload[0].lastData.humidity,\t\"WindSpeedMPH\": payload[0].lastData.windspeedmph\t}\t]",
                "tot": "jsonata"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 880,
        "y": 440,
        "wires": [
            []
        ]
    }
]

What is the reason for all this complexity ? What is wrong with writing the values to the database every x minutes, after which you can perform operations on the data ?

I figure there is no reason to have the data bloat of writing it every 5 minutes. Plus this way I do not have to worry about reading data out of Influx to then manipulate or make decisions on it.

Plus I wanted to learn how to use flow variables to pass data from iteration to iteration of the flow.

The end goal is to make a graph that has 52 days on it and for each the high and low. Lots of decisions on bee keeping involves kinda guessing what the weather might do and this could help in that.

I figure there is no reason to have the data bloat of writing it every 5 minutes.

Writing 3 values every 5 minutes will not fill up your harddrive, in addition: Influx has built-in retention policies, it will consolidate data over time based on the policy, specifically for this reason.

Plus this way I do not have to worry about reading data out of Influx to then manipulate or make decisions on it.

But this is exactly where the power of influx comes in, if you want the highs/lows grouped by day, it is a simple query, which is extremely fast with something like:

SELECT max("temperature") FROM "Temperature" WHERE time > now() - 7d GROUP BY time(1d) fill(null)

But to help you in your journey to learn flow variables, can you post an example input json object, and the expected output ?

Input is a weather data pull from my Ambient Weather station. that is triggered on each read of my hive temp sensors. that is a MQTT In node.

This is the weather get node minus my API and App Key:

[
    {
        "id": "d54134b42e557199",
        "type": "tab",
        "label": "Flow 3",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "210d04c5a4a6548a",
        "type": "http request",
        "z": "d54134b42e557199",
        "name": "HTTP Weather Request",
        "method": "GET",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "https://api.ambientweather.net/v1/devices?applicationKey=&apiKey=",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [],
        "credentials": {
            "user": "",
            "password": ""
        },
        "x": 370,
        "y": 260,
        "wires": [
            []
        ]
    }
]

The output is this (minus some redacted data):

[{REDACTED,"lastData":{"dateutc":1677273840000,"baromrelin":29.89,"baromabsin":28.73,"winddir":132,"windspeedmph":5.3,"windgustmph":6,"hourlyrainin":0,"dailyrainin":0,"weeklyrainin":0,"monthlyrainin":0.23,"yearlyrainin":0.74,"uv":2.1,"solarradiation":326.43,"maxdailygust":17.8,"tempf":23,"humidity":44,"battout":0,"temp6f":226.3,"humidity6":81,"batt6":1,"tempinf":70.5,"humidityin":30,"feelsLike":16.19,"dewPoint":4.37,"feelsLike6":2107.8,"dewPoint6":215.5,"feelsLikein":68.7,"dewPointin":37.6,"lastRain":"2023-02-14T12:59:00.000Z","tz":"America/Chicago","date":"2023-02-24T21:24:00.000Z"},REDACTED}}]

The only piece of data that I am worried about for this code is the tempf, which is the latest outside temp.

What I want the output to be for the database from the parser:
Time - Year and day (or can be in influx time, do not care)
High - Highest temp for that day
Low - Lowest temp for that day.

This would be my approach:

  1. Every x minutes get API data, append to flow context array.
  2. At 00:00 get the high/low from the context array, create influx object, send it and set the flow context back to an empty array (or delete it).

example flow:

[{"id":"0545a053efcb4e60","type":"inject","z":"b62adc6bfd3d40a9","name":"temp 19.4","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"lastData\":{\"dateutc\":1677273890000,\"baromrelin\":29.89,\"baromabsin\":28.73,\"winddir\":132,\"windspeedmph\":5.3,\"windgustmph\":6,\"hourlyrainin\":0,\"dailyrainin\":0,\"weeklyrainin\":0,\"monthlyrainin\":0.23,\"yearlyrainin\":0.74,\"uv\":2.1,\"solarradiation\":326.43,\"maxdailygust\":17.8,\"tempf\":19.4,\"humidity\":44,\"battout\":0,\"temp6f\":226.3,\"humidity6\":81,\"batt6\":1,\"tempinf\":70.5,\"humidityin\":30,\"feelsLike\":16.19,\"dewPoint\":4.37,\"feelsLike6\":2107.8,\"dewPoint6\":215.5,\"feelsLikein\":68.7,\"dewPointin\":37.6,\"lastRain\":\"2023-02-14T12:59:00.000Z\",\"tz\":\"America/Chicago\",\"date\":\"2023-02-24T21:24:00.000Z\"}}]","payloadType":"json","x":120,"y":740,"wires":[["4dbe7902e8e6e0fe"]]},{"id":"4dbe7902e8e6e0fe","type":"switch","z":"b62adc6bfd3d40a9","name":"flow context exists ?","property":"temperatures","propertyType":"flow","rules":[{"t":"null"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":380,"y":760,"wires":[["0c8f16e9e49e299d"],["18e1f7ccb2f86c02"]]},{"id":"0c8f16e9e49e299d","type":"change","z":"b62adc6bfd3d40a9","name":"","rules":[{"t":"set","p":"temperatures","pt":"flow","to":"[]","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":540,"y":700,"wires":[["18e1f7ccb2f86c02"]]},{"id":"5385366c893d8a4d","type":"change","z":"b62adc6bfd3d40a9","name":"append flow.temperatures","rules":[{"t":"set","p":"temperatures","pt":"flow","to":"$append($flowContext(\"temperatures\"), payload)\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":960,"y":780,"wires":[[]]},{"id":"18e1f7ccb2f86c02","type":"change","z":"b62adc6bfd3d40a9","name":"set timestamp & temperature","rules":[{"t":"set","p":"payload","pt":"msg","to":"{\t    \"tempf\": payload[0].lastData.tempf,\t    \"timestamp\":payload[0].lastData.dateutc\t}","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":680,"y":780,"wires":[["5385366c893d8a4d"]]},{"id":"a3affdebbdc2415e","type":"inject","z":"b62adc6bfd3d40a9","name":"@00:00","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"00 00 * * *","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":960,"wires":[["f689e96470b592c1"]]},{"id":"2db75be67f2f4ed5","type":"change","z":"b62adc6bfd3d40a9","name":"get high/low","rules":[{"t":"set","p":"payload","pt":"msg","to":"$flowContext(\"temperatures\").$^(tempf)[[-1, 0]]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":450,"y":960,"wires":[["4ba1ece5305472e6"]]},{"id":"4ba1ece5305472e6","type":"debug","z":"b62adc6bfd3d40a9","name":"debug 140","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1090,"y":960,"wires":[]},{"id":"ab48b076ea50e8ac","type":"comment","z":"b62adc6bfd3d40a9","name":"create influx object","info":"","x":630,"y":920,"wires":[]},{"id":"2ec542a6784ca6ae","type":"comment","z":"b62adc6bfd3d40a9","name":"reset flow.temperatures back to []","info":"","x":890,"y":900,"wires":[]},{"id":"f689e96470b592c1","type":"switch","z":"b62adc6bfd3d40a9","name":"context exist ?","property":"temperatures","propertyType":"flow","rules":[{"t":"nnull"}],"checkall":"true","repair":false,"outputs":1,"x":280,"y":960,"wires":[["2db75be67f2f4ed5"]]},{"id":"610feab29f7daf4a","type":"inject","z":"b62adc6bfd3d40a9","name":"temp 21.2","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"lastData\":{\"dateutc\":1677273900000,\"baromrelin\":29.89,\"baromabsin\":28.73,\"winddir\":132,\"windspeedmph\":5.3,\"windgustmph\":6,\"hourlyrainin\":0,\"dailyrainin\":0,\"weeklyrainin\":0,\"monthlyrainin\":0.23,\"yearlyrainin\":0.74,\"uv\":2.1,\"solarradiation\":326.43,\"maxdailygust\":17.8,\"tempf\":21.2,\"humidity\":44,\"battout\":0,\"temp6f\":226.3,\"humidity6\":81,\"batt6\":1,\"tempinf\":70.5,\"humidityin\":30,\"feelsLike\":16.19,\"dewPoint\":4.37,\"feelsLike6\":2107.8,\"dewPoint6\":215.5,\"feelsLikein\":68.7,\"dewPointin\":37.6,\"lastRain\":\"2023-02-14T12:59:00.000Z\",\"tz\":\"America/Chicago\",\"date\":\"2023-02-24T21:24:00.000Z\"}}]","payloadType":"json","x":120,"y":780,"wires":[["4dbe7902e8e6e0fe"]]},{"id":"f87b39f795161ac3","type":"inject","z":"b62adc6bfd3d40a9","name":"temp 22.6","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"lastData\":{\"dateutc\":1677273900000,\"baromrelin\":29.89,\"baromabsin\":28.73,\"winddir\":132,\"windspeedmph\":5.3,\"windgustmph\":6,\"hourlyrainin\":0,\"dailyrainin\":0,\"weeklyrainin\":0,\"monthlyrainin\":0.23,\"yearlyrainin\":0.74,\"uv\":2.1,\"solarradiation\":326.43,\"maxdailygust\":17.8,\"tempf\":22.6,\"humidity\":44,\"battout\":0,\"temp6f\":226.3,\"humidity6\":81,\"batt6\":1,\"tempinf\":70.5,\"humidityin\":30,\"feelsLike\":16.19,\"dewPoint\":4.37,\"feelsLike6\":2107.8,\"dewPoint6\":215.5,\"feelsLikein\":68.7,\"dewPointin\":37.6,\"lastRain\":\"2023-02-14T12:59:00.000Z\",\"tz\":\"America/Chicago\",\"date\":\"2023-02-24T21:24:00.000Z\"}}]","payloadType":"json","x":120,"y":820,"wires":[["4dbe7902e8e6e0fe"]]},{"id":"a9bb3e6472f13e71","type":"inject","z":"b62adc6bfd3d40a9","name":"temp 25.9","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"lastData\":{\"dateutc\":1677273980000,\"baromrelin\":29.89,\"baromabsin\":28.73,\"winddir\":132,\"windspeedmph\":5.3,\"windgustmph\":6,\"hourlyrainin\":0,\"dailyrainin\":0,\"weeklyrainin\":0,\"monthlyrainin\":0.23,\"yearlyrainin\":0.74,\"uv\":2.1,\"solarradiation\":326.43,\"maxdailygust\":17.8,\"tempf\":25.9,\"humidity\":44,\"battout\":0,\"temp6f\":226.3,\"humidity6\":81,\"batt6\":1,\"tempinf\":70.5,\"humidityin\":30,\"feelsLike\":16.19,\"dewPoint\":4.37,\"feelsLike6\":2107.8,\"dewPoint6\":215.5,\"feelsLikein\":68.7,\"dewPointin\":37.6,\"lastRain\":\"2023-02-14T12:59:00.000Z\",\"tz\":\"America/Chicago\",\"date\":\"2023-02-24T21:24:00.000Z\"}}]","payloadType":"json","x":120,"y":700,"wires":[["4dbe7902e8e6e0fe"]]},{"id":"6f63bc7c664df4ca","type":"comment","z":"b62adc6bfd3d40a9","name":"from api","info":"","x":120,"y":660,"wires":[]},{"id":"b758110e8200da82","type":"change","z":"b62adc6bfd3d40a9","name":"","rules":[{"t":"set","p":"temperatures","pt":"flow","to":"[]","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":860,"y":940,"wires":[[]]}]

Note I kept it verbose so that it might make more sense, but the number of nodes can be reduced.

So on your get high/low node, what is this code doing exactly?

$flowContext("temperatures").$^(tempf)[[-1, 0]]

I see it is getting the temperatures array. Is it sorted and getting the first and last element? Thus the high and low? I am just now getting around to playing with this code to see what it does for sure.

Should I be using flowContext instead of flow.get?

My parse node was doing:
"High": flow.get("high"),
Should it be instead:
"High": $flowContext("high"),

So that did it for saving high and low! Just using $flowContext instead of flow.get.

But now I get this error on the Date field:
Error: A 400 Bad Request error occurred: {"error":"unable to parse 'WeatherHighLow Date=Thu Feb 23 2023 15:17:08 GMT-0600 (Central Standard Time),High=49.6,Low=8.2': invalid boolean"}

I assume (maybe) I need to either convert back to Linux time to store or chage it to a string and store that?