Node-red freezing on deployment

General
1

Hello,

I am using node-red (currently on v3.1.9) for several years on a pi 3 for energy automation tasks like EV charging, Solar battery management etc. The installation has been working flawlessly so far.
Since few days it started to freeze on approx every 3rd deployment of code changes in a function node.

Here is what I have experienced/tested so far:

  • Using the systemctl status nodered command I see, that the flows are correctly stopped and restarted, but nothing afterwards.
  • The "top" command shows node-red at 100% cpu usage.
  • The only way to recover, is to restart (systemctl restart nodered)
  • I can deploy as many small changes (like moving items around, or adding a few caracters in a function node) I want without freezes
  • But when I make real code changes in the same functional node it freezes every third time.
  • No specific hints in any log file in /var/log

Any hints on where else to look for problems?

Many thanks

Heinz

2

If you modify the function so that you comment out all of the code inside of all of the tabs, does this still occur?

Is the function called periodically? How frequently?

What kind of deploy are you doing? (Full ,flows only, nodes only)?

Is there an inject node or something that triggers on startup that operates the function?

Does the function have code in the on start tab?

What is inside of the function? Can you share it by selecting only the function node and hitting CTRL+E to export it?

3

Many thanks Steve for your hints,

If you modify the function so that you comment out all of the code inside of all of the tabs, does this still occur?

no. However, this morning on that very same function node, I could do whatever deployment, it never happened! I will need to check it again this evening, at times when I normally have the problem. A bad client-server network connection during some hours???

Is the function called periodically? How frequently?

Once a day, but the problem never happens during operation, only when deploying

What kind of deploy are you doing? (Full ,flows only, nodes only)?

So far I always did full, but yesterday I deployed only the changed flow but it still happened

Is there an inject node or something that triggers on startup that operates the function?

Yes an inject node to trigger a flow once a day, but NOT on startup. However, there are other inject nodes which are kicked off at startup, but those are in an other flow.

Does the function have code in the on start tab?

no, only the standard commented out comments.

Many thanks for your time

Heinz

Admin edited for readability

4

That suggests 2 things.

  1. When you deploy, the function is being executed (i.e. a msg is arriving into its input)
    • This is easily proved by adding a debug (set to output full msg AND to go to the console) before the function (also, add a small delay immediately before the function to prevent the lockup affecting the debug output)
  2. Something in your function is maxing out the CPU
    • It may be your code depends on a context varaible or something external that has changed.

Please share your function as requested earlier. Alternatively, explain what it does (i.e. is there any loops? access to external stuff via fetch or global/flow/context? depend on any properties of the msg that might be missing?

5

Make a backup of your SD card in case it is starting to fail.

6

Yup, thanks, could of course also be a reason. The pi ist about 5 years old.
I do backups regularly on top of exporting all flows.

7

Many thanks for the hints Steve,

The first one I do not understand. Can you explain further? For the 2nd thought, I would expect the problem also to happen during normal operation, but it really only happens on deployment...

Pls find below the code of the entire flow. Note, that I am not a professional developper.. :slight_smile: I again checked every function node for any hidden startup code within the flow, but could not find anything. The suspected code is to be found in the node "Find best battery charging periode". I am currently working on the part which is commeted out towards the bottom.

What the flow does is the following:

  • Group 1 downloads a power tariff from supplier, checks it and makes it available for other nodes and flows
  • Group 2 finds the best tariff to charge the solar battery during the night, if necessary
  • Group 3 does the same for EV charging schedule
    All results are stored and consumed from a comprehensive Google Sheet.

Today it also froze, when I performed a change on a change node. Very strange....

Currently I am trying to perform deployements one to the changed node.....

Many thanks for your time.

With best regards

Heinz

[{"id":"6aeab476d56d9e19","type":"tab","label":"Energiemanagement","disabled":false,"info":"In this flow the energy consumption is \nmanaged and optimized for cost.\nThis is done by getting the daily \nvariable tariff data from the supplier, \nchecking the required energy and from this,\ncomputing the best charging stratgey and\nproviding resulting commands to the \ninverter/battery.","env":[]},{"id":"b0afcb9ac5170618","type":"group","z":"6aeab476d56d9e19","name":"Get variable Tariff from Groupe-e and write it to a global array for other flows and checks it for valid data","style":{"label":true},"nodes":["4375f62f6683b874","875e1405080e468e","e90dbd6cc14eb022","9328cd84712c2e12","506012f8794cc0dd","0343dde76c722438","ba52f74057ed5973","6190e38703b6dbf7","87a7e47bd15541b5","3e0d68d65e329f14","59a647cb19ed9b1e","3268c7be94f47475","243df198ce5d583d","108a51530df7c43c","8a3063c70a44f9e7","67b43a8259a32c41","320cdd08bd721d44"],"x":14,"y":19,"w":1092,"h":322},{"id":"9cb45dca8eb7fde1","type":"group","z":"6aeab476d56d9e19","name":"Turn of battery discharge during off peak hours when SoC too low  and replenish it from the grid in the night if necessary","style":{"label":true},"nodes":["234e2f2f25cb485e","0a836c56df47b457","45222a6546efcb55","8bda8025866b0eed","dc6e70b1c6c99b7e","5db17eda8e31308a","adf929db0a769057"],"x":14,"y":679,"w":1052,"h":182},{"id":"2ee7aa7e7c028a87","type":"group","z":"6aeab476d56d9e19","name":"Find best charging tariff and set charging times accordingly based on energy requirements","style":{"label":true},"nodes":["6a335c044679e7c8","0b288dd18072e683","1503edbdad81e0c6","471fe8e5840afe47","298839b92a45b81d","852b58d8624a6cab"],"x":14,"y":359,"w":1012,"h":142},{"id":"e410f11e486e1a99","type":"group","z":"6aeab476d56d9e19","name":"Find best EV charging time and required power","style":{"label":true},"nodes":["df42a0bf221fb393","099bcdc0dd04686c","ca88f5e82aac1196","342645aea1ab897c"],"x":14,"y":519,"w":1032,"h":142},{"id":"4375f62f6683b874","type":"inject","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"18:00 tariff load","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"00 18 * * *","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":100,"wires":[["875e1405080e468e","506012f8794cc0dd","3e0d68d65e329f14","108a51530df7c43c"]]},{"id":"875e1405080e468e","type":"function","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Compute URL","func":"//This function assembles the correct url to download\n//the tariff.\n//Details to be found at: https://www.groupe-e.ch/de/energie/elektrizitaet/privatkunden/vario\n\n  var TariffStartDate = new Date();\n  var TariffEndDate = new Date();\n\n  TariffStartDate.setDate(TariffStartDate.getDate()+1);\n  TariffStartDate.setHours(2); //to start at 00:00\n  TariffStartDate.setMinutes(0);\n  TariffStartDate.setSeconds(0);\n\n  TariffEndDate.setDate(TariffEndDate.getDate()+2);\n  TariffEndDate.setHours(2); //to end at 00:00\n  TariffEndDate.setMinutes(0);\n  TariffEndDate.setSeconds(0);\n\n\n  msg.url = \"https://api.tariffs.groupe-e.ch/v1/tariffs?start_timestamp=\" + TariffStartDate.toISOString().substring(0,19) + \"+02:00&end_timestamp=\" + TariffEndDate.toISOString().substring(0,19) + \"+02:00\";\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":360,"y":200,"wires":[["e90dbd6cc14eb022"]]},{"id":"e90dbd6cc14eb022","type":"http request","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Get tariff data","method":"GET","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":560,"y":200,"wires":[["9328cd84712c2e12","87a7e47bd15541b5"]]},{"id":"9328cd84712c2e12","type":"change","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Store tomorrows tariff globally","rules":[{"t":"set","p":"SupplierTariffData","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":790,"y":200,"wires":[["6190e38703b6dbf7"]]},{"id":"234e2f2f25cb485e","type":"mqtt in","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"Battery Emulator","topic":"battery-emulator_esp32-507060/info","qos":"2","datatype":"auto-detect","broker":"7a4dd03f.f90ad","nl":false,"rap":true,"rh":0,"inputs":0,"x":120,"y":760,"wires":[["0a836c56df47b457","adf929db0a769057"]]},{"id":"0a836c56df47b457","type":"function","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"Manage battery SoC","func":"//This routine checks every \"Threshold\" minutes\n    //if we are in off peak periode and if the State of Charge (SoC) is above \"MinSoc\"\n    //if below MinSoc during off peak periode, discharge is stopped\n    //if the battery needs to be replenished from the grid (typically during winter)\n    //if Google Sheet Status = NotStop, do nothing\n\nconst CutOffSoC = 6; //SoC in % when battery is considered empty (inverter has turned of battery) \nconst Threshold = 5; //return only every 5min\nconst RemoteControl = global.get(\"GoogleInstructions\")[0][0]; //to get operating instructions from Google Sheet\nconst ChargePower = parseInt(global.get(\"GoogleInstructions\")[0][1]); //to get charging power from Google Sheet;\nconst MinSoC = parseInt(global.get(\"GoogleInstructions\")[1][1]); //to get MinSoc from Google Sheet\nconst OffPeakStartHour = parseInt(global.get(\"GoogleInstructions\")[2][1]); //ex. 21:00\nconst OffPeakEndHour = parseInt(global.get(\"GoogleInstructions\")[3][1]); // this should be in line with the end time of EV charging to prevent, that battery will be depleted through EV charging\nconst ReplenishStartTime = new Date(global.get(\"GoogleInstructions\")[4][1]); //ex. 03:00 because of low usage of grid\nconst ReplenishEndTime = new Date(global.get(\"GoogleInstructions\")[5][1]); //ex. 05:00 because grid load starts here\n\nvar ActionArray = [];\nvar SoC = parseFloat(msg.payload[\"SOC_real\"]);\nvar BatPower = parseFloat(msg.payload[\"stat_batt_power\"]);\nvar Hour = new Date().getHours();\nvar CurrentTime = new Date();\nvar TimerSec = parseInt(new Date().getSeconds());\n//var TimerMin = (new Date().getMinutes() / Threshold) % 1; // %: Modulo to check full \n\n\n//Execute only at full minute and only if in Google Sheet OpMode is not in \"NotStop\" or \"Laden\" state\nif ((6 > TimerSec || TimerSec > 58) && RemoteControl != \"NotStop\") {\n\n        //if during off peak hours SoC gets below minimal value, turn off battery\n        if (SoC < MinSoC + 5 && (Hour >= OffPeakStartHour || Hour < OffPeakEndHour)) { //shall stop earlier than MinSoC used to replenish\n            if (BatPower < 0) { //prevent update when battery is already disabled (0 Amp) or \n                                 //charging from grid (positive Amp value)\n                msg.payload = [{ \"OpMode\": \"KeineRegelung\" }];\n                return msg;\n            }\n\n        } else { //if we are out of Off Peak or SoC is higher than MinSoC\n            if (SoC > CutOffSoC && RemoteControl != \"Laden\" && RemoteControl != \"Entladen\" && RemoteControl != \"Automatik\") { //only update status, if battery has been set to idle before\n                msg.payload = [{ \"OpMode\": \"Automatik\" }];\n                return msg;\n            }\n        }\n\n        //if during replenish hours SoC is below minimal value and battery is not already set to charge\n        if (SoC < MinSoC && ReplenishStartTime <= CurrentTime && CurrentTime < ReplenishEndTime) {\n            if (RemoteControl != \"Laden\") { //write action only if status is not already set to \"Laden\"\n                msg.payload = [{ \"OpMode\": \"Laden\" }]; //Charge battery\n                return msg;\n            }\n\n        } else { //if we are out of replenish hours or SoC is higher than MinSoC\n            if (RemoteControl == \"Laden\") { //only update status, if battery has been set to charge before\n                //we disable the battery here as it will be enabled when peak hours start by \"Check Soc\" function\n                msg.payload = [{ \"OpMode\": \"KeineRegelung\" }];\n                return msg;\n            }\n        }\n  }\n   ","outputs":1,"timeout":"","noerr":0,"initialize":"","finalize":"","libs":[],"x":440,"y":760,"wires":[["45222a6546efcb55","dc6e70b1c6c99b7e","8bda8025866b0eed"]]},{"id":"45222a6546efcb55","type":"debug","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"Op Mode","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":720,"y":720,"wires":[]},{"id":"8bda8025866b0eed","type":"google-spreadsheet","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"Set Operation Mode","auth":"4ea411a2.fe143","sheet":"1UKvqEo3yGbNzWyS3AA8jb4CNSL0H4-wk_99dpqQC_2U","range":"Fernsteuerung_WR!A2","method":"new","direction":"line","action":"set","clear":false,"line":false,"column":false,"fields":"all","save":"_sheet","selfields":[""],"cell_l":"","cell_c":"","input":"payload","output":"payload","saveType":"msg","inputType":"msg","outputType":"msg","sheetType":"str","rangeType":"str","cell_lType":"str","cell_cType":"str","x":760,"y":760,"wires":[[],[]]},{"id":"dc6e70b1c6c99b7e","type":"function","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"Format output","func":"//for testing purposes only\n\nvar TempVar = msg.payload[0];\nvar Aktion = TempVar[\"OpMode\"];\nvar Datum = new Date().toLocaleString();\n\nmsg.payload = Datum + \" \" + Aktion;\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":740,"y":820,"wires":[["5db17eda8e31308a"]]},{"id":"5db17eda8e31308a","type":"file","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"Write emulator logfile","filename":"/var/log/emulator.log","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":940,"y":820,"wires":[[]]},{"id":"506012f8794cc0dd","type":"delay","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"","pauseType":"delay","timeout":"30","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":150,"y":300,"wires":[["0343dde76c722438"]]},{"id":"0343dde76c722438","type":"function","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Validate tariff","func":"//This function checks the downloaded tariff data for errors\n//and issues error notification if a problem occured\n\nconst TariffData = global.get(\"SupplierTariffData\");\nvar ErrorCount = 0;\nvar ErrorLocation = 0;\nmsg.topic = \"Problem beim Laden des variablen Tarifs!\"\nmsg.payload = \"ACHTUNG: Der variable Tarif konnte entweder nicht geladen werder, oder aber der Download ist fehlerhaft! Bitte kontrollieren.\"\n\n//check if array is not empty\nif (TariffData.length == 0){\n    ErrorCount++;\n    ErrorLocation = 0;\n    }\n\n//check, if every tariff position is valid and increase\n//error counter if not\nfor (var i= 0; i<TariffData.length; i++){\n    if (isNaN(new Date(TariffData[0][\"start_timestamp\"]))){\n        ErrorCount++;\n        ErrorLocation = i;\n    }\n\n    if (isNaN(new Date(TariffData[0][\"end_timestamp\"]))){\n        ErrorCount++;\n        ErrorLocation = i;\n    }\n\n    if (isNaN(TariffData[0][\"vario_plus\"])){\n        ErrorCount++;\n        ErrorLocation = i;\n    }\n\n    if (isNaN(TariffData[0][\"vario_grid\"])){\n        ErrorCount++;\n        ErrorLocation = i;\n    }\n\n    if (isNaN(TariffData[0][\"dt_plus\"])){\n        ErrorCount++;\n        ErrorLocation = i;\n    }\n}\n\nmsg.topic = \"Problem beim Laden des variablen Tarifs!\";\nmsg.payload = \"ACHTUNG: Der variable Tarif konnte entweder nicht geladen werder, oder aber der Download ist fehlerhaft! Bitte kontrollieren. Anzahl Fehler: \" + ErrorCount + \" Fehlerstelle: \" + ErrorLocation;\n\n\nif (ErrorCount > 0) {return msg};\n","outputs":2,"timeout":"","noerr":0,"initialize":"","finalize":"","libs":[],"x":350,"y":300,"wires":[["ba52f74057ed5973"],[]]},{"id":"ba52f74057ed5973","type":"e-mail","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","server":"smtp.gmail.com","port":"465","authtype":"BASIC","saslformat":false,"token":"oauth2Response.access_token","secure":true,"tls":true,"name":"heinz.ruffieux@ondemandmanagement.net","dname":"Send error message","x":580,"y":300,"wires":[]},{"id":"6a335c044679e7c8","type":"inject","z":"6aeab476d56d9e19","g":"2ee7aa7e7c028a87","name":"Initiate tariff finder at 23:00","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"00 23 * * *","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":180,"y":400,"wires":[["471fe8e5840afe47"]]},{"id":"0b288dd18072e683","type":"debug","z":"6aeab476d56d9e19","g":"2ee7aa7e7c028a87","name":"debug 47","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":720,"y":460,"wires":[]},{"id":"6190e38703b6dbf7","type":"debug","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"debug 48","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1000,"y":200,"wires":[]},{"id":"1503edbdad81e0c6","type":"google-spreadsheet","z":"6aeab476d56d9e19","g":"2ee7aa7e7c028a87","name":"Write best charge periodes to Google Sheet","auth":"4ea411a2.fe143","sheet":"1UKvqEo3yGbNzWyS3AA8jb4CNSL0H4-wk_99dpqQC_2U","range":"Fernsteuerung_WR!B6","method":"new","direction":"line","action":"set","clear":false,"line":false,"column":false,"fields":"all","save":"_sheet","selfields":[""],"cell_l":"","cell_c":"","input":"payload","output":"payload","saveType":"msg","inputType":"msg","outputType":"msg","sheetType":"str","rangeType":"str","cell_lType":"str","cell_cType":"str","x":830,"y":400,"wires":[[],[]]},{"id":"df42a0bf221fb393","type":"inject","z":"6aeab476d56d9e19","g":"e410f11e486e1a99","name":"Start at 19:00","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"00 19 * * *","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":560,"wires":[["ca88f5e82aac1196"]]},{"id":"87a7e47bd15541b5","type":"file","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"write tariff to log file","filename":"/var/log/tariff_data.log","filenameType":"str","appendNewline":true,"createDir":false,"overwriteFile":"false","encoding":"none","x":750,"y":240,"wires":[[]]},{"id":"099bcdc0dd04686c","type":"debug","z":"6aeab476d56d9e19","g":"e410f11e486e1a99","name":"debug 50","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":720,"y":620,"wires":[]},{"id":"3e0d68d65e329f14","type":"file in","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Read tariff file","filename":"/var/log/tariff_data.log","filenameType":"str","format":"lines","chunk":false,"sendError":false,"encoding":"utf8","allProps":false,"x":360,"y":100,"wires":[["59a647cb19ed9b1e"]]},{"id":"59a647cb19ed9b1e","type":"json","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"","property":"payload","action":"obj","pretty":false,"x":530,"y":100,"wires":[["3268c7be94f47475"]]},{"id":"3268c7be94f47475","type":"change","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Store todays data","rules":[{"t":"set","p":"TodaysData","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":750,"y":100,"wires":[["243df198ce5d583d"]]},{"id":"243df198ce5d583d","type":"debug","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"debug 51","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1000,"y":100,"wires":[]},{"id":"108a51530df7c43c","type":"delay","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":140,"y":200,"wires":[["875e1405080e468e"]]},{"id":"ca88f5e82aac1196","type":"function","z":"6aeab476d56d9e19","g":"e410f11e486e1a99","name":"Define best EV charging periode","func":"//This function combines today's evening tariff and tomorrows tariff and\n//finds the best overnight charging periode as most cars are not plugged in\n//during day time\n\nconst ChargeParam = global.get(\"GoogleInstructions\"); //get instructions from Google Sheet\nconst ChargingEnergy = parseFloat(ChargeParam[6][1]);\nconst ChargingDurationSlow = parseFloat(ChargeParam[7][1]); //for vhc charging with single phase\nconst ChargingDurationFast = parseFloat(ChargeParam[8][1]); //for vhc charging with 3 phase\n\nconst TodaysTariff = flow.get(\"TodaysData\");\nconst TomorrowsTariff = global.get(\"SupplierTariffData\");\nconst MaxAmps = 16; //max allowed amps at the wallbox\nvar CombinedTariff = [];\nvar ChargingPeriode = [];\nvar ChargingSegmentsSlow = ChargingDurationSlow * 4-1; //4 15min segments per hour\nvar ChargingSegmentsFast = ChargingDurationFast * 4-1; //4 15min segments per hour\nvar AvgSegmentTariff = [];\nvar CarChargeStartTime = {};\nvar CarChargeEndTime = {};\n\n//compute the required charging speed in amps required to\n//charge the required energy during the required duration\n//This part could also be implemented on a per wallbox basis\n//but here it is calculated only once per day whil at the wallbox\n//every minute.\nvar ChargingPower = ChargingEnergy / ChargingDurationSlow; //Energy in kWh divided by duration in h leads to the required power in kW\nvar ChargingAmps = Math.round(ChargingPower *1000 / (238 *0.9));  //I = P/(U*effective power) (0.9 assumed)\n\n//if the calculated Amps would be too high set it to the max and issue a warning\nif (ChargingAmps > MaxAmps){ \n    ChargingAmps = 16;\n}\nglobal.set(\"ChargingAmpsSlow\", ChargingAmps);\n\n\n//combine tariffs and create one common overnight tariff from 21:00 until 06:00 next day\nfor (var i=84; i< TodaysTariff.length; i++){ //from tariff 84 (21:00) to the end of the day\n    CombinedTariff.push(TodaysTariff[i]);\n}\nfor (var i=0; i<=23; i++) { //from tariff 0 (midnight) to tariff 23 (06:00)\n    CombinedTariff.push(TomorrowsTariff[i]);\n}\n\n//for slow chargers (1 and 2 phase charging)\n\n    //for ChargingDuration hours lets find the best time window in the \n    //combined array containing 15mins periodes\n    for (var i=0; i<CombinedTariff.length-ChargingSegmentsSlow; i++) {\n        var AvgSegTar =0;\n        for (var y=i; y<i+ChargingSegmentsSlow; y++) {\n            AvgSegTar += parseFloat(CombinedTariff[y][\"vario_plus\"]);\n        }\n        AvgSegTar = AvgSegTar / ChargingSegmentsSlow;\n        AvgSegmentTariff.push([i, y, AvgSegTar]);\n    }\n\n    //sort along the cost of the window, the first will be the cheapest\n    AvgSegmentTariff.sort((a, b) => a[\"2\"] - b[\"2\"]);\n\n    //now we know which charging window is the best\n    //so, lets define charging start and end time of that window [0]\n    CarChargeStartTime = CombinedTariff[AvgSegmentTariff[0][0]][\"start_timestamp\"];\n    CarChargeEndTime = CombinedTariff[AvgSegmentTariff[0][1]][\"end_timestamp\"];\n\n    ChargingPeriode.push({CarChargeStartTime}); \n    ChargingPeriode.push({CarChargeEndTime});\n\n//for 3 phase chargers:\n    var AvgSegmentTariff = [];\n    //for ChargingDuration hours lets find the best time window in the \n    //combined array containing 15mins periodes\n    for (var i = 0; i < CombinedTariff.length - ChargingSegmentsFast; i++) {\n        var AvgSegTar = 0;\n        for (var y = i; y < i + ChargingSegmentsFast; y++) {\n            AvgSegTar += parseFloat(CombinedTariff[y][\"vario_plus\"]);\n        }\n    AvgSegTar = AvgSegTar / ChargingSegmentsFast;\n    AvgSegmentTariff.push([i, y, AvgSegTar]);\n    }\n\n    //sort along the cost of the window\n    AvgSegmentTariff.sort((a, b) => a[\"2\"] - b[\"2\"]);\n\n    //now we know which charging window is the best\n    //so, lets define charging start and end time of that window [0]\n    CarChargeStartTime = CombinedTariff[AvgSegmentTariff[0][0]][\"start_timestamp\"];\n    CarChargeEndTime = CombinedTariff[AvgSegmentTariff[0][1]][\"end_timestamp\"];\n\n    ChargingPeriode.push({ CarChargeStartTime });\n    ChargingPeriode.push({ CarChargeEndTime });\n\n\nmsg.payload = ChargingPeriode; \n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":470,"y":560,"wires":[["099bcdc0dd04686c","342645aea1ab897c"]]},{"id":"342645aea1ab897c","type":"google-spreadsheet","z":"6aeab476d56d9e19","g":"e410f11e486e1a99","name":"Write best car charge periode to Google Sheet","auth":"4ea411a2.fe143","sheet":"1UKvqEo3yGbNzWyS3AA8jb4CNSL0H4-wk_99dpqQC_2U","range":"Fernsteuerung_WR!B11","method":"new","direction":"line","action":"set","clear":false,"line":false,"column":false,"fields":"all","save":"_sheet","selfields":[""],"cell_l":"","cell_c":"","input":"payload","output":"payload","saveType":"msg","inputType":"msg","outputType":"msg","sheetType":"str","rangeType":"str","cell_lType":"str","cell_cType":"str","x":840,"y":560,"wires":[[],[]]},{"id":"8a3063c70a44f9e7","type":"comment","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Read last tariff for later use (Define best EV charging periode) and store it in a flow variable","info":"","x":370,"y":60,"wires":[]},{"id":"67b43a8259a32c41","type":"comment","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Read tomorrows tariff store it in a global variable and to a log file","info":"","x":290,"y":160,"wires":[]},{"id":"320cdd08bd721d44","type":"comment","z":"6aeab476d56d9e19","g":"b0afcb9ac5170618","name":"Validate tariff array and report if erroneous or empty","info":"","x":250,"y":260,"wires":[]},{"id":"adf929db0a769057","type":"debug","z":"6aeab476d56d9e19","g":"9cb45dca8eb7fde1","name":"debug 54","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":400,"y":800,"wires":[]},{"id":"471fe8e5840afe47","type":"function","z":"6aeab476d56d9e19","g":"2ee7aa7e7c028a87","name":"Find best battery charging periode","func":"\n\n//this function finds the lowest cost periode to charge the battery within the supplier tariff\n\nconst ChargeParam = global.get(\"GoogleInstructions\"); //get instructions from Google Sheet\nconst MinSoC = parseInt(ChargeParam[1][1]); // Min State of Charge (SoC)\nconst CurrentSoc = parseFloat(ChargeParam[2][5]); // Current SoC \nconst MaxCapacity = 39*parseInt(ChargeParam[5][5])/100; //net battery capacity reduced by SoH\nconst ChargingPower = parseInt(ChargeParam[0][1]); //charging power in W\nconst OffPeakStartHour = parseInt(ChargeParam[2][1]);\nconst OffpeakEndHour = parseInt(ChargeParam[3][1]);\nvar EnergyRequirement = 15; //energy (in kWh) needed to charge to the required level (default value)\nvar LastTariffNbr = 31; //last tariff number to be considered for the LowTariff evaluation (31 = next morning 08:00)\nvar BatReplenishStartTime = {};\nvar BatReplenishEndTime = {};\nvar ReplenishPeriode = []; //Start and End Time for Battery replenishment depending of tariff (prices)\n\nvar JsonData = global.get(\"SupplierTariffData\"); //GetTariffData and get back the Json values\n\nvar ReferenceTariff = JsonData[0][\"dt_plus\"];  //get offpeak tarif\nvar LowTariffArray = []; //contains the required tariff information\nvar UltraLowTariffArray = []; //contains a copy of the LowTariffArray, in case there are very low tariffs\nvar TariffLength = 15; //length of tariff periode in minutes\nvar UltraLowTariffRate = 9; //Rate below which we want to fill up the battery in any case\n                            //because bad weather is forcasted\n \n  //build array with all tariffs lower than the reference\n  var y=0;\n  for (var i = 0;i<LastTariffNbr; i++) {\n    //var TariffHour = new Date(JsonData[i][\"end_timestamp\"]).getHours();\n\n    //select all tariff periodes where the tariff is lower than the reference tariff\n    if (JsonData[i][\"vario_plus\"] < ReferenceTariff ) {\n      LowTariffArray[y] = {};\n      LowTariffArray[y] = { \"TariffNumber\": i, \n                            \"StartTime\": JsonData[i][\"start_timestamp\"], \n                            \"EndTime\": JsonData[i][\"end_timestamp\"],\n                            \"Tariff\": JsonData[i][\"vario_plus\"]\n                          } ;\n      y++;\n    }\n  }\n\n  //Store those values in a global array for other flows to charge cars\n  flow.set(\"LowTariffArray\", LowTariffArray);\n\n\n  //now we calculate how many tariff periods are required to get from current charge level to the required charge level\n  if (MinSoC > CurrentSoc) { //only when current state of charge is lower than minimal state of charge, as we recharge the battery only in this case\n    EnergyRequirement = (MaxCapacity*(MinSoC - CurrentSoc)/100).toFixed(2); //is max capacity multiplied by the % to be charged\n  } else {\n    EnergyRequirement = 0;\n  }\n  var ChargingPeriodes = Math.ceil(EnergyRequirement * 1000 / ChargingPower * 60 / TariffLength);\n\n  //now sort the array along the tariff \"vario_plus\" to get the periodes sorted along the ascending rates\n  LowTariffArray.sort((a, b) => a[\"Tariff\"] - b[\"Tariff\"]);  \n\n\n  //save this state for a later use case (charge battery on very low tariffs)\n  UltraLowTariffArray = LowTariffArray;\n\n  //now we strip the number of array entries which are not used as we have more low cost periodes available than required\n  //and then we sort it along the tariff number to be in line with the original tariff number to match it with a future \n  //tariff structure\n  if (LowTariffArray.length >= ChargingPeriodes) {\n    for (var i= 0; ChargingPeriodes - LowTariffArray.length; i++) {\n      LowTariffArray.pop();  //removes always the last element in the array (here the most expensive)\n    }\n\n  } else { //if there are more periodes required than identified as beging on low tariff\n    //do nothing for the moment\n  }\n\n  //sort along the tariff number\n  LowTariffArray.sort((a, b) => a[\"TariffNumber\"] - b[\"TariffNumber\"]);\n\nif (LowTariffArray.length > 0){ //if no charge needed we don't output and leave the date from yesterday\n  BatReplenishStartTime = LowTariffArray[0][\"StartTime\"];\n  BatReplenishEndTime = LowTariffArray[LowTariffArray.length-1][\"EndTime\"];\n\n  ReplenishPeriode.push({BatReplenishStartTime});\n  ReplenishPeriode.push({BatReplenishEndTime});\n\n} else { //when there is no replenishment necessary use a very old date for clarity\n\n  var OldDate = new Date(1970, 1, 0, 1, 0, 0, 0).toISOString(); // 1. Jan. 1970 00:00\n\n  ReplenishPeriode.push({ OldDate }); //for start and end time the same date invalidates charging\n  ReplenishPeriode.push({ OldDate });\n}\n\n/*\n//if we got Ultra Low Tariffs (UltraLowTariffRate) during the night it means that there \n//is bad weather tomorrow therefore we want to fill up the battery to the max \n//for this we take the \"UltraLowTariffArray\" from above, which is already sorted along\n//the rates\n\n//count how many \"Ultra Low tariff rates\" are available\nvar y=0;\nfor (var i=0; UltraLowTariffArray.length-1; i++){\n //if (parseInt(UltraLowTariffArray[0][\"Tariff\"]) < UltraLowTariffRate) {\n    y++;\n //} \n}\n\n//redefine battery charging start and end time\nif (y > 0) { //if there are Ultra low charging rates available\n  ReplenishPeriode.length = 0; //empty the array from values defined above\n  BatReplenishStartTime = UltraLowTariffArray[0][\"StartTime\"];\n  BatReplenishEndTime = UltraLowTariffArray[y][\"EndTime\"];\n\n  ReplenishPeriode.push({ BatReplenishStartTime });\n  ReplenishPeriode.push({ BatReplenishEndTime });\n\n}\n\nmsg.payload = JsonData;\n*/\n\nmsg.payload = ReplenishPeriode;\n\nreturn msg;\n\n\n","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":480,"y":400,"wires":[["0b288dd18072e683","1503edbdad81e0c6"]]},{"id":"298839b92a45b81d","type":"inject","z":"6aeab476d56d9e19","g":"2ee7aa7e7c028a87","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":120,"y":460,"wires":[["852b58d8624a6cab"]]},{"id":"852b58d8624a6cab","type":"change","z":"6aeab476d56d9e19","g":"2ee7aa7e7c028a87","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"SupplierTariffData","tot":"global"}],"action":"","property":"","from":"","to":"","reg":false,"x":330,"y":460,"wires":[["0b288dd18072e683"]]},{"id":"7a4dd03f.f90ad","type":"mqtt-broker","name":"OpenWB","broker":"127.0.0.1","port":"1883","clientid":"","autoConnect":true,"usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"autoUnsubscribe":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""},{"id":"4ea411a2.fe143","type":"google-service-account","name":"Energiestatistik","scope":["https://www.googleapis.com/auth/spreadsheets"],"way":"json","check_dialogflow":"","check_speech":""}]
8

While node-red is running, open a command window and run top. Is anything hogging the processor? Also check for shortage of available RAM.

Leave top running, and when node-red freezes again check what it then shows.

1 Like
9

Hello all,

I finally found the (stupid) error:

2 errors in one line:

  1. no condition set AND
  2. doing such math calculation within a condition does not seem to work either

wrong:

correct:

Of course a very stupid error, but what I do not understand is, why this leads to a freeze when deploying. The code should not execute on deployment without an external injection- correct?

10

Assuming that the code did not contain the asterisks then, since ChargingPeriodes - LowTariffArray.length is always truthy then it will loop forever.

11

If you use the more modern syntax of forEach() then you will make your code simpler and avoid the possibility of this happening.

12

Thanks Colin,

Yes, the Asterix' are formating error and were NOT in the code.

Yes, I fully understand the consequences of the error DURING EXECUTION, But what I do not understand is, why this led to freezes ON DEPLOYMENTS, as this code is not executed during startup of the node.

Any thoughts on that? I am just trying to hone my node-red skills a bit... :slight_smile:

Many thanks

13

Perfect! Many thanks.

Yes, my coding skills became a bit rusty..... :slight_smile:

Will start, using foreach() immediately. Seems to be MUCH simpler....

Heinz

14

It will be executed when the node receives its next message.
What sort of deploy were you doing, Full, Modified Flows or Modified Nodes, as configured in the drop down from the Deploy button.

15

Thanks Colin,

Yes, I would fully understand if it run into problems on the next execution, but it froze the entire node-red system on about every 3rd deployment.

No more freezes since I corrected the for loop. Very strange.

I tried deployments on all levels: Full, flow and node level. With same result.

16

Only thing I can think of is the runtime was already "stuck" before you hit deploy?

The runtime is completely separate from the frontend editor and so you would not notice the backend is stuck / struggling (purely due to timing)?

17

I suspect Steve's suggestion is correct. Did you try what I had suggested, which was to leave top running and see what it shows? If Steve is correct then you would have seen the CPU go to 100% before doing the deploy.

18

Agree, could well be, as also Colin stated earlier on.

When I think about now, I was mislead by the fact that the function node get's called only once per day at 23:00. BUT I was debugging the problem and then of course I kicked it off manually and did not realize, that with every "kick-off" I caused the system to be frozen already, which you do not see immediately, but only on next deploy.

So, next time I will run top in such situations the whole time - as Colin pointed out.

I think THAT's it! Many thanks guys. I learned a lot, by this!

1 Like