Question: Optimize msg traffic

General
1

Hi,

One thing I have noticed is that NR or HA, but in NR at least the message layout differs wildly, sometimes the entity_id is in the payload, sometimes it IS the payload and sometime it is located in the msg.data, also a lot of data is superfluous (but maybe that's so for me being a newbie and all). And I'm using entity_id in this post as placeholder because it can be any field.
For me it causes a hassle, so I'm standardizing all messages coming out of nodes, even if the called node would overwrite incoming payloads (and I would not want that) I will shift parts of the needed data either in the msg structure (i.e. moving payload into data etc) or I'll use flow and global context.
Since I don't need all data of most nodes I will also drop data I do not need. So I use topic to be able to read what's going on without opening all debugs. The payload always has the basics, id, state/service(wheter reporting or calling),
ofcourse in case I'm calling i.e. a service the payload wil be more. etc.

But it got me wondering, is there no option doing it in the called node it self or is there a way to use a template, other the what I'm doing now.

[{"id":"f39243689c25bde9","type":"server-events","z":"593fb70375108267","name":"","server":"2fba4297.e4145e","version":2,"eventType":"state_changed","exposeToHomeAssistant":false,"eventData":"","haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"waitForRunning":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"$outputData(\"eventData\").event_type","valueType":"jsonata"}],"event_type":"","x":120,"y":1920,"wires":[["1ff4baa0ee2d364c"]]},{"id":"1ff4baa0ee2d364c","type":"switch","z":"593fb70375108267","name":"1=Light,2=Presence","property":"payload.entity_id","propertyType":"msg","rules":[{"t":"cont","v":"light","vt":"str"},{"t":"cont","v":"input_boolean.presence","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":340,"y":1920,"wires":[[],["e444904265f02b70"]]},{"id":"e444904265f02b70","type":"function","z":"593fb70375108267","name":"Std msg layout  + update house Array (hard coded)","func":"var entity = msg.payload.entity_id;\nvar state = msg.payload.event.new_state.state;\nvar parent_id = msg.payload.context.parent_id; \nvar user_id = msg.payload.context.user_id;\nvar last = msg.payload.event.new_state.last_updated;\nvar newload = \"\";\nvar newdata = \"\";\nvar newcontext = { parent_id: parent_id, user_id: user_id, last: last }\nif(state == \"on\" || state == true){\n    state = \"on\";\n    var msg1 = {}; \n    newload = { entity_id: entity, state: state, context: newcontext };\n    msg1.topic = entity + \" \" + state;\n    msg1.payload = newload;\n    msg1.data = newdata;\n    } \n    else{\n    state = \"off\";\n    var msg2 = {};\n    newload = { entity_id: entity, state: state, context: newcontext };\n    msg2.topic = entity + \" \" + state;\n    msg2.payload = newload;\n    msg2.data = newdata;\n}\n\nif (entity.includes(\"woonk\")){\nglobal.set (\"house.room[1].presence\", state);\n}\nif (entity.includes(\"badk\")) {\n    global.set(\"house.room[2].presence\", state);\n}\nif (entity.includes(\"werkk\")) {\n    global.set(\"house.room[4].presence\", state);\n}\nif (entity.includes(\"greenr\")) {\n    global.set(\"house.room[6].presence\", state);\n}\n\nreturn [msg1, msg2];","outputs":2,"noerr":0,"initialize":"","finalize":"","libs":[],"x":650,"y":1940,"wires":[["e13dc2bf246b38d2"],["70c996af57c0e279"]]},{"id":"e13dc2bf246b38d2","type":"link out","z":"593fb70375108267","name":"Present Light on","mode":"link","links":["4c2e5c492dd799e3","7efaa58af6d5cf4f"],"x":925,"y":1900,"wires":[]},{"id":"bb8b0e1f7f2825bc","type":"comment","z":"593fb70375108267","name":"Will be drepreceated","info":"","x":960,"y":1860,"wires":[]},{"id":"70c996af57c0e279","type":"link out","z":"593fb70375108267","name":"Not Present, lights out?","mode":"link","links":["4c2e5c492dd799e3","7efaa58af6d5cf4f"],"x":925,"y":1980,"wires":[]},{"id":"a7c71ccc8eb5ca8f","type":"comment","z":"593fb70375108267","name":"Triggers shutting down the room with warnings","info":"","x":920,"y":2020,"wires":[]},{"id":"2fba4297.e4145e","type":"server","name":"Home Assistant BasAdmin","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":false,"cacheJson":true,"heartbeat":true,"heartbeatInterval":"30","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]

In the above example I use the 3rd node to standardize it all and I'll make it a function node template, but now I have to tweak it for every type of nod.

Any ideas?

2

FYI Node-RED is not affiliated with Home Assistant, most folks here don't use home assistant.

If the json objects are inconsistent, the best path to follow is to create an issue against the HA nodes.

But looking at the documentation, quite some thought went into this by the looks of it, and it seems like you can use msg.payload.data, with some prerequisites.

3

Each node will deal with a msg object and depending on the node different information will be created and passed onto the next node. Lets look at a simple flow, an inject node connected to a debug node.

The inject node starts with a payload that is set to a timestamp and a topic that is an empty string. Once it fires (by setting the inject and/or the repeat option) a _msgid is created that is unique to each msg it sends off. You can also add other parts to the msg in the inject node as you determine your flow needs.

When the msg gets to the dbug node, you can have id display one or all of the msg object.

Now if you have a temperature sensor, it would probably have a temperature and maybe a humidity reading that would be passsed in the sensor while a light sensor might just have lum's that it passes on.

Each node will create parts of the msg that report what that node needs to pass on. Then there is the join node that will take information from two or more msgs and combine them into a single msg. Then there is the change node where you can remove, move data to different parts of the msg or remove parts of the msg altogether.

This is the way it works.

I recommend watching this playlist: Node-RED Essentials. The videos are done by the developers of node-red. They're nice & short and to the point. You will understand a whole lot more in about 1 hour. A small investment for a lot of gain.

4

If you could describe why this is so, it would go a long way to helping us understand why you need it this way. Perhaps there is a better means of accomplishing what you're trying to do? I don't know. But a lot of people who watch this forum are full of ideas and ways of doing things and it's a good place to pick up something new.

And while trying to standardize your messages may be a noble cause, it may also be a hill you can't take. I'll use zenofmud's idea and go a step further. The HTTP request and response nodes are two nodes that must go together. Whenever you receive a request, you must send a response. The request node creates two message objects that must stay intact between receive and send, req and res. If these nodes are altered in any way, it destroys the match and will make the responder unable to respond. But they're the only nodes I know of that require these two properties. So it can't be standardized.

But then again, maybe I'm missing the point of the standardization. Can you clarify? That would help a bunch.

EDIT: After looking at your flow function, I can see some of what you're doing. But it doesn't make a lot of sense in the small context of what's there. Perhaps you don't feel comfortable with sharing your whole flow as there may be personal details in it you don't want shared. That's fine. But would you be able to give an example of the whole flow or detail what goes in, what comes out and what it does in the middle to make that happen? Building your message and creating objects is something that almost every flow has to do. But if you could tell us why you have to have it a certain way or why you have to have globals (for instance), perhaps we could tell you a way that will make it easier on you that would eliminate some or all of your need to standardize.

5

First, sorry it took so long but I have recovered from a crash this weekend due to a faulty SSD cable, so I have lost some work and had to go back to an older project, but I think it will give more insight. Okay, as to the flow of this node, it is part of a routine that, based on presence will turn lights on or off and when turning them off will provide a warning first.

[{"id":"70fef1d0c5ee98b6","type":"comment","z":"593fb70375108267","name":"Disable Perms if user_id = user","info":"Zodra er in in (automation) groep wordt ingegrepen Perm eraf","x":670,"y":1840,"wires":[]},{"id":"d8cf07494168e65c","type":"server-events","z":"593fb70375108267","name":"","server":"2fba4297.e4145e","version":2,"eventType":"state_changed","exposeToHomeAssistant":false,"eventData":"","haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"waitForRunning":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"$outputData(\"eventData\").event_type","valueType":"jsonata"}],"event_type":"","x":160,"y":1920,"wires":[["4e5f7f800423eecf"]]},{"id":"4e5f7f800423eecf","type":"switch","z":"593fb70375108267","name":"1=Light,2=Presence","property":"payload.entity_id","propertyType":"msg","rules":[{"t":"cont","v":"light","vt":"str"},{"t":"cont","v":"input_boolean.presence","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":380,"y":1920,"wires":[["7c9af3e0e9d7a2fd"],["b5e8e2180d5bbb56"]]},{"id":"7c9af3e0e9d7a2fd","type":"function","z":"593fb70375108267","name":"What permission to turn off","func":"var entity = msg.payload.entity_id;\nvar user_id = msg.payload.context.user_id;\nvar parent_id = msg.payload.context.parent_id;\n\nvar bas_id = \"011714f9a5034d9ea20d8c731238a231\";\nvar ellen_id = \"b95355d6c20b42f090420c0c4a71f303\";\nvar supervisor_id = \"c073a75c7cbf4ff896c4043e4710d8ab\";\nmsg.data = \"\"\nconst area_arr =[\n/**1 */    {naam: \"woonkmr\", area: \"woonkamer\"},\n/**2 */    {naam: \"badkmr\", area: \"badkamer\"},\n/**3 */    {naam: \"slaapkmr\", area: \"slpkamer\"},\n/**4 */    {naam: \"werkkmr\", area: \"werkkamer\"},\n/**5 */    {naam: \"kastenkmr\", area: \"kastenkamer\"},\n/**6 */    {naam: \"logeerkmr\", area: \"logeerkamer\"},\n/**7 */    {naam: \"greenrm\", area: \"greenroom\" },\n         ];\nvar len = area_arr.length;      \nif (entity.includes(\"light\")) {\n    for (let k = 0; k < len; k++) {\n        var tmpstrg = area_arr[k].naam;\n        if (entity.includes(tmpstrg)) {\n            var area = area_arr[k].area;\n            entity = \"input_boolean.automate_perm_\" + area_arr[k].naam;\n        }\n        /** Als Bas of Ellen iets instelt dan automation gelijk af. */\n        if (user_id == bas_id || user_id == ellen_id) {\n\n            msg.topic = \"\";\n            msg.payload = { domain: \"input_boolean\", service: \"turn_off\", data: { \"entity_id\": entity } };\n            msg.topic = entity;\n            node.send(msg);\n        } /** Als Node-red het doet is user_id die van supervisor */\n        else if (user_id == supervisor_id) {\n            /** do something */\n        } /** Als HA automation het doet is user_id en PArent_id null */\n        else if (user_id == null) {\n            /** do something */\n        }\n    }\n}\nreturn null;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":660,"y":1880,"wires":[[]]},{"id":"dfc3913b4f315354","type":"link out","z":"593fb70375108267","name":"Not Present, lights out?","mode":"link","links":["4c2e5c492dd799e3","85f3456b6b8d2242"],"x":915,"y":2000,"wires":[]},{"id":"652a8a42b0afd895","type":"comment","z":"593fb70375108267","name":"Listen to HA","info":"","x":130,"y":1840,"wires":[]},{"id":"b5e8e2180d5bbb56","type":"function","z":"593fb70375108267","name":"update house Array (hard coded)","func":"var loaded = msg.payload;\nvar entity = msg.payload.entity_id;\nvar state = msg.payload.event.new_state.state;\nif(state == \"on\" || state == true){\n    state = \"on\";\n   var msg1 = {}; \n   msg1.topic = \"start_light\";\n   msg1.payload = entity;\n} else{\n    state = \"off\";\n    var msg2 = {};\n    msg2.topic = msg.topic;\n    msg2.payload = loaded;\n}\n\nif (entity.includes(\"woonk\")){\nglobal.set (\"house.room[1].presence\", state);\n}\nif (entity.includes(\"badk\")) {\n    global.set(\"house.room[2].presence\", state);\n}\nif (entity.includes(\"werkk\")) {\n    global.set(\"house.room[4].presence\", state);\n}\nif (entity.includes(\"greenr\")) {\n    global.set(\"house.room[6].presence\", state);\n}\n\nreturn [msg1, msg2];","outputs":2,"noerr":0,"initialize":"","finalize":"","libs":[],"x":680,"y":1980,"wires":[[],["dfc3913b4f315354","556575b152c63244"]]},{"id":"d29859edde27580f","type":"comment","z":"593fb70375108267","name":"Verdeel op inhoud","info":"","x":370,"y":1880,"wires":[]},{"id":"eb350e0cb674eae7","type":"comment","z":"593fb70375108267","name":"Status Aanwezigheid","info":"","x":630,"y":1940,"wires":[]},{"id":"85f3456b6b8d2242","type":"link in","z":"593fb70375108267","name":"not present = lights out","links":["4b833fecc1ca8d63","de1b0ff328a7a137","b46d50c732b6cbf3","dfc3913b4f315354"],"x":175,"y":2220,"wires":[["556575b152c63244"]]},{"id":"4a08ca0a53c8eedc","type":"function","z":"593fb70375108267","name":"Warn and off msg","func":"var entity = msg.payload.entity_id;\nvar staat = msg.payload.event.new_state.state;\nmsg.data = \"\"\nconst area_arr = [\n/**1 */    { naam: \"woonkmr\", area: \"woonkamer\" },\n/**2 */    { naam: \"badkmr\", area: \"badkamer\" },\n/**3 */    { naam: \"slaapkmr\", area: \"slpkamer\" },\n/**4 */    { naam: \"werkkmr\", area: \"werkkamer\" },\n/**5 */    { naam: \"kastenkmr\", area: \"kastenkamer\" },\n/**6 */    { naam: \"logeerkmr\", area: \"logeerkamer\" },\n/**7 */    { naam: \"greenrm\", area: \"greenroom\" },\n];\nvar len = area_arr.length;\nif (entity.includes(\"input_boolean.presence\")) {\n    for (let k = 0; k < len; k++) {\n        var tmpstrg = area_arr[k].naam;\n        if (entity.includes(tmpstrg)) {\n            var area = area_arr[k].area;\n        }\n    }\n\n    msg.topic = \"no_pres_warn1 \" + area;\n    msg.payload = { domain: \"light\", service: \"turn_on\", data: { \"area_id\": area, \"brightness_step_pct\": 10 } };\n    node.send(msg);\n    \n    msg.topic = \"warn2\";\n    msg.payload = { domain: \"light\", service: \"turn_on\", data: { \"area_id\": area, \"brightness_step_pct\": -10 } };\n    node.send(msg);\n    \n    msg.topic = \"dim \" + area;\n    msg.payload = { domain: \"light\", service: \"turn_on\", data: { \"area_id\": area, \"brightness_step_pct\": -40, \"transition\": 10 } };\n    node.send(msg);\n\n    msg.topic = \"out \" + area;\n    msg.payload = { domain: \"light\", service: \"turn_off\", data: { \"area_id\": area } };\n    node.send(msg);\n\n\n}\nreturn null;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":260,"y":2140,"wires":[["8318d08ea3351352"]]},{"id":"8318d08ea3351352","type":"switch","z":"593fb70375108267","name":"warn/dim/out","property":"topic","propertyType":"msg","rules":[{"t":"cont","v":"warn","vt":"str"},{"t":"cont","v":"dim","vt":"str"},{"t":"cont","v":"out","vt":"str"}],"checkall":"true","repair":false,"outputs":3,"x":480,"y":2140,"wires":[["76459aeb039958ae"],["9df9feb3b4cee5c8"],["bb7ced8334b965f5"]]},{"id":"76459aeb039958ae","type":"trigger","z":"593fb70375108267","name":"Warning!","op1":"","op2":"","op1type":"pay","op2type":"payl","duration":"5","extend":false,"overrideDelay":false,"units":"s","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":670,"y":2100,"wires":[["d92e97d64e7d7d0c"]]},{"id":"bb7ced8334b965f5","type":"delay","z":"593fb70375108267","name":"","pauseType":"delay","timeout":"4","timeoutUnits":"minutes","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":670,"y":2180,"wires":[["0101d8586298b625"]]},{"id":"556575b152c63244","type":"switch","z":"593fb70375108267","name":"","property":"payload.event.new_state.state","propertyType":"msg","rules":[{"t":"eq","v":"off","vt":"str"},{"t":"eq","v":"on","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":260,"y":2220,"wires":[["4a08ca0a53c8eedc","2a14de80f6032584"],["54d148dee81c090b"]]},{"id":"0101d8586298b625","type":"q-gate","z":"593fb70375108267","name":"","controlTopic":"control","defaultState":"open","openCmd":"open","closeCmd":"close","toggleCmd":"toggle","queueCmd":"queue","defaultCmd":"default","triggerCmd":"trigger","flushCmd":"flush","resetCmd":"reset","peekCmd":"peek","dropCmd":"drop","statusCmd":"status","maxQueueLength":"100","keepNewest":false,"qToggle":false,"persist":false,"storeName":"default","x":800,"y":2240,"wires":[["d92e97d64e7d7d0c"]]},{"id":"d92e97d64e7d7d0c","type":"link out","z":"593fb70375108267","name":"Turn Off Empty Area","mode":"link","links":["f7901627d7e8fd14"],"x":835,"y":2160,"wires":[]},{"id":"2a14de80f6032584","type":"change","z":"593fb70375108267","name":"open","rules":[{"t":"set","p":"topic","pt":"msg","to":"control","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"open","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":2180,"wires":[["0101d8586298b625"]]},{"id":"54d148dee81c090b","type":"change","z":"593fb70375108267","name":"close","rules":[{"t":"set","p":"topic","pt":"msg","to":"control","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"close","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":460,"y":2220,"wires":[["0101d8586298b625"]]},{"id":"9df9feb3b4cee5c8","type":"delay","z":"593fb70375108267","name":"","pauseType":"delay","timeout":"2","timeoutUnits":"minutes","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":670,"y":2140,"wires":[["0101d8586298b625"]]},{"id":"3ddea2940ecd53f3","type":"server-state-changed","z":"593fb70375108267","name":"Motion Greenroom","server":"2fba4297.e4145e","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"binary_sensor.pir_greenrm_motion","entityidfiltertype":"exact","outputinitially":false,"state_type":"str","haltifstate":"on","halt_if_type":"str","halt_if_compare":"is","outputs":2,"output_only_on_state_change":false,"for":"0","forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"on","valueType":"str"},{"property":"topic","propertyType":"msg","value":"","valueType":"triggerId"},{"property":"data","propertyType":"msg","value":"","valueType":"triggerId"}],"x":170,"y":1720,"wires":[["de04624d97ed95f1"],["5443bddcce39016f"]]},{"id":"de04624d97ed95f1","type":"change","z":"593fb70375108267","name":"Set presence.greenrm","rules":[{"t":"set","p":"house.room[6].presence","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":760,"y":1720,"wires":[["a6765c68a5e2a079"]]},{"id":"eaed52b88d0969cf","type":"countdown","z":"593fb70375108267","name":"","topic":"presence greenrm","payloadTimerStart":"","payloadTimerStartType":"nul","payloadTimerStop":"off","payloadTimerStopType":"str","timer":"1800","resetWhileRunning":true,"outputOnReset":false,"setTimeToNewWhileRunning":true,"startCountdownOnControlMessage":true,"x":570,"y":1740,"wires":[["de04624d97ed95f1"],[]]},{"id":"5443bddcce39016f","type":"function","z":"593fb70375108267","name":"Set Greenroom timer","func":"msg.data = msg.data;\nmsg.topic = \"control\"\nvar pay = global.get(\"house.room[6].presence_timer\");\npay = parseFloat(pay) *60;\nmsg.payload = pay;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":380,"y":1740,"wires":[["eaed52b88d0969cf"]]},{"id":"a6765c68a5e2a079","type":"function","z":"593fb70375108267","name":"Trigger SC","func":"var service = msg.payload;\nif (service == true || service == \"on\"){\n    service = \"on\";\n} else if (service == false || service == \"off\"){\n    service = \"off\";\n}\nvar entity = msg.data;\nif (entity == undefined){\n    entity = msg.topic\n}\nflow.set(\"pres_enti\", entity);\nmsg.payload = null;\n\nconst area_arr = [\n/**1 */    { naam: \"woonkmr\", area: \"woonkamer\", objnr: 1 },\n/**2 */    { naam: \"badkmr\", area: \"badkamer\", objnr: 2 },\n/**3 */    { naam: \"slpkmr\", area: \"slaapkamer\", objnr: 3 },\n/**4 */    { naam: \"werkkmr\", area: \"werkkamer\", objnr: 4 },\n/**5 */    { naam: \"kastenkmr\", area: \"kastenkamer\", objnr: 5 },\n/**6 */    { naam: \"logeerkmr\", area: \"logeerkamer\", objnr: 7 },\n/**7 */    { naam: \"greenrm\", area: \"greenroom\", objnr: 6 },\n/**8 */    { naam: \"primelamp\", area: \"greenroom\" },\n/**9 */    { naam: \"bureaulamp\", area: \"greenroom\" }\n];\nvar lengte = area_arr.length;\nfor (let k = 0; k < lengte; k++) {\n    var chkstrg = area_arr[k].naam;\n    \n    if (entity.includes(chkstrg)) {\n        var naam = area_arr[k].naam;\n        var area = area_arr[k].area;\n        entity = \"input_boolean.presence_\" + naam;\n        flow.set(\"entity[\" + k + \"]\", entity);\n        msg.topic = \"bool\";\n        msg.payload = { domain: \"input_boolean\", service: \"turn_\"+ service, data: { \"entity_id\": entity } };\n        node.send(msg);                \n    }\n}\nreturn null;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":970,"y":1760,"wires":[["aae1d9c349cfdeb8"]]},{"id":"aae1d9c349cfdeb8","type":"api-call-service","z":"593fb70375108267","name":"Input Boolean {Payload}","server":"2fba4297.e4145e","version":5,"debugenabled":false,"domain":"input_boolean","service":"{{service}}","areaId":[],"deviceId":[],"entityId":[],"data":"","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"data"}],"queue":"none","x":1190,"y":1800,"wires":[[]]},{"id":"2fba4297.e4145e","type":"server","name":"Home Assistant BasAdmin","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":false,"cacheJson":true,"heartbeat":true,"heartbeatInterval":"30","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]

(it is a bit messy, it's old)
The motion sensor triggers presence and a "off" timer, the events node listens to changes of the boolean (Home Assistant Helper), when nobody is in the room (according to PIR) the lights will flash, then dim and the the whole area will switch off. If after first flash or dim movement is detected, the routine will stop and lights will restore.

To provide a key example of where this hangs to most, is in the wait step, using a countdown (right now I'm thinking of going for the delay, but have to recalculate the timer settings then). The countdown node will overwrite your payload, without any option (besides context) of retaining that data.

And this wil go for more nodes, so in order to maintain information from begin to end, without using context which poses it's own set of challenges, I have to move payload to data, plus, when there already is data in msg.data find a way to "layer" that retention. The payload contains the actionable data, the data is meta in my view and as such it may not be needed in one hop, from one node to another, it may well be needed later in the flow. So whilst I do use the context for more of the non-volatile data ( and data that is needed in more flows), the use of the msg construct is needed, simply because using context for that many variables would be madness. So now you need a switch or function node after an "action" node to move around data. Why not provide an option to "retain original message" and move msg.payload to msg.data, at which time the results of the node can be stored in payload again.

6

I tried to dissect what your flow is doing, but I got lost. If this is the way you have to deal with home assistant data, it reconfirms my choice not to use HA :')

Perhaps you should draw out what your flow is supposed to do, because I have the gut feeling that this can be way more simple. Note that node-red's mantra is: Low-code programming for event-driven applications - but you started with lots of code.

The reverse of "wait" is "trigger", use trigger nodes to perform actions based on events.

I am pretty sure that you can request all the information from HA when you need it without maintaining arrays of objects with the areas/rooms and people (unless this is not defined in HA ?).

7

:grinning: not lot's of code, but some, yeah, anyway, I have simplified the flow already and have found the delay node more compliant.

[{"id":"5feae811b2bf1936","type":"server-state-changed","z":"593fb70375108267","name":"Motion Greenroom","server":"2fba4297.e4145e","version":4,"exposeToHomeAssistant":false,"haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"entityidfilter":"binary_sensor.pir_greenrm_motion","entityidfiltertype":"exact","outputinitially":true,"state_type":"str","haltifstate":"on","halt_if_type":"str","halt_if_compare":"is","outputs":2,"output_only_on_state_change":false,"for":"0","forType":"num","forUnits":"minutes","ignorePrevStateNull":false,"ignorePrevStateUnknown":false,"ignorePrevStateUnavailable":false,"ignoreCurrentStateUnknown":false,"ignoreCurrentStateUnavailable":false,"outputProperties":[{"property":"topic","propertyType":"msg","value":"house.room[6].presence","valueType":"str"},{"property":"delay","propertyType":"msg","value":"$number($globalContext('house.room[6].presence_timer'))*60000","valueType":"jsonata"},{"property":"payload","propertyType":"msg","value":"","valueType":"entityState"},{"property":"data.event","propertyType":"msg","value":"","valueType":"eventData"},{"property":"data.entity_id","propertyType":"msg","value":"","valueType":"triggerId"},{"property":"data.state","propertyType":"msg","value":"","valueType":"entityState"}],"x":270,"y":2900,"wires":[["59bd31c7c85e29e6"],["30b52cf25744bad3"]]},{"id":"30b52cf25744bad3","type":"delay","z":"593fb70375108267","name":"","pauseType":"delayv","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":480,"y":2940,"wires":[["59bd31c7c85e29e6"]]},{"id":"59bd31c7c85e29e6","type":"function","z":"593fb70375108267","name":"Trigger SC","func":"var room_presence = msg.topic;\nvar state = msg.payload;\nglobal.set(room_presence, state);\nvar entity = msg.data.entity_id;\n\nconst area_arr = [\n    { naam: \"woonkmr\", area: \"woonkamer\", objnr: 1 },\n    { naam: \"badkmr\", area: \"badkamer\", objnr: 2 },\n    { naam: \"slpkmr\", area: \"slaapkamer\", objnr: 3 },\n    { naam: \"werkkmr\", area: \"werkkamer\", objnr: 4 },\n    { naam: \"kastenkmr\", area: \"kastenkamer\", objnr: 5 },\n    { naam: \"logeerkmr\", area: \"logeerkamer\", objnr: 7 },\n    { naam: \"greenrm\", area: \"greenroom\", objnr: 6 },\n    { naam: \"primelamp\", area: \"greenroom\", objnr: 6 },\n    { naam: \"bureaulamp\", area: \"greenroom\", objnr: 6 },\n    { naam: \"konijn\", area: \"woonkamer\", objnr: 1}\n];\nvar lengte = area_arr.length;\nfor (let k = 0; k < lengte; k++) {\n    var chkstrg = area_arr[k].naam;\n    \n    if (entity.includes(chkstrg)) {\n        var naam = area_arr[k].naam;\n        entity = \"input_boolean.presence_\" + naam;\n        msg.topic = msg.topic + \" \" + state;\n        msg.payload = { domain: \"input_boolean\", service: \"turn_\"+ state, data: { \"entity_id\": entity } };\n        node.send(msg);                \n    }\n}\nreturn null;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":670,"y":2900,"wires":[["e49a17ac2bfb0348"]]},{"id":"e49a17ac2bfb0348","type":"api-call-service","z":"593fb70375108267","name":"Input Boolean {Payload}","server":"2fba4297.e4145e","version":5,"debugenabled":false,"domain":"input_boolean","service":"{{service}}","areaId":[],"deviceId":[],"entityId":[],"data":"","dataType":"json","mergeContext":"","mustacheAltTags":false,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"data"}],"queue":"none","x":870,"y":2900,"wires":[[]]},{"id":"18f5fd7b1e7f97f1","type":"function","z":"593fb70375108267","name":"Warn and off msg","func":"var entity = msg.payload.entity_id;\nvar staat = msg.payload.event.new_state.state;\nmsg.data = \"\"\nconst area_arr = [\n/**1 */    { naam: \"woonkmr\", area: \"woonkamer\" },\n/**2 */    { naam: \"badkmr\", area: \"badkamer\" },\n/**3 */    { naam: \"slaapkmr\", area: \"slpkamer\" },\n/**4 */    { naam: \"werkkmr\", area: \"werkkamer\" },\n/**5 */    { naam: \"kastenkmr\", area: \"kastenkamer\" },\n/**6 */    { naam: \"logeerkmr\", area: \"logeerkamer\" },\n/**7 */    { naam: \"greenrm\", area: \"greenroom\" },\n];\nvar len = area_arr.length;\nif (entity.includes(\"input_boolean.presence\")) {\n    for (let k = 0; k < len; k++) {\n        var tmpstrg = area_arr[k].naam;\n        if (entity.includes(tmpstrg)) {\n            var area = area_arr[k].area;\n        }\n    }\n\n    msg.topic = \"no_pres_warn1 \" + area;\n    msg.payload = { domain: \"light\", service: \"turn_on\", data: { \"area_id\": area, \"brightness_step_pct\": 10 } };\n    node.send(msg);\n    \n    msg.topic = \"warn2\";\n    msg.payload = { domain: \"light\", service: \"turn_on\", data: { \"area_id\": area, \"brightness_step_pct\": -10 } };\n    node.send(msg);\n    \n    msg.topic = \"dim \" + area;\n    msg.payload = { domain: \"light\", service: \"turn_on\", data: { \"area_id\": area, \"brightness_step_pct\": -40, \"transition\": 10 } };\n    node.send(msg);\n\n    msg.topic = \"out \" + area;\n    msg.payload = { domain: \"light\", service: \"turn_off\", data: { \"area_id\": area } };\n    node.send(msg);\n\n\n}\nreturn null;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":770,"y":3020,"wires":[["52f8d8647a437540"]]},{"id":"52f8d8647a437540","type":"switch","z":"593fb70375108267","name":"warn/dim/out","property":"topic","propertyType":"msg","rules":[{"t":"cont","v":"warn","vt":"str"},{"t":"cont","v":"dim","vt":"str"},{"t":"cont","v":"out","vt":"str"}],"checkall":"true","repair":false,"outputs":3,"x":950,"y":3020,"wires":[["2b4748addd4df2e9"],["13bd190a54b37756"],["450a8e0ec11ca940"]]},{"id":"2b4748addd4df2e9","type":"trigger","z":"593fb70375108267","name":"Warning!","op1":"","op2":"","op1type":"pay","op2type":"payl","duration":"5","extend":false,"overrideDelay":false,"units":"s","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":1140,"y":3000,"wires":[["fdd88af0a6a4935d"]]},{"id":"450a8e0ec11ca940","type":"delay","z":"593fb70375108267","name":"","pauseType":"delay","timeout":"4","timeoutUnits":"minutes","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1140,"y":3080,"wires":[["05b7e9017090f18e"]]},{"id":"c5ef771705420896","type":"switch","z":"593fb70375108267","name":"","property":"payload.event.new_state.state","propertyType":"msg","rules":[{"t":"eq","v":"off","vt":"str"},{"t":"eq","v":"on","vt":"str"}],"checkall":"true","repair":false,"outputs":2,"x":570,"y":3040,"wires":[["18f5fd7b1e7f97f1","c83dfba5e2d242b1"],["f64b4ad5b6f5989c"]]},{"id":"05b7e9017090f18e","type":"q-gate","z":"593fb70375108267","name":"","controlTopic":"control","defaultState":"open","openCmd":"open","closeCmd":"close","toggleCmd":"toggle","queueCmd":"queue","defaultCmd":"default","triggerCmd":"trigger","flushCmd":"flush","resetCmd":"reset","peekCmd":"peek","dropCmd":"drop","statusCmd":"status","maxQueueLength":"100","keepNewest":false,"qToggle":false,"persist":false,"storeName":"default","x":1310,"y":3140,"wires":[["fdd88af0a6a4935d"]]},{"id":"fdd88af0a6a4935d","type":"link out","z":"593fb70375108267","name":"Turn Off Empty Area","mode":"link","links":["f7901627d7e8fd14"],"x":1425,"y":3080,"wires":[]},{"id":"c83dfba5e2d242b1","type":"change","z":"593fb70375108267","name":"open","rules":[{"t":"set","p":"topic","pt":"msg","to":"control","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"open","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":3060,"wires":[["05b7e9017090f18e"]]},{"id":"f64b4ad5b6f5989c","type":"change","z":"593fb70375108267","name":"close","rules":[{"t":"set","p":"topic","pt":"msg","to":"control","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"close","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":3100,"wires":[["05b7e9017090f18e"]]},{"id":"13bd190a54b37756","type":"delay","z":"593fb70375108267","name":"","pauseType":"delay","timeout":"2","timeoutUnits":"minutes","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1140,"y":3040,"wires":[["05b7e9017090f18e"]]},{"id":"fab188fade824c91","type":"server-events","z":"593fb70375108267","name":"","server":"2fba4297.e4145e","version":2,"eventType":"state_changed","exposeToHomeAssistant":false,"eventData":"","haConfig":[{"property":"name","value":""},{"property":"icon","value":""}],"waitForRunning":true,"outputProperties":[{"property":"payload","propertyType":"msg","value":"","valueType":"eventData"},{"property":"topic","propertyType":"msg","value":"$outputData(\"eventData\").event_type","valueType":"jsonata"}],"event_type":"","x":260,"y":3040,"wires":[["20ca0ca5b68985af"]]},{"id":"20ca0ca5b68985af","type":"switch","z":"593fb70375108267","name":"presence","property":"payload.entity_id","propertyType":"msg","rules":[{"t":"cont","v":"input_boolean.presence","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":440,"y":3040,"wires":[["c5ef771705420896"]]},{"id":"2fba4297.e4145e","type":"server","name":"Home Assistant BasAdmin","version":5,"addon":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":false,"cacheJson":true,"heartbeat":true,"heartbeatInterval":"30","areaSelector":"friendlyName","deviceSelector":"friendlyName","entitySelector":"friendlyName","statusSeparator":"at: ","statusYear":"hidden","statusMonth":"short","statusDay":"numeric","statusHourCycle":"h23","statusTimeFormat":"h:m","enableGlobalContextStore":true}]

Now, as to using HA(*) and the code, that seems a bit besides the point, my original question concerned the way msg object organizes data going in and coming out of nodes. A lot of data seems to be "volatile" and control over what is lost is limited, but it might be I'm looking at it wrong.
Because of the low-code idea, I would suspect there to be an easier way to store data, without using context and thus lot's of variables.

(*)
HA is quite good at storing data and integrates with almost everything "out-of-the-box" (which I need for redundancy (ZHA,Tradfri/Dirigera, HUE, SmartThings, KNX, etc, etc) , however the global "homeAssistant" is so big as it contains all of HA's data, so I created a subset to facilitate easier working. For me HA is data repo and GUI, NR is logic. But even NR cannot function "alone".

8

There are a number HA nodes, you are now listening for all events - see the warning in the documentation, while there is also an 'events:state' node, where you can listen for specific things (presence in your case) and within your flow, you can get the values you need using the 'current state' node ?

Node-red can work with most, if not more devices you listed. There are thousands of nodes available.
I think the nice thing about HA, is that it is a state machine - it knows the state of all devices: query it when you need it.

9

I don't think you're looking at it wrong, but just need to think about it differently. Bakman2 is making a really good point that HA is a state machine and not a processing machine. With HA, you check and change states. It's really simple in concept.

The power of Node-Red comes in when you think about how the nodes interact with each other. In most cases of a trigger, there is a parallel operation being performed that the trigger activates. If you have a message queue, there is a node sending a message to the queue and another node sending the trigger to the queue. If you have a problem with information being lost node to node, the answer may not be in how to preserve a message from end to end. The answer may be found in asking the question of can you circumvent all the issues or just use something different (like the delay node) and eliminate all the issues at once?

It looks like you're on a good track now and are getting this more and more resolved. A suggestion I would pose in this case is if you're having problems with nodes, try using something built to be completely manipulated like a function node. I know the point of Node-Red is to be able to link nodes together with minimal configuration and make things work. That's a great concept. But when you can branch out and add custom capabilities that fit your application, you can really take charge of your flow and make it do exactly what you want. Just something to think about.

And BTW, I like the idea of blinking the lights as a warning. Hadn't thought about something like that before.

10

Thanks all, to be sure, I'm gaining more and more insight into the how-to-use of NR. To be clear, to me HA is indeed basically a state machine (and UI). And btw @bakman2 thanks for the suggestion regarding the events:all node, I changed it to {event:state ( regex: input_boolean.presence).} As to the current state node, in this case it's not the best option. Also because I'm trying not to create an entire forest of "similar" nodes.

@madhouse I make use of the function node and I'm seeing that using it allows me to get a better grip if NR as a whole too, streamlining msgs and moving functionality out of the function node as a result.

11

Sounds great. Come on back here if/when you get stuck again and we'll see what we can do to help.

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.