Really strange behavior: some values from msg.payload wont be displayed on NR dashboard

Hello Folks :smiley:

Description:
I have a really strange behavior with node-red dashboard (i think) - I am not even sure if its a "mistake" or "error"...
First of all: All I will explain only happen when I open a new browser session and access node-red Dashboard.

Problem: I visit my dashboard first time and 3 different values from node arent shown on dashboard, but other messages will displayed correctly (I think its cached?).
This problem only happens on the exact same 3 values (and everytime - problem dont rotates to other nodes...)

Image example:
pic

Here you only see one of the three values (third one was cutted) - If I visit the dashboard first time (or incognito mode): Status line wont show up and is also empty (checked it) - "Verbundet seit" shows up correctly...
If I open the node-red editor and redeploy all nodes then all 3 values shown up or a new value comes in (cron)....

I am really confused about - cant find out where the problem is (like i said, dont even know if this is an error? )

First time I realised the problem: node-red-dashboard: 2.21.0 - was installed
Didnt had this problem before.

What was tested:
Completly other browser engine: exact same result
checked cron logs: no errors
checked journalctl: no errors
checked node-red systemd status: no errors
Problem reproducable: Yes - every new session on NR dashboard

tested the flow and nodes connected with this 3 values (see below for details): works as expeced and no missing values

I tested all nodes and flows connected with this 3 values because all these 3 values comes from my fritzbox (avm) and I thought maybe avm implemented some security stuff or ddos protections or else...
So I spammed (on all 3 nodes) 40 times or so in 3 seconds... and I got everytime the correct output
no error - no missing value from my fritzbox - nothing
SO I think my fritzbox isn the issue ?

Anyway enough text, I hope some gurus can help me out of this or can give a hint to debug it :smiley:

Have a nice day and also stay healty !
greetings

EDIT forgot some info:
Node-red: v.1.0.6
dashboard: node-red-dashboard v.2.21.0

The 3 "missing" values using the following nodes from:
node-red-contrib-fritzapi - 0.2.11
node-red-contrib-fritz - 1.3.9

EDIT2 - recognized every new session on node-red Dashboard reproduce this problem

I think to help yourself, you are going to have to export the part of the flow which displays the message/data on the dashboard.

I think to help yourself, you are going to have to export the part of the flow which displays the message/data on the dashboard.

[{"id":"41524b0c.38dcd4","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"c8c6b900.829f5","type":"fritzbox-in","z":"41524b0c.38dcd4","device":"49e6695a.2a6fb","name":"Fritzbox 7590","service":"urn:schemas-upnp-org:service:WANIPConnection:1","action":"GetStatusInfo","arguments":"{}","x":320,"y":120,"wires":[["9ec32405.da51a","1dbcb2b6.731ced"]]},{"id":"3efa8652.e4ae82","type":"inject","z":"41524b0c.38dcd4","name":"MAC address","topic":"","payload":"{\"NewMACAddress\":\"44:XX:XX:XX:XX:89\"}","payloadType":"json","repeat":"900","crontab":"","once":true,"onceDelay":"11","x":120,"y":119,"wires":[["c8c6b900.829f5"]]},{"id":"f72adc7c.f5fa2","type":"function","z":"41524b0c.38dcd4","name":"prepare Status info for dashboard","func":"var WANstatus = {};\n\nvar engtoger = function(translate){\nif (translate == \"Connected\"){\nreturn \"Verbunden\";\n  }else if (translate == \"Disconnected\"){\nreturn \"Nicht Verbunden\";\n  }else{\nreturn \"No data fetched\"; \n  }\n}\n\nvar color = function(colorizing){\n   if (colorizing == \"Connected\"){\n       return \"#3ADF00\"; \n   }else{\n       return \"red\";\n   }\n}\n\nWANstatus.topic = \"Status from Internet Connection\";\nWANstatus.payload = engtoger(msg.payload);\nWANstatus.color = color(msg.payload);\n\nreturn WANstatus;","outputs":1,"noerr":0,"x":840,"y":120,"wires":[["ea83d1bd.bf9a5"]]},{"id":"9ec32405.da51a","type":"change","z":"41524b0c.38dcd4","name":"change to msg.payload","rules":[{"t":"move","p":"payload.NewConnectionStatus","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":550,"y":120,"wires":[["f72adc7c.f5fa2"]]},{"id":"ea83d1bd.bf9a5","type":"ui_template","z":"41524b0c.38dcd4","group":"a17f3bc6.017e98","name":"Status Info","order":1,"width":6,"height":1,"format":"<style>\n    span.right {\n    display:table;\n}\n</style>\n\n<div layout=\"row\" layout-align=\"start center\">\n  <span flex>&nbsp;Status:</span>\n  <span flex class=\"right\"><b><font color={{msg.color}}>{{msg.payload}}&nbsp;</font></b></span>\n</div>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":false,"templateScope":"local","x":1110,"y":120,"wires":[[]]},{"id":"1dbcb2b6.731ced","type":"change","z":"41524b0c.38dcd4","name":"change to msg.payload","rules":[{"t":"move","p":"payload.NewUptime","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":550,"y":179,"wires":[["a55df022.847438"]]},{"id":"a55df022.847438","type":"function","z":"41524b0c.38dcd4","name":"prepare Status info for dashboard","func":"var wantime = {};\nvar totalseconds = Number(msg.payload);\n\nNumber.prototype.toHHMMSS = function() {\n  var hours = Math.floor(this / 3600) < 10 ? (\"00\" + Math.floor(this / 3600)).slice(-2) : Math.floor(this / 3600);\n  var minutes = (\"00\" + Math.floor((this % 3600) / 60)).slice(-2);\n  var seconds = (\"00\" + (this % 3600) % 60).slice(-2);\n  return hours + \":\" + minutes + \":\" + seconds;\n}\n\nwantime.topic = \"Converted time to HH:MM:SS\";\nwantime.payload = totalseconds.toHHMMSS();\n\nreturn wantime;\n","outputs":1,"noerr":0,"x":840,"y":179,"wires":[["f607c38b.162c58"]]},{"id":"f607c38b.162c58","type":"ui_template","z":"41524b0c.38dcd4","group":"a17f3bc6.017e98","name":"connected since","order":2,"width":6,"height":1,"format":"<style>\n    span.right {\n    display:table;\n}\n</style>\n\n<div layout=\"row\" layout-align=\"start center\">\n  <span flex>&nbsp;Verbunden seit:</span>\n  <span flex class=\"right\"><b>{{msg.payload}}&nbsp;</b></span>\n</div>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":false,"templateScope":"local","x":1120,"y":179,"wires":[[]]},{"id":"d2253ce4.91ae28","type":"comment","z":"41524b0c.38dcd4","name":"Issue 1 (MAC address masked)","info":"","x":220,"y":60,"wires":[]},{"id":"be089e9a.60bc78","type":"fritzbox-in","z":"41524b0c.38dcd4","device":"49e6695a.2a6fb","name":"guest wifi status","service":"urn:dslforum-org:service:WLANConfiguration:3","action":"GetInfo","arguments":"{}","x":260,"y":420,"wires":[["d1b32c6e.12e39"]]},{"id":"1d577308.5d0da5","type":"inject","z":"41524b0c.38dcd4","name":"","topic":"","payload":"true","payloadType":"bool","repeat":"1800","crontab":"","once":true,"onceDelay":"11","x":90,"y":420,"wires":[["be089e9a.60bc78"]]},{"id":"d1b32c6e.12e39","type":"change","z":"41524b0c.38dcd4","name":"NewStatus -> payload","rules":[{"t":"move","p":"payload.NewStatus","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":420,"wires":[["36c0ab4c.34fff4"]]},{"id":"36c0ab4c.34fff4","type":"function","z":"41524b0c.38dcd4","name":"check enabled or not (button)","func":"if (msg.payload == \"Up\")\n{\n    delete msg.topic;\n    msg.payload = \"0\";\n    msg.background = 'green';\n    msg.label = 'Deaktivieren';\n}\nelse if (msg.payload == \"Disabled\")\n{\n    delete msg.topic;\n    msg.payload = \"1\";\n    msg.background = '#b30000';\n    msg.label = 'Aktivieren';\n}\nelse\n{\n    delete msg.payload; \n    delete msg.topic; \n    msg.background = '#B55A00';\n    msg.label = 'NO DATA';\n}\n\nreturn msg;","outputs":1,"noerr":0,"x":750,"y":420,"wires":[["2b3a8265.c0fb66"]]},{"id":"1a227726.dce3f9","type":"change","z":"41524b0c.38dcd4","name":"","rules":[{"t":"set","p":"background","pt":"msg","to":"green","tot":"str"},{"t":"set","p":"label","pt":"msg","to":"Deaktivieren","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"0","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":720,"y":479,"wires":[["2b3a8265.c0fb66"]]},{"id":"4978af67.7629c8","type":"function","z":"41524b0c.38dcd4","name":"Button switch","func":"var state = context.state || false;\nstate = !state;\ncontext.state = state;\nif (state) { return [msg,null]; }\nelse { return [null,msg]; }\n","outputs":2,"noerr":0,"x":510,"y":486,"wires":[["1a227726.dce3f9"],["23573eed.45e5ba"]]},{"id":"23573eed.45e5ba","type":"change","z":"41524b0c.38dcd4","name":"","rules":[{"t":"set","p":"background","pt":"msg","to":"#b30000","tot":"str"},{"t":"set","p":"label","pt":"msg","to":"Aktivieren","tot":"str"},{"t":"set","p":"payload","pt":"msg","to":"1","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":720,"y":520,"wires":[["2b3a8265.c0fb66"]]},{"id":"2b3a8265.c0fb66","type":"ui_template","z":"41524b0c.38dcd4","group":"3e295110.f2bf86","name":"Gast button","order":3,"width":3,"height":2,"format":"<md-button style=\"background-color: {{msg.background}};\" class=\"button\" ng-click=\"send({payload: msg.payload})\">\n   <!--<md-tooltip>Ein- und Ausschalten des Gäste-WLAN</md-tooltip>-->\n   <span class=\"fa fa-wifi\" aria-hidden=\"true\"></span>\n   {{msg.label}}\n</md-button>","storeOutMessages":true,"fwdInMessages":false,"resendOnRefresh":false,"templateScope":"local","x":1010,"y":420,"wires":[["4978af67.7629c8","21f62e23.7e9e1a","f9a042f6.57df38"]]},{"id":"7f51392f.a58d08","type":"fritzbox-in","z":"41524b0c.38dcd4","device":"49e6695a.2a6fb","name":"enable guest wifi","service":"urn:dslforum-org:service:WLANConfiguration:3","action":"SetEnable","arguments":"{\"NewEnable\":\"value\"}","x":1290,"y":500,"wires":[[]]},{"id":"21f62e23.7e9e1a","type":"change","z":"41524b0c.38dcd4","name":"payload -> NewEnable","rules":[{"t":"move","p":"payload","pt":"msg","to":"payload.NewEnable","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1070,"y":500,"wires":[["7f51392f.a58d08"]]},{"id":"49b0c968.d05388","type":"ui_toast","z":"41524b0c.38dcd4","position":"bottom right","displayTime":"10","highlight":"red","sendall":true,"outputs":0,"ok":"OK","cancel":"","raw":false,"topic":"Gast WLAN","name":"","x":1290,"y":360,"wires":[]},{"id":"f9a042f6.57df38","type":"change","z":"41524b0c.38dcd4","name":"","rules":[{"t":"change","p":"payload","pt":"msg","from":"1","fromt":"str","to":"erfolgreich aktiviert !","tot":"str"},{"t":"change","p":"payload","pt":"msg","from":"0","fromt":"str","to":"erfolgreich deaktiviert !","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":1020,"y":360,"wires":[["49b0c968.d05388"]]},{"id":"ed92c4ed.0bea8","type":"comment","z":"41524b0c.38dcd4","name":"Issue 2","info":"","x":120,"y":300,"wires":[]},{"id":"7a62be27.1155","type":"ui_template","z":"41524b0c.38dcd4","group":"d5db950f.db284","name":"Soll Wert","order":4,"width":3,"height":1,"format":"<style>\n    span.right {\n    display:table;\n}\n</style>\n\n<div layout=\"row\" layout-align=\"start center\">\n  <span >(Soll:&nbsp;</span>\n  <span class=\"right\"><b>{{msg.payload}}</b>)</span>\n</div>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":false,"templateScope":"local","x":1100,"y":820,"wires":[[]]},{"id":"7fb6da94.f36b94","type":"function","z":"41524b0c.38dcd4","name":"prepare Status info for dashboard","func":"var targettemp = {};\n\nvar engtoger = function(translate){\nif (translate == \"on\"){\nreturn \"Boost\";\n  }else if (translate == \"off\"){\nreturn \"Aus\";\n  }else{\nreturn msg.payload + \" °C\"; \n  }\n}\n\ntargettemp.payload = engtoger(msg.payload);\n\nreturn targettemp;","outputs":1,"noerr":0,"x":820,"y":820,"wires":[["7a62be27.1155"]]},{"id":"f2ccd080.5794f8","type":"fritz-thermostat","z":"41524b0c.38dcd4","connection":"ef4f9b3b.e9564","name":"Schlafzimmer","action":"getTemperature","x":530,"y":760,"wires":[["84d00bb1.410648"]]},{"id":"3a313290.c4046e","type":"change","z":"41524b0c.38dcd4","name":"payload -> ain","rules":[{"t":"move","p":"payload","pt":"msg","to":"ain","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":320,"y":760,"wires":[["f2ccd080.5794f8","f35dd180.b90dc"]]},{"id":"84d00bb1.410648","type":"ui_gauge","z":"41524b0c.38dcd4","name":"","group":"d5db950f.db284","order":1,"width":6,"height":6,"gtype":"gage","title":"Temperatur","label":"°C","format":"{{value}}","min":0,"max":"30","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":710,"y":760,"wires":[]},{"id":"f35dd180.b90dc","type":"fritz-thermostat","z":"41524b0c.38dcd4","connection":"ef4f9b3b.e9564","name":"Schlafzimmer","action":"getTempTarget","x":530,"y":820,"wires":[["7fb6da94.f36b94"]]},{"id":"54c31d49.fe6e7c","type":"inject","z":"41524b0c.38dcd4","name":"Schlafzimmer","topic":"","payload":"XXXXXXX284","payloadType":"str","repeat":"600","crontab":"","once":true,"onceDelay":"11","x":120,"y":760,"wires":[["3a313290.c4046e"]]},{"id":"38d209b8.8a77c6","type":"comment","z":"41524b0c.38dcd4","name":"Issue 3 (AIN masked)","info":"","x":190,"y":660,"wires":[]},{"id":"49e6695a.2a6fb","type":"fritzbox-config","z":"","name":"Sascha Fritzbox","host":"fritz.box","port":"49443","ssl":true},{"id":"a17f3bc6.017e98","type":"ui_group","z":"","name":"Internet","tab":"ed44971.11cb4e8","order":8,"disp":true,"width":"6","collapse":false},{"id":"3e295110.f2bf86","type":"ui_group","z":"","name":"Gast WLAN","tab":"ed44971.11cb4e8","order":7,"disp":true,"width":"5","collapse":false},{"id":"d5db950f.db284","type":"ui_group","z":"","name":"Heizung Schlafzimmer","tab":"ed44971.11cb4e8","order":5,"disp":true,"width":"6","collapse":false},{"id":"ef4f9b3b.e9564","type":"fritz-api","z":"","name":"Sascha Fritzbox","host":"https://fritz.box","strictSSL":false},{"id":"ed44971.11cb4e8","type":"ui_tab","z":"","name":"Home","icon":"home","order":2,"disabled":false,"hidden":false}]

Here you have the 3 flows - which I have issues with
Note: MAC Address and AIN address are masked

Sascha,

Thanks, but the whole thing falls apart when I don't have these...... fritzbox nodes.

What I suggest you do is put a debug node after those nodes and set it to show the complete message.

That will go a long way to showing you what is going on.

If the messages being sent by these fritzbox nodes doesn't always have the parts needed, it will fail.

Give that a go and when you get/see their outputs, post them here and we may get further down the track.

Hi thanks for your help - but please watch the details of the dashboard nodes (i.e. template nodes)...
The flow itself working 100%

Its only these 3 values dont show anything within 15 minutes (other values otherwise will!) - only if the next interval triggers the flow - then values come up!

for example: look at issue 1 in my exported flows, there u have "connected since" and "status info"...
and then whatch my screenshot (first post)

status (screenshot) -> Status info (template node)
"Verbunden seit" (screenshot) -> connected since (template node)

IF the flow wont work then I have no values on both in screenshot :smiley: sorry but not my issue :smiley:

It may not be your issue but you are the one who is saying it isn't working.

So it kind of is your issue.

I didn't keep your flow as it is meaningless to me, as I did say.

Yes, screen shots when it fails may be handy, but there again, no promises.

It may be worth researching how those nodes work. What they put out and when.

If they only put out information which is different to last time (just an example) it won't work if invoked when nothing has changed since last time.

sure - know what you mean - but trust me debugged 6hrs and more - the flows are working fine on debug nodes
dont know how to show...

like I said the values are setup for refreshing every 15 minutes...
I visiting NR Dashboard first time when machine is up no value - waited for next refresh interval value up !

If something is horribly wrong with nodes or flow itself then I think i should never get an value (regardless of refresh time.

For Me it seems its more of an caching problem or MAYBE anything in node-red itself is limited for preventing any leaks or so, i guess
I am no developer but thats whats my feeling about

Oh, ok. Then maybe the needed value isn't in the original message.......

Again, you need to look at the messages.

Suggestion:

(Needs a bit of skill)

Write the messages to a file so you can see what the values are originally and what they become.
(Storage/file)
I think the node is standard in NR.

So connect the node to the output of the fritzbox node and set the node to write to a file you can find easily.

Test it to make sure it works. (Stick an inject node to inject a test message into the file node.)

Inject it and check the file has what was injected.

If it works, reboot and then see what is being sent out of the node.

That is the best I can suggest at this point as it seems that without knowing what is being sent: you are stuck.

Again I have all my values internally - but wont show on dashboard.
dont know to explain that :smiley:

but anyway maybe I can just reboot the machine where node-red is located at ?
maybe something has hickups - but dont have any hope for work ^^

EDIT: rebooted my node-red machine without luck - but now 4 values are missing on new browser session instead of 3 :confused:

Sascha,

It would seem this is beyond my skill set. Sorry.

@MisterT Sascha, since your data comes out of the fritz devices, no one who doesn't have those devices can run your flow. That said if you attach a debug node (set to display the "Complete msg object") you could provide that msg.payload and try putting it info an inject node to run thru your flow. That way you could make sure the flow is working the way you expect.

You could also provide the debug information so someone looking at your issue has the data to test it with.

also what happens if you decrease the Inject once after nn seconds, then option to 1 second?

Hm Yea I know @zenofmud - but I dont think the flow is the Problem - The flow is working since 6 month or so without touching it ? I know some updates can kill...

But lets try another way - maybe my Issue is more clear:

I already have a open tab with active NR dashboard right ?
Now all values are there and nothing is missing ok ? (Also status from screenshot above - value is visible)

Now... I open a second (new) firefox Tab and visit my NR dashboard now 4 values is missing (Status from screenshot above also)...
When I go to first tab again: I see all values - then going back to last opened browser tab: values missing...

Like I said I dont think the flow itself is the problem... seems something like weird cacheing stuff is going on ?
I mean why are 3 values (now 4) gone if I open a new tab and the other (old) tab all values are still there ? :wink:

You could try and cache the events server side in the context of a function node and resend them when an ui control node registers a connect event. But this seems like a stop gap measure.
To further debug this you could connect something like https://flows.nodered.org/node/node-red-contrib-flogger to the output of the avm nodes and after your functions/change nodes to see if maybe sometimes they send an empty value, an unexpected key or something else that could cause this.
Just a few ideas.
Johannes

Actually i think I know what happens. You use a template node to display your data. This template node will not if im right cache anything. So it will be empty if you open a new session. That is until a value gets send from the avm nodes.
So yes I think you will either have to trigger a request to the avm with a ui control on connect event or use the function node caching i described above.

Yes I will test this - but another question maybe it has something to do with my issue also ?

last few days (last time where I updated my node-red plugins and suff)...
npm told me to do a "npm audit fix" do solve vulnerabilities...

I did that but I dont know anymore the exact output from it - The Only one what I can remember was:
"npm audit fix" -> fixed 8 packages because of vulnerabilities in package "minimist" or something like that ?
and all these 8 packages was from node-red dashboard and plugins

So you think the npm audit fix was not the prob ?

Nope. Just tested.
Have a look at this:

[{"id":"76fe5319.039fa4","type":"ui_template","z":"7df8e457.799894","group":"f789b1cf.67e25","name":"Status Info","order":1,"width":6,"height":1,"format":"<style>\n    span.right {\n    display:table;\n}\n</style>\n\n<div layout=\"row\" layout-align=\"start center\">\n  <span flex>&nbsp;Status:</span>\n  <span flex class=\"right\"><b><font color={{msg.color}}>{{msg.payload}}&nbsp;</font></b></span>\n</div>","storeOutMessages":true,"fwdInMessages":true,"resendOnRefresh":false,"templateScope":"local","x":950,"y":760,"wires":[[]]},{"id":"b76ecafb.55799","type":"inject","z":"7df8e457.799894","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":360,"y":760,"wires":[["b55e7fa1.972a78"]]},{"id":"b55e7fa1.972a78","type":"function","z":"7df8e457.799894","name":"red","func":"msg.color = \"red\";\nreturn msg;","outputs":1,"noerr":0,"x":510,"y":760,"wires":[["7d3fdbd5.2cc544"]]},{"id":"7d3fdbd5.2cc544","type":"function","z":"7df8e457.799894","name":"cache last message","func":"let lastValue = {};\nif(msg.payload == \"resend\"){\n    lastValue = context.get(\"lastValue\") || \"no message yet\";\n    msg = lastValue;\n    return msg;\n} else {\n    context.set(\"lastValue\", msg);\n    return msg;\n}","outputs":1,"noerr":0,"x":720,"y":760,"wires":[["76fe5319.039fa4"]]},{"id":"547c244.7da125c","type":"ui_ui_control","z":"7df8e457.799894","name":"","events":"all","x":320,"y":820,"wires":[["a5761408.725d08"]]},{"id":"a5761408.725d08","type":"change","z":"7df8e457.799894","name":"resend","rules":[{"t":"set","p":"payload","pt":"msg","to":"resend","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":470,"y":820,"wires":[["7d3fdbd5.2cc544"]]},{"id":"f789b1cf.67e25","type":"ui_group","z":"","name":"Internet","tab":"1ec6f22f.d0591e","order":8,"disp":true,"width":"6","collapse":false},{"id":"1ec6f22f.d0591e","type":"ui_tab","z":"","name":"Home","icon":"home","order":2,"disabled":false,"hidden":false}]

something like this should solve the problem. Look at the cache last message function and the ui control and change node.

1 Like

Thanks for great work - but what I am wondering: "Verbunden seit" is also a template not and there I have no problems with missing values ? so why ? confusing

is also in my exported flow above...

There is one pretty new feature for ui_template node which should affect behaviors about the area your issue seems to be.
Check for it's configuration for every ui_template you have

2 Likes

you have to do a little change to the cache function in the get line. There was an error when there was no message yet, should be like this:

let lastValue = {};
if(msg.payload == "resend"){
    lastValue = context.get("lastValue") || {payload:"no message yet",color:"orange"};
    msg = lastValue;
    return msg;
} else {
    context.set("lastValue", msg);
    return msg;
}

Dont know about that one.
I would really probably do some long time logging to see if the message from the avm node always include the expected keys in the object.