I have a scenario with multiple Raspberry Pi using an Aeotec Gen5 USB stick to communicate with Z-Wave devices, with some of them close enough to be in range of devices from other sticks. Each one of them has a preconfigured set of attributes based on the tuple [nodeid,cmdclass,instance,cmdidx].
Periodically I send a refreshValue command for each attribute and, as long as the value has changed enough from the previous notification, the device publishes the new value (I can see a message sent to the zwave: value changed topic). In the end, while everything works ok, I have a good-enough value (might not be 100% accurate) that I can forward to my server.
However sometimes I get no response at all. I mean, no matter if I restart nod-red or even the whole Raspberry Pi, all I get are timeout notifications. From my understanding, it'd make sense if the devices were too far (not the case) or multiple networks were interfering with each other, but I think each stick should create a unique network and, once the devices are paired with the stick, they should live in their own world so to speak.
At this point I've tried to force a network healing (by injecting a healNetwork command) with no success. Although I can't tell for sure, I think I'm not receiving the scan complete message in some of these cases as there is still no communication with all the devices, and I'm not sure if trying to send the requests (refreshValue) before the scan complete is received might be breaking something.
So, which is the right sequence that must be followed in this case? What if I don't get a scan complete message at all?
Hi @Mamonetti!
What is the kind of devices you are trying to interview?
Are those battery powered or mains powered?
If battery powered, you might consider that the devices are (usually) sleeping ... so you need to wake them up & keep them awake to perform the interview or a network heal.
BR, R.
BTW #1: I've no experience w/ nodered-contrib-openzwave, yet can confirm that node-red-contrib-zwave-js works like a charm.
BTW #2: What version of nodered-contrib-openzwave are you working with? There seem to be severals in the flows library...
Hate to hijack a thread. But a major update (V7) of my ZWave Node is tirelessly being worked on.
haven't been around for sometime. - so thought i'd mention it.
Working closely with the zwave protocol, how close a device is (whilst helps) isn't always a 'its close, it should work' get out clause - Network heals also.
Network heals can take hours/days potentially. - especially if battery operated (see below).
and whilst Network Heals are great at fixing most communication problems, they are certainly not always the answer.
Has there been any movement in the surrounding area (stick end or device end) - metallic surfaces as an example?
Like above - if these are battery operated devices, they sleep, and only submit values when needed.
Battery devices wake up on a schedule, to receive config updates, submit battery reports.
Triggering a device is not waking it up.
a motion detector being triggered, doesn't mean It will fetch any pending updates, or is prepared to get healed, they need to be woken up via a routine of some sort, or wait for the wake up interval.
Querying a device mid heal, should not cause an error, as messages should hopefully be managed by the module in question - but I can't answer that one.
I understand that waiting for these devices to communicate might take some time but I expect to see some activity sooner or later. The problem is that in some of my rbpis, that's not happening for too long (a week or more).
The most relevant problem I can think of here has to do with actuation as I won't be able to update values (when actuation is supported) whenever I want. Actually it's hard to tell when the setValue command must be sent as devices might go back to sleep right after sending their value giving me no time to react and send my update. It's hard to say what to do in this specific case TBH.
Ok, I've made a couple simple tests with Z-Wave JS and attached I have a simple flow where I've been communicating with an Aeotec Smart Switch 6.
After that, I have some questions:
The obvious one, do I have to wait for the ALL_NODES_READY event before trying to communicate with the devices? I guess the answer is yes, but the question has to do with whether I might generate some issue in the controller if I try to send something for some reason before having received this event.
As you know, after having reached the CMD CLASS level inside a node, devices provide one or more instances (3-phase power meters usually have 1 for the global consumption and 3 more, one per phase) and with each one of them providing different properties such as energy, power, etc. How can I get (or set, depending on the device) the Xth property of the Yth instance? I've seen the concept of endpoint but that's not enough to map all the properties. In my case all the properties have endpoint=0 and what seems to be different is the propertyKey (65537 for energy and 66049 for power), but it looks like adding this attribute doesn't work and I keep receiving the energy while I'm trying to get the power:
Should I use the forceUpdate option in other to update battery-powered devices? How long is the controller going to be waiting for the device to wake up before dropping the request? What if I send several requests before the device actually wakes up? I'd expect for the controller to overwrite pending updates every time you send a new one.
Considering I intend to use the CC API instead of the Value API, which is the method in the CC API equivalent to pollValue? And similar to the previous question, what if it takes too long before the device wakes up and I send multiple requests to the controller? Is it going to queue them? For me it would make sense to drop new gets/polls if there's one pending but I'm not sure if that's how it works.
When you configure a Device Node in Specific Node mode, it forces you to choose one existing node and if you don't do that, it goes back to All Nodes mode. Although the idea is good, I'd find useful if I was allowed not to choose the node. I know I'd be forced to set the node property in the incoming message but that's the key point here, it would provide a more flexible solution (knowing that you'll get an error if you don't set the property). Or, to put it in a different way, what if I try to clone my flow after having chosen node X in the combo and it turns out there's no such a node paired with the controller in another node-red that I'm trying to deploy? Will I get a deploy error?
What is the responseThroughEvent property I've shown in the example above? I've copied from somewhere but I've no idea whether it's important or not.
Why do I keep receiving the error "TypeError: Cannot read properties of undefined (reading 'controller')" every time I send a get/set request? It's generated by the controller but the request is processed correctly (v6.5.5).
The obvious one, do I have to wait for the ALL_NODES_READY event before trying to communicate with the devices?
You won't implode anything - but yes, you really should
As you know, after having reached the CMD CLASS level inside a node, devices provide one or more instances (3-phase power meters usually have 1 for the global consumption and 3 more, one per phase)
Each ValueID represents exactly what cc, property, endpoint a value belongs to propertyKey is only used by the ValueAPI - I suggest using it - its much better.
See Here
The updates you get contain the ValueID also - commandClass, Property, Endpoint (and sometimes propertyKey)
The CCAPI is VERY low level - so it does not have abstraction
Should I use the forceUpdate option in other to update battery-powered devices? How long is the controller going to be waiting for the device to wake up before dropping the request? What if I send several requests before the device actually wakes up? I'd expect for the controller to overwrite pending updates every time you send a new one.
Some Devices DO NOT report an updated value, when its changed by the user.
Keeping this in mind.....
When Using CCAPI - no attempt is made to 'confirm' it has changed - again : The CCAPI is VERY low level.
However, some users do prefer the CCAPI Whilst at the same time, have devices that don't report an updated value - forceUpdate tries to mimic the ValueAPI - why? the ValueAPI does all the hard work of confirming values have been updated - where as the CCAPI (low level API) naturally does not.
The controller and / or driver (ZWave JS) has an outgoing queue, that will empty when the device wakes up, this queue lives as long as the process is running.
forceUpdate is not needed for ValueAPI
In a nut shell: forceUpdate will poll a value, after setting a value when using CCAPI - to mimic the behaviour of the ValueAPI - but is not as robust as the ValueAPI
See here:
When you configure a Device Node in Specific Node mode, it forces you to choose one existing node and if you don't do that, it goes back to All Nodes mode. Although the idea is good, I'd find useful if I was allowed not to choose the node. I know I'd be forced to set the node property in the incoming message but that's the key point here, it would provide a more flexible solution (knowing that you'll get an error if you don't set the property). Or, to put it in a different way, what if I try to clone my flow after having chosen node X in the combo and it turns out there's no such a node paired with the controller in another node-red that I'm trying to deploy? Will I get a deploy error?
you won't get an error during deploy - it just won't receive anything, and will respond with an invalid node message when trying to send something it.
if you want to set the node later - you can use As Specified
What is the responseThroughEvent property
Some GET methods return directly from the call you make, some are returned from an event - which is detached from your executed method, so for us to "wait" for the returned value (as it may not come as an event) we need to say responseThroughEvent: false (default is true)
this is not needed for CCAPISET methods or the ValueAPI
See here:
Why do I keep receiving the error "TypeError: Cannot read properties of undefined (reading 'controller')" every time I send a get/set request? It's generated by the controller but the request is processed correctly (v6.5.5).
I am not aware of any issues with this message currently.
could you create a test flow for me?
However I'd add a couple comments regarding the documentation:
There's no reference on the wiki about where you can get the valueId and its format (at least I can't find it).
Based on my tests, some attributes are optional because there are multiple ways to add them and therefore you can actually reduce the size of the json object. For example you can choose between commandClass or commandClassName (or both), same with property and propertyName and so on.
This extra documentation would definitely be helpful.
That's exactly my point. How can you confirm that an update of the value has been received by the device? I don't know if this functionality is provided by the Z-Wave protocol itself and I can think of situations where even if you modify a value (let's say you switch something on) and try to read it later, someone has locally switched the device off again before you actually send the read request (either manually or automatically inside the driver) and the system might enter a loop trying to force the switch-on operation again and again.
But then what would happen if multiple updates were sent from a node? Would all of them be queued and later actually sent to the device? Depending on the scenario, and speaking of battery-powered devices, this might end up becoming a problem as the flow may add tons of updates that are useless without knowing the status of this queue. From my perspective, and even though it requires some extra work, each time a new update is added to the queue, I'd look for another pending one that's related to the same device/property in order to overwrite it whenever it's possible, but that's just an improvement, I know.
In the end that would mean you'd never have 2 updates for the same device/property in the queue.
Ok, As Specified is the option I was looking for.
Then responseThroughEvent = true (default) is the option that provides the behavior I was expecting to get.
Hmm.. truth to be told, I can't replicate this error . After having removed the non-responding nodes from the stick's memory, the error is gone, and even if I try to go back to this same situation (pair a new node so that the stick knows about it but switch it off so that a failure in the communication is detected), the error doesn't show up again.. If I find this problem again, I'll save the flow and I'll post the issue.
There's no reference on the wiki about where you can get the valueId and its format (at least I can't find it).
Noted
Based on my tests, some attributes are optional because there are multiple ways to add them and therefore you can actually reduce the size of the json object. For example you can choose between commandClass or commandClassName (or both), same with property and propertyName and so on.
At least commandClass & property are required, endpoint & propertyKey are dependent on target property / device, Z-Wave JS may use just the name if it's the only thing provided - I'm not entirely sure.
See here:
That's exactly my point. How can you confirm that an update of the value has been received by the device?
If using the CCAPI you won't, Unless the device responds with an unsolicited Value update after.
The ValueAPI will (when setting a value):
SetValue -> Wait for a few ms (after the ACK)
-------Time----->Unsolicited Received -> You get the VALUE_UPDATED event
-------Time----->No Unsolicited Received -> ZWave JS wil poll the value -> You get the VALUE_UPDATED event.
For the system to respond promptly, it will optimistically give you a VALUE_UPDATED event after
the ACK - this can be disabled (at the cost of the slight delay, whilst the value is first checked)
the controller setting is: No Optimistic
VALUE_UPDATED will be sent to your flow in either case
this should be enough to let you know the value has been set
Would all of them be queued and later actually sent to the device?
Currently yes - The dev behind Z Wave JS has a task to 'void' the first queued message, for the same ValueID
EDIT
There's no reference on the wiki about where you can get the valueId and its format (at least I can't find it).
So the ValueAPI guarantees you that, once the device wakes up (it may take time), the update will be sent. That's certainly useful as higher layers can focus on their stuff after sending an update request, although they should have to wait for the VALUE_UPDATED event in case they needed to work asynchronously without moving forward before the ACK has been confirmed (in the form of an event).
If I understood you correctly, all the new updates will be queued but every time you send a new one and there's another one active for the same ValueID (the scenario I was describing in my previous post), this older entry will remain in the queue but won't be sent as it'll be voided. That's equivalent to removing it before adding the new one, am I right?
However this may lead the queue to have a big amount of useless entries especially if the target device is offline. To be honest, I don't see why these old updates aren't directly removed but maybe I just misunderstood your comment.
So the ValueAPI guarantees you that, once the device wakes up (it may take time), the update will be sent. That's certainly useful as higher layers can focus on their stuff after sending an update request, although they should have to wait for the VALUE_UPDATED event in case they needed to work asynchronously without moving forward before the ACK has been confirmed (in the form of an event).
That's correct, you will get VALUE_UPDATED when:
The device produces an unsolicited update
As a result of the user changing a value (if the device is kind enough to do so)
A naturally occurring event on the device (air temp changes as an example)
The driver getting fed up of waiting for the unsolicited change to come back (therefore polling it)
The ACK being received (if opted to get optimistic updates - which we do by default)
Sometimes, duplicated VALUE_UPDATED events are received 1xOptimistic, 1xUnsolicited.
hence why disabling optimistic is favoured.
optimistic disabling was added to the driver as some UI's (HA for instance), needed very quick turn arounds, but then it wasn't needed by all UI's
If I understood you correctly, all the new updates will be queued but every time you send a new one and there's another one active for the same ValueID (the scenario I was describing in my previous post), this older entry will remain in the queue but won't be sent as it'll be voided. That's equivalent to removing it before adding the new one, am I right?
Sorry I didn't explain this one well.
Currently ALL messages will be sent (Evan for a change on the same ValueID)
The developer has a task at hand to change this, In that any set message for a valueID that already has a queued set, will replace the old message.
This has been discussed with various frontends using ZWave JS
Delayed updates are always a problem as you might have queued a command that was valid the moment you sent it but not some time later. Replacing the old message looks like a good idea for me also from this point of view.
Your high-level module, as long as it's aware of the current value and the last command sent, can always undo an operation by overwriting the last command sent to the queue. So if you queued a switch-on some time ago and you notice that right now it makes no sense, you just need to add a switch-off and, assuming the device was off at the beginning of the exchange, nothing wrong will happen (your device will receive a switch-off when it was off, nothing more) but your system should remain stable.
Knowing how Z-Wave JS works underneath, I guess I'll create my own queue to replicate this behavior at least until it's operational. And knowing that it's just a matter of time before it's ready, I'll add a property in my config file to enable/disable my own queues.
This module is a direct passthrough to ZWave JS (well... pretty much )
It has no 'out-of-band' storage/message queuing, as its not needed. - ZWave JS manages all internals of messaging and storage.
Node RED Flow -> setValue -> ZWave JS -> Magic
Magic -> ZWave JS -> Node RED Flow.
Node RED Flow -> getValue -> ZWave JS -> Magic -> Node RED Flow
Evan the values in the UI are fetched from ZWave JS before being displayed.
the values held by ZWave JS are synced as and when devices tell us of updates - and they survive reboots.
Now that I think about it, it would be great to have the option to export the configuration of a device from the controller's panel including all the relevant information for the node.
Next there's an example of the configuration of a node including a subset of the options that should be included (just a few ValueID, but the idea is to include all of them). There's no need to use this specific format but I'm sure you know what I mean.
In fact, I'd also add the option to export the configuration of the full network which would generate an array of objects following the previous format, that is:
[
{
"node": 14,
...
},
...
{
"node": 21,
...
}
]
In my case I have my own json config file that filters the data received from devices and knowing which information is actually available would help quite a lot. Right now I'm using something like that based on the xml file generated by openzwave when you send a writeConfig command to a Z-Wave output node.
Note: The output format has changed in V7 for getValueDB and a few other changes for other events.
please see the upcoming change log - if you are planning to sculpture around the current format namely getValue and getValueMetadata
Ok, I understand. I'll try these commands you've suggested.
And regarding V7, when is the expected release date? Based on the changelog and considering that my first goal is to use get/set commands and receive events, I think I can live with the current version and maybe I'll take a look to the new format of getValueDB if it ends up providing what I'm looking for.
. After having removed the non-responding nodes from the stick's memory, the error is gone, and even if I try to go back to this same situation (pair a new node so that the stick knows about it but switch it off so that a failure in the communication is detected), the error doesn't show up again.. If I find this problem again, I'll save the flow and I'll post the issue.
)
