[Announce] node-red-contrib-homie-convention

[Christian-Me]

update to version 0.0.2

Hello,
I released version 0.0.2-beta of a custom node to handle communication over mqtt via the homie convention.

node-red-contrib-homie-convention

This is my FIRST custom node! Any feedback appreciated.

goals

• Handling all communication
• making full use of the discovery functionality of the homie convention
• easy interfacing with the standard dashboard nodes using ui_control features
• Node-RED acts as a controller
• Concept of predicted state of parameters: Changes on actuators should be confirmed by the actuator. The predicted should be visible in the dashboard until the new value is confirmed.
• basic support of homie extensions $stats, $fw and $implementation
• ToDo: detecting offline devices
• ToDo: managing status updates
• ToDo: managing modified or deleted homie devices, nodes or parameters
• ToDo: make Node-RED acting like a homie device
• ToDo: full extensions support
• ToDo: bugfixes
• ToDo: code cleanup and streamlining

known issues

• mqtt security is not implemented yet (need some help to perhaps use the build in mqtt-config-node)
• sending messages to homie parameters using wildcards is implemented but not tested
• some config UI issues (lists sometimes not correctly refreshed lists). Close the config and open it again. Tried to limit getHTTP requests ...
• some format conversions (true/false, ON/OFF and others) might not working correctly
• make sure that most messages has to be sent as retained messages with QoS=2 according to the homie convention. Some devices can only sent QoS=0 and retained=true. Configure your broker to store these messages on non volatile memory. Keep this in mid if you use tools like the fantastic MQTT Explorer (The Explorer shows the values in the tree structure even if the don`t exists on the broker any more)
• Now tested and developed with Node-RED 1.0.x
• perhaps the node is a little bit overloaded ... perhaps split the dashboard part in a separate node? Or get the dashboard to do this naively?

For development I used the several ESP8266 running ESPEasy framework using the C014 & P086 plugins I developed too (all Beta!). I`m using mosquitto with some tweaks in the mosquitto.ini file to store the messages due to the limitation of the pubsub-library on the ESP8266 not able to send QoS=2 messages.

[kuema]

Nice work!

As soon as I am done wiring the rest of my new flat, this will be one of next items on the to-do list.
Besides Bart and Steve's (@BartButenaers and @Steve-Mcl) awesome SVG node and Julian's (@TotallyInformation) uibuilder, of course.

(edited by the wish of a single Bart )

[bohtho]

I really like how detailed and encompassing your node is. I’ve read up on Homie and see some benefits in my setup (states, autodiscovery, standardisation simplifying my other nodes, easily set up UI), but I haven’t taken the node to use yet.

Is there an estimate for when the usage of the native MQTT client, or node’s own client improved with authentication, would be testable?

[Christian-Me]

You are right, security is important, will work on this asap.

[bohtho]

I noticed in the "todo" that you considered interfacing with the "native" node-red MQTT client instead?

[Christian-Me]

There was a discussion a wile ago resulting in the conclusion (as I remember) that the mqtt node is not really designed for reuse in A custom node. So I have to role out my own. Basic username/password is done but too basic and more symbolic as username/password is transferred in plain text. For TLS I try to reuse the ssl/tls config node to make live a little bit easier. But first I have to find how and where it is implemented.

[kuema]

The TLS config node is provided by the core as tls-config, so you can use it like any other config node.

So I think all you need to do is just add an optional node property, like

tls: {type:"tls-config",required: false}

The core MQTT node is a good example.

[Christian-Me]

Thank you for pointing me in the right direction. Hope I can implement it over the weekend.

[Christian-Me]

So here it is ...

tested against mosquitto on windows with a self signed ca.crt uploaded to the tls-config node.

Perhaps somebody can test it directly form my github before i push the new version to npm? Thank you.

[bohtho]

That was a quick! I will only be able to test the simple authentication, and a bit later.

[Magnus_P]

I trying to integrate the Homey to the Node-RED using the MQTT Hub app that sends Homie Convention (v3.0.1).

The problem I encounter is the Node and Property in the Node-Red Homie Node is not retained as selectable items in the drop down list. To find a new device I need to broadcast all devices again from the MQTT Hub application in Homey, then they show up, for a short while!

Additional I have another problem with the Input to the Node-Red Homie Node, the value is not updated (have checked by MQTT Explorer), is this related to above if there is no defined Node definition?

[Christian-Me]

Hello,
thank you for using my node and sorry for the inconvenience that it is still beta.

From what you describe it looks like there is a problem with retained messages. First please take a look if MQTT Hub is sending retained messages as QoS>0 using mqtt explorer (or the tool of your choice). If not there is a solution to configure your broker of choice to handle QoS=0 messages with the retained flag set as persistent data. Take a look to this document on my github

Every restart the homie node starts with a fresh database. You should see information during the init process on the console / log. By design the Homie convention relies on the mqtt broker as a persistent database,

Hope this helps (I have to include this issue prominent in the documentation)

Chris

[Magnus_P]

Thank you for guiding me...I trying to check with MQTT Explorer if the messages is retained or not, which i belive they are, or?

[Christian-Me]

Nope you should see the orange flag on the right panel as in the screenshot

If you see this the QoS is not important any more. But check if the data survive a restart of your broker too.

[Magnus_P]

Seems the MQTT message not is set to retained - I have open a discussion topic in the Homey forum about this. https://community.athom.com/t/mqtt-hub-gateway/6766/703?u=magnus_p

[Christian-Me]

Take a look at the homie convention:

3.3 QoS and retained messages

The nature of the Homie convention makes it safe about duplicate messages, so the recommended QoS for reliability is QoS 1 . All messages MUST be sent as retained , UNLESS stated otherwise.

So if your device can’t do it you have to configure your broker to do it. Storing and saving QoS=0 Messages with retained flag is within the MQTT spec 3.3(?). The broker even „should“ keep those messages.

[Magnus_P]

Now I have moved over to another MQTT Broker on my Synology NAS drive and things starting to be much better .

I have another problem now... Everything works fine to import the data to Node-RED, the switches, buttons updates fine. But when I try to change a light status from Node-Red the correct Message is not sent. Only the set parameter is changing and anything else such the onoff property. Any Idea? I have configured switch according to below and connected this to the input

MQTT Explorer

Node-RED switch config

[Christian-Me]

Hi Magnus,
It seams like the homue implentation on the Homey side is bot correct. A command is transferred (i.e. from Node-RED) via the set topic and should be executed and then acknowledged via the property topic. Either your controller is not receiving the /set command or it is not responding properly:

From the Homie convention:

Property command topic

• homie / device ID / node ID / property ID / set : The device must subscribe to this topic if the property is settable (in case of actuators for example).

A Homie controller publishes to the set command topic with non-retained messages only.

The assigned and processed payload must be reflected by the Homie device in the property topic homie / device ID / node ID / property ID as soon as possible. This property state update not only informs other devices about the change but closes the control loop for the commanding controller, important for deterministic interaction with the client device.

To give an example: A kitchen-light device exposing the light node with a settable power property subscribes to the topic homie/kitchen-light/light/power/set for commands:

homie/kitchen-light/light/power/set ← "true"

In response the device will turn on the light and upon success update its power property state accordingly:

homie/kitchen-light/light/power → "true"

[Magnus_P]

Something is a little bit strange since this configuration works fine when using ordinary MQTT in/out nodes! Now I can both receive and send the commands where the light turn on/off correctly.

I will continuing to fault trace further…

[{"id":"d6672b1f.e58f48","type":"ui_switch","z":"f6a0cce1.578a1","name":"Light Switch","label":"Light Switch","tooltip":"","group":"af4d5500.1311d8","order":9,"width":"3","height":"1","passthru":false,"decouple":"true","topic":"homie/homey-topic/lucasfonster/onoff/set","style":"","onvalue":"true","onvalueType":"bool","onicon":"","oncolor":"","offvalue":"false","offvalueType":"bool","officon":"","offcolor":"","x":610,"y":740,"wires":[["d312f4e2.7a9058"]]},{"id":"d312f4e2.7a9058","type":"mqtt out","z":"f6a0cce1.578a1","name":"MQTT Transmitter","topic":"","qos":"","retain":"","broker":"4709a5ff.a5e08c","x":830,"y":740,"wires":},{"id":"c2b2758f.cc9f78","type":"mqtt in","z":"f6a0cce1.578a1","name":"MQTT receiver light","topic":"homie/homey-topic/lucasfonster/onoff","qos":"2","datatype":"auto","broker":"4709a5ff.a5e08c","x":150,"y":740,"wires":[["83ef5d25.1bb9f"]]},{"id":"83ef5d25.1bb9f","type":"function","z":"f6a0cce1.578a1","name":"Convert String to Boolean","func":"if(msg.payload === "true"){\n msg.payload = true; \n}else{\n msg.payload = false;\n}\nreturn msg;","outputs":1,"noerr":0,"x":390,"y":740,"wires":[["d6672b1f.e58f48"]]},{"id":"af4d5500.1311d8","type":"ui_group","name":"Group 4","tab":"30048f8.763b8f","order":4,"disp":true,"width":6},{"id":"4709a5ff.a5e08c","type":"mqtt-broker","z":"","name":"magnus_p","broker":"192.168.1.189","port":"1883","clientid":"Node-Red","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"30048f8.763b8f","type":"ui_tab","z":"","name":"Home","icon":"dashboard","order":1,"disabled":false,"hidden":false}]

[Christian-Me]

Good Morning from Berlin.

I have to dig a little bit deeper. I could not import your flow (there is something missing at the end and an error in between:

So perhaps you can "resend" both flows (homie version & "manual" version). You wrote that the homie/homey-topic/lucasfonster/onoff/set topic changes between true/false when you use the switch. So this should be the correct behavior in both versions.

A Boolean $datatype of the homie node should accept true/false, 0/1, "ON"/"OFF", "on"/"off", "true"/"false";

Side note: I personal use the "enum" $datatype for my DIY switches with $format "ON,OFF" because switches are normally not true or false. But this is because homie brings a high level of abstraction and do not want to be dependent on the device type. But this is more a cosmetic issue.