MQTT Node: Subscribing to `+/+/$homie` does not work?

Sorry but you have to do the mosquitto.conf modifications to work with QoS=0 and retained messages. You find hints in the comments of the mosquitto.conf - or in my readme.md

$state comes through because it is updates every 60seconds.

FYI accoring to the HIVEMQ site (https://www.hivemq.com/blog/mqtt-essentials-part-5-mqtt-topics-best-practices/0


Topics beginning with $

Generally, you can name your MQTT topics as you wish. However, there is one exception: __ Topics that start with a symbol have a different purpose.__ These topics are not part of the subscription when you subscribe to the multi-level wildcard as a topic (#). __The -symbol topics are reserved for internal statistics of the MQTT broker.__ Clients cannot publish messages to these topics. At the moment, there is no official standardization for such topics. Commonly, $SYS/ is used for all the following information, but broker implementations varies. One suggestion for $SYS-topics is in the MQTT GitHub wiki. Here are some examples:

$SYS/broker/clients/connected
$SYS/broker/clients/disconnected
$SYS/broker/clients/total
$SYS/broker/messages/sent
$SYS/broker/uptime


I wonder if that is part of the issue (or this might just be a red herring)

Don't think so because homie messages usually start with homie/deviceName and only on third level they (some) start with $

There are a couple of issues here:

  • There appears to be a bug in the homie esp library as it should be using QoS 1 I believe.
  • There is definitely a problem in Node-RED because all the other MQTT browsers happily show the result for +/+/$homie, only Node-RED doesn't.

Interesting! I've never had a problem with Homie convention previously and I've manually output topics with $ happily in the past.

Good point.

:sigh: never quite as easy as you want it to be is it!

I will check out the Mosquitto configuration to see if that helps. I may also try and run up a test MQTT subscription node.js server to see if I get the same issue with that.

I also see that MicroPython has an ESP Homie library and I wonder if that is worth a try, certainly there are a few limitations in the one I'm using via the Arduino IDE.

In terms of using Homie from Node-RED, the biggest limitations are:

  • The Homie convention nodes have no method for dynamically specifying the device, I certainly don't want to have to manually create configuration nodes for each device.

    I note that Homie is meant to be self-discoverable and the homie nodes don't support that.

  • There is no node to automatically subscribe to 1 or more Homie device topics and grab all of the sub-topics which would be massively useful and overcome one of the potential issues with Homie.

A lot of info:

This is not a bug ... its a feature. As RAM is limited there is no good way to implement QoS>0 as you have to store messages for resent as long as you don`t have confirmation. The pubsub library explicit states this.

Not sure, let's see what happens if you reconfigure mosquito. You don't mess up with the MQTT spec this way because the MQTT spec says the broker "should store" but "could drop" ... strange to use so many "possibility forms" in a spec where people relies on.

Really. I use "my" node quite dynamically (in the end you can use "[any]" to get "anything". Tell me if you need "anything" else :slight_smile: . Happy to improve the node:

image

You need more data?

I think I may not be understanding it properly then.

Let me start a new topic and ask you the question directly then you will understand what I want to do.

Thanks for everyone's help by the way. Great support as always.

You mean the root topic? But this is somehow fixed by the convention to homie (for use with autodetect)

image

The devise is defined by the node or [any] thing.

Didn't thought that subscribing to # and passing all data through the node is a good idea, perhaps I was wrong.

The device Name in the config node is only the name Node-RED exposes itself as a homie device

image

Fine - I developed a kind of passion to boost the homie convention ... Think it is worth to be used more to streamline the communication between all kind of devices. It could really simplify how fows can be exchanged.

Then I'm really glad I started this thread :grin: - I'm happy to feed your passion!

No, that's OK, I use "devices" as my root as that makes more sense. You will see from the new thread what I'm trying to achieve.

I think that was what I was confused about. I probably didn't read the docs properly.

I suspect that we may have some more conversations about the best use of the convention. I'll also be interested in your thoughts about best use of nodes vs properties. But I know I need to read up more about v4 of the convention - lets take things to the other thread.

Great to hear (or read) that Homie gets a new power user with Julian. :smile:

I'm in the process of moving my devices to Homie as well, bit by bit at least. My passion is for standards, patterns and generic solutions, and Homie fits quite well into my abstract world. :nerd_face:

1 Like

I like the idea of it & have implemented a slightly modified version of it but in the workplace, I simply cannot start topics with "homie".

Is the standard strict on this? Or does the standard permit alterative start topic?

1 Like

As I understood it, you can choose any base topic you want.

The root topic in this document is homie/ . If this root topic does not suit your needs (in case of, e.g., a public broker or because of branding), you can choose another.

From: https://homieiot.github.io/specification/ (point 3)

Thanks. Its been a while since I read that and was slightly put of when on another thread some time back I asked why this new node being developed was forcing the user to start all topics with "homie" (it was hard coded in the node) and the developer said it's a standard. (Or at least that was what I took from the response - off I go looking for that thread - to clear my head)

I guess I should have read the spec more closely.

Cheers.

Edit...
So this was the thread & now everything becomes clear. I missed one sentence from @Christian-Me response where he said "he would eventually make it configurable". I missed that (shame on me :slightly_frowning_face:)

It wouldn't be a very flexible convention if it was fixed to homie/. :sweat_smile:

The actual implementations would have to support changing the root topic, of course.

Like Julian, I'm actually using devices, but that's been that way years before homie. I had a similar approach to Homie regarding the topic structure. Now I'm just restructuring it to fit the Homie convention, so I can streamline everything. :nerd_face:

1 Like

Me too! Please see my recent comment on this in the other thread.

You do not have to use this and I am not. I am using devices as my root. The convention states that controllers don't have to be able to auto-discover the root - however, this is exactly what started this thread since the way to auto-discover the root is to subscribe to +/+/$homie, if you get anything back then you know that homie is in use and can work out the root.

Yes, my original use of Homie quickly ignored the conventions which were too restrictive for me at the time. So I have a very mangled version implemented on my current HA system.

The desire to re-work all of my slowly evolved system into something rather more neat and sensible is what kicked off this discussion in the first place.

Please do join the other thread as we are exploring more of this over there. At the moment, there are still several things that are restricting me from fully adopting the convention and it would be absolutely fantastic if we could agree on how to resolve those things.

In the meantime, this thread is finished now since I know what is causing the problem even though something is still odd because Node-RED's mqtt-in node seems to be more sensitive than other tools and so giving me a slightly different result. I don't really have time to track that down right now and it is clearly an edge-case so hopefully not something that others are likely to hit.

Could also be a problem with the library used by the MQTT nodes.
It's this version: https://www.npmjs.com/package/mqtt/v/2.18.8

Some edge case when you have multiple overlapping subscriptions and using wildcards... just a thought.

Should we close this thread then? Before we stray too far away... :smile:

1 Like

I did consider that but thought it might be rude :grin:

But now you've suggested it as well ... why not, I think that it has served its purpose.

As a final reminder, here is a link to the other thread where we have started to talk about extended use-cases and standards.

https://discourse.nodered.org/t/mqtt-homie-convention-node-questions-about-usage/26541/11

1 Like

Julian, forgive me for writing on your closed topic - perhaps I missed something.

Is this solved?

You said

so I'm a little confused?

Are you now able to sub to +/+/$homie or is there an actual issue in node-red that needs to be resolved?

Sorry if i missed that bit.

1 Like

Not really solved. There is certainly an odd interaction between the esp homie framework and Node-RED's mqtt-in node. I just no longer think it important enough to worry about for now.

At least part of the issue is what Christian mentioned about the default Mosquitto settings and using QoS=0 for retained messages. Something that accounts for some other previous issues I had (months ago) that was causing some of my lighting to go wonk. I "fixed" that by adjusting my MQTT output settings in Node-RED.

Unfortunately, I just don't have any spare time or capacity to work through the many test cases logically in order to track down whether or not there is a bug or just my misunderstanding of how things should be working. Too many variables.

I can work around the issue so there is little point in continuing this thread. However, if others want to do any more testing, I'd be happy for it to be reopened. It is just that the other thread is, I believe, a lot more important.