Repeated messages with Vernemq mqtt broker

General
8

You are hitting an "interesting" edge case in the MQTT spec. I say "interesting" because I spent hours upon hours discussing it as part of the MQTT standards group.

It all comes down to how a broker handles overlapping subscriptions.

In your scenario, you have a subscription to tydwr/testit/xxx and tydwr/testit/#. Because of the wildcard in the second topic, these two topic filters overlap.

Before MQTT 3.1.1 was standardised, the old spec did not spell out what a broker should do if a client subscribed to overlapping topics. As such, implementations took two different approaches. They would either:

  1. handle each subscription as a separate 'thing' - so when the broker is checking to see if a message is on a topic a client is subscribed to, it checks every single subscription and sends a copy of the message for each matching subscription.

  2. handle the subscriptions as a whole - a client is either subscribed to a topic or it isn't. It doesn't matter how many different individual subscriptions were made that may encompass the topic. In this case, the broker will send one copy of the message to the client. This is the behavior of Mosquitto.

Unfortunately we were not able to decide on a single, well-defined, behaviour for this due to the number of existing implementations of both methods. So the spec says that both behaviours are valid and its up to the broker to pick.

Now, here comes the issue: there is no way for a client to know ahead of time which behaviour the broker is going to follow.

When a message arrives in Node-RED, the broker node has to check all of its MQTT In nodes to see which ones match the arriving message. It can then pass the message to those nodes.

If the broker sends one message for the overlapping subscriptions, then that message will get passed to each MQTT node. Everyone is happy.

But if the broker sends two messages, then the Node-RED broker node will match each one to both nodes, so both nodes will send on both messages. No-one is happy.

The best advice I can give is to avoid overlapping subscriptions.

1 Like