Repeated messages with Vernemq mqtt broker

General
1

I am using Vernemq MQTT broker with node-red both running in Docker in a Traefik Stack (though I don't expect that is the problem). I am seeing a strange symptom with repeated messages when subscribing to a topic. If I have a simple flow as in the one below with an inject node publishing a topic via MQTT Out node and an MQTT In node picking up that topic then all is well. However if I add a second MQTT In node subscribing to the topic with the last element replaced by # then I see two received messages (from the first MQTT In node) for each Inject. In this example the first topic is tydwr/testit/xxx and to make it repeat I additionally subscribe to tydwr/testit/#. Note that the additional subscription node is not connected to anything. If I delete the second subscription then the repeat stops.
Running mosquitto_sub at the same time does not see any repeat.
Using the same flow, just changing the broker to a mosquitto broker removes the problem.
image
image

[{"id":"c20136d0.1ab618","type":"inject","z":"bdd7be38.d3b55","name":"","topic":"tydwr/testit/xxx","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":297,"y":1003,"wires":[["4cbf2a29.a1d304"]]},{"id":"c6eba3b4.a73048","type":"debug","z":"bdd7be38.d3b55","name":"test","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":487,"y":1085,"wires":[]},{"id":"4cbf2a29.a1d304","type":"mqtt out","z":"bdd7be38.d3b55","name":"","topic":"","qos":"1","retain":"false","broker":"bbdb09c8.d3d0a","x":509,"y":1002,"wires":[]},{"id":"49f76222.230f34","type":"mqtt in","z":"bdd7be38.d3b55","name":"","topic":"tydwr/testit/xxx","qos":"1","datatype":"auto","broker":"bbdb09c8.d3d0a","x":284,"y":1078,"wires":[["c6eba3b4.a73048"]]},{"id":"b8aa9ddb.2ce188","type":"mqtt in","z":"bdd7be38.d3b55","name":"","topic":"tydwr/testit/#","qos":"1","datatype":"auto","broker":"bbdb09c8.d3d0a","x":282,"y":1152,"wires":[[]]},{"id":"bbdb09c8.d3d0a","type":"mqtt-broker","z":"","name":"","broker":"x201-vps.local","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""}]
2

I have seen this long ago when i was new to node-red (so didnt report it - thought it was my fault);

In the end I put an RBE after the subscribe node. This nipped it in the bud but it was not the real solution.

Since then I have began to think somehow there are 2 event handlers linked up to the subscribe node (in the background) - perhaps due to modifications and deploys.

Would you be in a position to try something?

  1. check it is still happening
  2. stop node red
  3. start node red
  4. try again

Let us know if it is still occurring.

3

I have tried restarting node-red and Vernemq.
There was a bug a year ago that caused similar issues, but it was not broker dependent I don't think. It was fixed in 0.20.4 apparently.

I can't use an RBE as a workaround as I want to receive all the messages even if they are the same, but I am working round it using a Delay node in Rate Limit mode with a limit of 1/sec and drop intermediate, so if two are received effectively at the same time then one is chucked away.

4

So is this persistent with vernemq?

e.g. if you change to a local or other remote broker, does the issue go away?

5

Yes, I think I mentioned that if I switch to a mosquitto broker that there is no problem. It returns if I go back to Vernemq.

6

Any thoughts on this @dceejay, @knolleary?

7

... err maybe don't use Vernemq ?

1 Like
8

You are hitting an "interesting" edge case in the MQTT spec. I say "interesting" because I spent hours upon hours discussing it as part of the MQTT standards group.

It all comes down to how a broker handles overlapping subscriptions.

In your scenario, you have a subscription to tydwr/testit/xxx and tydwr/testit/#. Because of the wildcard in the second topic, these two topic filters overlap.

Before MQTT 3.1.1 was standardised, the old spec did not spell out what a broker should do if a client subscribed to overlapping topics. As such, implementations took two different approaches. They would either:

  1. handle each subscription as a separate 'thing' - so when the broker is checking to see if a message is on a topic a client is subscribed to, it checks every single subscription and sends a copy of the message for each matching subscription.

  2. handle the subscriptions as a whole - a client is either subscribed to a topic or it isn't. It doesn't matter how many different individual subscriptions were made that may encompass the topic. In this case, the broker will send one copy of the message to the client. This is the behavior of Mosquitto.

Unfortunately we were not able to decide on a single, well-defined, behaviour for this due to the number of existing implementations of both methods. So the spec says that both behaviours are valid and its up to the broker to pick.

Now, here comes the issue: there is no way for a client to know ahead of time which behaviour the broker is going to follow.

When a message arrives in Node-RED, the broker node has to check all of its MQTT In nodes to see which ones match the arriving message. It can then pass the message to those nodes.

If the broker sends one message for the overlapping subscriptions, then that message will get passed to each MQTT node. Everyone is happy.

But if the broker sends two messages, then the Node-RED broker node will match each one to both nodes, so both nodes will send on both messages. No-one is happy.

The best advice I can give is to avoid overlapping subscriptions.

1 Like
9

In fact it also seems to happen if you have two nodes both subscribing to the same topic (so again they are overlapping). For example two nodes subscribing to tydwr/testit/xxx. That is something that I do a lot in order to keep separation of function across flows. So for example if the value is a temperature and is used in some control flow to control room heating for example then there would be an MQTT In node for it there, but it may also be needed on the dashboard which is in a different flow tab so that the control logic is not mixed up with the dashboard.

@dceejay the reason I am using Vernemq is that I am using Traefik to provide internet facing security, with a Docker stack for node-red and Vernemq (with wss) and was able to get that up and running from a position of no knowledge of Docker or Traefik by starting with a working setup discussed in a thread last week. It isn't necessarily trivial to slot mosquitto into that, though I think I may have gained sufficient knowledge to have a go at that. I will stick with my workaround of using a Delay node to suppress the repeats till I have time to try it with Mosquitto. I will post that in the Projects section if I get it going.

Thanks both.

10

Or an RBE node ?

11

I can't use an RBE node as I do need a message to come through every time, even if it is the same value as the previous one. Two reasons, firstly it goes into a node-red-contrib-pid which expects regular process values even if there is no change, and secondly it is going to an influx database and I don't want gaps in the data under normal operation. If there is a sensor failure, for example, then that will cause a gap, but i don't want gaps the rest of the time.

1 Like
12

Are you in control of the format of the incoming data?

If it was JS(ON) with .value and .ts you could solve this with an RBE.

e.g...

image
image903Ă—245 20.3 KB

13

Sorry, I don't see how that distinguishes between two messages that happen to have the same values and a repeated message?

I have a workaround that functions perfectly. Since I know that good messages for each topic are never going to be closer together than at least a few seconds then a Delay node configured like this does it.
image

14

Was that in response to me @Colin?

If so, then to explain - if your source data is JSON & has a timestamp then we can assume each NEW value has a NEW timestamp. That makes the JSON unique (not a duplicate) & thus passes the RBE. If it IS a duplicate then the RBE will see ALL fields (value and ts) are the same and it will be dropped. And consider this - if the data IS new and was sent on the same timestamp - you only lose a "same value" that is < 1ms different - hardly an issue (unless you need to count messages)

The rate limit may well work but if messages appear faster than 1sec, you will lose data. (i do appreciate that may be ok for this instance).

With the above RBE solution, you only lose data (i.e. RBE only blocks it) if the data has the same timestamp (which is ms resolution)

15

Yes, sorry.

I didn't realise that you meant that ts was a timestamp. If the data did have a timestamp then yes, the RBE would do it. Most of my mqtt data is immediate and therefore doesn't need a timestamp. The Delay node works in either case.

16

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.