Identifying the duplicate value in queue and dropping that particular value out of queue

General
1

is there any ways to identifying the duplicate messages in queue and drop that duplicate message of queue . i have some messages from azure iot hub in JSON format (attached the payload below) .i have kept simple queue node ahead of azure iot hub node .it will queue all messages and when and all i get signal from PLC this messages will be triggered from queue and send to plc tags . i am having same messages repeated twice sometimes but there are not stored one after other .is there any way to identify those duplicates and delete them from queue

2

Possibly you can use a Filter node to stop then going into the queue.

3

this is the json message i get from azure iot hub . like this ill receive 100 messages with different "binId" and other parameter .there are some cases same messages is getting received after some messages .i.e assume the below is my first message in queue and i have 49 different messages in queue but 50th message in queue is duplicate of 1st message i should kick this 50th message .how can i take one message and compare with other message in queue and if not duplicate then load to queue
{"binId":"BPTC02-37-1-5-1","commandType":2,"sRequestId":"fbb79de2-bc7b-40a3-b5e1-6095627c19af","rRequestId":"81902f1c-be22-4ff2-8d2a-f0e1ba8a81ab","toX":1,"toY":1,"toZ":1,"fromX":0,"fromY":0,"fromZ":0}

4

Do you mean that only one message should be allowed in the queue for each binId?

There is node-red-contrib-multiple-queue which may do what you want.
You can configure it to maintain a seperate queue for each msg.binId and allow only one entry per queue.
I don't think you can trigger the oldest queued message regardless of binId though.

5

I have written several nodes for message queueing and scheduling (not all of them published), and I might be able to help if I understood your issue better. I am assuming a few things, so correct me if I am wrong.

  1. Two messages are "duplicates" if every property, not just the binId is identical.
  2. The difficulty is not with the size of the queue. A few hundred messages of the size you describe should not create a problem, and a few percent of duplicates will not matter.
  3. When you say "send to plc tags," you mean that a message is sent from the queue to a PLC.
  4. For some reason, you don't want the PLC to see duplicate messages. If this is the case, please explain why. Filtering needs to be done either before or after the queue, and if you want the PLC to never see a duplicate, you may have to do much more work than is done by the queue.
6

That's correct, but IIRC you can send the oldest message from each queue at the same time by using the trigger command with the Queue Selector set to All Queues.

7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.