Deduplicate Messages based on specific property

Hello,

I have sensors being heard by several gateways. Those gateways push those messages up to the cloud Node-Red server.

Node-Red then has two very similar messages incoming with the exact same telemetry, but slightly different other properties. I'd like to pick only the 'best' one to proceed forward in the flow.

Example Message 1
topic: 1234
{
sensor_id: 1234
temperature: 72
gateway: r419
rssi: 40
}

Example Message 2
topic: 1234
{
sensor_id: 1234
temperature: 72
gateway: r403
rssi: 67
}

I'd like only the message with the better RSSI (lower is better) (in this case the first one) to go through.
I tried the deduplicate node, and am infact using that right now. It deduplicates the messages, but makes no consideration, it seems, for mathmatically choosing which message to pass through, rather it seems time based, in that it just passes whichever one gets there first.

I thought I could use the join node, and maybe I can, but in this same stream I also have hundreds of other sensor transmissions coming through as well. if I join, manual mode, and combine each msg.payload, using the value of the msg.topic, it ends up just choosing the last transmission for a given msg.topic (where msg.topic is the sensor_id), and not necessarily the best rssi.

Example Message 3
topic: 1222
{
sensor_id: 1222
temperature: 65
gateway: r409
rssi: 67
}

If I just join on gateway, unfortunately join just gave me the last transmission as heard by each gateway, which ended up with lost data.

I feel like I am missing something obvious in regards to joining a stream of messages received in an amount of time, based on a key, and then looping over them to choose which one has the best value.

Does anyone else have a similar issue or could give me some advice?

I think I found a solution with

Heres one way using standard nodes , context, and JSONata

[{"id":"aa0649dac3dba56c","type":"inject","z":"b9860b4b9de8c8da","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"1234","payload":"{\"sensor_id\":1234,\"temperature\":72,\"gateway\":\"r419\",\"rssi\":40}","payloadType":"json","x":170,"y":360,"wires":[["e3f35f846117a96b"]]},{"id":"e3f35f846117a96b","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"set","p":"rssi","pt":"msg","to":"$string($$.payload.rssi)","tot":"jsonata"},{"t":"set","p":"sensors[msg.topic][msg.rssi]","pt":"flow","to":"payload","tot":"msg"},{"t":"set","p":"payload","pt":"msg","to":"sensors[msg.topic]","tot":"flow","dc":true}],"action":"","property":"","from":"","to":"","reg":false,"x":400,"y":480,"wires":[["87ae20b55283ba3b"]]},{"id":"f9ad367d388dfe9d","type":"inject","z":"b9860b4b9de8c8da","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"1222","payload":"{     \"sensor_id\": 1222,     \"temperature\": 72,     \"gateway\": \"r420\",     \"rssi\": 88 }","payloadType":"json","x":170,"y":520,"wires":[["e3f35f846117a96b"]]},{"id":"54028d3c3bc688c7","type":"inject","z":"b9860b4b9de8c8da","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"1234","payload":"{     \"sensor_id\": 1234,     \"temperature\": 72,     \"gateway\": \"r419\",     \"rssi\": 57 }","payloadType":"json","x":170,"y":400,"wires":[["e3f35f846117a96b"]]},{"id":"8ca13bcb4974be68","type":"inject","z":"b9860b4b9de8c8da","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"1222","payload":"{\"sensor_id\":1222,\"temperature\":72,\"gateway\":\"r420\",\"rssi\":44}","payloadType":"json","x":170,"y":560,"wires":[["e3f35f846117a96b"]]},{"id":"87ae20b55283ba3b","type":"trigger","z":"b9860b4b9de8c8da","name":"","op1":"","op2":"","op1type":"nul","op2type":"payl","duration":"2000","extend":false,"overrideDelay":false,"units":"ms","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":420,"y":540,"wires":[["fbced18d73bf89c1"]]},{"id":"fbced18d73bf89c1","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"delete","p":"rssi","pt":"msg"},{"t":"delete","p":"sensors[msg.topic]","pt":"flow"},{"t":"set","p":"payload","pt":"msg","to":"$$.payload.*^(rssi)[0]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":640,"y":460,"wires":[["60cbf1a4e853ad44"]]},{"id":"60cbf1a4e853ad44","type":"debug","z":"b9860b4b9de8c8da","name":"debug 250","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":690,"y":580,"wires":[]}]

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.