Deduplicate payload based on url

General
1

I would like deduplicate a payload but my payload has content (which changes) and an url (which stays the same). I would like to deduplicate based on URL

This is the function I'm using for the payload. Is there any chance I can use the node-red-contrib-deduplicate to only check on the specific payload which contains the url? Or is the only possibility to check the entire message?

var results = "Locatie: " + msg.payload.location + " | Prijstype: " + msg.payload.prijstype + " | Prijs: " + msg.payload.prijs / 100 + "\n \n" + "Beschrijving: " + msg.payload.description + "\n \n" + "Afbeelding: " + msg.payload.PictureUrl + "\n \n" + "AdvertentieURL: " + msg.payload.vipurl ;
msg.payload = results
return msg;
2

Do you mean that you only want to pass messages that have a different URL to the previous message?

3

If that is what you mean then you can do it with a Filter node (previously called RBE) configured like this

image

4

The thing is that the URL always changes. So it wil go from A,B,C,D,E,F,G,A,A,H,I,A,J,K... So sometimes a duplicate URL will occur. I want to filter out the double over a long period of time.

Think I need the node-red-contrib-deduplicate but I have no clue how to make it look for a specific part of the payload. I think it only checks for the complete message/payload

edit:

founded a solution. I needed to use "Payload Key Property for analysis" in the node-red-contrib-deduplicate module :slight_smile: it works now

5

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.