What i can say about it is that i in fact was able to comprehend how to make this flow work.
The only issue left is that, at least the node that i was using (feedparser) send all feed items (regardless of being already sent or not) when the flow starts up.
That would be okay if i ran node-red in a "stable" (not prone to reboots and downs) environment, that isn't absolutely the case of my Raspberry Pi. So, reboots and flow restarts are something that will be pretty frequent here, and the feedparser node sending all past feed items at startup could ruin the experience of my Discord server.
So, what i was looking for was a way to prevent the feed node to send those past items at startup, only monitoring for new changes during the time that the flow is running. This is the behavior of the rsstail tool that i mentioned earlier, and in that case i even was able to pipe the information to another script that posts to webhooks via bash. But that is a awful way to do that since i would end up cluttering my pi's systemd with a multitude of the same bash script (only differing in the rss url to be listened, and the webhook url to be posted).
That's why i looked for a free (because i'm unemployed atm and don't have funds to pay for such kind of service) alternative to IFTTT or Zapier.
After the incident in this thread i installed n8n, the UI is good and such (like i said), but i also ran into the same issue (feed flow sends everything at startup).
So just to check, missing data while your flow isnât running is ok in your scenario ? IE if your pi blips out for 5 mins and some stories arrive on the feed, then you donât need them ?
A small question: does read and write nodes read/write files from ~/.node-red directory?
It says that, if the path isnt absolute, "it will be relative to the working directory of the Node-RED process."
I'm studying similar flows on n8n and one that i saw actually stores all "posted" items into a table, to be read later to filter-out the already sent ones.
Oh thank you so much! I will try this addition ASAP!
EDIT: Just reinstalled the module, i noticed on the log the following output:
-----------------------------------------------------------
2022-06-02T20:02:06.256Z Install : node-red-node-feedparser 0.3.0
2022-06-02T20:02:08.421Z npm install --no-audit --no-update-notifier --no-fund --save --save-prefix=~ --production --engine-strict node-red-node-feedparser@0.3.0
2022-06-02T20:02:18.248Z [err] npm
2022-06-02T20:02:18.250Z [err] WARN deprecated har-validator@5.1.5: this library is no longer supported
2022-06-02T20:02:18.833Z [err] npm
2022-06-02T20:02:18.833Z [err] WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See https://v8.dev/blog/math-random for details.
2022-06-02T20:02:19.054Z [err] npm
2022-06-02T20:02:19.055Z [err] WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142
2022-06-02T20:02:19.873Z [out]
2022-06-02T20:02:19.873Z [out] added 64 packages in 11s
2022-06-02T20:02:19.919Z rc=0
No - nothing for you to worry about. The node is using a deprecated library under the covers - but its a very stable library with close to 20million weekly downloads. We should update it to use a maintained library - but there hasn't been a critical need to yet.
Thank you guys so much for your patience and to not let me down! With @knolleary's kindness and @dceejay readiness to change the node adding that option, i was able to make a fully working flow to post items to my Discord server!
I only need to realize what of the posts i should tick as the accepted answer to solve the topic...
Notice that it posts twice after one hour, idk if this have some relation with the 30min setting that i set.
If one wants to check out yourself, here is the exported flow (i just edited manually the webhook url to a dummy one for obvious reasons, but the rest of the json is straight from my node-red instance): GIST - since posting here would be cumbersome due to the size of the .json.
The internal mechanism uses the article.date property as a unique key. Maybe if they edit or update the article that property is changing and so looks like a new story.
sorry to ask but: why don't you just subscribe to the youtube webhooks instead?.
There're situations in which the approach you took at the beginning of this thread makes sense: a regular RSS feed is a good example. But parsing a feed to create a webhook, when there's already a pub/sub option available on the source (you-tube) that is pushing events already, doesn't look like the right approach to me.
I do not believe they are inconsistent - @dceejay is saying he believes that if a post gets modified that the date of the posting is updated.
You would need to check that to see if that is in fact what is happening - possibly by creating your own YT channel posting some content and then modifying in some way to see if the date/time does get updated
At least then you would know why it is happening and could work out logic to suppress them (if thats what you require)
