How can i make a flow that checks for RSS feeds and then posts on Discord (via Webhook)?

Hello! I'm a really new users of Node-Red, always wanted to try to selfhost my "automation" solution (instead of rely in Zapier, IFFTT and similars).

Just installed it on my Pi 3b+ (with Rasp OS Lite 64 bits) and i'm trying to make a simple flow that checks periodically a RSS feed (managed to install feedparser node), and when new items in the feed are detected, it sends a message to my Discord server via Webhook. I'm aware webhooks are a kind of HTTP "interaction", but i'm not sure if any of the preinstalled network nodes can do that for me. Plus, it requires a JSON payload. More info Here and Here.

So how could i do that?

The reason i'm asking specifically for a Webhook solution is:

  1. First, i had troubles installing node-red-contrib-discord-advanced in my setup. It requires node v16 but the bash script available on the user guide of node-red website installs v14, and i'm very afraid to mess up my installation trying to update node version.

  2. Second, i prefer Webhooks because for each webhook URL i can set its "nickname" and "profile picture" without the need to create multiple bots in Discord's application page (that is a hurdle if one have 2FA enabled). Its even possible to send the nick and avatar of the webhook straight in the payload, what can allows, depending of the RSS feed, to dynamically change its name for better organization at the server.

Anyway, thanks a lot for this tool!

You can rerun it with --node16 to perform the upgrade.

Don't worry about breaking anything, it's very robust & we are always around to assist ( that's on top of the huge forum history to scour too) :metal:

Thanks, i will try to update it asap.

About the webhook thing, its feasible just using the built-in network nodes? Judging from the curl examples seems to be a POST request. Tried here with http request node but i didnt figured out how i can specify the JSON payload.

Yes. Perfectly feasible. The HTTP request node is what you need. Have a look at the built in examples (CTRL-I) and the cookbook on the node-red site.

1 Like

The http node understands when you send a payload as an object and converts it into JSON. So simply create the necessary object
using dot notation and send it.

I've been "playing" around with a http-request node in Node-RED to send commands to a Discord channel using webhook. Here a screen-shot of my experimental flow.

This is what I've put in one of the function nodes.
shutdown_discord_bot

And this is what's in the http-request node.
http_webhook
So what happens is... I can send a command like $stop or $snap or just plain text - which ends up in a Discord channel. My Discord bot will react to the command, so for example $snap will trigger a series of still photos to be taken on my home cameras and sent back to Discord. $stop will shutdown the bot.

2 Likes

Thank you guys. With @dynamicdave's hints i managed to make a working webhook that sends "bar! " with node!

The only thing i wasn't able to make was output an arbitrary text (like [INFO] Webhook sent!) into the debug window.

I tried appending the "Preparar Console" function node with the following settings:
image
image
...but it only gives me errors like (cant attach due to my account limitations in the forum :frowning:) :

function : (error)
"Function tried to send a message of type string"

What i'm doing wrong this time?

you always have to return an object
so
return {payload: "[INFO] webhook enviado"};

[edit] nothing will be returned from On start only On message.
Try
node.send({payload:"string"}); or node.warn("the string message")

1 Like

I did it, but now it doesn't return anything to the debug window. :frowning:

Current settings of the debug node (renamed "console"):
image

EDIT: Oh i'm very sorry. I think i deactivated the debug node and didnt realized it. Now it works correctly, thanks.

Now i will try to switch the Inject node by a feedparser to see if i can achieve what i'm trying to make.

Guys, now i reached another wall.

Seems like feedparser node doesn't do (or allow to do) any checks about the time of the posts (at least in the first run). The only possible variables to this node is the URL of the feed and the interval to check for new items.

image

The logic that i'm attempting to do in this test flow is the following:

  1. Check if is there any feed item newer than the last time we checked the feed, or in case of being the first run any item posted since the time the flow started - i know that in this case the check would be milliseconds after the start of the flow, but just to now allow the flow to spill old posts to the webhook;

    • If something new exists, prepare the msg object to be sent via webhook, one request for each item;
    • If nothing new exists, wait a predetermined time to check again.

This is how I send plain text to Discord using http-request and webhook.

text_in_fn_node

1 Like

I think i nailed this part of the flow.

Now i'm really struggling with the RSS part...

The timestamp should be in each article of the feed. So you probably need to loop through each (or use the split node) and check each timestamp.

A more accurate approach might be to record each article id or timestamp as you process it. For each one, compare against the variable of already processed entries. Just remember to clear out old entries that you will never see again (maybe after 24hrs?) and also remember that some RSS/ATOM feeds may update an article.

Unfortunately feedparser seems to not process each feed entry into a JS object to access those properties. Seems to just output each entry's XML raw code.

That was another thing that made me scratch my head. Is even possible to make a variable (other than msg) to persist across nodes (and even iterations)?
I honestly can't find any docs or good examples about this.

Context storage

Can be accessed in function node and change node

Try passing it through the xml node.

I have zero experience with RSS so I won't be able to help you.

Sorry, I thought you knew that you needed to pass the incoming feed through the XML node which will turn it into JavaScript objects.

Mostly, you attach data to the msgs you pass to the next node(s). However, the context, flow and global variables are available which you can either set from a function node or a change node.

So now i decided to give up using feedparser and instead i'm testing replacing it with a combination of inject+http request+xml parser nodes.


Seems to work more elegantly that feedparser.
But my question, before returning to the timestamp checking is how could i loop through each of the entries of the feed (in the image they are under [0-9], [10-19] and [20-29] in the debug window).and execute the next node for each of them?

Use the "copy path" button (see canned text below) to get the path to the entry array & use that path in a change node to move (or set) msg.payload to the value of msg.payload.feed.entry (or whatever it is)

Then once entry is in msg.payload, feed that though a split node to create a stream of objects that you can then use/modify. If you need them back into a single array, then send them through a join node.

inject :point_right: http request :point_right: xml :point_right: change node (move (or set) msg.payload to the value of msg.payload.feed.entry :point_right: do stuff with that entry (function node/change node/whatever) :point_right: join node :point_right:

At this point you will have looped through each item & "done stuff" then back to a single array of items.


Canned text...

Getting the path to an item

There’s a great page in the docs (Working with messages : Node-RED) that will explain how to use the debug panel to find the right path to any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

BX00Cy7yHi

1 Like