Thanks. Tried your advice.
Click Deploy...
...still sends the whole object (and process once).
Wrong way around
..this?
1 Like
Joining here from the Reddit thread.
To loop through the rss items, you can use the node-red-contrib-loop-processing node like below.
Flow
...more
1 Like
...continued
loop body
This is where you process every item. msg.payload will be the actual item from the array.
1 Like
There is no need for a contrib node & looping is ill advised (for risk of infinite loop)
The split node is all you need.
Just checking the code for the feedparser node.
It could easily be amended to spit out a message per article.
Also, it could probably output JavaScript objects rather than XML which would be a lot better for most people.
From there, it could probably be amended to only emit new/amended articles rather than emitting everything each time.
They would be options of course to ensure backwards compatibility.
It is a pretty simple node so would make an easy starter project for someone.
1 Like
OK, I need to test, looking at the feedparser code, it appears to be capable out outputting each article individually at least. There are other libraries that do output JSON directly as well.
Personally, I would also like to find a better way to handle both incoming feeds and feed merging & republishing. Not been using feeds so much over the last couple of years, partly due to the limited good options for offline aggregation/reading with a complete article not the truncated ones so many feeds produce. And partly because I'm no longer regularly travelling by public transport.
I greatly miss Yahoo! Pipes which made feed manipulation really easy. Really, it should be possible to reproduce some of those features in Node-RED.
Thanks for the concise explanation. I will try to reproduce this flow here in my instance.
In that part, why is Key Variable defined in that way (msg.alf99226....)?
Excuse my sincerity, but I'll follow his example at first simply because he was more clear and illustrative about how trying to make that work with loop nodes. I didn't really understand your explanation using split node.
If there was an example similar to his (with images, a concrete example, etc), but with split node, it would facilitate my comprehension (remember that i'm newbie in Node-RED)...
But... how? I need an example flow about how i could achieve that.
Here is an example
Demo Flow (with fake discord webhook)
[{"id":"6523be4cd6ad3b7b","type":"function","z":"ab64ded2af71d680","name":"format item","func":"msg.payload = {\n content: msg.payload.description[0],\n title: msg.payload.title[0],\n link: msg.payload.link[0],\n guid: msg.payload.guid[0],\n pubDate: msg.payload.pubDate[0],\n creator: msg.payload[\"dc:creator\"][0],\n}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":2010,"y":180,"wires":[["31e0f0a9064d2468","05568558b0ef648e"]]},{"id":"0935a26242e12522","type":"inject","z":"ab64ded2af71d680","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1660,"y":100,"wires":[["3c7c1ee8da1f377e"]]},{"id":"3c7c1ee8da1f377e","type":"http request","z":"ab64ded2af71d680","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"http://feeds2.feedburner.com/canaltechbr","tls":"","persist":false,"proxy":"","authType":"","senderr":false,"x":1830,"y":100,"wires":[["d861c7895ea1a567"]]},{"id":"10a057bd40734f89","type":"debug","z":"ab64ded2af71d680","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":2190,"y":100,"wires":[]},{"id":"800ea657428c94d3","type":"change","z":"ab64ded2af71d680","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.rss.channel[0].item","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1680,"y":180,"wires":[["07de5594d5f3359c"]]},{"id":"07de5594d5f3359c","type":"split","z":"ab64ded2af71d680","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":1850,"y":180,"wires":[["6523be4cd6ad3b7b"]]},{"id":"31e0f0a9064d2468","type":"debug","z":"ab64ded2af71d680","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":2190,"y":180,"wires":[]},{"id":"d861c7895ea1a567","type":"xml","z":"ab64ded2af71d680","name":"","property":"payload","attr":"","chr":"","x":1990,"y":100,"wires":[["10a057bd40734f89","800ea657428c94d3"]]},{"id":"21a58233a74d0d36","type":"http in","z":"ab64ded2af71d680","name":"","url":"/fake_discord_webhook","method":"post","upload":false,"swaggerDoc":"","x":1800,"y":360,"wires":[["ef14c51c7057a935"]]},{"id":"3ee0a2dc1762bf32","type":"http request","z":"ab64ded2af71d680","name":"post to discord webhook","method":"POST","ret":"obj","paytoqs":"ignore","url":"http://localhost:1880/fake_discord_webhook","tls":"","persist":false,"proxy":"","authType":"","senderr":false,"credentials":{},"x":1970,"y":260,"wires":[["e4227aab1d8a63bd"]]},{"id":"8434c1b401af6b6f","type":"http response","z":"ab64ded2af71d680","name":"","statusCode":"","headers":{},"x":2190,"y":360,"wires":[]},{"id":"e4227aab1d8a63bd","type":"debug","z":"ab64ded2af71d680","name":"","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"statusCode","statusType":"msg","x":2190,"y":260,"wires":[]},{"id":"ef14c51c7057a935","type":"function","z":"ab64ded2af71d680","name":"prepare reply data","func":"msg.payload = {\n result: \"success\",\n title: msg.payload.title\n}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":2030,"y":360,"wires":[["8434c1b401af6b6f"]]},{"id":"05568558b0ef648e","type":"delay","z":"ab64ded2af71d680","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"2","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1750,"y":260,"wires":[["3ee0a2dc1762bf32"]]}]
1 Like
Thank you so much! I will try that asap!
I imported your demo flow, just changing the fake webhook url to a real one and the rss feed to Lorem RSS (to try to test with a highly changing feed) and got some JSON parse errors like in the image:
Set the faulting http request to return a string instead of parsed JSON and you won't get an error.
Give me a chance! 
You can't make the current node do it as it isn't written that way. I started to prepare a test node but totally sidetracked myself by setting up a new GitHub repo that will let me more easily do experiments like this. It also has some build tools (which is what took me so long to get working, the whole evening).
Now I have it ready and installed, I can copy the core feedparser node and experiment with it.
Hopefully it will also be useful to other people who want to get started on creating new nodes.
1 Like
Right, so it seems that my memory is even worse than I thought!
I've reproduced the node-red-node-feedparse node and it does, in fact return a msg for each article in a fetched feed.
msg.pubDate contains the timestamp of when the article was published and so you should be able to filter on that using a change node.
Sorry about the earlier mislead. Too tired to sort out the remaining logic tonight. May have a go tomorrow.
Thank you so much for this examples!
EDIT: I noticed that your feedparser node had a entry point (point at left), that the feedparser node that i installed (node-red-node-feedparser) didn't had.
Got intrigued and searched for feedparser in the Manage Pallete "store". Ends up that there is some "forks" of this node available as well, and some have the left point like the yours.
So... what feedparser node you tested exactly?
Ah, I meant to mention that but it was late and I forgot. That was because I wanted to be able to trigger it manually. So I hacked that into my test version. To be honest, it doesn't make a lot of difference. The library that is used actually takes into account a cache so you have to restart node-red anyway if you want the full feed gain.
The test version is available in TotallyInformation/node-red-experimental-nodes: An occasionally changing set of experiments for Node-RED (github.com) if you want to try it. You will need to install from the command line though:
cd ~/.node-red
npm install totallyinformation/node-red-experimental-nodes
As mentioned, that is my new experiments package so nodes will come into and go out of it over time.
Got rid of that and installed the core node-red-node-discord node instead. Seems to work perfectly.
Connected the discord message send node to the output of the feedparser node and it just worked.
So now I'm a bit confused as to what you actually want to achieve since the basics work as anticipated.
Could you clarify?
