Please note that i don't want to send the message to Discord via a BOT (so no need to use node-red-node-discord), but in fact i want to send it via Webhooks (HTTP POST). In this way i can send each feed with a specific avatar/nick without need to create multiple bots (i dont want to do that).
So i'm currently trying to replicate this flow you did.
I just want to know what are the configs of the change and switch nodes to make it equal to yours.
So...my bad. My biggest problem was not knowing what each node returned (msg.whatever).
Now i just realized that is there an help tab besides debug and it have all methods and variables returned by each node (including installed ones). I'm so dumb to not realize this.
So i discovered there is a pubDate field that can be compared with a timestamp (but i don't know how exactly can i compare since the datetime format is different and i don't know how to get the timestamp of each iteration (at least the original feedparser that self-activates and dont need a inject node).
Thanks for letting my know why. Unfortunately, I couldn't work out how the discord webhooks work in the limited time I had today which is why went back to the node so that at least I could get some output into discord. If you can give some guidance on the URL to call and what to include in the POST body then I'll try again. I didn't find the Discord API docs to be of much help.
Sorry, I deleted it because I was trying to get something else to work. I realised that, because the feedparse node sends out a msg for each article, that flow didn't work. Instead, I think that you need to try and retain an entry for each article you've already processes (I was trying to use msg.article.guid).
The debug node is the most important node of all 
With a node like feedparser though, make sure that you set the debug node to show the whole message and not just the msg.payload.
Yes, both Steve and myself showed you that earlier in the thread.
The timestamp in the msg.article.pubDate is in ISO 8601 format which is the international standard for exchanging timestamps. The easiest way to do the comparison is using a function node and a bit of JavaScript. let pubdate = new Date(msg.article.pubDate) will convert the string timestamp into a JavaScript one. You can then directly compare that with another JavaScript date.
Yes, and that is the complexity. Weirdly, it would actually be easier if the feedparser node did actually output a single XML or JSON object. Then it would be easy to compare the timestamps.
So here is where I got up to before I ran out of time.
The function node updates a context variable that is persisted to filestore. It is an object variable where each property is a msg.article.guid. There is some code that should delete any entries older than 24hrs but please note that in the example below, that isn't correctly coded, the function doesn't quite work and still needs some tweaking. Sadly, I'm out of time to do that right now.
//flow bbcFeedLastRun
const bbcFeed = context.get('bbcFeed', 'file') || {}
const article = msg.article
// We will keep the feed history for 5 days
let d = new Date()
d.setDate(d.getDate() - 5)
if ( !bbcFeed[article.guid] ) {
bbcFeed[article.guid] = new Date()
node.send([{
rich: {
title: article.title,
author: {
name: 'BBC',
url: article.link,
},
description: article.description,
}
}, null])
} else if (bbcFeed[article.guid] < d) {
node.warn('no')
delete bbcFeed[article.guid]
}
context.set('bbcFeed', bbcFeed, 'file')
return [null, bbcFeed]
Well, webhooks for Discord need to be sent in JSON format in a POST request. When i did a test flow only with the webhook part i needed to "assemble" the webhook JSON using a function node and then link to a http request node to finally send to Discord.
This website can help you to output a valid JSON to use in the webhook's http request.
EDIT: Basically this is the basic valid JSON needed to be sent to the webhook:
{
"content": "the proper message goes here. Links, etc",
"embeds": null,
"attachments": []
}
TBH i only want this checking because feedparser spills the entire feed each time the flow is restarted. That would fill the target channel with duplicate messages.
IF only feedparser had options/arguments like rsstail to filter out any entry older than the current timestamp at startup (and other interesting options), everything would be easier.
So... any conclusions?
I realized that the heart of the problem is just that when starting the flow feedparser node always spills with old items, regardless of whether they have already been sent before or not.
It just needs to monitor future updates, not the past ones (that are spilled at the startup). Those should be discarded and not being sent again to the webhook. Is exactly this that i don't know how to do in a flow, both using feedparser or Inject+HTTP+XML. i don't know how to block only the first iteration output (that is the problematic since it sends the whole xml file), the subsequent are fine because they surely contain only new items.
I have been following through this discussion.
Surely as there seems to be a msg.pubdate you could track in global context the last one you sent and just restart from there i.e. loop through the array of messages in the feed until you find one that has pubdate > your context stored date ?
Craig
I just went back and looked at Julians last post - he gave you a function node to use to do exactly what you are asking for
I think you need to install that and put a bit of effort into playing with it and trying to understand what it is doing and then ask specific questions/post issues rather than expecting him to write the whole thing for you
Craig
1 Like
First of all, who is "Julian"? I cant locate anyone with this name commenting on this thread. If you can be more clear about what answer you mean, i would be grateful.
Well... i wasn't expecting such kind of... rude comment here. I'll assume English is not your native language like me (it's quite common for non-English speakers to sometimes sound "rude" due to translation issues) in that case.
I suppose you don't know how Discord (and sending webhooks) work. I just can't lose time experimenting in the way you want me to do because, well... i will flood important channels with test messages, and i absolutely want to do it. I already lose a couple of members testing things in my server and i just canāt afford to do it again, that's it.
Second... isn't Node-Red suppose to facilitate the setup of such flows? Why it cant be so intuitive like... IFFTT? Or Zapier? Those tools have RSS feed triggers (how nodes are called on that) that aren't so bothersome and didn't post the whole XML file at once... they actually have options to only send new items published after the start of the Flow.
Why Node-Red can't work in the same way and instead assigns the user (who often installs Node-RED thinking they won't need to code anything) the unfortunate task of figure what why the hell the tool is working in a counter-intuitve way?
If you check the user icons you will often see the person's actual name. You will certainly see mine.
You may wish to consider that most of the people helping here are doing so freely of their own time. This isn't a paid support channel after all. We will go out of our way to help people, particularly those who are willing to have a go. But we all have busy lives and perhaps have less patience if we think people are coming for free support on a commercial project.
No, not necessarily, it is a low-code programming platform. Not an IFTTT clone.
Perhaps because those are commercial services with paid plans supporting them. Node-RED is open source and free. Should you or someone else wish to contribute money or expertise to the project, I expect it could move forward in new ways. Indeed, sponsoring someone or a group to write a more advanced feed reader would be an excellent way to get what you want.
Well, you are free to pay for those services if you wish. Indeed, I have a paid IFTTT subscription for some tasks that it is excellent at that would take more effort in Node-RED. That is my option and I'm happy to do so since, for some things my time is more valuable that the money.
In addition, the feedparser node doesn't post everything at once. It somehow caches the feed and doesn't repeat articles unless you restart Node-RED. That is a function of the underlying node.js library, the node is a wrapper around that. So questions about how the library work are probably better directed to that libraries author via GitHub. As already demonstrated, there are plenty of people here who would happily help improve the node if some positive suggestions were made.
Because Node-RED is a programming tool and not another IFTTT. It does things that IFTTT will never be able to do whereas IFTTT does some things better than Node-RED because it has a different focus. No point in whining. As already said, you can certainly use both tools together as I've regularly done in the past.
I note that this thread has now reached 50 responses which is a lot. A number of suggestions have been made and I'm not sure that the last responses are really getting us anywhere I'm afraid. It certainly won't get further if people start insulting each other.
2 Likes
I think Julian has answered everything here - yes i am a native english speaker (well Australian anyway)
You are maybe using the wrong tool here - you seem to assume that NR is another service like IFTTT etc etc - it is not - as Julian has said it is a low code visual programming environment.
This thread is now 50 posts long most of which are by active members trying to help you - but it appears that you are mistaken in that you think this is some form of obligation and you can just keep requesting more things without getting your hands dirty.
I actually do know how discord and webhooks work - i would suggest you setup a test channel and let your members (sounds like some form of paid service) know that these are exactly that - test channels until you get this right.
For some of my more advanced stuff i wanted to do early in the piece for my home automation i went out and contracted a person to do the work for me - i broke it down into blocks, specified what i expected and what the end result should look like - he then wrote this for me in NR for a set price - this might be a better approach for you to try if you do not want to get into the programming yourself.
regards
Craig
So this is the way you guys treat unfamiliarized users? That's revealing.
By the way, i will not spend any more time banging my head when those "experienced users" aren't able to explain to a layman person how their software works. I didn't insulted anyone here and was just not getting the point the other users were trying to show - maybe because your guys were just initially just spilling terminologies like i already knew how to use that, and i needed to insist in telling you to show pictures to better visualize what are you meant to show here.
It's easy to brush off me since i'm just a weed in this garden... changing behavior to better acclimatize, oh no, that's difficult... i bet its not more difficult that use a half-baked software like this, with horrible UI.
n8n is also free and have an excellent UI... other open-source software also have. Choose a better excuse next time.
So, i'm lefting this forum and wiping nodered from my pi. Zero interest in work with cocky people like you. Feel free to ban me - or better, please remove my data from this forum, will try other solutions (like n8n) or even code myself something straight in Node.js, since in that way i will not need to interact with toxic people. Bye.
Hey @vico I fear you might have misunderstood the guys responding to you. I will attempt to explain if you are still around and willing to listen...
The approach many of the forum members take (including myself) is to offer the user advice and direction (and sometimes complete solutions) but it is almost always with the intention to help the user to learn - which I believe is the best way. By asking a user to spend some time working/playing with their own flow is not meant to be rude - it is really meant as a nudge to the user to try for themselves and learn in the process - after all, "the best way to learn is by doing" right?
You are of course free to move to another platform but in my experience the quality and amount of free help that users get on this forum is (in my opinion) quite simply amazing and even superior to enterprise (paid for) help in many products that I have used over the years.
Good luck either way.
3 Likes
Unfortunately a rather sad case of someone expecting everything to be spoon fed to them and for free.
I guess you can't help everyone.
BTW, for anyone else coming across this thread, the other services mentioned here such as n8n, IFTTT, Zapier - are not entirely free. n8n for example as a permissive non-commercial license. You can't run your own server for a commercial venture.
I'll close this thread as it clearly has come to an end.
1 Like
I'm reopening this topic in case @vico is still interested in making progress here. There are plenty of people in this forum who are more than willing to try to help.
1 Like
Well, I came back here just out of consideration for @knolleary and @Steve-Mcl, who were polite to me and didn't want to harass me like some other members (including a moderator) tried to do.
Unfortunately, having those "rotten oranges" in this community, I just can't feel confident to remain anymore. In fact i kindly ask you guys to remove my account from the platform (doesn't seems to be an option in user preferences for that).
Plus, if was wrong about the purpose of Node-RED, in comparison to IFTTT and Zapier, a large reason for that is because in every post on r/selfhosted about alternatives to IFTTT and Zapier have a reply recommending Node-RED - including from the same rude moderator that closed my thread, he said above that Node-RED isn't exactly a drop-in replacement to those platforms, but in Reddit he was one of the people that always recommends it for people looking for... a drop-in for IFTTT and Zapier. That's a very misleading thing to do. Plus... his behavior replying other users that recommended other pieces of softwares in that linked thread was very aggressive. I hate to point others but in this case the other devs should have a look about this. That would in fact marrow down the reputation of the project as a whole.
I'll try to follow my attempts on n8n, a similar platform but with a more intuitive UI for beginners, more integrations that i can use (not only RSS but also Discord, Twitter, Google and other websites that isn't related to HA) and where I had a more warming and clearer (and respectful) answer from a moderator when I posted about the same problem I had there as well. I hope in this case i will be able to achieve what i want to do.
So, to resume, i'm sorry if i'm mistake the purpose of Node-RED. I'm not a sorcerer with a crystal ball to understand at first what is the focus of a software that i never used before, and i think in the moment the more veteran users noticed that i was a little bit "off-course" from what i was expecting the software to do, they just could say me that maybe this software could not be exactly what i was expecting to be. I just came from advice for users from Reddit that told me that it was a real drop-in to IFTTT and Zapier. But it isn't, and there is no problem with that. Every software have its focus and it was never my intention to change that.
But... What happened here was ridiculous. Next time you want to reprehend an user because he didn't understand a thing about your software think it can be a layman person like your parents. Be kind. Everyone have the right to be wrong. But harass people in that way... its a big NO. And for those saying "it's a free software...", character isn't something you buy. It came from home. From raising. Treating users bad because the software is free isn't a excuse.
Hi @vico
I'm truly sorry for the experience you've had in the forum. I'm the creator of Node-RED and have worked hard to build a welcoming community. This thread is not a good example of that - and you have highlighted some very valid points of criticism in how you were replied to. I hope everyone hears your feedback and reflects on it properly.
What you were trying to do is a perfectly reasonable task to expect to be able to do in Node-RED. It is true to say Node-RED can be a bit more DIY than IFTTT and Zapier. But we need to learn from feedback like yours to improve things.
I'll have a proper read back through the whole thread (as it appeared to be making some progress initially) to see where things broke down. I appreciate this is too late for you and we have lost you as a Node-RED convert. But if you were interested, I'd be happy to work through it with you directly.
But of course, if you are certain you'd like you account removed, I can do that for you.
1 Like
