[ANNOUNCE] node-red-contrib-sse-client

[BartButenaers]

@Eelco

I have not modified my NR settings file to enable https

That is not relevant for this case. That is when you want to secure incoming directions, e.g. when you navigate from your pc to your Node-RED flow editor via https.
In this demo my node calls a third-party server via https, so the third party has done the SSL setup.

@Eelco

Hence, I assume my Nose Red does not know how to call an https service

That is not correct. Node-RED can send http requests to third party services as long as your system is connected to the internet.

@Eelco

did look for green for (connection established)

Then your internet connection is ok ...

@Eelco

There it fails.

Do you have some errors in the node-red log perhaps?

[Eelco]

I start from fresh, so we share same information

step 1 is to validate the demo through curl in command prompt of my W10-laptop, new per mid March
curl -X GET --header "Accept:text/event-stream" "https://proxy.streamdata.io/http://stockmarket.streamdata.io/prices/"

Hmm, that is a new response. Did not see that one before but I can imagine it is blocking

Same curl command now on my NR host

So there is something fishy about the demo. Are you able to inspect?

As for switch from http to https is base NR: the default http-request node can be used for https traffic, but then you need to set-up a tls config (client side!). I know how to do it for self-signed, but not for chain from commercial CA re-using what is already in OS (Mozilla)

[Eelco]

And this is view from NR editor, Flow 3 is created fresh from import of SSE client demo. The connection error may be down to the certificate issue on proxy server

Here is a working curl (sorry to hide account details). That one should go into SSE-client

Here is essential flow from your example, populated with working curl details

Nothing special in pm2 logs node-red

4 headers are present, same as in curl

[afelix]

@Eelco

As for switch from http to https is base NR: the default http-request node can be used for https traffic, but then you need to set-up a tls config (client side!). I know how to do it for self-signed, but not for chain from commercial CA re-using what is already in OS (Mozilla)

I might be extremely wrong here, but if the websites you’re attempting to reach have valid signed certificates that are acknowledged by CAs, you don’t need to use the tls config. Just like you don’t have to supply certificates yourself to your webbrowser (chrome/firefox/whatever) when you try to access the same page.

[BartButenaers]

• So the website used in my demo has an expired certificate. Your curl in Windows doesn't mind about this, but on the NR host curl fails. I'm not using curl myself, but google shows me that indeed you might have to setup ca certs on your server to manage this. Or you need to use curl -k to disable certificate validation. But that is a bit too much off topic for this discussion about the sse-client node.

• However curl indeed gives some useful information about what might be going wrong with the demo.

• I see that you are also using another "schiphol" service. But there you haven't a certificate problem with curl on the NR host. So this is another issue I assume. Is that correct?

• I'm using this module under the hood, so cannot simply integrate logic from that https-node with it (although identical https settings 'might' be passed to the EventSource library). But I'm seeing in the EventSource repository two things I could try:

  • I could try to add a checkbox "allow unauthorized https requests" as suggested here. This is from the NodeJs documentation:

    If not false a server automatically reject clients with invalid certificates.
    

  • I could add functionality to add keystores (as in their unit test case), but that is for client side certificates. So probably won't help you with your issue I assume.

  The first option should be fairly easy to implement, so will try to do that this week. For the second one I have no time at the moment.

• But I have no clue why my demo works here on my system, and not on yours.

[Eelco]

Well, I took my knowledge from configuring the node and there is this box to tick which then prompts you for TLS-config.
But on your nudge I just gave it a try .
Flow is like inject-httpRequest-debug (so a plain http request, all settings on default, injected payload is "help")
Case 1 is google.com. It does not flinch and returns code 200 plus unrendered content. Works as you suggest.
Case 2 is https://discourse.nodered.org/. Same finding
Case 3 is klm.com. I cannot get it working (nor through contribute-https), which tells me world of http calls is not trivial.
So my hunch that the s in https is blocking may be wrong. Thanks for the lesson

So much for intuitive understanding

[Eelco]

Thanks for thinking along.

• The expired certificate is actually blocking curl both on Windows and Ubuntu (all platforms in test). I would love to see your demo working again, would you have a tip who can I approach to get the cert updated?
• "Event source" is probably something server side in the demo. Change should not be required to get demo flowing again
• There might be another issue in accessing the "schiphol" service. For now I have no clue where to start looking why NR fails so I focus on a running SSE-client as step 1.
• Perhaps, as sidestep to fixing the demo, would you know of another SSE-emitter I can play with?

[BartButenaers]

Hi @Eelco,

I have been reading some background information, and it seems I can now (in the latest version of the EventSource module) implement my own server certificate validation (based on CA root/intermediate certificates that a user needs to specify). However I don't have time to implement that now.

Don't know if this will solve your case, but I have created a new 0.2.2 version on Github which contains following changes:

• I have upgraded to the latest version of EventSource, which contains a number of improvements and bugfixes.

• I have added two new options (which are explained on my readme page):

  

This version has not been published on NPM yet, so you can install it directly from my Github account:

npm install bartbutenaers/node-red-contrib-sse-client

In this new version when I use my demo I get indeed this error:

But that is solved as soon as I deselect the "Reject unauthorized https requests", since that skips the server certificate validation. So I hope this workaround solves your problem(s).

Remark: I'm aware that it would be more safe to add all the features to specify the CA certs, so you would be able to keep the server certificate validation enabled. If anybody wants to create a pull request, that would be fine because I have a lack of time ...

[BartButenaers]

Don't expect any feedback anymore whether my new version has fixed the problem, so I have published version 0.2.2 on npm:

[hansju-11]

http post methode available?

I have this request:
curl https://....mynetworkprovider..../api/v3/events -X POST -H "Authorization: mypassword token" -H "Accept: text/event-stream" --data '{"identifiers":[{"ids":{"id":"12345678"}}]}'

can I do this ?

[BartButenaers]

Hi @hansju-11,

Not quite sure what you mean with this. The server pushes events to the clients, and this node allows Node-RED to act as one of those SSE clients. What do you want to post?

[hansju-11]

Hello @BartButenaers,
for me it looks like that the SSE Clients sent the request (URL) as GET method, my server will accept only POST method. http GET VS POST. Is there a way to change the request method from get to post?

[BartButenaers]

@hansju-11,
Ah ok now I understand your question. That makes sense indeed.

But I am using in my node the eventsource package as a dependency. You see in their code that they use the NodeJs http(s) module to send their GET requests. To send a POST request they need to change their code a little bit, as you can seen in the NodeJs documentation of the http module.

So if you can ask them (by registering an issue on their Github repo) if they can somehow make it possible to choose between get or post, then I will update my node afterwards to make this selectable in the config screen. Unfortunately I don't have time at the moment to digg into this myself.

[hansju-11]

HEllo Bart,

request is done.

want feature http post request · Issue #316 · EventSource/eventsource (github.com)

We will see what will happen. Thank you for your support

[BartButenaers]

Hi @hansju-11,
What a pity you still got no response at all. Got today a similar request as your on Github...