Websocket with authentication

Core Development Feature Requests
1

I am using the data service of an automated storage system. Historical data is provided with HTTP, while live data is sent via websocket.

While accessing the historical data with an API key for authentication was straightforward, I struggled to access live data. How can I authenticate for websocket access with my API key?

Thanks a lot for any help!

pbr

2

I may be wrong but I'm not sure you can. Typically you would send the API key in an http header even for websocket connections. The initial ws handshake happens over http and then the connection gets "upgraded" to websocket at which point you can no longer send extra headers.

As far as I know, there is no way of sending custom headers with the websocket nodes.

3

Thanks @TotallyInformation. I believe this goes in the right direction.

I have used an inject node with msg.headers set to

{
"API-Authorization": "<my API Authentication Token",
"Connection": "Upgrade",
"Upgrade": "websocket",
"Sec-WebSocket-Key": "YXNkJiY4OWFicnRlcXM3Zw==",
"Sec-WebSocket-Protocol": "chat",
"Sec-WebSocket-Version": "13"
}

followed by an HTTP request node.

However, I get a "JSON parse error" in the HTTP request node and, as a consequence I suppose, the server rejects my request with status code 500.

Can somebody help me to understand the JSON parse error?

Thanks a lot!

4

Don't know whether that will work but an interesting test anyway.

It would seem that the web endpoint you are sending to wants some data in the body. So you need to be doing a POST rather than a get and you need to send something in msg.payload that corresponds to the JSON it is looking for.

5

@TotallyInformation Thanks - I have changed the method to POST. Without having made any other changes, the status is changing to 400 "no valid HTTP request". Unfortunately, the API description is very vague, just stating "Pass the API token". Nothing is mentioned about what should be in the msg.payload.

I'll try to connect to the storage system manufacturer for some hints.

6

Small step forward.

This is what I have passed in the header of the GET request to the website (it does not allow a POST)

{
"API-Authorization": “<my API token>"
}

This is what I have passed in the payload of the message:

{
"Connection": "Upgrade",
"Upgrade": "websocket",
"Sec-WebSocket-Key": "YXNkJiY4OWFicnRlcXM3Zw==",
"Sec-WebSocket-Protocol": "json",
"Sec-WebSocket-Version": "13"
}

I get a response with status code 200 (which is a success message) but also the information that the connection has been closed again.

Does this ring a bell to someone?

Thanks in advance

7

Honestly, that does not surprise me, I didn't think you would be able to force a connection that way. The server is correct in that it has seen the connection close because the http-request node cannot deal with the ws upgrade.

From here, I think you have a couple of options, neither of which are likely to be ideal for you.

  1. Dig into the code for the websocket nodes and see if there is anything in there that would allow you to pass the auth header even if it isn't documented.
  2. Create a stand-alone node.js service that creates a web-socket connection directly, with the auth. and that then acts as a communication broker with Node-RED.
8

Thanks @TotallyInformation. You are right, both options go beyond what I can do. Anyway, the company selling the automated storage system has asked the company providing the webservice for their systems to look into my issue in detail. If they can't make out something we can achieve from Node-red I most likely have to drop this project.

9

Sorry to hear that, hopefully there will be a resolution. Please do let the forum know. In truth, it sounds like there should be an enhancement to the websocket nodes to allow for custom headers to be passed in. If I get time, I'll have a look at the code to see if there is anything hidden away already.

It won't make you feel any better but I dealt with a related issue regarding websockets as work today where our cloud security broker is blocking access to a web service because of websockets. They are sometimes poorly understood. Of course, as I am not an operational IT person any more, I'm able to simply pass on the problem to someone else to deal with having identified it. :rofl:

Just had a quick look at the code and I can't see anything. Perhaps @knolleary or @Steve-Mcl can comment on whether this might be a useful addition?

10

I ran into something like this a few years ago. Ended up using 'curl' in the exec node. Had to make function nodes to handle the whole thing. I don't know if it would work for a WebSocket. I just needed it to do a post for auth. Just a thought. Might want to ask the provider if they have a curl example.

11

The issue with websockets is that they need more than one interaction, firstly an http(s) call followed by an upgrade call that actually establishes the socket.

12

Finally, I got a response from the company that provides the data service on behalf of the manufacturer of the automated storage system.

The http API requests always requires authentication header in order to retrieve data. There is currently no way to achieve a switching from regular HTTP API calls to web socket API calls. That means, if you want to utilize the live data from a web socket connection, you will need to connect to web socket in a secure manner. There are two ways you can achieve this, one is using the a authentication header, the other is specifying the protocol to use when connecting to the web socket, e.g. specifying sec-websocket-protocol. However, this is also transmitted as part of the header.

If there is no chance to tweak the websocket in node in a way that allows sending the authorization token in the header, I, unfortunately, have to drop the Node-Red idea.

As it is a project of my students (who are logistics engineers with a limited background in coding) I will be having a hard time convincing them to code this in Python.

Thanks @TotallyInformation for the kind help!

13

OK, we've not seen any input from @knolleary or @Steve-Mcl on this. Can I suggest that you change the category for the thread from general to Core Development Feature Requests to see if we can at least see whether the core devs would be willing to add the ability to pass custom headers to the websocket node?

Sadly won't solve your current issue since that will take some time to get to a live version I'm sure but at least we could fix forward for when this comes up again.

14

Unfortunately I have not had a moment to delve into this. It is on my radar.

@pbr in the meantime, you could try using the websocket class in a function node. but TBH, this company should explore something far better - like MQTT for their streaming data. Have you asked if they support MQTT?

15

Unfortunately, they do not support MQTT either @TotallyInformation. We are the first university to have been granted access to the system and I will deliver a kind of user experience report with my students. One issue definitely will be "Think from the user's perspective", i.e., make it as easy as possible for users to prototype dashboards.

16

Then not supporting MQTT is a BIG issue (IMO)

17

I have asked for changing the thread category to Core Development.

The key question is: Would it be possible to change the websocket in node in a way that allows passing an authentication token, actually two items: keyword, token.

18

No MQTT access, unfortunately, @Steve-Mcl . Using the websocket class in a function node - doesn't that need a "require" statement (which I thought is not possible within Node-Red)?

19

Correct, require/import is not supported in the function node

HOWEVER: you can add requires in the setup tab (of the function node).

20

Changed.

One for Steve.

You can turn on the feature in the startup tab that allows Node-RED to install and require a module for you.

Or, as Steve says, you can add require: require, to the globals section of settings.js. However, that does mean that anyone with access to the editor can then use/abuse require in function nodes. I have that set on my dev instance so that I can (ab)use Node-RED in ways that might not be entirely sensible in a University setting with students :grin:

Alternatively, you can simply require the websockets library in settings.js so that it is available to Editors and you don't then have to open up other potential issues by allowing people to install other random modules (assuming that you've locked down the settings.js file of course).