Digest authentication support in HttpRequest node

[BartButenaers]

Hi folks,

I got a question last week from a poor bastard who had bought a box full of chinese IP camera's. He could access the IP camera's from within the browser without problems. However from within a Node-RED flow (e.g. httprequest node) he received an unauthorised exception, although he entered the SAME url/username/password combination ...

I found out that his camera expected digest authentication, while the httprequest node only offers basic authentication. Here is the difference in a nutshell:

• Basic authentication: The client sends a HTTP request with an 'authorization' header that contains the word Basic followed by a space and a base64-encoded string username:password. For example:

  Authorization: Basic ZGVtbzpwQDU1dzByZA==
  

  When such a request is send across an unsecure http connection, a hacker can see the username and password (by simple base64 decoding). This means a secure https connection should be used instead ...

• Digest authentication: The client sends a HTTP request with an 'authorization' header that contains an MD5 hashed password. For example:

  Authorization: Digest username="Mufasa",
                    realm="testrealm@host.com",
                    nonce="dcd98b7102dd2f0e8b11d0f600bfb0c093",
                    ...
  

  Such a password can safely be exchanged across an http connection.

Fortunately @HirokiUchikawa has recently reworked the httprequest node, to make use of the 'request' library. And that library offers digest authentication:

And that works fine! When I set the sendImmediately to true, the basic authentication still works on my own IP camera and the digest authentication works fine on the chinese IP camera's. Only a single line of code need to be added:

Some questions about this change:

• Is this an acceptable change?
• Does the node's config screen need to show the 'digest' somewhere?
  

Thanks !
Bart

[knolleary]

@BartButenaers

When I set the sendImmediately to true,

I assume you mean false as per the screenshot?

When its sent to false, the library sends the request without any auth info attached. The 401 response it gets back from the server will include the type of auth required, so the library can resubmit the request with the appropriate auth header.

So it works, but it takes two requests over the network. So I don't think we can afford to hard code it in - by definition, all flows that successfully use auth with the node today will be penalised.

That does mean it would need to be exposed as an option in the UI, but at this point of a Friday night, I don't know what that should look like.

A tick box for 'send immediately' doesn't really help the user understand what it means - I had to read the 3 paragraphs of the request module's readme a couple times to understand what it meant.

Maybe a select box for the type of auth - which could be expanded to include Bearer (something that can be done today by setting msg.headers.authorization to the right value before the HTTP Request node). Needs more thought about what the different options mean and what they would require the node to actually do.

[BartButenaers]

@knolleary

I assume you mean false as per the screenshot?

Indeed, that was a type error... By setting it to 'false' the digest authentication works fine.

@knolleary

So it works, but it takes two requests over the network. So I don't think we can afford to hard code it in

Ok, I agree with that explanation.

@knolleary

Maybe a select box for the type of auth

Have been thinking about that also, but I thought I would get lapidated when suggesting it ...

[rjandsam]

Hi,
I am new to Node-Red and I am interested in the topic you have discussed.

I am using the HTTP Request node that is part of Node-Red to communicate with a Hikvision PTZ camera and I am doing this successfully but only with basic authentication, I would like to use digest as it is more secure and was wondering if you could explain how I change the options to do this.

do I need to install something extra as I have read all of the above and noticed you have added a line of code but I have no idea how this is done. do you have any pointers or a link to where I can learn more on how to do this type of modification.

Thanks

Rich

[BartButenaers]

Hi Richard (@rjandsam

This is indeed a feature that some other folks have already been asked me about, since it is used quited a lot for multiple branches of IP camera's. However as Nick already explained, that single line of code is not a good solution. There would be a better solution, e.g. a dropdown with authentication methods...

All proposals are welcome!!
Bart

[rjandsam]

My programmer Darren has recreated both files to do exactly that and it works a treat is this something that you would like me to share if so what is the protocol, if any for this?

[knolleary]

@rjandsam

My programmer Darren has recreated both files to do exactly that and it works a treat is this something that you would like me to share if so what is the protocol, if any for this?

A pull-request to allow us to see the proposed changes and to review them would be a next step.

Also some discussion around the actual implementation that covers some of the questions I raised in my previous post would be useful.

[BartButenaers]

Hey Nick (@knolleary),
I thought (from the above discussion) that you wanted to add also Bearer authentication. Which would make sense, since then the HttpRequest node would support all 3 available authentication methods.

Something like this (with checkbox description changed and dropdown added):

Containing following options in the dropdown:

With 'Basic authentication' default selected, to avoid impact on existing flows ...

Bart

[1iveowl]

@BartButenaers Looks very nice and handy. Is there an ETA ?

As a related side note:

I'm struggling with Digest Authentican using http-request. Since it's not currently supported I manage the flow manually through headers and function and md5.

I can get it to work, however, I've run into an annoying issue. For my manual implementatio of Digest Authentication to work, I disable "Use basic authentication". The annoying issue is that it keep enabling itself again, messing up my manual Digest Auth implementation. Here are my observations:

1. If I open the Http Reqeust node and exit with Done, it will re-enable the "Use BAsic Authentication".

2. If I open the Http Request node and exit with Cancel, it will stay disabled.

Seems like a bug to me.

[BartButenaers]

@1iveowl

Is there an ETA

@1iveowl :
Since the http-request node is a standard node, it is adviced to wait for feedback from @knolleary (before creating a pull-request). Because the above proposal might have some drawbacks, that I haven't thought about. And once we have an agreement, hopefully Darren (the programmer from @rjandsam) has some time left to implement it

And at the end, I will create an identical solution for my node-red-contrib-multipart-stream-decoder node, since users have the same problem there (to get an mjpeg stream from a camera with digest authentication) ...

[knolleary]

@BartButenaers yes, your suggestion is closer to what I had in mind than what @rjandsam shared - but I wasn't sure if you were offering it up as a contribution or just a mocked up UI to clarify to @rjandsam what was needed.

Either way, there's only a few days left to get anything in 0.20.

[BartButenaers]

@knolleary

but I wasn't sure if you were offering it up as a contribution or just a mocked up UI to clarify to @rjandsam what was needed.

@knolleary: Well I 'thought' you had something like that in mind, but I was not sure. Will next time try to be a bit more specific when I need your opinion...

@knolleary

Either way, there's only a few days left to get anything in 0.20.

I can try to do the contribution myself, but I haven't got a device that supports digest authentication. Will need someone else to test that for me ... Will see what I can do. And if it is too late for the release, no problem. We can't keep going on pushing you to add new stuff into the release.

[BartButenaers]

Nick (@knolleary),

I have installed version 0.20.0-beta.5 and changed the config screen behaviour:

• By default the 'basic authentication' is selected, to make sure we don't break existing flows.
• The username and password fields are displayed both for basic and digest authentication.
• When bearer authentication is selected, a bearer token can be entered (which is stored in the credentials section).

Is the config screen ok for you?
My only doubt is whether the "bearer token" label should be changed to simply "token"??

[knolleary]

@BartButenaers looks good - but ultimately we'll review it fully once there's a PR. Thanks

[BartButenaers]

@rjandsam,
I have created a fork of Node-RED, with an implementation of this feature request. Seems all to be working fine, even for existing flows. Everything is ready (incl. test flows) for a pull request. I have also added the Bearer Authentication.

But now I have been loosing quite some time installing grunt, but it keeps giving me errors. Don't know how that grunt thing works. Contributing to the Node-RED core is not easy for normal hobbyists like myself ...

My time is up for today. Will try to get grunt running this evening.

[knolleary]

@BartButenaers if you have code that is working, stick the PR. You can tussle with Grunt in parallel.

[dceejay]

( I usually have to install grunt-cli as a global.... sudo npm i -g grunt-cli so that the grunt command is available everywhere... )

[BartButenaers]

@dceejay

I usually have to install grunt-cli as a global

Hey Dave (@dceejay),
I had also installed it global (for same reason), but still get a lot of errors. For example:

Error: Cannot find module 'should'
_ at Function.Module.resolveFilename (internal/modules/cjs/loader.js:603:15)
_ at Function.Module.load (internal/modules/cjs/loader.js:529:25)
_ at Module.require (internal/modules/cjs/loader.js:657:17)_
...

Could you please explain a bit more which parameters I have to add to the grunt command, for a pull-request? Because I saw last night here that there are multiple options available...

@knolleary

if you have code that is working, stick the PR. You can tussle with Grunt in parallel.

Hi Nick (@knolleary),
I have created a pull request, but I wasn't able to run grunt on it. How stupid ...

[dceejay]

usually you only need either grunt build if you want to build the runtime for running Node-RED... or just grunt - which will build it then run all the tests.

[Jude.Robise]

Dear all,

I have some camera units which has http alertStream. I can use some API GET requests to get those Stream. Authentication mode is basic and I can only use node red Multipart decoder to get that stream continuously. No issue.

Now I have another device which has only Digest authentication. Still I have the alertStream but I cannot use Multipart decoder node for this case because it has only basic authentication.

Even though Node red http request node has both basic and digest authentication modes, it cannot handle those Streams. Only multipart decoder could handle that stream with basic auth.

Question is How can I use multipart decoder for digest authentication mode. ?

Correct if I'm wrong at any point.
Thanks in advance.
-Jude