i'm programming an application with node-red that builds a dashboard with status information from an API providing information of that API-Server.
So i build a lot of flows generating API request URLs and send them to a central http request node. Actually there are about 6 request per secound.
The API description says i have to use digest authentication with user and password. And while the API is https encrypted i also activated "use SSL/TLS" and configured a basic TLS-configuration for that connection.
In general this works really well.
But now the http server of that API-Server has changed from lighttp to nginx and now i get a lot of 503 erros because the nginx server thinks that my node-red authentications are Spam requests and blocks the authentication for some secounds.
Logs in nginx look like this:
ngx_slab_alloc() failed: no memory
auth_digest ran out of shm space.
ngx_http_auth_digest_send_challenge: ngx_http_auth_digest_next_nonce failed,
In my node-red application every http request is like a new authentication.
Is this the way digest authentication works or should i program a general login request where i get some hash-values from, that i store in some flow-context and re-use im my api request node?
has anybody an idea?
I could use the "keep alive" option. But nevertheless nodered sends always a new authentication request so it always creates a new session instead of using "the old one".
Question is: How to use the first digest auth instead of re-authenticate everytime?
I think this is a web server configuration error. It looks like the server can't cope with the number of logins you are sending it.
The server response should include a "nonce" that the server will accept for a short period for further requests. I think the default is very short though - maybe 30sec?
I've never used digest auth though with Node-RED so I'm not really sure how the http-request node does it. You'd probably have to trawl through the code of the node itself to find out.
Digest auth is really not recommended in any case and even NGINX don't recommend the use of that module.
The server should implement a proper session management service to avoid these issues.
thanks for your reply.
The company told me that my implementation with node-red always creates new connections and this is interpreted by the (nginx)server as an attack and so it sometimes stops responding HTTP 200 answers. Unfortunately they don't plan to replace the digest auth by a proper session manager so i have to deal with it.
What i see in the default server GUI by monitoring the network-traffic with firefox developer tools,
after logging in via web-ui all request always have the same "nonce"-string.
If i create a digest-auth request in node-red i always get a new nonce-response in the debug-node even if i tick "teep alive" in node-settings.
So the question might be:
Can i store this nonce-value in a flow-context and reuse it in my requests? Would it work and what would be the best way to do that or would i then have to implement the whole digest auth process in my own way instead of using the pre-implemented on in the node-settings?
At least i'm not good enough to understand the code of the http-request-node itself, i'm not a programmer thats why i'm using the wonderful node-red
Worth a go. Try grabbing/storing the appropriate header values from a successful request and then, in a separate HTTP Request node set the appropriate headers (leaving Auth unchecked of course).
Agreed. Of course, you will certainly also have to check for an auth failure so that you can loop back to your original digest auth request to get the next valid nonce.
If all of that works, I wonder @Steve-Mcl whether it might be worth this being raised as an issue/enhancement-request against the request node?
I was wrong.
If the digest authentication build in the http request node had a successful response, it doesn't give back the used nonce. In the debug-node only the cnonce is provided with that it's not possible to recreate a self created digest authentication.
So in this case i totaly have to self build the complete digest auth process which might be possible but first i have to checkout how to create the hashes for user and password.
But this is really a big thing to do. Not a simple one.
First regex all information from the first 401-response, then creating this multible MD5 hashes from all responsible inputs an the finally creating the successful login.
Now i'm hoping that it will even for a longer period.
I think a native implementation inside the http-request node would be much nicer
Normally i'm not a programmer. So everything in JS i do is "common sense of technology" .
To get this running i copied a lot of stuff from here and to learn how it should work was wikipedia and the node-code on github.
Reading the node-code was the hardest part because there are a lot of of referencing functions and some object chains I don't understand yet.
But the first question would be how to implement this in the native http-request-node. I have no idea how that should work.
And my work was just for my specific case. As I understand from the wikipedia article, there are some more "if then" cases in digist-auth
So to conclude:
I would absolutely not feel comfortable porting this implementation into native node code
