Add HTTP Persistence (connection reuse) to http-request node

Hi,

I was setting up a front-end that interfaces with an API backend however I couldn't use that API due to the limitation on HTTP keep-alive or HTTP persistence feature were not possible on Node-RED "http request" core node.

The backend system requires an API login first in order perform subsequent API requests. It expects me to first send the credentials to the url http://10.10.10.10:8888/, then it reply with a new URL in the location header (sample response http://10.10.10.10:8888/lskdh234hv02n), this new URL I must use in subsequent requests.

In order to use the newly provided URL by the response, the first HTTP request should keep the initial HTTP request open (TCP connection should be kept) so next requests use the same TCP session. However this is not happening when sending HTTP request from the "http request" node on Node-RED due to the nature of this node was designed. I confirmed this using wireshark and compared the capture with another capture using postman when doing the same API activity.

On Node-RED, I tried to inject and enforce the header "connection: keep-alive" in the before the http request node, however Node-Red is overwriting this header value with "connection: close\r\n" specifically. I confirmed this behavior by sending other headers like User-Agent, Test-Header, I can see them in my request and they arrive to the back-end but the connection header is still coming with "close\r\n"

Here is what I'm injecting before the HTTP request node to manipulate the connection header:

msg.headers = {
"connection" : "keep-alive",
"my-custom-header": "samplevalue",
"User-Agent": "My User Agent"
}
return msg; 

I see "User-Agent" and "my-custom-header" correctly set but the "connection" is still close.

How I made it work:

Simply I used the www-request node (alternative to the http-request node), thanks to the builders and maintainers of this node. However, I had to modify the node code, I added the forever options to the opts in the www-request.js file and that fixed the entire issue.

I have also modified the .html .js and locale files and added this option to the node, now it is fully functional and available on my local www-request node. I think this feature is useful in similar cases. The code is ready, shall I pull a request for it?

1 Like

Sounds like some solid work there.

I assume you mean this node: node-red-contrib-http-request?

If so, you should raise a PR on that Node's GitHub site - In fact, someone (you?) has done that 16 minutes ago. However, sadly there are outstanding PR's from 18 months ago & the README states that this node is deprecated. So I wouldn't expect much of a response I'm afraid.

What might be nice would be for Nick or Dave to chip in to say whether or not it would be good to include this in the core http request node.

Thanks @TotallyInformation for your response. Yes, I meant the node-red-contrib-http-request. And you're right, I did a PR few minutes ago, hopefully this can be added to the www-request node or would be great if persistence feature is added to the http-request core node as well.

Actually I used the alternative (enhanced) www-request node because I couldn't find and modify the core node http-request.

The http-request node is buried in the @node-red/nodes package I think. But it would be better to create your own version until @knolleary or @dceejay confirms whether or not they'd like to see it as a PR for the core node.

I think they may be offline at the moment, probably having a well-earned family Sunday afternoon if they've any sense.

1 Like

I published made a pull request on the www-request node at GitHub and got it merged. Hopefully this will be added to the core http-request node as well in the future.

Hi @Ibrahim.nezar

If you wanted to propose a change along these lines, then you could raise a PR against the core HTTP Request node here: https://github.com/node-red/node-red/blob/master/packages/node_modules/%40node-red/nodes/core/io/21-httprequest.js

3 Likes

Hi Knolleary,

Done :slight_smile:

Thank you.