How to send insecureHTTPParser Request with the http request node

[mickym2]

Hello i'm not a WEB specialist - but I want to know, if I can send insecureHTTPParser requests with the http-Request Node. It seems that "Disable strict HTTP parsing" does not work.

Using the search function guides me to this thread and posting #16, but as I am a dummy user - I understood nothing.

With the axios library I can set the insecureHTTPParser parameter to true and then it is possible to get data from the URL.

axios.get(url, { insecureHTTPParser: true }).then((response) => response.data);

So my question is: Is it possible to send this parameter somehome with the http-request node and how?

An example would be great. Thanks in advance.

[bakman2]

If you look at the documentation of the node it states:

rejectUnauthorized
If set to false, allows requests to be made to https sites that use self signed certificates.

So you can put a change node before the http-request node:
set: msg.rejectUnauthorized
with a boolean set to false

*assuming this is your question.

[mickym2]

No my question was how to set
{ insecureHTTPParser: true }

[bakman2]

What does that option do?

[mickym2]

This option let you open sites which using an insecure HTTP Parser.

Node.js release fixes a critical HTTP security vulnerability | Snyk

So these are web sites which are "insecure" - but I do not know - what internally is wrong with these sites.

[hardillb]

You tick this box

The fix went into 3.0.2

p.s we don't use axios, the http-request node uses GOT.

And the problem is that the HTTP headers returned by the server do not match the spec (they break each header with only a New Line char and not a New Line + Carriage return), this allows for HTTP Request Smuggling.

[mickym2]

I tried this option - but doesn't work. As I, in person, cannot test this anymore - I take it - and if really needed - I will take axios in a function node. - So many thanks so far.

[hardillb]

There are tests for this option in the unit test suite that tests both the positive and negative cases so I'm pretty confident it is settings the option correctly.

But it is is possible your target server is doing something different, we would need a network capture of the request to see what the response header actually looks like to see why it's not getting parsed.

[mickym2]

This URL is from a local IP address and I have not his system. So you get these errors above and the Node shows

I do not have this system - therefore I can't give any details. It is a local sytem to measure power consumption resp. production . The system may only available in Germany: EWE Elmo Smartmeter.

So I am not able to deliver here more information. I can only say that with this parameter {insecureHTTPParser: true} - we could get the information (a JSON) from the internal website, without we got these error messages.

Therefore my question was - if we can achieve the same with the http request node.

Sorry that I am not able to provide additional information.

I saw in another post - the response was:

{'bytesParsed':47,'code':'HPE_CR_EXPECTED','reason':'Missing expected CR after header value','rawPacket':{'type':'Buffer','data':[72,84,84,80,

or

{'bytesParsed':46,'code':'HPE_INVALID_HEADER_TOKEN','reason':'Invalid header value char','rawPacket':{'type':'Buffer','data':[72,8

The data buffer is exactly the same
If look to the data - I cant see any irregularities:

[hardillb]

If we could get all of the rawPacket in the those 2 responses we could see what's missing (all 127 bytes).

The formatted string does not show the newline and carriage return chars which is what the actual problem is.

Capture with something like wireshark or pcap would be best, or you can set the debug not to log to the console as well so it doesn't get cropped for the debug sidebar so we get all of the rawPacket.data array

[mickym2]

I can only copy what is provided:

 {'bytesParsed':46,'code':'HPE_INVALID_HEADER_TOKEN','reason':'Invalid header value char','rawPacket':{'type':'Buffer','data':[72,84,84,80,47,49,46,49,32,50,48,48,32,79,75,10,67,111,110,116,101,110,116,45,116,121,112,101,58,32,97,112,112,108,105,99,97,116,105,111,110,47,106,115,111,110,10,67,97,99,104,101,45,67,111,110,116,114,111,108,58,32,110,111,45,115,116,111,114,101,44,32,110,111,45,99,97,99,104,101,44,32,109,117,115,116,45,114,101,118,97,108,105,100,97,116,101,44,32,109,97,120,45,97,103,101,61,48,10,80,114,97,103,109,97,58,32,110,111,45,99,97,99,104,101,10,10]}}

 {'bytesParsed':47,'code':'HPE_CR_EXPECTED','reason':'Missing expected CR after header value','rawPacket':{'type':'Buffer','data':[72,84,84,80,47,49,46,49,32,50,48,48,32,79,75,10,67,111,110,116,101,110,116,45,116,121,112,101,58,32,97,112,112,108,105,99,97,116,105,111,110,47,106,115,111,110,10,67,97,99,104,101,45,67,111,110,116,114,111,108,58,32,110,111,45,115,116,111,114,101,44,32,110,111,45,99,97,99,104,101,44,32,109,117,115,116,45,114,101,118,97,108,105,100,97,116,101,44,32,109,97,120,45,97,103,101,61,48,10,80,114,97,103,109,97,58,32,110,111,45,99,97,99,104,101,10,10]}}

[Steve-Mcl]

If you fire up wireshark or TCPdump or similar, you can capture what is travelling down the wire. This is what Ben will need to understand the issue.

[mickym2]

Ok - this is not possible as I dont have the hardware. I wanted to help someone else with a NodeRed Flow. - As the user is using the axios library now - and doesn't want to follow up with NodeRed, I must close this topic. But thank you all for your support so far.

[hardillb]

Apologies, while the bulk of the feature made it into 3.0.2 the one key fix missed the cut so will be in 3.1.0 when it ships.

I was testing on the git head, which does have the fix and the tests.