Content-Type of HTTP In node

augjoh · 26 July 2020 10:23

Our project runs node-red standalone and provides an OCSP (RFC 6960) service. The OCSP requests are received using an HTTP In node. When receiving an HTTP POST request with Content-Type: application/ocsp-request, the body of the HTTP request is modified by node-red resp. express. Unfortunately this corrupts the DER (Distinguished Encoding Rules) encoded body of the request. When the same request is posted with Content-Type: application/octet-stream there is no modification of the request body.

How can node-red be configured to pass HTTP POST requests with Content-Type: application/ocsp-request as-is?

Steve-Mcl · 26 July 2020 11:24

There is an example of setting headers in the cookbook

augjoh · 26 July 2020 11:31

Hi Steve, this issue is not about creating HTTP headers, but receiving broken content within node-red, based on the value of HTTP Content-Type header.

Steve-Mcl · 26 July 2020 11:43

It was my opinion from this statement....

... That you needed to know how to set a header in your post request.

Anyhow...

I seen a post a few days ago about a similar issue and if memory serves me it was accepted that the request type in question should probably be returned as binary.

See this thread perhaps this header should also be handled as binary?

janvda · 26 July 2020 12:04

... yes indeed this problem is similar to the problem I noticed in that thread. The HTTP in is stringifying the body in case content type is application/XXX where XXX is different from octet-stream.

So as work around (if possible) use Content-Type: application/octet-stream.

augjoh · 26 July 2020 12:14

There is no way to change the client's implementation to use a different Content-Type.

I've figured that it might be possible to use the following code to configure the express application:

app.use(bodyParser.raw({ type: 'application/ocsp-request' }))

But I couldn't figure out how to get a reference to the express application in node-red.

Steve-Mcl · 26 July 2020 12:24

You could edit node-red core as a trial and then perhaps raise a PR (with approval from Dave or Nick)

This is the place you need to handle your header type...

github.com

node-red/node-red/blob/82677c304e7640a5ea640bac407eb7ac719d6c78/packages/node_modules/@node-red/nodes/core/network/21-httpin.js#L39



function rawBodyParser(req, res, next) {
    if (req.skipRawBodyParser) { next(); } // don't parse this if told to skip
    if (req._body) { return next(); }
    req.body = "";
    req._body = true;

    var isText = true;
    var checkUTF = false;

    if (req.headers['content-type']) {
        var contentType = typer.parse(req.headers['content-type'])
        if (contentType.type) {
            var parsedType = mediaTyper.parse(contentType.type);
            if (parsedType.type === "text") {
                isText = true;
            } else if (parsedType.subtype === "xml" || parsedType.suffix === "xml") {
                isText = true;
            } else if (parsedType.type !== "application") {
                isText = false;
            } else if (parsedType.subtype !== "octet-stream") {

knolleary · 26 July 2020 13:47

It isn't sustainable to keep adding exceptions to the logic that is trying to do the right thing for 90% of cases.

It would be better to add an option to the node to explicitly leave the payload as the raw Buffer and skip all the parsing.

Steve-Mcl · 26 July 2020 13:51

Makes sense.

@augjoh / @janvda perhaps you could raise a PR to handle this like Nick suggests - to get it moving?

augjoh · 21 August 2020 22:55

@steve / @janvda I cannot provide a PR for this issue, but I'll test if someone creates one.

I've found a workaround for this problem, in settings.js add:

    httpNodeMiddleware: function(req,res,next) {
        switch (req.get('content-type')) {
            case 'application/timestamp-query':
            case 'application/ocsp-request':
                req.headers['content-type'] = 'application/octet-stream';
                break;
	}
        next();
    },