Accessing Raw Request Body in Node-RED

Hello Node-RED community,

I've been trying to access the raw request body in a Node-RED project, specifically for processing Stripe webhook events, which require access to the raw body to verify their signature. I've tried several approaches, but none have been successful so far. I am seeking your guidance and expertise to help me solve this issue.

Here is what I've tried so far:

  1. Using the httpNodeMiddleware in the settings.js file to intercept the request and parse the raw body using the raw-body package:
const getRawBody = require('raw-body');
module.exports = {
  // ...
  httpNodeMiddleware: function (req, res, next) {
    if (req.originalUrl === '/stripe/webhook' && req.method === 'POST') {
      req.skipRawBodyParser = true;

      getRawBody(req, {
        length: req.headers['content-length'],
        limit: '1mb',
        encoding: 'utf8'
      }, function (err, string) {
        if (err) {
          return next(err);
        }
        req.body = string;
        next();
      });
    } else {
      next();
    }
  },
  // ...
};

This approach resulted in the following error: InternalServerError: stream is not readable.

  1. Trying different stream handling approaches like on-finished and busboy packages, but they either produced the same error or were not suitable for handling raw JSON payloads.
const getRawBody = require('raw-body');
const onFinished = require('on-finished');

module.exports = {
  // ...
  httpNodeMiddleware: function (req, res, next) {
    if (req.originalUrl === '/stripe/webhook' && req.method === 'POST') {
      req.skipRawBodyParser = true;
      console.log('httpNodeMiddleware', req.originalUrl);

      // Ensure the request is fully processed before parsing the raw body
      onFinished(req, function (err) {
        if (err) {
          return next(err);
        }

        // Parse the raw body
        getRawBody(req, {
          length: req.headers['content-length'],
          limit: '1mb',
          encoding: 'utf8'
        }, function (err, string) {
          if (err) {
            return next(err);
          }
          req.body = string;
          next();
        });
      });
    } else {
      next();
    }
  },
  // ...
};

I am looking for a solution to access the raw request body in Node-RED without relying on additional dependencies or packages. Any suggestions, guidance, or examples would be greatly appreciated. Thank you in advance for your help!

I seem to remember that you cannot access the raw body because of one of the ExpressJS middleware libraries loaded that interprets it.

You might be able to use Node.js's native request library though:

HTTPS | Node.js v18.15.0 Documentation (nodejs.org)

1 Like

to do what exactly :slight_smile: ?

The linked article has an example of how to do a http request via code. You could do this in a function node. Just be sure to add a reference to the node lib in the function node setup tab

2 Likes

@Steve-Mcl @TotallyInformation
Thank you for your suggestions, but it seems there may be a misunderstanding. I am not looking to make an HTTP request using Node.js native libraries or the function node. My goal is to access the raw request body of an incoming HTTP request in Node-RED itself, specifically for processing Stripe webhook events.

I appreciate your input, but I am seeking a solution that allows me to work with the raw body within Node-RED without creating new requests. If you have any other ideas or suggestions, I would be grateful for your help.

Just passing by this thread.

Could you not use the Standard TCP nodes? - I'm sure there are cleaner ways, maybe even a Node, have not searched.

[{"id":"3a7824226fcfa66d","type":"tab","label":"Flow 2","disabled":false,"info":"","env":[]},{"id":"833db30a8cb93f9c","type":"http in","z":"3a7824226fcfa66d","name":"","url":"/foo","method":"post","upload":false,"swaggerDoc":"","x":180,"y":180,"wires":[["cb9236ba739c664e"]]},{"id":"f0a3fead9252c48a","type":"http response","z":"3a7824226fcfa66d","name":"","statusCode":"","headers":{},"x":670,"y":180,"wires":[]},{"id":"798019ce7bc287c6","type":"tcp in","z":"3a7824226fcfa66d","name":"","server":"server","host":"","port":"1882","datamode":"stream","datatype":"utf8","newline":"","topic":"","trim":false,"base64":false,"tls":"","x":200,"y":320,"wires":[["c2ab47e157cea238","93d1a184c7381891"]]},{"id":"eeacbf575d4041f8","type":"debug","z":"3a7824226fcfa66d","name":"debug 3","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":680,"y":320,"wires":[]},{"id":"e8d7a1526a7361cf","type":"tcp out","z":"3a7824226fcfa66d","name":"","host":"","port":"","beserver":"reply","base64":false,"end":false,"tls":"","x":230,"y":460,"wires":[]},{"id":"c2ab47e157cea238","type":"template","z":"3a7824226fcfa66d","name":"Send 200 OK","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"HTTP/1.1 200 \nConnection: Close\nContent-Length: 2\n\nOK","output":"str","x":220,"y":400,"wires":[["e8d7a1526a7361cf"]]},{"id":"93d1a184c7381891","type":"function","z":"3a7824226fcfa66d","name":"Parse Request","func":"let rawBody = msg.payload.split('\\r\\n\\r\\n')[1]\nnode.warn(msg.payload)\nlet r = httpZ.parse(msg.payload);\n\nlet h = {}\nfor (let i = 0; i < r.headers.length; i++) {\n    let n = {}\n    n[r.headers[i]['name']] = r.headers[i]['value']\n    Object.assign(h, n);\n}\nr.headers = h\nr.rawBody = rawBody\nmsg.payload = r\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[{"var":"httpZ","module":"http-z"}],"x":420,"y":320,"wires":[["eeacbf575d4041f8"]]},{"id":"cb9236ba739c664e","type":"function","z":"3a7824226fcfa66d","name":"Redirect to Port 1882","func":"let hostname = msg.req.headers.host.split(\":\")[0]\nmsg.headers ={}\nmsg.headers.location = `http://${hostname}:1882${msg.req.url}`\nmsg.statusCode = 307\nmsg.payload = ''\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":400,"y":180,"wires":[["f0a3fead9252c48a"]]}]

Source:
https://flows.nodered.org/flow/12569b0e9e05c773ce54aa923f68c110

1 Like

Ah, my bad. :grin:

Though I think that the same applies because the middleware that interprets the body is applied quite early on. However, you should probably try Node-RED's custom middleware feature (see settings.js) to see if it applies early enough for you to grab the raw body. I don't think it does but worth a quick test.

I am rather skewed in my thinking about incoming HTTP towards uibuilder for some odd reason :wink: But one thing you can do with it is tell it to use a custom ExpressJS instance. Now to be honest, I'm still not sure whether that instance will also apply the body interpretation but I don't think it does.

uibuilder allows custom middleware as well so you should be able to access it.

What uibuilder does not currently have is the ability to set a parameterised URL I'm afraid. So the only way to deal with that if you need to (to be fair, you haven't said that you do) would again be via custom middleware which would let you intercept the inbound request and add a custom header that you could use in your flows.

1 Like

No bother.

You should probably remove the http-request tag as it implies the question is related to the http-request node.

Have you tried any of the existing stripe contrib nodes?

1 Like

Hey @Steve-Mcl, thank you so much for your suggestion, it works. This node does its job.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.