I am developping a node as part of a package.
The node is a WebHook (much like what you can imagine exist in SLACK... so, the user subscribes to events and the target app will deliver those events as POST to the Webhook endpoint).
To create the endpoint I use the following in my code:
NOW, I need to catch the UNPARSED BODY coming from the target application, before it is parsed.
Experimenting blindly (I'm not an expert) it seems that the presence or absence of the bodyParser.json(), parameter actually does not change the results: "mySuccessCallback" ALWAYS receives a parsed req.body object... WHich is not what I need
I tried to use something like this:
RED.httpNode.post('/thisIsMyEndpoint', myOwnFunction, mySuccessCallback, myErrorCallback);
But my own function also only receives a req, res, next and *req is already parsed.
Do you have any idea how to
either intercept the raw body, process it and, eventually PREVENT further processing
provide a req.raw_body attribute to the "mySuccessCallback"
Although I've never tried it, there is an optional middleware section (commented out) in the node-red settings.js file:
// The following property can be used to add a custom middleware function
// in front of all http in nodes. This allows custom authentication to be
// applied to all http in nodes, or any other sort of common request processing.
httpNodeMiddleware: function(req,res,next) {
// Handle/reject the request, or pass it on to the http in node by calling next();
// Optionally skip our rawBodyParser by setting this to true;
req.skipRawBodyParser = true;
next();
},
So it appears that you can set up some http in flows in node-red, which will receive the raw data from the calling application. Of course, this setting will apply to ALL of your exposed endpoints...
A bit more context on the request - Stefan is trying to handle an http request that includes a checksum of its body in a header. In order to verify the checksum, Stefan needs the raw body.
Unfortunately we don't currently provide any way to prevent the JSON body parser from running if the content-type properly identifies it as JSON.
i haven't understood what you mean by saying that "express doesn't provide it either any more".
Do you mean that using Express it is no more possible to add Middleware that can be set to intercept the raw body? I think that some of my colleagues were actually able to do it.
Isn't there the possibility (and, if yes, HOW) in the code of my web hook to program directly an Express endpoint instead of using the RED.httpNode..post ?
I mean, if I could use the app.use and app.post directly....
See the quoted article. Express used to have the raw body returned but according to that post, doesn't any more.
No, middleware is a really important concept in Express and I use it extensively in my uibuilder node.
Check here:
This gives you access to Express directly & immediately after that, it shows the use of Express middleware.
Note that Node-RED has two different Express instances to work with. One for the admin UI and one for everything else. In truth, I think that my code is serious overkill but I took it from core code. When the admin UI gets split from the runtime, that kind of defensive coding may be more useful.
As you can tell from my var name, this gives you an Express app reference to work with. The middleware can be "use", "get", "post", etc. "use" is the most generic.
uibuilder is very heavily commented (the only way I can ever follow my own code later!) so hopefully everything is pretty well explained. I also have an example repo on GitHub that may be easier to follow. Not all of the example node's code is necessarily correct now as I've not updated it but it might be easier to start with it as it works through and shows each stage of the node's process.
By the time a node gets access to RED.httpNode, the default set of body parsers have already been applied.
As I've said a few times, as the person who wrote the code, there is no way for you to get ahold of the raw request in the current code.
If I wasn't snowed under with high priority work right now, I'd be able to spend some time devising a way to expose it in a future release of Node-RED. But right now, I don't have time to do that. If someone wants to help on that - please do.
Sorry, I probably confused things there. I was only answering the question about getting the reference to the Express app. Of course, most of the middleware has already been applied.
I also mentioned previously that Express no longer provides access to the raw return as indicated in the StackOverflow question that I referenced.
@stefanopog, if you want to confirm whether Express provides some form of access to the raw return. I suggest creating a simple Express app, you will be able to easily examine the data directly without the complexity of Node-RED wrapped around it.
OK, so maybe that StackOverflow answer isn't correct. Never-the-less, as Nick says, Node-RED itself doesn't currently expose it anyway. It would need changes to the Node-RED core.
An easier way right now, if Express is exposing the raw body would be to set up your own Express server. It isn't hard to create a simple server and you could easily then link it to Node-RED by one of the many comms methods ranging from MQTT, WS, HTTP, TCP, UDP or UNIX sockets. But maybe this isn't possible in your setup?
@stefanopog , did you find a solution in the meantime? I have a quite similar requirement - i have to hmac-sha256 the raw body for authentication purposes...
I've already tested with the httpNodeMiddleware but it seems the body already has been parsed here...
we receive messages from internet connected systems. To authenticate the validity of these messages the originator builds a hmac-SHA256 checksum over the body with a shared secret. And to validate this message we need to access the raw body as the transmitted json has some line-breaks and other formatting applied that also influences the checksum...
i had some time digging into it and now it might i've found a solution by picking up @shrickus hint with the httpNodeMiddleware.
In the first tests with the httpNodeMiddleware I discovered that the request already has ben parsed at this point. So I started tracking down where it got parsed and found it: It seems to be a default handler from the admin-UI at this point:
After discovering this line 36 gave me the right hint: We just get the admin UI away form the default path by setting something like httpAdminRoot: '/admin', in settings.js and voila - the request is unparsed in httpNodeMiddleware.
As next step I've created an own jsonParser setting me the raw body. I've also placed this code into settings.js just before the module.exports = { line:
var bodyParser = require('body-parser');
var jsonParserWithRawBody = bodyParser.json({
verify: function (req, res, buf, encoding) {
req.rawBody = buf;
}
})
This parser stores the body into req.rawBody for later usage...
some lines down in settings.js i've commented in the httpNodeMiddleware and conditionally added my parser:
@chrisnie - good digging! I'm currently looking to see if there's a way we can expose a way to do this more gracefully without having edit the settings file - as that makes it harder to just import a node and use it. But glad to see you've found a solution for the time being.
