I need to get the RAW body from a listener node

[chrisnie]

Hi @knolleary
I already started extending the 21-httpin.js with an additional option to expose the rawBody (same concept with the body-parser's verify function) but it didn't work. The current solution works just by modifiing the settings.js so no patching of Node-RED itself was necessary. But when you have a way / idea how to deal with the admin interface i could make out a PR out with my extensions...

Cheers, Chris

[Toastgeraet]

Some late praise @chrisnie, great work! Exactly what i was looking for. Also the use case of verifying sha256-hmac of the request, for auth purposes. In the mean time, has any progress been made integrating this into NodeRED proper @knolleary?

[knolleary]

@Toastgeraet

has any progress been made integrating this into NodeRED proper @knolleary?

No - this dropped off my radar and no-one else picked it up.

[Toastgeraet]

Alright, thx for the update!

[alexk111]

One more vote for the raw body option in http in nodes. Having the same issue with the request verification.

[srk78]

Another bump for this functionality. Request verification for webhooks fired from WooCommerce is also done by a hmac-SHA256 checksum over the body with a shared secret.

[MarkPartlett]

A bump for this functionality from me too. I’m trying to use the rawBody for validation of the Nextcloud webhooks app : GitHub - kffl/nextcloud-webhooks: Webhooks app for Nextcloud using SHA256.

[stathisaska]

@chrisnie

After discovering this line 36 gave me the right hint: We just get the admin UI away form the default path by setting something like httpAdminRoot: '/admin', in settings.js and voila - the request is unparsed in httpNodeMiddleware.

@chrisnie Did not work for me. I am still getting parsed body

[stathisaska]

@chrisnie

As next step I've created an own jsonParser setting me the raw body. I've also placed this code into settings.js just before the module.exports = { line:

Added that and now node red won't load

[Colin]

You may have a syntax error in settings.js. What error do you see when you try to start node red?

[stathisaska]

Step 1: Identify current state

node-red before starting anything

22 Jun 16:39:41 - [info] Node-RED version: v2.2.2
22 Jun 16:39:41 - [info] Node.js  version: v12.22.12
22 Jun 16:39:41 - [info] Linux 5.13.0-1031-aws x64 LE
22 Jun 16:39:41 - [info] Loading palette nodes
22 Jun 16:39:42 - [info] Settings file  : /home/ubuntu/.node-red/settings.js
22 Jun 16:39:42 - [info] Context store  : 'default' [module=memory]
22 Jun 16:39:42 - [info] User directory : /home/ubuntu/.node-red
22 Jun 16:39:42 - [warn] Projects disabled : editorTheme.projects.enabled=false
22 Jun 16:39:42 - [info] Flows file     : /home/ubuntu/.node-red/flows.json
22 Jun 16:39:42 - [warn] 

---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.

You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------

22 Jun 16:39:42 - [error] Unable to listen on http://127.0.0.1:1880/
22 Jun 16:39:42 - [error] Error: port in use

Note I am not using the default port

Step 2: Enable httpNodeMiddleware

    httpNodeMiddleware: function(req,res,next) {
    //    // Handle/reject the request, or pass it on to the http in node by calling next();
    //    // Optionally skip our rawBodyParser by setting this to true;
       req.skipRawBodyParser = true;
       next();
    },

...restart node red pm2 restart node-red

node-red provides same result

Running a flow with http in... input still parsed object

Step 3: Change the admin panel from my.domain.com to my.domain.com/admin

in settings.js, just enabled the

httpAdminRoot: '/admin',

... restart node red pm2 restart node-red

and check node-red

The terminal output is now different

22 Jun 16:51:47 - [info] Node-RED version: v2.2.2
22 Jun 16:51:47 - [info] Node.js  version: v12.22.12
22 Jun 16:51:47 - [info] Linux 5.13.0-1031-aws x64 LE
22 Jun 16:51:48 - [info] Loading palette nodes
22 Jun 16:51:48 - [info] Settings file  : /home/ubuntu/.node-red/settings.js
22 Jun 16:51:48 - [info] Context store  : 'default' [module=memory]
22 Jun 16:51:48 - [info] User directory : /home/ubuntu/.node-red
22 Jun 16:51:48 - [warn] Projects disabled : editorTheme.projects.enabled=false
22 Jun 16:51:48 - [info] Flows file     : /home/ubuntu/.node-red/flows.json
22 Jun 16:51:48 - [warn] 

---------------------------------------------------------------------
Your flow credentials file is encrypted using a system-generated key.

If the system-generated key is lost for any reason, your credentials
file will not be recoverable, you will have to delete it and re-enter
your credentials.

You should set your own key using the 'credentialSecret' option in
your settings file. Node-RED will then re-encrypt your credentials
file using your chosen key the next time you deploy a change.
---------------------------------------------------------------------

22 Jun 16:51:48 - [error] Unable to listen on http://127.0.0.1:1880/admin/
22 Jun 16:51:48 - [error] Error: port in use

Catching a new webhook... msg.payload is again an object

After changing the root directory, I also get a 404 error from Postman. The error is due to cryptojs library I manually added in node red. I assume the library cannot be accessed from the /admin directory

Step 4: Try the adding the code

Coded Added

var bodyParser = require('body-parser');
var jsonParserWithRawBody = bodyParser.json({
    verify: function (req, res, buf, encoding) {
        req.rawBody = buf;
    }
})

in the very beginning - just above module.exports

Coded removed

Commented out the httpNodeMiddleware that I enabled in Step 2 and added the code above instead

httpNodeMiddleware: function(req,res,next) {
        if (req.url === '/my/httpin/url/where/i/want/to/have/the/raw/body') {
           jsonParserWithRawBody(req, res, next);
        } else
           next();
    },

...restart ... node-red

Error loading settings file: /home/ubuntu/.node-red/settings.js
ubuntu@ip-172-31-32-243:~$

Nothing works, I am exhausted

[Colin]

@stathisaska

Error: port in use

You have to stop the current node red instance from running before you start another one.

[stathisaska]

@Colin

I restarted the server and I am getting the same error. Maybe pm2 ? Is this relevant to the issue ?

[Colin]

If the port is already in use then almost certainly node red is already running.

[Colin]

The settings.js error means you have messed it up. Restore the previous version from your backup and start again.

[stathisaska]

Ok will try starting from scratch

[stathisaska]

@Colin Getting the same error on Ubuntu 20.04.4 LTS

The error appeared after installing pm2

[Colin]

Which error exactly?

[stathisaska]

I could not make this work. I highly discourage anyone trying it as it turned out an outrageous amount of time waste for me.

I used n8n instead