Please help with Xero webhook to Node Red

General
1

I have been dealing with cryptography craziness since yesterday, I 've literally lost my sleep over this.

I am implementing a node red solution to get webhooks from Xero to be written in a custom app. I have experienced a lot of issues with the payload and how it needs to be stringified and how needs to be hashed, but eventually figured it out thanks to a Github fellow that posted this code to get the body to its 'raw' state

let msgPayloadRaw = JSON.stringify(msg.payload).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"');

I then create a sha256 base64 hash to check against the header value using the following js code. I ended up importing the cryptojs library, because I could not find a way to make this output with the nodes. HMAC needs to be binary before returning to base64, whereas the nodes only return string.

var cryptojs = context.global.cryptojs;

const webhookKey = 'MyWebhookKeyHere';

let msgPayloadRaw = JSON.stringify(msg.payload).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"');


let bdata = new Buffer(msgPayloadRaw).toString();

let ciphertext = cryptojs.HmacSHA256(bdata, webhookKey );

let base64encoded = cryptojs.enc.Base64.stringify(ciphertext);


msg.payload = base64encoded;

return msg;

Now everything should work great, but I get a crazy result showcased in this recording, where the web hooks intent status turns to 'OK', and some seconds later returns to this error:

Retry
We haven’t received a successful response on the most recent delivery attempt and will retry sending with decreasing frequency for an overall period of 24 hours.
Response not 200. Learn more
Last sent at 2022-06-22 11:48:28 UTC

Anyone experienced that ? Any suggestions ?

My guess is that the stringify code works only for payloads that have no events, because by debugging the flow, I can see that the last call is the only one with events count > 0. In addition, the 'false' tries are only a letter different than the hash, whereas the try with the events object count > 0 contains a completely different hash against the one I am computing

Is there anyway I can get the raw string body of the HTTP request, instead of a JSON object ? The JSON.Stringify is not returning an identical result (spaces), which means the hash is different and the workaround mentioned above is not working in every case.

2

I don't know what is Xero, never set up a webhook for that service... that said, reading their doc it says they're expecting these 5 things from your node-red:

It uses HTTPS on the standard 443 port
It responds within 5 seconds with a 200 O.K status code
There is no body in the response
There are no cookies in the response headers
If the signature is invalid a 401 Unauthorised status code is returned

Is the flow dealing with the webhook able to handle all these conditions?

If yes, maybe you can share the flow and we can have a look at it.

3

Hello @lu4t

Thank you for participating in this post.

I have now confirmed that the issue is related to the body.

Just forget the Xero part, the problem is very simple:

Body needs to be hashed. The body in the http request is like that

{"events":[],"firstEventSequence": 0,"lastEventSequence": 0, "entropy": "IVMMHNWPBAZYRZJRCUAQ"}

Notice the spaces after each :

Node Red converts that body to JSON object. When I do JSON.stringify(msg.payload); I will get the following

{"events":[],"firstEventSequence":0,"lastEventSequence":0, "entropy":"IVMMHNWPBAZYRZJRCUAQ"}

which is obviously the same, but when hashed it isn't because of the spacing.

The GitHub fellow did that that walkround

JSON.stringify(msg.payload).split(':').join(': ').split(': [').join(':[').split(',"entropy"').join(', "entropy"');

But this works only when the events array is empty. That's because it's a walkaround, not the actual solution. The solution would be to get the raw string of the http request, is that possible using Node Red ?

4

All they're requesting could be done with node-red. I have several webhook receivers working that are very similar to the one you're trying to make.

What I don't understand is: why you say the problem is in the body, when they clearly say the body has to be empty?

As I said, I am happy to help but without the flow is hard to tell what could be wrong.

5

The response body should be empty. Not the input.

The input is as explained above and it needs to be verified using a hash and comparing to a header value. I know it is the body, because I have made it partially work using that GitHub fellow's walkaround.

What I need is either get the raw body of the request or found a proper text replacement space adding walkaround. I 've found that in php the have solved that idiotic problem using the solution quoted below

The only solution is to edit p h p . i n i and set:
always_populate_raw_post_data = -1

You can't turn it off by setting error_reporting(0) or by using ini_set() to change that setting.

I discovered this when watching this helpful video, Let’s play (web)hooky with PHP. Over a million small businesses, and… | by Sid Maestre | Xero Developer, and figuring out that it's the same method that I was using and it failed on my system as well.

In the process I also learned something new. I've been a huge fan of ngrok for awhile, but I didn't know until tonight that it has a response/request viewer built-in. (localhost:4040).

6

Do you mean from an HTTP Request node? If so then the node has an option to return a string.

7

Hi @Colin

Thank you for your contribution.

This flow is a backend service consuming web hooks. I am using 'http in' unfortunately, which does not support to get the body into msg.payload as UTF-8 string

8

Seems like my issue is same with this

9

Have you looked at the Xero Developer Docs - they have Node.JS libraries available ?

They also provide a number of example apps to show both basic and advanced functionality in their Github repository.

Lastly if you look at n8n and make.com - they both have a number of integrations/recipes that you can access (and fairly active forums also) that address a lot of the issues - specifically on n8n i was looking (just yesterday) at a forum thread that discussed using their http-in node for some additional functionality

Craig

10

Hi @craigcurtin

I am heavily using all API services, including make, Zapier, Claris Connect and recently n8n.

For this specific implementation, Node Red is the best fit.

11

Yes i was more pointing out the fact that the two specific services i mentioned gave http methods that could be adapted to use in NR that could assist you with your issue

Craig

12

After a lot of research, I found out that it might be possible to get the RAW body, but I could not make it work

13

I ended up using n8n for this, since it was not possible to be developed in Node Red, at least without crazy hocus pocus that I could not recreate.

14

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.