Webhook Relay 0.3.0 node release (now you can respond to API requests)

TL;DR (short version)

With node-red-contrib-webhookrelay node you can now receive HTTP requests and send back responses to the caller without having public IP/NAT/domain (Node-RED can be running on your laptop or under a desk without any router config). You just need to return a custom JSON object back to the webhookrelay node with your desired response body, status code and headers.

Ideal when you want to expose a secure, restricted API without exposing whole Node-RED instance to the internet.

The longer version

In the previous article about controlling gadgets via IFTTT and Node-RED I showed a new way to receive webhooks without public IP or configuring NAT and then performing certain actions. However, sometimes you need to respond back to the webhook producers or just other applications that expect success/error responses to properly function. Up until now you would have needed to use Webhook Relay tunnels but with the recent release, we allow sending dynamic responses back to the caller.

Pros:

  • No need to expose your Node-RED to the internet.
  • Respond with carefully crafted HTTP responses choosing your status code, headers and body.

This feature transforms Webhook Relay webhook forwarding feature from unidirectional-only to a much more powerful tool.

How it works

Webhook responses work by pausing HTTP response for up to 10 seconds while waiting for the response. The rules are simple:

  1. You have to explicitly declare on the Webhook Relay output (via relay CLI or web dashboard) that it should wait for the response.
  2. Your application has to send response back to the incoming webhook within 10 seconds.
  3. Response cannot be larger than 3MB (usually API responses are a lot smaller).
  4. Response has to include original meta object that you have received with the webhook (it contains unique request ID and bucket ID that are required by Webhook Relay to correctly respond)

Creating an API with Node-RED and Webhook Relay node

We will create a simple API backend that will return current weather information for a selected city:

Here we will use several custom nodes nodes:

Other nodes are from the standard palette.

Note that: We need to preserve payload.meta object from the original Webhook Relay message as we will be using it to reply to the correct request. Your application has 10 seconds to send a reply and there might be several request in-flight that Node-RED is dealing with.

Webhook Relay node configuration

  1. Go to buckets page and create a new bucket.
  2. Once you have a bucket, open Input settings and select that it should wait for a reply from "Any output":

  1. Go to tokens page and get your key/secret pair.
  2. Add bucket name and token key/secret pair to the node-red-contrib-webhookrelay node.

Configuring the flow

  1. grab request metadata for later response

We will have to join this later with the rest of the data to correctly respond to the caller.

  1. parse URL query

now, since we HTTP requests with a query like http://example.com/weather?city=London&country=GB, we need to get these details into an object that openweather node will understand. Here's the code:

function getJsonFromUrl(url) {
  if(!url) url = location.search;
  var query = url.substr(0);
  var result = {};
  query.split("&").forEach(function(part) {
    var item = part.split("=");
    result[item[0]] = decodeURIComponent(item[1]);
  });
  return result;
}

return {
    location: getJsonFromUrl(msg.payload.query)
}
  1. request weather

  1. encode JSON data - here we just take the response from openweather response and encode it into a JSON string. This payload will be returned to the caller.

  2. put message into a "path" for later join - same as number 1:

  1. join metadata and data - time to join weather data and request metadata:

  1. form API response - use "function" node to grab "meta" and "data" values from the previous node
return {
  meta: msg.paths["meta"].meta,  // this is original meta field from the payload (it's important to include it so we have the message ID)
  status: 200,   // status code to return (200, 201, 400, etc)
  body: msg.paths["data"].data, // body
  headers: {
    'content-type': ['application/json']  // good practice to include content type, browsers do their best to display it nicely
  }
};

That's it, connect the response node back to the Webhook Relay node and open your Bucket's input URL in your browser or just use curl (if you are on Linux or Mac):

curl https://my.webhookrelay.com/v1/webhooks/YOUR-INPUT-UUID?city=London&country=GB

What's next

Try out integrating different APIs. If free tier is too low for you, message me and I will bump up your limits :slight_smile:

Here's the flow itself, feel free to import and play with it.

[{"id":"44a1295a.6a99d","type":"tab","label":"Node-RED API","disabled":false,"info":""},{"id":"5b239f76.3f86d8","type":"webhookrelay","z":"44a1295a.6a99d","buckets":"node-red-responses","x":160,"y":320,"wires":[["329e9d6b.728d6a","8858f09a.c7aee8"]]},{"id":"ee1d7dc4.e7c358","type":"function","z":"44a1295a.6a99d","name":"create response","func":"return {\n    meta: msg.paths[\"meta\"].meta,  // this is original meta field from the payload (it's important to include it so we have the message ID)\n    status: 200,   // status code to return (200, 201, 400, etc)\n\tbody: msg.paths[\"data\"].data, // body\n\theaders: {\n\t  'content-type': ['application/json']\n\t}\n};","outputs":1,"noerr":0,"x":1280,"y":320,"wires":[["5b239f76.3f86d8"]]},{"id":"c3ea1e3b.a8c708","type":"openweathermap","z":"44a1295a.6a99d","name":"","wtype":"current","lon":"","lat":"","city":"","country":"","language":"en","x":510,"y":460,"wires":[["27524c01.87056c"]]},{"id":"329e9d6b.728d6a","type":"function","z":"44a1295a.6a99d","name":"get city name","func":"function getJsonFromUrl(url) {\n  if(!url) url = location.search;\n  var query = url.substr(0);\n  var result = {};\n  query.split(\"&\").forEach(function(part) {\n    var item = part.split(\"=\");\n    result[item[0]] = decodeURIComponent(item[1]);\n  });\n  return result;\n}\n\nreturn {\n    location: getJsonFromUrl(msg.payload.query)\n}","outputs":1,"noerr":0,"x":250,"y":460,"wires":[["c3ea1e3b.a8c708"]]},{"id":"27524c01.87056c","type":"json","z":"44a1295a.6a99d","name":"encode","property":"payload","action":"","pretty":false,"x":720,"y":460,"wires":[["13242f86.520e8"]]},{"id":"cf7bfe30.6a52b8","type":"wait-paths","z":"44a1295a.6a99d","name":"wait for meta and data","paths":"[\"data\",\"meta\"]","timeout":15000,"finalTimeout":60000,"x":1020,"y":320,"wires":[["ee1d7dc4.e7c358"]]},{"id":"13242f86.520e8","type":"change","z":"44a1295a.6a99d","name":"paths[\"data\"]","rules":[{"t":"move","p":"payload","pt":"msg","to":"paths[\"data\"].data","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":930,"y":460,"wires":[["cf7bfe30.6a52b8"]]},{"id":"8858f09a.c7aee8","type":"change","z":"44a1295a.6a99d","name":"paths[\"meta\"]","rules":[{"t":"move","p":"payload.meta","pt":"msg","to":"paths[\"meta\"].meta","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":570,"y":320,"wires":[["cf7bfe30.6a52b8"]]},{"id":"78312bed.0624fc","type":"comment","z":"44a1295a.6a99d","name":"1. grab request metadata for later response","info":"","x":660,"y":260,"wires":[]},{"id":"d13cd56d.3e6a08","type":"comment","z":"44a1295a.6a99d","name":"2. parse URL query","info":"","x":270,"y":400,"wires":[]},{"id":"b0668370.e60a88","type":"comment","z":"44a1295a.6a99d","name":"3. request weather","info":"","x":510,"y":400,"wires":[]},{"id":"a9a0d2e1.527298","type":"comment","z":"44a1295a.6a99d","name":"4. encode json","info":"","x":740,"y":400,"wires":[]},{"id":"fe57aa5.43ee4d8","type":"comment","z":"44a1295a.6a99d","name":"5. put message into a \"path\" for later join","info":"","x":1020,"y":400,"wires":[]},{"id":"d0d96c0d.930398","type":"comment","z":"44a1295a.6a99d","name":"6. join metadata and data","info":"","x":1030,"y":260,"wires":[]},{"id":"f25f94a5.7ad5d8","type":"comment","z":"44a1295a.6a99d","name":"7. form API response","info":"","x":1300,"y":260,"wires":[]}]

Originally published here https://webhookrelay.com/blog/2020/03/02/responding-to-api-calls-on-nodered/ but thought it makes sense to add it here as well :slightly_smiling_face:

A question for Node-RED community

What other ways we could use to pass the metadata from the original request back? maybe rely on the _msgid (or something like that which is set by Node-RED itself and just have internal cache inside webhookrelay node of IDs and metadata?)

3 Likes

Hi, thanks for sharing this. Always good to see another method for connecting Node-RED to the Internet.

One point though, it would I think be better to make it clearer that the free tier does not support HTTPS. Meaning that traffic could be intercepted. Since you mention secure connections above, it is important for people to understand that they need to be on a paid tier for that.

Hi, thanks for the feedback, free tier for webhooks provides HTTPS as well (all endpoints https://my.webhookrelay.com/v1/webhooks/... are encrypted). The only limitation on free tier is the quantity of the webhooks.

I think you had in mind HTTPS tunnels (https://webhookrelay.com/v1/guide/http-tunnels.html) which require a separate Docker container or relay agent running on the host.

I thought the same (screenshot below).
However, the free tier is just really a POC, as 150 webhooks per month is insufficient to use even for iot home use.

1 Like

Yeah, I guess once the responses are now supported for the webhooks it might be a good time to also update feature descriptions.

As for pricing, paid tier starts at $4.5 and as long as you don't reach ~8-10K reqs/month it won't be an issue, however going further it would be nice if users upgraded :slight_smile:

Also, for most Node-RED/Home Assistant users I increase the limits as the initial 150/month was mostly for CI/CD workflows where users would be integrating it into Jenkins, Drone, Gitlab, there 150 is good enough for a lot of companies. Can't do a blanket increase in limits as I might lose paid subscriptions from that potential user base :smiley:

1 Like

OK, perhaps this section of your web page might be slightly clearer for those of us who find it challenging to read through to the end of a paragraph properly! :slight_smile:

image

5 a day or thereabouts. Not a lot but would suffice for some things.

Certainly another interesting option. It would be enough for the garage door opening issue that came up the other day for example.

Isn't that what I showed in the screenshot - my last post above.
Who is finding it challenging? :wink:

I answered before reading your post Paul.

2 Likes

on another note - any better ideas how would it be possible to connect back two flows where I need to get some data from the original request for response? :smiley: Looked at split/join nodes but it seemed like they have a bit different use-case

This topic was automatically closed after 60 days. New replies are no longer allowed.