# Http-in node & payload too large

**URL:** <https://discourse.nodered.org/t/http-in-node-payload-too-large/79077>\
**Category:** General\
**Tags:** docker\
**Created:** [9 June 2023 09:42 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077 "2023-06-09T09:42:44Z")\
**Posts on this page:** 16\
**Page:** 1

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [9 June 2023 09:42 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/1 "2023-06-09T09:42:44Z")

</div>

Hi There!

I'm using a http-in node to post data to my flow. Sometime when I post somethin that is larger than `apiMaxLength` (i.e. 5mb), I get a PayloadTooLarge error in the console of Node-RED.

i.e. my node-red docker console:

```auto
rednode-nrin_1 | PayloadTooLargeError: request entity too large
rednode-nrin_1 | at readStream (/usr/src/node-red/node_modules/raw-body/index.js:156:17)
rednode-nrin_1 | at getRawBody (/usr/src/node-red/node_modules/raw-body/index.js:109:12)
rednode-nrin_1 | at read (/usr/src/node-red/node_modules/body-parser/lib/read.js:79:3)
rednode-nrin_1 | at urlencodedParser (/usr/src/node-red/node_modules/body-parser/lib/types/urlencoded.js:116:5)
rednode-nrin_1 | at Layer.handle [as handle_request] (/usr/src/node-red/node_modules/express/lib/router/layer.js:95:5)
rednode-nrin_1 | at trim_prefix (/usr/src/node-red/node_modules/express/lib/router/index.js:328:13)
rednode-nrin_1 | at /usr/src/node-red/node_modules/express/lib/router/index.js:286:9
rednode-nrin_1 | at Function.process_params (/usr/src/node-red/node_modules/express/lib/router/index.js:346:12)
rednode-nrin_1 | at next (/usr/src/node-red/node_modules/express/lib/router/index.js:280:10)
rednode-nrin_1 | at jsonParser (/usr/src/node-red/node_modules/body-parser/lib/types/json.js:119:7)

```

What I would like to get instead is an exception in the flow that I can capture with the catch node and deal with in the flow. Something like this:

 ![Screen Shot 2023-06-09 at 11.40.19](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/0/4/042786612100e251239d7ae8b4b24fb29c9ac282.png)

But that catch does not capture anything unfortunately. I am blissfully unaware that the post http-in node failed with 413 payload too large.

Is there a fix for this?

Cheers!

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [9 June 2023 11:30 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/2 "2023-06-09T11:30:43Z")

</div>

What is the return code in returned msg?

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [9 June 2023 11:42 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/3 "2023-06-09T11:42:31Z")

</div>

I think you will find the error is not thrown in the endpoint. So the catch can not see it, I think node-reds http server is returning the request as to big.  
@dceejay 413, as stated in OG post

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [9 June 2023 11:49 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/4 "2023-06-09T11:49:24Z")

</div>

> [@E1cid](#):
>
> I think you will find the error is not thrown in the endpoint. So the catch can not see it, I think node-reds http server is returning the request as to big.

Exactly, the frontend code in the [screenshot.html](https://github.com/gorenje/node-red-contrib-introspection/blob/c6f398e9eb78974dfb3df0bd12e36b8647f67378/nodes/15-screenshot.html#L246-L260) file is this:

```auto
        $.ajax({
          type: "POST",
          url: "/screenshot",
          dataType: "image/svg+xml;charset=utf-8",
          data: {
            ...data,
            d: svgdata
          },
          complete: (obj) => {
            switch( obj.status ) {
            case 413:
              // apiMaxLength in the settings.js is set to 5mb by default,
              // if svg larger than that, need to increase the limit.
              // Provide a hint to that setting in the error message.
              RED.notify(
                "Screenshot too large, increase apiMaxLength in settings.js.", {
                  type: "error"
                }
              );
              break;
.... snippet ....

```

and that's where I'm getting the 413 - the http-in node produces nothing. I've got a debug node after it and there is no output.

---

<div class="post-metadata">

**Author:** ![hardillb](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hardillb/32/12373_2.png) [@hardillb](https://discourse.nodered.org/u/hardillb)\
**Post date:** [10 June 2023 20:30 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/5 "2023-06-10T20:30:36Z")

</div>

The size error is thrown by express middleware long before it gets to the http-in handler.

There is no way to get notified of this error in the flow.

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [11 June 2023 08:45 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/6 "2023-06-11T08:45:30Z")

</div>

> [@hardillb](#):
>
> There is no way to get notified of this error in the flow.

Is that a bug or a feature? If I setup an http-in and it generates such an error, should I be informed or not? If I control the code that is using the http-in endpoint (as in this case), I notice it. If OTOH its third party software and I don't regularly check the Node-RED console log, I would remain blissfully unaware of any error.

I would argue that since the fix is increasing the `apiMaxLength` setting, this error should be noticeable on the flow side since the fix is on the flow side. So that error should be caught and passed up to the flow as an exception IMHO.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [11 June 2023 09:04 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/7 "2023-06-11T09:04:40Z")

</div>

> [@gregorius](#):
>
> that error should be caught and passed up to the flow as an exception IMHO.

That does not seem like an unreasonable request. Let's see what other folk think.

Note: while more of a new feature than a bug fix, my expectation would be - should this be implemented, it would be in the next major version.

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [11 June 2023 10:22 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/8 "2023-06-11T10:22:31Z")

</div>

> [@Steve-Mcl](#):
>
> That does not seem like an unreasonable request. Let's see what other folk think.

Cheers! 👍 Should I create an github issue for it?

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [11 June 2023 10:33 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/9 "2023-06-11T10:33:58Z")

</div>

> [@hardillb](#):
>
> The size error is thrown by express middleware long before it gets to the http-in handler.

I think from an end-user perspective, it's too much to require me to know that too, i.e., _Node-RED uses an express middleware for its http-in and it could also generate errors_. As an end-user I should be informed about any component Node-RED depends on that fails or Node-RED should take care of the failure for me. Else I can just use expressjs directly!

> [@hardillb](#):
>
> There is no way to get notified of this error in the flow.

True but I think that should change so that it does hit the flow for the reasoning above!

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [11 June 2023 10:52 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/10 "2023-06-11T10:52:47Z")

</div>

> [@gregorius](#):
>
> Should I create an github issue for it

Yes please. And if you have the capacity to contribute, please branch from (and target your PR to) the `dev` branch.

_A gentle note, to manage expectations: while I am not against this, it is not a given that it will be accepted (and that would be for good reason of course) tho it seems like a desirable feature request._

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [11 June 2023 11:10 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/11 "2023-06-11T11:10:40Z")

</div>

> [@Steve-Mcl](#):
>
> A gentle note, to manage expectations: while I am not against this, it is not a given that it will be accepted (and that would be for good reason of course) tho it seems like a desirable feature request.

Fair enough, and I would not approach this if the idea would not be acceptable to the developer team, hence I wish to have the discussion before the development (disc b4 dev!) 👍

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [12 June 2023 12:22 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/12 "2023-06-12T12:22:16Z")

</div>

Just to summaries our discussion on Slack:

- the exception happens in the [`body-parser`](https://www.npmjs.com/package/body-parser) module which is used (at least) twice in the Node-RED middleware
- first usage is in the [admin middleware stack](https://github.com/node-red/node-red/blob/5d698d66d0857e65b08be41b212b7bcafe5f8261/packages/node_modules/%40node-red/editor-api/lib/index.js#L78)
- second usage is in the [http-in node](https://github.com/node-red/node-red/blob/7c79ca787850e932a6ab3cca0b0b5a81a3d2d95b/packages/node_modules/%40node-red/nodes/core/network/21-httpin.js#L230)

The http-in node [adds itself to the middleware](https://github.com/node-red/node-red/blob/7c79ca787850e932a6ab3cca0b0b5a81a3d2d95b/packages/node_modules/%40node-red/nodes/core/network/21-httpin.js#L263-L273) so the error might happen in the first usage or the second if there is a http-in involved in the flow.

The issue remains that the `body-parser` consumes the error\*. Secondly the association to the node that caused the error is not explicit therefore its difficult to generate a debug entry in the debug panel. If anything, it would be a case of having a log window with the same output as the console window in which Node-RED was started.

\*=it consumes the error but it also passes it along to the middleware chain using [`http-errors`](https://www.npmjs.com/package/http-errors) and a call to `next(createError(...))`

As there doesn't seem to be an elegant solution to this, IMO there is no real point in creating an issue that will simply remain open (except perhaps to point to this discussion).

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [12 June 2023 13:30 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/13 "2023-06-12T13:30:05Z")

</div>

> [@gregorius](#):
>
> Secondly the association to the node that caused the error is not explicit therefore its difficult to generate a debug entry in the debug panel

In the HTTP-IN node, the routes are setup (and their middlewares) dynamically, at runtime, in the HTTP-IN node. It may be feasible to wrap the [bodyparser](https://github.com/node-red/node-red/blob/master/packages/node_modules/%40node-red/nodes/core/network/21-httpin.js#L230) (and catch any errors inside the HTTP-IN node) before [adding it to the route](https://github.com/node-red/node-red/blob/master/packages/node_modules/%40node-red/nodes/core/network/21-httpin.js#L264-L272) and therefore would be in the right context to raise an error from the actual HTTP-IN node.

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [12 June 2023 13:31 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/14 "2023-06-12T13:31:32Z")

</div>

> [@gregorius](#):
>
> Just to summaries our discussion on Slack:

Can you link to the discussion please?

---

<div class="post-metadata">

**Author:** ![gregorius](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/gregorius/32/73816_2.png) [@gregorius](https://discourse.nodered.org/u/gregorius)\
**Post date:** [12 June 2023 13:37 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/15 "2023-06-12T13:37:02Z")

</div>

> [@Steve-Mcl](#):
>
> Can you link to the discussion please?

Yep - discussion starts here [Slack](https://node-red.slack.com/archives/C9RSU6S56/p1686482594694639)

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [11 August 2023 13:37 UTC](https://discourse.nodered.org/t/http-in-node-payload-too-large/79077/16 "2023-08-11T13:37:58Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.