Catch node connected to a http response gets in a loop when timeout occurs

[celo]

Hi everyone, how are you all doing?

I have a basic flow where a request to an endpoint on the internet is made by an http request node and its response is sent to a http response. I'm forcing a timeout on purpose on that request (by setting msg.requestTimeout to 1ms, to try my error catch functionality.
I have a Catch node listening to "all nodes", which then connects to a change node and then to the http response node, so it returns a formatted response to the requester.

Problem is, when the timeout occurs, the http response node throws a: "Error [ERR_HTTP_HEADERS_SENT]: Cannot set headers after they are sent to the client"
exception, which is caught again by the Catch node, who redirects the exception to the http response again, and so it goes until it hits that exception 9 times. I'd like to avoid that loop on my flow. Could anyone help?

The code to the scenario is here, to help clarify what I'm trying to describe:

[{"id":"9c4c2f99.21c6e","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"57104a42.b45f84","type":"http in","z":"9c4c2f99.21c6e","name":"","url":"/test","method":"get","upload":false,"swaggerDoc":"","x":160,"y":100,"wires":[["793eb631.aa4418"]]},{"id":"52dca73c.c740a8","type":"http request","z":"9c4c2f99.21c6e","name":"Call external API","method":"use","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":590,"y":240,"wires":[["8ad6211e.aac7"]]},{"id":"8ad6211e.aac7","type":"http response","z":"9c4c2f99.21c6e","name":"response","statusCode":"","headers":{"content-type":"application/json"},"x":820,"y":340,"wires":[]},{"id":"6a312f65.0d599","type":"catch","z":"9c4c2f99.21c6e","name":"Error Catch","scope":null,"uncaught":false,"x":230,"y":360,"wires":[["d78a3b27.fa12e8"]]},{"id":"793eb631.aa4418","type":"function","z":"9c4c2f99.21c6e","name":"Forcing timeout on a request","func":"msg.url = 'http://viacep.com.br/ws/01001000/json/'\nmsg.method = \"GET\"\nmsg.requestTimeout = 1\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":340,"y":180,"wires":[["52dca73c.c740a8"]]},{"id":"d78a3b27.fa12e8","type":"change","z":"9c4c2f99.21c6e","name":"Internal Server Error","rules":[{"t":"set","p":"payload","pt":"msg","to":"{}","tot":"json"},{"t":"set","p":"payload.errorType","pt":"msg","to":"INTERNAL_SERVER_ERROR","tot":"str"},{"t":"set","p":"payload.errorCode","pt":"msg","to":"INTERNAL_SERVER_ERROR","tot":"str"},{"t":"set","p":"payload.errorMessage","pt":"msg","to":"An error has ocurred.","tot":"str"},{"t":"set","p":"statusCode","pt":"msg","to":"500","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":360,"wires":[["8ad6211e.aac7"]]}]

[E1cid]

The error message from the catch node has a count, use a switch node to check if count is <= to 1. (msg.error.source.count)

[{"id":"6a312f65.0d599","type":"catch","z":"9c4c2f99.21c6e","name":"Error Catch","scope":["52dca73c.c740a8"],"uncaught":false,"x":230,"y":360,"wires":[["4fcae190.53b61"]]},{"id":"4fcae190.53b61","type":"switch","z":"9c4c2f99.21c6e","name":"","property":"error.source.count","propertyType":"msg","rules":[{"t":"lte","v":"1","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":400,"y":360,"wires":[["d78a3b27.fa12e8"]]},{"id":"d78a3b27.fa12e8","type":"change","z":"9c4c2f99.21c6e","name":"Internal Server Error","rules":[{"t":"set","p":"payload","pt":"msg","to":"{}","tot":"json"},{"t":"set","p":"payload.errorType","pt":"msg","to":"INTERNAL_SERVER_ERROR","tot":"str"},{"t":"set","p":"payload.errorCode","pt":"msg","to":"INTERNAL_SERVER_ERROR","tot":"str"},{"t":"set","p":"payload.message","pt":"msg","to":"An error has ocurred.","tot":"str"},{"t":"set","p":"statusCode","pt":"msg","to":"500","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":610,"y":360,"wires":[["8ad6211e.aac7"]]},{"id":"8ad6211e.aac7","type":"http response","z":"9c4c2f99.21c6e","name":"response","statusCode":"","headers":{"content-type":"application/json"},"x":820,"y":340,"wires":[]},{"id":"52dca73c.c740a8","type":"http request","z":"9c4c2f99.21c6e","name":"Call external API","method":"use","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":660,"y":180,"wires":[["8ad6211e.aac7"]]},{"id":"793eb631.aa4418","type":"function","z":"9c4c2f99.21c6e","name":"Forcing timeout on a request","func":"msg.url = 'http://viacep.com.br/ws/01001000/json/'\nmsg.method = \"GET\"\nmsg.requestTimeout = 1\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":340,"y":180,"wires":[["52dca73c.c740a8"]]},{"id":"57104a42.b45f84","type":"http in","z":"9c4c2f99.21c6e","name":"","url":"/test","method":"get","upload":false,"swaggerDoc":"","x":160,"y":100,"wires":[["793eb631.aa4418"]]}]

[celo]

Thanks for the response! Doing what @E1cid suggested helps, since I'd be limiting how many times the loop happens, but that by itself does not stop the problem from happening.

I started to get the following exceptions:

• "no response from server": The one I wanted to capture in the first place and return a formatted response to the client, through the http response node

• "Invalid status code: ETIMEDOUT": apparently an exception does NOT halt the execution of the flow (is that an expected behavior?), meaning that on my example, the http request node threw the timeout exception but continued its way directly into the http response node (that got a 'ETIMEDOUT' string instead of an Integer in the msg.statusCode property, throwing another exception).

• "Cannot set headers after they are sent to the client": apparently, since a response was already processed and sent by the first exception to an http request node (its headers were set), this last one happened because the second exception, which tried to send another response and set headers again to the http response node. That's the scenario that causes the loop.

This troubleshoot made me think that the root of the problem might be that an exception doesn't halt the execution of the flow, but I'd really appreciate more help to investigate it. Thanks in advance!

[E1cid]

If you wish to check the api's response then use the statusCode.
You can send different messages depending on the api return status

[{"id":"57104a42.b45f84","type":"http in","z":"5a245aa1.510164","name":"","url":"/test","method":"get","upload":false,"swaggerDoc":"","x":70,"y":3500,"wires":[["793eb631.aa4418"]]},{"id":"793eb631.aa4418","type":"function","z":"5a245aa1.510164","name":"Forcing timeout on a request","func":"msg.url = 'http://viacep.com.br/ws/01001000/json/'\nmsg.method = \"GET\"\nmsg.requestTimeout = 1\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":250,"y":3580,"wires":[["52dca73c.c740a8"]]},{"id":"52dca73c.c740a8","type":"http request","z":"5a245aa1.510164","name":"Call external API","method":"use","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":570,"y":3580,"wires":[["6fe5da70.d43d4c"]]},{"id":"6fe5da70.d43d4c","type":"switch","z":"5a245aa1.510164","name":"","property":"statusCode","propertyType":"msg","rules":[{"t":"eq","v":"200","vt":"num"},{"t":"else"}],"checkall":"false","repair":false,"outputs":2,"x":300,"y":3660,"wires":[["8ad6211e.aac7"],["d78a3b27.fa12e8"]]},{"id":"8ad6211e.aac7","type":"http response","z":"5a245aa1.510164","name":"response","statusCode":"","headers":{"content-type":"application/json"},"x":750,"y":3720,"wires":[]},{"id":"d78a3b27.fa12e8","type":"change","z":"5a245aa1.510164","name":"Internal Server Error","rules":[{"t":"set","p":"payload","pt":"msg","to":"{}","tot":"json"},{"t":"set","p":"payload.errorType","pt":"msg","to":"INTERNAL_SERVER_ERROR","tot":"str"},{"t":"set","p":"payload.errorCode","pt":"msg","to":"INTERNAL_SERVER_ERROR","tot":"str"},{"t":"set","p":"payload.message","pt":"msg","to":"An error has ocurred.","tot":"str"},{"t":"set","p":"statusCode","pt":"msg","to":"500","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":430,"y":3740,"wires":[["8ad6211e.aac7"]]}]

[celo]

Thanks for the response E1cid, I'm able to check the integrations' responses by checking the msg.statusCode property, but what I'm trying to understand is if that behavior of the Http Request and Catch nodes is expected.

If an exception is thrown by any node, shouldn't the execution of the flow stop at that node, be handled by the Catch node (when set) and only continue following the wires from the Catch node beyond? Cause otherwise, the flow execution splits in two, and that might not lead to the correct http response node, for example.

[dceejay]

as usual it depends... in the case of http returns like 404 not found are "good" return codes in that the far end responded and told you it can't find a document - so it's not broken per se - but your flow may not be happy, vs not being able to connect to a server.

[celo]

Hey @dceejay, thanks for the response. I understand that it would be difficult to distinguish a "good" return code from a "bad" one, since that will depend entirely on a developer's need for a specific scenario.

For anyone who might stumble upon the same thing one day, as an alternative for that problem I solved it by putting a Switch node (or a function) to check the integration's status, and stopped the Catch node from retrying any operation or following any paths. I connected it to a debug node only, so that if any exception is thrown, I'll have that logged and monitored on my application.