Http in - Get - module is returning a timeout error

General
1

RemoteDisconnected: Remote end closed connection without response.

The request is a long request and the time of finish is more than 2 minutes.

nodered

The operating logic is to receive a request from an external system, carry out queries in subsystems within Node-RED and then unite all the data and return it to the system that requested it (external system).

So how can it be configured so that the initial query of the external system with Node-RED is not completed?

2

I believe this depends more on the client system than the HTTP nodes in Node-Red. Those will receive and send messages based on what they get and when. I don't believe they do anything with timeouts. So it comes down to whatever is processing on the client side (i.e. browser or program) to determine an appropriate timeout for the request return to be handled.

That said, there's some things you can do to mitigate what's happening. First of all, if you're not already doing so, can you perform sub-system queries in parallel so they run concurrently instead of consecutively? If you know that a query is coming, is it possible to cache the results periodically? For instance, if data is good for say five minutes, you could set a timer to query the sub-system every five minutes so the data is cached and ready to pull instantly when requested. Or maybe you just cache whatever query takes a long time and allow the rest to be live queries.

Another option is instead of using one node pair to run the entire query set, why not split it out and run multiple nodes for individual queries?

Another question, is there anything in the client side of the system you could used to "deposit" the data after a query is completed? You could have the client send the request and immediately send the acknowledgement while the queries begin running. Once they're done, the server sends the data packet to a receiver on the client side. Two separate transactions but both are handled without any wait delays.

Again, I don't know the specifics of your setup, but there are ways of spanning the long wait period. If there's something that can be done besides lengthening the timeout, you would be better off implementing those so that any problems with the connection can be detected sooner (the real purpose behind the timeout function).

Are any of those possibilities?

3

Where and when are you seeing that?

4

Do you mean it takes node red 2 minutes to prepare the response to send back to the remote system?

5

Can the external system have a http timeout value defined as per your needs?
I dont think you can handle it in node red (afaik).

6

Exactly, for this specific application, it will be necessary to carry out subqueries in other systems, aggregate the data and return the data treated with some rules to the requestor. The time between the request and the return must be greater than 2 minutes, and it is necessary to adjust Node-Red for this. A test installation I performed on Node-Red did not show this error, I inserted a 3-minute delay between http in and http response, and it worked normally. Is there any parameter in the settings that can be adjusted this time?

delay3m

7

I believe that the error is in Node-Red, even because I simulated an Http Resquest module in node-red requesting data from http in. The error also occurs.

Error: Remote end close connection without response

8

What is the difference between that and the system that fails? Are you requesting the API from the same client? I expected it to be the client that has timed out and closed the connection. You did not reply to the question as to where you are seeing the error, if it is in node-red then which node is it coming from?

9

There is a server in production where after 2 minutes the error occurs. The test server is default, I believe the production one too, using the settings file of the production server on the test server, the result does not change. I believe it is the server that is closing the connection because requesting with the http_request module (node-red) the error also occurs, even increasing the timeout of this module. The Error appears on the client, stating that the server closed the connection with no response.

10

I don't this this is cause by node-red, but by the program that is requesting the url.
Is this a python program by any chance ?

11

It's a simple python request, in another Node-Red installation the problem doesn't occur

def buscar_dados():
warnings.filterwarnings('ignore')
headers = {'content-type': 'application/json','X-Requested-With':'message/http'}
response = requests.get(f"http://127.0.0.1:1066/teste")

#dados = json.loads(response.content)  
return response
12

A lot has been written about timeouts with python requests. I would suggest to try this code instead, under the "condensed" header.

in another Node-Red installation the problem doesn't occur

this all depends on the variables at play.

13

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.