Http request node misbehaving

General
1

I have been using a http request node in node red to access my rooftop solar forecast successfully for some time now using the following url.

https://api.solcast.com.au/rooftop_sites/[resource_id]/forecasts?hours=336&format=json&api_key=[api_key]

The request usually returns a json object and a status code of 200.

Recently, approx. 24 hrs ago, the node now only returns an empty string with a status code of 429 indicating too many requests, however when I use the same url in a browser, the expected data is returned.

Any clues on what might be causing this and how to correct it would be appreciated.

2

A two thoughts

  • check if the site has started limiting the number of requests allowed per hour/day or what ever
  • Reduce the number of requests you make
3

Also try adding a browser user-agent header to the http request node.

4

Thanks Paul. I have a request for info in with Solcast. In the mean time i will turn the process off for a few days and see what happens.

5

Is the browser running on the same machine that is running node-red?

6

No. Node-red is running on a headless raspberry pi and the browser call was from a windows machine that I was using to access the raspberry pi.

7

add a debug immediately before the http-request node to verify you are not hammering it causing the 429.

Also, if you add a catch node, you can output the error to a debug set to show the "full msg object" here you should be able to see the rate limit headers (if set) to understand when you can next call the endpoint. You could automate a cool-down guard based on the values!

8

Thank you Steve. I had already checked both of those with no success.

9

And as luck would have it, with no apparent change to the input, the process is now working again.
It may have been a glitch on the service provider's side but thank you all for your assistance in any case.
I will watch it for a few days and then mark the topic as solved.

10

Look at the msg.headers after the http-request, a 429 error is likely to include details about what the rate limit is and when you can access it again.

e.g.

Retry-After: 3600

Would me don't try again for the next hour

1 Like
11

Thank you all for your thoughts. The solcast help desk just confirmed that the problem was on their end as a result of another user making too many requests.
But i will pay more attention to the information in the headers from now on.

1 Like
12

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.