# Website opens in chrome but gives error message when using node red http node

**URL:** <https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071>\
**Category:** General\
**Created:** [4 November 2022 04:32 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071 "2022-11-04T04:32:31Z")\
**Posts on this page:** 17\
**Page:** 1

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [4 November 2022 04:32 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/1 "2022-11-04T04:32:31Z")

</div>

If i open the following link in chrome it works fine and displays all data,

[http://www.bom.gov.au/fwo/IDN60801/IDN60801.94767.json](http://www.bom.gov.au/fwo/IDN60801/IDN60801.94767.json)

but if i use same in node red http request node I get the following in debug.

"

Your access is blocked due to the detection of a potential automated access request. The Bureau of Meteorology website does not support web scraping: if you are trying to access Bureau data through automated means, you should stop. You may like to consider the following options:

- An anonymous FTP channel: [http://www.bom.gov.au/catalogue/anon-ftp.shtml](http://www.bom.gov.au/catalogue/anon-ftp.shtml) - this is free to access, but use is subject to the default terms of the Bureau's copyright notice: [http://www.bom.gov.au/other/copyright.shtml](http://www.bom.gov.au/other/copyright.shtml) 
- A Registered User service for continued use of Bureau data if your activity does not comply with the default terms: [http://reg.bom.gov.au/other/charges.shtml](http://reg.bom.gov.au/other/charges.shtml) noting charges apply to most data products. Please contact webreg@bom.gov.au to discuss your requirements.

If you still nee..."

can anyone help, thanks

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [4 November 2022 05:25 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/2 "2022-11-04T05:25:35Z")

</div>

It requires headers to be set.

Can you try with this http request:

```auto
[
    {
        "id": "3a0493d9fee81171",
        "type": "http request",
        "z": "92d9e3127ad54058",
        "name": "",
        "method": "GET",
        "ret": "obj",
        "paytoqs": "ignore",
        "url": "http://www.bom.gov.au/fwo/IDN60801/IDN60801.94767.json",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [
            {
                "keyType": "other",
                "keyValue": "User-Agent",
                "valueType": "other",
                "valueValue": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.0 Safari/605.1.15"
            },
            {
                "keyType": "other",
                "keyValue": "Accept",
                "valueType": "other",
                "valueValue": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
            },
            {
                "keyType": "other",
                "keyValue": "Upgrade-Insecure-Requests",
                "valueType": "other",
                "valueValue": "1"
            },
            {
                "keyType": "other",
                "keyValue": "Connection",
                "valueType": "other",
                "valueValue": "keep-alive"
            }
        ],
        "credentials": {},
        "x": 450,
        "y": 280,
        "wires": [
            [
                "09dab6285439ee8f"
            ]
        ]
    }
]

```

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [4 November 2022 07:07 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/3 "2022-11-04T07:07:06Z")

</div>

> [@wassa350](#):
>
> The Bureau of Meteorology website does not support web scraping

Are you asking for help with breaking the bureau's terms of use?

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [4 November 2022 19:23 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/4 "2022-11-04T19:23:37Z")

</div>

Is opening the site with node Red scraping ????

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [4 November 2022 21:36 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/5 "2022-11-04T21:36:38Z")

</div>

> [@wassa350](#):
>
> Is opening the site with node Red scraping

Yes, unless, having read it, you are just going to display it in the dashboard, or similar. In which case you would be better to use an iframe.

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [6 November 2022 02:50 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/6 "2022-11-06T02:50:19Z")

</div>

Yes, I only want to view the data on my dashboard. Would that be termed scraping???

I could display it from the info that I get when I open the page in chrome. I am struggling to see what is different to opening directly in node Red. Morally/ legally/ or any other code I mean.

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [6 November 2022 05:49 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/7 "2022-11-06T05:49:45Z")

</div>

Hi Colin, please advise what is iframe that you mention and how is it used for getting and displaying the json data from this site???  
thankyou

---

<div class="post-metadata">

**Author:** ![zenofmud](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/zenofmud/32/316_2.png) [@zenofmud](https://discourse.nodered.org/u/zenofmud)\
**Post date:** [6 November 2022 08:57 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/8 "2022-11-06T08:57:24Z")

</div>

> [@wassa350](#):
>
> what is iframe

google `what is iframe` and you will find lots of information.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [6 November 2022 09:03 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/9 "2022-11-06T09:03:50Z")

</div>

Also search the forum for  
dashboard iframe  
and you will find numerous posts about doing it.

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [6 November 2022 11:22 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/10 "2022-11-06T11:22:31Z")

</div>

if i copy this script above and paste into http request the node indicates that there is an error showing the red triangle.

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [6 November 2022 11:25 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/11 "2022-11-06T11:25:57Z")

</div>

thanks Colin, i can see the json data in my dashboard using iframe node. Is it possible to extract specific pieces of the json data from the iframe node to then display using graph or gauge in dashboard?????

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [6 November 2022 11:35 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/12 "2022-11-06T11:35:53Z")

</div>

> [@wassa350](#):
>
> i can see the json data in my dashboard using iframe node

No it isn't possible. Normally the iframe would be used to show the webpage, not the json.

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [6 November 2022 12:08 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/13 "2022-11-06T12:08:18Z")

</div>

The code IS the http request, tested it, it works.

Import the code into node red and a request node with the correct properties will be added to the flow.

---

<div class="post-metadata">

**Author:** ![andro](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andro/32/70042_2.png) [@andro](https://discourse.nodered.org/u/andro)\
**Post date:** [6 November 2022 15:51 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/14 "2022-11-06T15:51:41Z")

</div>

Hello from Australia.

I get the same error using your URL. Something not right about that endpoint. Where did you get that from?

Try this:

[http://reg.bom.gov.au/fwo/IDV60901/IDV60901.95936.json](http://reg.bom.gov.au/fwo/IDV60901/IDV60901.95936.json)

I note the URL has reg.bom not www.bom.

That's the weather for Melbourne. Works great.

if that works for you I can point you to the page where you get these endpoints.

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [7 November 2022 01:38 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/15 "2022-11-07T01:38:23Z")

</div>

Thankyou Andrew, this link works perfectly. I adjusted the IDV stuff and now can see my local weather.

Again, thanks for you assistance

---

<div class="post-metadata">

**Author:** ![wassa350](https://avatars.discourse-cdn.com/v4/letter/w/51bf81/32.png) [@wassa350](https://discourse.nodered.org/u/wassa350)\
**Post date:** [7 November 2022 01:41 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/16 "2022-11-07T01:41:45Z")

</div>

I imported into node red, but I still get the errors about scraping as previously

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [6 January 2023 01:42 UTC](https://discourse.nodered.org/t/website-opens-in-chrome-but-gives-error-message-when-using-node-red-http-node/70071/17 "2023-01-06T01:42:36Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.