# Extracting data from web page

**URL:** <https://discourse.nodered.org/t/extracting-data-from-web-page/46769>\
**Category:** General\
**Created:** [6 June 2021 11:07 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769 "2021-06-06T11:07:30Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [6 June 2021 11:07 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769/1 "2021-06-06T11:07:30Z")

</div>

Hallo all,  
I have been trying to get a value from an extranal webpage with the http request node. I just dont seem to get it working. I dont understand how to get the correct url from the web page.

 ![Capture](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/d/e/ded97e5bde8c68d258c4edc6db6a1eb75534c341.jpeg)

can you please help?

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [6 June 2021 11:17 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769/2 "2021-06-06T11:17:49Z")

</div>

> [@Duncan](#):
>
> I dont understand how to get the correct url from the web page

You obviously already know the URL (as you are looking at the page src)

Do you mean you dont know how to get specific the data from the page into node-red?

What have you tried so far?

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [6 June 2021 12:03 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769/3 "2021-06-06T12:03:28Z")

</div>

HI steve, thanks for your reply. I have the info in front of me but I don't exactly understand fow to filter down to the variable I need. If I get a little help to grasp the concept I think I will be on my way.

```auto
[{"id":"a2b07ec9.73957","type":"inject","z":"ae9889c1.d28c78","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":180,"y":640,"wires":[["ca3e73c1.fc47d"]]},{"id":"ca3e73c1.fc47d","type":"http request","z":"ae9889c1.d28c78","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"http://duco001/boxinfo.html","tls":"","persist":false,"proxy":"","authType":"","x":470,"y":640,"wires":[["ad4f898b.d61258","979c94ca.eea358"]]},{"id":"ad4f898b.d61258","type":"debug","z":"ae9889c1.d28c78","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1030,"y":640,"wires":[]},{"id":"3cac8f4.795947","type":"debug","z":"ae9889c1.d28c78","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1030,"y":700,"wires":[]},{"id":"979c94ca.eea358","type":"html","z":"ae9889c1.d28c78","name":"","property":"payload","outproperty":"payload","tag":"div.boxinfotable1>table>tbody>tr>td","ret":"text","as":"single","x":780,"y":700,"wires":[["3cac8f4.795947"]]}]

```

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [6 June 2021 12:24 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769/4 "2021-06-06T12:24:42Z")

</div>

It is entirely possible the values are NOT in the HTML but are "added" to the page via JavaScript

Try writing the contents of the http request payload to a file - is the value you want inside the file?

If not, then open devtools, select the network tab & refresh the page - you will see several (or many) requests to other data sources & the value you want might be in one of those. With any luck, it will be in a JSON response - then you can user that URL instead of the webpage.

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [6 June 2021 13:08 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769/5 "2021-06-06T13:08:01Z")

</div>

Amazing, just like you said. I could find the values in a JSON file. It's working, thanks so much!

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [20 June 2021 13:08 UTC](https://discourse.nodered.org/t/extracting-data-from-web-page/46769/6 "2021-06-20T13:08:19Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.