# Web scraping with java Script

**URL:** <https://discourse.nodered.org/t/web-scraping-with-java-script/44593>\
**Category:** General\
**Created:** [23 April 2021 10:15 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593 "2021-04-23T10:15:01Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![vikozo](https://avatars.discourse-cdn.com/v4/letter/v/858c86/32.png) [@vikozo](https://discourse.nodered.org/u/vikozo)\
**Post date:** [23 April 2021 10:15 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/1 "2021-04-23T10:15:02Z")

</div>

Hello  
i would like to get information from a homepage like this

[https://tinygs.com/station/Test\_TinyGS\_Schampi@927667720](https://tinygs.com/station/Test_TinyGS_Schampi@927667720)

basicly the Listening with the status

and the information about  
Norby time and all the information

but if i add the to a html i write in the file it need javascript

so how do i manage this?

have a nice day  
vinc

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [23 April 2021 10:36 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/2 "2021-04-23T10:36:08Z")

</div>

I don't fully understand what you want but I suggest you start at the beginning by fetching the data from the website. To do that you need first to understand how the data is available on the site. Is it in html or is it accessed via javascript, for example? Once you know what you need to do then you should be able to use an http request node to fetch it.

---

<div class="post-metadata">

**Author:** ![vikozo](https://avatars.discourse-cdn.com/v4/letter/v/858c86/32.png) [@vikozo](https://discourse.nodered.org/u/vikozo)\
**Post date:** [23 April 2021 10:44 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/3 "2021-04-23T10:44:34Z")

</div>

@Colin  
thanks for you fast feedback  
done like you said and in the text are data's missing but a information like

**We're sorry but tinygs-webapp doesn't work properly without JavaScript enabled. Please enable ...**

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [23 April 2021 12:09 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/4 "2021-04-23T12:09:39Z")

</div>

@vikozo as you might have realised, the values displayed are generated by javascript - meaning the HTML that you get from a HTTP Request contains none of the data you require.

If you can identify a URL to an API that is called you might be able to locate the values of interest.

example...

- Open devtools (F12)
- select network tab
- refresh (f5)

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/2/4/24ef4897028a83c3aab5b5475d052c98d7883817.png)

---

<div class="post-metadata">

**Author:** ![paulkeates](https://avatars.discourse-cdn.com/v4/letter/p/9de053/32.png) [@paulkeates](https://discourse.nodered.org/u/paulkeates)\
**Post date:** [23 April 2021 16:29 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/5 "2021-04-23T16:29:39Z")

</div>

Hi,

Worst-case you can install WDIO nodes and navigate through the web UI which also executes JS & renders everything. You can use WDIO to extract information from the page that way.

Cheers,

Paul

---

<div class="post-metadata">

**Author:** ![vikozo](https://avatars.discourse-cdn.com/v4/letter/v/858c86/32.png) [@vikozo](https://discourse.nodered.org/u/vikozo)\
**Post date:** [23 April 2021 18:26 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/6 "2021-04-23T18:26:54Z")

</div>

Hello @Steve-Mcl  
thanks a lot - it took me longer to have the same view then you haven - then after to get my datas!  
Now some Node Red Magic - bat there will be other question.

have a nice day  
vinc

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [7 May 2021 18:27 UTC](https://discourse.nodered.org/t/web-scraping-with-java-script/44593/7 "2021-05-07T18:27:02Z")

</div>

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.