Web scraping with java Script

General
1

Hello
i would like to get information from a homepage like this

https://tinygs.com/station/Test_TinyGS_Schampi@927667720

basicly the Listening with the status

and the information about
Norby time and all the information

but if i add the to a html i write in the file it need javascript

so how do i manage this?

have a nice day
vinc

2

I don't fully understand what you want but I suggest you start at the beginning by fetching the data from the website. To do that you need first to understand how the data is available on the site. Is it in html or is it accessed via javascript, for example? Once you know what you need to do then you should be able to use an http request node to fetch it.

3

@Colin
thanks for you fast feedback
done like you said and in the text are data's missing but a information like

We're sorry but tinygs-webapp doesn't work properly without JavaScript enabled. Please enable ...
4

@vikozo as you might have realised, the values displayed are generated by javascript - meaning the HTML that you get from a HTTP Request contains none of the data you require.

If you can identify a URL to an API that is called you might be able to locate the values of interest.

example...

  • Open devtools (F12)
  • select network tab
  • refresh (f5)

image

5

Hi,

Worst-case you can install WDIO nodes and navigate through the web UI which also executes JS & renders everything. You can use WDIO to extract information from the page that way.

Cheers,

Paul

6

Hello @Steve-Mcl
thanks a lot - it took me longer to have the same view then you haven - then after to get my datas!
Now some Node Red Magic - bat there will be other question.

have a nice day
vinc

7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.