Extract elements from the retrieved html document

#1

1/ To begin, I followed this cookbook to pick a value in the Nodered website :

Great ! I managed to display the version of nodeRed (with some modifications of the flow)
["v0.19.6"].

2/ Now I'm going to do the same thing with another html address:

https://www.suncalc.org/#/48.8584,2.2946,15/2019.03.07/17:55/1/1

3/ The goal is to retrieve the position of the sun: altitude and azimuth
=> I inspect the azimuth value of the page and I see
<span class =" Bold "id =" azimuth "> 253.90 ° </ span>

4/ I try in the node Edit html node:
span [id =" azimuth "] or #azimuth or .azimuth but the answer on the debug is"..." or [empty]

Does anyone have another idea of the css-selector that I have to put?

0 Likes

#2

Hi @SuperNinja

the page you are looking at is inserting the azimuth value using javascript that is run after the page has been loaded. If you View Source on the page (rather than inspect) and search for the element, you'll see if looks like this:

<span class="Bold" id="azimuth">...</span>

That is the text on the page before the javascript kicks in to replace is with the actual value.

The HTTP Request node just gets the page - it doesn't run any scripts or similar the page may contain - so it can only see the value being ....

So to get the value you want, you'll have to figure out where the page is getting it from - there's probably an HTTP request the page is making to get that information. Using the chrome tools I can see the page actually makes 158 network requests to load - one of those will contain the information you want and you might be able to call that url instead. It's just a case of hunting it down and seeing if their api allows you to call it directly.

0 Likes

#3

Hi @knolleary ,
ho, Yes , i understand.
looking a little I saw in a folder scripts, a calculation in which appear altitude and azimuth but it is really very complex . https://www.suncalc.org/scripts/somotools.min.js?v=1 ......

instead, I found another way out: https://midcdmz.nrel.gov/apps/solpos.pl?syear=2019&smonth=3&sday=7&eyear=2019&emonth=3&eday=7&step=1&stepunit=1&latitude=48.85800&longitude=2.29473&timezone=1&press=1013.0&temp=15&aspect=180&tilt=0&solcon=1367&sbwid=7.6&sbrad=31.7&sbsky=0.04&interval=0&field=2&field=11&zip=0
this allows thanks to the coordinates of position, to have in answer a table of value of the position of the sun from 0:00 to 24:00:

array [145]
[0 ... 9]
0: object
col1: "Date"
col2: "Time"
col3: "Azimuth angle"
col4: "Elevation (refracted)"
1: object
col1: "3/7/2019"
col2: "0:00:00"
col3: 271.9514
col4: -8.9642

It's up to me to put all this in a table that I update every day. Then I would have to look for the position of altitude and azimuth, according to the time I want ... a lot of work lol

0 Likes

#4

Alternatively, you could use one of the many contrib nodes that provide sun calculations. Most of them are based on suncalc.js

1 Like

#5

Thanks @Andrei,

i choose this very simple node: https://flows.nodered.org/node/node-red-contrib-sunpos

and got my azimuth/altitude :grinning:

it does not really answer the question of the beginning (thanks to Nick) but we could put the topic as resolved.

1 Like