Get data from a homepage

[vikozo]

Hello
from

MeteoSchweiz

i would like to collect the 3 altitude "Nullgradgrenze"

so from the
i have a html node and the Selektor is
table.table_simple heights-table

i would expect some output but got just nothing - so what do i wrong?
Could the problem be the space between table_simple heights-table?

[
    {
        "id": "36240c5e5341146e",
        "type": "inject",
        "z": "d753f1f6adf829e0",
        "name": "",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "",
        "payload": "",
        "payloadType": "date",
        "x": 240,
        "y": 3060,
        "wires": [
            [
                "cb6935655871fe12"
            ]
        ]
    },
    {
        "id": "f3a54f2556d33c09",
        "type": "html",
        "z": "d753f1f6adf829e0",
        "name": "",
        "property": "payload",
        "outproperty": "payload",
        "tag": "table.table_simple heights-table",
        "ret": "html",
        "as": "multi",
        "x": 790,
        "y": 3060,
        "wires": [
            [
                "8c398d9d94eafc32"
            ]
        ]
    },
    {
        "id": "cb6935655871fe12",
        "type": "http request",
        "z": "d753f1f6adf829e0",
        "name": "",
        "method": "GET",
        "ret": "txt",
        "paytoqs": "ignore",
        "url": "https://www.meteoschweiz.admin.ch/home.html?tab=temperature",
        "tls": "",
        "persist": false,
        "proxy": "",
        "authType": "",
        "x": 490,
        "y": 3060,
        "wires": [
            [
                "f3a54f2556d33c09"
            ]
        ]
    },
    {
        "id": "8c398d9d94eafc32",
        "type": "debug",
        "z": "d753f1f6adf829e0",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "statusVal": "",
        "statusType": "auto",
        "x": 730,
        "y": 3100,
        "wires": []
    }
]

[zenofmud]

While testing something like this, the first thing I do is add a write file node and dump the data from the http request to a file on my device.

inject -> write file

This allows you to then use a read file node to get the page without sending multiple requests to the server and - possible - gettng you banned as a spammer.

inject -> read file -> html -> debug

you can also use a text editor to look at th html, search for "Nullgradgrenze" to see how to address it.

In addition, it looks like this page is filled by some of the scripts that are built into it. You are going to follow them too

[Steve-Mcl]

The values you want are likely populated by script from a separate JSON request. Open your browsers devtools, refresh the page while looking at the "network" tab - you will see json data requests - what you are looking for will likely be inside one of those...

[vikozo]

@Steve-Mcl it worked - but the server don't like it - access denied after a couple of time. then the path to the json file
https://www.meteoschweiz.admin.ch/product/output/altitude-levels/version__20220814_1431/heights_temperature_de.json

has a version with a time stamp ;-(

[UnborN]

You can try two requests in sequence .. one to get the valid json url path and the second request to the actual file

Demo Flow:

[{"id":"36240c5e5341146e","type":"inject","z":"54efb553244c241f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":140,"y":4360,"wires":[["f9b2a0a4d11f74ef"]]},{"id":"cb6935655871fe12","type":"http request","z":"54efb553244c241f","name":"","method":"GET","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":950,"y":4380,"wires":[["8c398d9d94eafc32"]]},{"id":"8c398d9d94eafc32","type":"debug","z":"54efb553244c241f","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1110,"y":4380,"wires":[]},{"id":"f9b2a0a4d11f74ef","type":"http request","z":"54efb553244c241f","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://www.meteoschweiz.admin.ch/home.html?tab=temperature","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":330,"y":4360,"wires":[["24a06e3c3958cf59"]]},{"id":"246ae5ad2bab3f4a","type":"debug","z":"54efb553244c241f","name":"debug 17","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":780,"y":4480,"wires":[]},{"id":"49971f9af646b89d","type":"function","z":"54efb553244c241f","name":"function","func":"let jsonUrl = msg.payload[0][\"data-json-url\"]\n\nmsg.url = \"https://www.meteoschweiz.admin.ch\" + jsonUrl;\n\nmsg.headers = {\n    \"Accept\": \"application/json, text/javascript, */*; q=0.01\",\n    \"X-Requested-With\": \"XMLHttpRequest\",\n    \"Referer\": \"https://www.meteoschweiz.admin.ch/home.html?tab=temperature\"\n}\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":780,"y":4380,"wires":[["dc96e67f1d8bc840","cb6935655871fe12"]]},{"id":"24a06e3c3958cf59","type":"html","z":"54efb553244c241f","name":"","property":"payload","outproperty":"payload","tag":".temperature__heights","ret":"attr","as":"single","x":560,"y":4380,"wires":[["246ae5ad2bab3f4a","49971f9af646b89d"]]},{"id":"dc96e67f1d8bc840","type":"debug","z":"54efb553244c241f","name":"debug 18","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":910,"y":4280,"wires":[]}]