Schema

# Grab details from password protected website **URL:** \ **Category:** General\ **Created:** [2 January 2020 16:58 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832 "2020-01-02T16:58:31Z")\ **Posts on this page:** 7\ **Page:** 1

**Author:** ![ebnerjoh](https://avatars.discourse-cdn.com/v4/letter/e/ebca7d/32.png) [@ebnerjoh](https://discourse.nodered.org/u/ebnerjoh)\ **Post date:** [2 January 2020 16:58 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/1 "2020-01-02T16:58:31Z")

Hi, I have a DataLogger for my Solar System and I want to extract values from the internal Webserver of the solar heating system. It is a system from ta.co.at (UVR1611) and as Datalogger I am using the CMI from the same company. This CMI has an integrated API and I can get most of the needed information via the API. For the "not accessable" values I can create a so called schema, which I can access via a webbrowser + user/pass. It is a website, where you get the Popup from the browser to enter the username/password. If I want to directly access this Schema, I need to go to "[http://192.168.1.19/schema.html#1](http://192.168.1.19/schema.html#1)". I am getting a popup for User/Password and I only need to enter "gast" as user. Then I am able to see the website. I tried the following: ```auto [{"id":"da5794f7.b89e78","type":"http request","z":"d2043ed8.8e731","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"http://192.168.1.19/schema.html#1","tls":"","proxy":"","authType":"basic","x":430,"y":3040,"wires":[["79ecd2cf.2ebacc"]]},{"id":"44fe52c7.a9fb2c","type":"inject","z":"d2043ed8.8e731","name":"","topic":"","payload":"","payloadType":"date","repeat":"60","crontab":"","once":true,"onceDelay":"60","x":210,"y":3040,"wires":[["da5794f7.b89e78"]]},{"id":"79ecd2cf.2ebacc","type":"debug","z":"d2043ed8.8e731","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":3040,"wires":[]}] ``` It seems that I am getting connected, because with a wrong username I am getting back a 404. With the correct username I am getting back a website, but unfortunatley without values. It seems that the website is displaying the values via javascript which seems not to be handeled by the http request node. This is what I am getting back: ```auto Schema

CHANGE

OK Abbrechen ``` Any ideas? Br, JOhannes ---

**Author:** ![afelix](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/afelix/32/9743_2.png) [@afelix](https://discourse.nodered.org/u/afelix)\ **Post date:** [2 January 2020 17:01 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/2 "2020-01-02T17:01:57Z")

> [@ebnerjoh](#): > > It is a website, where you get the Popup from the browser to enter the username/password. This type of authentication is called Basic Auth, and is supported out of the box by the `http request` core node. Using that node, set the auth type to basic, enter `gast` in the `username` field, save and try to request it. From the flow that you posted it appears you are doing that. Are you 100% sure that the URL you are requesting is correct? ---

**Author:** ![ebnerjoh](https://avatars.discourse-cdn.com/v4/letter/e/ebca7d/32.png) [@ebnerjoh](https://discourse.nodered.org/u/ebnerjoh)\ **Post date:** [2 January 2020 17:43 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/3 "2020-01-02T17:43:46Z")

Hi, Yes, I am pretty sure the URL is correct, and also authentication is working. If I open the page via chrome and then "right click" on the page to check the source, then it looks identically: ```auto Schema

CHANGE

OK Cancel ``` I think the values are inserted via the javascript and this is maybe not working with the http request node?` This is when analyising the website via chrome developer mode: ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/f/f55b9d116b20cc83cd9f3192ac72a9f6d893160a.png) ---

**Author:** ![afelix](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/afelix/32/9743_2.png) [@afelix](https://discourse.nodered.org/u/afelix)\ **Post date:** [2 January 2020 18:10 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/4 "2020-01-02T18:10:16Z")

Yes it is. Open the page while looking at the network tab of the developer tools. That should show which resources are loaded with the actual data. ---

**Author:** ![ebnerjoh](https://avatars.discourse-cdn.com/v4/letter/e/ebca7d/32.png) [@ebnerjoh](https://discourse.nodered.org/u/ebnerjoh)\ **Post date:** [2 January 2020 18:19 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/5 "2020-01-02T18:19:31Z")

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/0/0a47492635fa8dad8adf59cdd3a01fbe75ddac8c.png) ---

**Author:** ![ebnerjoh](https://avatars.discourse-cdn.com/v4/letter/e/ebca7d/32.png) [@ebnerjoh](https://discourse.nodered.org/u/ebnerjoh)\ **Post date:** [2 January 2020 18:21 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/6 "2020-01-02T18:21:35Z")

This are the values I am looking for: ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/6/6f837ab1a69f18b0631d71d1264d9cc2cd4f554d.png) ---

**Author:** ![afelix](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/afelix/32/9743_2.png) [@afelix](https://discourse.nodered.org/u/afelix)\ **Post date:** [2 January 2020 18:36 UTC](https://discourse.nodered.org/t/grab-details-from-password-protected-website/19832/7 "2020-01-02T18:36:19Z")

well guess that's your answer, the link for `1.cgi` with 1 URL parameter called `_` (used to prevent caching) that is the timestamp you can get from an inject node. So you set it to use the payload as URL parameters, then previously in a change node set the payload to `{"_": }`, for example inject set to mode timestamp, change node with "move" `msg.payload` to `msg.payload._`.