Getting data from web api - cookies issue?

[unclebob]

Hi all,

First time user of Node-Red and maybe biting off more then I can chew!

I want to connect to the following website, and pull down my data, it is a two step process on the website.
first go to
https://homebrew.n3rgy.com/consumer-data/

enter the password and press set. That creates a cookie which is stored in the browser.

Then i want to go to
https://homebrew.n3rgy.com/cgi-bin/n3rgy-consumer-data.py?
and get the data. (eventually i want to write it off into mysql but small steps...)

I can make the call to the second page, but get not authorised, i assume because the cookie can not be found?

I initially tried on python, but don't understand web site calls enough to get that working either!

Any hints/tips or outright information on steps? I don't need anything fancy to input the password - happy to hard code that into the process, but essentially i'm stuck at the process of putting the password in the box, pressing the set (or the coding equivalent) and storing the cookie for the second call...

[TotallyInformation]

You need to look at what the site is expecting to receive in the way of headers and cookies.

Try using your browser to do a normal login and look at the developer tools network tab. Clicking on the entries there will show you the sent headers.

Knowing that, you then need to set up a flow that captures the data from the first page (may just be a cookie or may need other headers such as an authorization header). When you have that, you can add the headers to the 2nd request.