I'm complete noob when it comes to Node-Red, but i do have a background in Java and C#, my HTML is mediocre at best. So background out of the way, here it is:
I am trying to log into a website that i need to scrape a data table from, but the http request and https request nodes don't seem to do the trick. I do know that there is a token exchange as the login is on a session that has an exipry. I'm just not sure what process to follow or nodes to use to login and extract the data table.
Any help would be great
It depends on the website's authentication model. The HTTP Request node can be used for Basic or Digest Authentication however it is more likely the website has a login flow that involves exchanging username/password for a token, and that token must be provided with each request.
There are too many possible ways for it to work to give you a single answer. If you can provide details and the site you want to scrape, we may be able to give you some pointers.
Thanks for responding so quickly.
The final page that i need to scrape from is Coupa Home , i'm not quite comfortable with providing login details on a public forum. If you need them we can exchange them over DM or email.
I will need credentials for testing - temporary access will of course be fine - you can DM detail by clicking my avitar and clicking the "message" button.
Additional info needed - provide a screenshot showing 1 of the values you are attempting to scrape
I click on your avatar, but there is no message button. Or am I looking in the wrong place?