I think, this question is beyond NR, but asking anyway? So, trying to web page scrap dynamic content is just possible within NR? At least I don't see how a GET via http request/response can invoke dynamic content. Just like curl or wget can't as well, right? Example https://repo.saltproject.io/p…

To get dynamic data from a website, you need to run a scraper capable of executing the front-end JavaScript. I am aware of only 1 node that will do that and it presents a full headless browser so is a massive installation. However, it should work. OK, actually, there seem to be a few variations: L…

I think, this question is beyond NR, but asking anyway?

General

Steve-Mcl 13 April 2021 03:23 3

As @Bobo has said there are a few topics on the forum covering this. I know I have went through it with the op. Search @Steve-Mcl scrape on this forum. Hopefully that will turn up the threads

Edit...
So this thread show how the value was obtained by monitoring the networks tab in devtools.

Topic		Replies	Views
Dynamic HTML scraping General http-request	6	661	11 October 2022
HTML Scrape - missing elements General	12	373	17 December 2022
Http-request node does not get all the page data (AKA How to scrape dynamic data from a web page) FAQs	3	594	16 November 2022
HTTP GET - rendering problem General	11	542	23 September 2020
Scrapping a dynamic (AJAX) page General	5	108	30 May 2023

I think, this question is beyond NR, but asking anyway?

Related Topics