Hy guys ... I would like to know if there is any method to get the price from this product... The link for the product.. Product Here is my code : [ { "id": "90fe156d.97b8e8", "type": "inject", "z": "cd02b28.20baf5", "name": "", "topic": "", "payload": "", "payloadType": "date", "repeat": "", "crontab": "", "once": false, "onceDelay": 0.1, "x": 140, "y": 740, "wires": [ [ "ecae983f.47f5a8" ] ] }, { "id": "5c05c01d.dc5cc8", "type": "html", "z": "cd02b28.20baf5", "name": "", "property": "", "outproperty": "", "tag": ".tabs-content", "ret": "html", "as": "single", "x": 460, "y": 740, "wires": [ [ "249aa5e1.5b7aa2" ] ] }, { "id": "249aa5e1.5b7aa2", "type": "change", "z": "cd02b28.20baf5", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "msg.payload[0]", "tot": "msg" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 680, "y": 740, "wires": [ [ "bde8ccfb.b9e438" ] ] }, { "id": "bde8ccfb.b9e438", "type": "switch", "z": "cd02b28.20baf5", "name": "", "property": "payload", "propertyType": "msg", "rules": [ { "t": "cont", "v": "343", "vt": "str" }, { "t": "else" } ], "checkall": "true", "repair": false, "outputs": 2, "x": 850, "y": 740, "wires": [ [], [ "ff823761.4e5a4" ] ] }, { "id": "ff823761.4e5a4", "type": "change", "z": "cd02b28.20baf5", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "YESS!!!!", "tot": "str" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 1020, "y": 740, "wires": [ [ "55c3909.a1bf6f" ] ] }, { "id": "ecae983f.47f5a8", "type": "https-node", "z": "cd02b28.20baf5", "name": "", "method": "GET", "ret": "txt", "url": "https://www.worten.pt/produtos/ar-condicionado-becken-bac2323-24-m2-12000-btu-branco-5723895", "authorized": false, "agent": true, "x": 290, "y": 740, "wires": [ [ "5c05c01d.dc5cc8" ] ] }, { "id": "55c3909.a1bf6f", "type": "debug", "z": "cd02b28.20baf5", "name": "", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "false", "x": 1210, "y": 740, "wires": [] } ] can any one show me where i'm wrong ?

Sadly, that isn't valid to import I'm afraid, you will need to try again.

Sorry Guys :wink: Fixed @Paul-Reed and @TotallyInformation ...

Thanks, but could you simplify it as well? There are 3 nodes that I don't have installed and it is a pain to install/uninstall nodes all the time when helping people. As you only are asking about how to get a specific value, an inject node, an http-request node, the html node and the debug node are all you need.

Here you have it @TotallyInformation [ { "id": "90fe156d.97b8e8", "type": "inject", "z": "cd02b28.20baf5", "name": "", "topic": "", "payload": "", "payloadType": "date", "repeat": "", "crontab": "", "once": false, "onceDelay": 0.1, "x": 140, "y": 740, "wires": [ [ "ecae983f.47f5a8" ] ] }, { "id": "5c05c01d.dc5cc8", "type": "html", "z": "cd02b28.20baf5", "name": "", "property": "", "outproperty": "", "tag": ".tabs-content", "ret": "html", "as": "single", "x": 460, "y": 740, "wires": [ [ "249aa5e1.5b7aa2" ] ] }, { "id": "249aa5e1.5b7aa2", "type": "change", "z": "cd02b28.20baf5", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "msg.payload[0]", "tot": "msg" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 680, "y": 740, "wires": [ [ "bde8ccfb.b9e438" ] ] }, { "id": "bde8ccfb.b9e438", "type": "switch", "z": "cd02b28.20baf5", "name": "", "property": "payload", "propertyType": "msg", "rules": [ { "t": "cont", "v": "343", "vt": "str" }, { "t": "else" } ], "checkall": "true", "repair": false, "outputs": 2, "x": 850, "y": 740, "wires": [ [], [ "ff823761.4e5a4" ] ] }, { "id": "ff823761.4e5a4", "type": "change", "z": "cd02b28.20baf5", "name": "", "rules": [ { "t": "set", "p": "payload", "pt": "msg", "to": "YESS!!!!", "tot": "str" } ], "action": "", "property": "", "from": "", "to": "", "reg": false, "x": 1020, "y": 740, "wires": [ [ "55c3909.a1bf6f" ] ] }, { "id": "ecae983f.47f5a8", "type": "https-node", "z": "cd02b28.20baf5", "name": "", "method": "GET", "ret": "txt", "url": "https://www.worten.pt/produtos/ar-condicionado-becken-bac2323-24-m2-12000-btu-branco-5723895", "authorized": false, "agent": true, "x": 290, "y": 740, "wires": [ [ "5c05c01d.dc5cc8" ] ] }, { "id": "55c3909.a1bf6f", "type": "debug", "z": "cd02b28.20baf5", "name": "", "active": true, "tosidebar": true, "console": false, "tostatus": false, "complete": "false", "x": 1210, "y": 740, "wires": [] } ]

Thanks. I got the link manually anyway. That is one horrid web page! By the way, to do this yourself, make use of your browser's developer console. Switch to the elements tab and use the element selector. You can then look at the tags and copy out the CSS Selector for the appropriate tag. Here is the relevant HTML: [image] As you can see, if you can tease the content attribute from #panel1c > div > div.w-product__actions.iss-product-stock-button > div.w-product__price.iss-product-price-container > span.w-product__price__current.iss-product-current-price then you are good to go :wink: So, something like this should work: [{"id":"8cab92ce.9323d","type":"debug","z":"4e0def05.d4415","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":850,"y":260,"wires":[]},{"id":"5479aade.0fd914","type":"inject","z":"4e0def05.d4415","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":220,"y":260,"wires":[["c2e2b434.9eddf8"]]},{"id":"c2e2b434.9eddf8","type":"http request","z":"4e0def05.d4415","name":"","method":"GET","ret":"txt","paytoqs":false,"url":"https://www.worten.pt/produtos/ar-condicionado-becken-bac2323-24-m2-12000-btu-branco-5723895","tls":"","proxy":"","authType":"","x":370,"y":260,"wires":[["34cfeab1.5effd6"]]},{"id":"34cfeab1.5effd6","type":"html","z":"4e0def05.d4415","name":"","property":"payload","outproperty":"payload","tag":".iss-product-current-price","ret":"html","as":"single","x":590,"y":260,"wires":[["8cab92ce.9323d"]]}] But it doesn't because the page includes some nasty javascript checks and the http-request node fails them and triggers a Google challenge/response as you will see if you replace the selector in the html node with body div.

Weird but if you wget on console it works... it's possible to wget using console and read as a file ? I think that the problem is the user agent....

You should be able to set the user agent. I think it is possibly looking for a cookie as well so you might try capturing that from a browser session and see if you can add it back in, the http-request node lets you do that. Yes, you should also be able to use wget, you can use it with the exec node and feed its output back to your flow. You should then be able to process it using the http node.

Node-RED

Node-RED Forum

How to scrap with node-red when http request fails?

General

Paul-Reed 31 August 2019 21:07 3

Please read this - How to share code or flow json

Topic		Replies	Views
Http Request not responding correctly General	4	1359	29 December 2020
Http request help from web General	12	377	11 June 2022
Web scraping issue General	21	3303	16 January 2021
Grab text value from HTML General	8	2650	13 May 2020
Node-RED with http request from local Serverpage General	20	4257	7 January 2019

Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

How to scrap with node-red when http request fails?

Related topics