I made a scraping sensor for HA yesterday and all the data is collected as it should. However, i want to repeat this whole process for several URLs. For the moment, i've entered the inject node payload manually, but what i want is for that payload to be collected from a list of several URLs. I've created a non-connected template node with all the URLs i want, but i don't know how to integrate it, or if that's the way to do it. The 2nd smaller issue, is that i want the HA Sensor node to have the msg.payload.title as HA Config name. I've tried to sneak in "valueType":"msg" in the code, but it's rejected, and naming it msg.payload.name will only name it exactly that. I've tried with double and triple {} brackets, but no luck. [{"id":"85776cea.2a018","type":"template","z":"4f7a68fe.44caa8","name":"Funds","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"{\n \"global\": \"F00000JORS\",\n \"norge\": \"F00000JORR\",\n \"usa\": \"F0GBR04OTI\",\n \"healthcare\": \"F00000ZFFV\",\n \"teknologi\": \"F00000ZFG1\"\n}","output":"json","x":170,"y":200,"wires":[[]]},{"id":"30cdee7.0c63112","type":"inject","z":"4f7a68fe.44caa8","name":"POST id","topic":"","payload":"F0GBR04NGU","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":140,"wires":[["34ea4898.030bb8"]]},{"id":"34ea4898.030bb8","type":"change","z":"4f7a68fe.44caa8","name":"","rules":[{"t":"set","p":"post","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":250,"y":140,"wires":[["eff86bb0.4dd408","fbd3baa.4f2d348"]]},{"id":"eff86bb0.4dd408","type":"http request","z":"4f7a68fe.44caa8","name":"GET Main","method":"GET","ret":"txt","paytoqs":false,"url":"https://www.morningstar.no/no/funds/snapshot/snapshot.aspx?id={{{post}}}","tls":"","persist":false,"proxy":"","authType":"","x":420,"y":100,"wires":[["b4130d6.7e481f","c4aa1a4a.19cac8","a52fdedc.cc052"]]},{"id":"fbd3baa.4f2d348","type":"http request","z":"4f7a68fe.44caa8","name":"GET History","method":"GET","ret":"txt","paytoqs":false,"url":"https://www.morningstar.no/no/funds/snapshot/snapshot.aspx?id={{{post}}}&tab=1","tls":"","persist":false,"proxy":"","authType":"","x":430,"y":180,"wires":[["a576ce16.8b778","5120fffe.dd8b1","9cd280d1.6cc38","9f3328a5.e32f78"]]},{"id":"c4aa1a4a.19cac8","type":"html","z":"4f7a68fe.44caa8","name":"Title","property":"payload","outproperty":"payload","tag":"h1","ret":"text","as":"multi","x":590,"y":20,"wires":[["7bac34f9.8a751c"]]},{"id":"a52fdedc.cc052","type":"html","z":"4f7a68fe.44caa8","name":"Date","property":"payload","outproperty":"payload","tag":"#overviewQuickstatsDiv tr:nth-child(2) td.line.heading span","ret":"text","as":"multi","x":590,"y":60,"wires":[["4a8bac12.fe9be4"]]},{"id":"b4130d6.7e481f","type":"html","z":"4f7a68fe.44caa8","name":"Price","property":"payload","outproperty":"payload","tag":"#overviewQuickstatsDiv tr:nth-of-type(2) td:nth-of-type(3)","ret":"text","as":"single","x":590,"y":100,"wires":[["32496c07.ab8f64"]]},{"id":"a576ce16.8b778","type":"html","z":"4f7a68fe.44caa8","name":"1 Day","property":"payload","outproperty":"payload","tag":"#returnsTrailingDiv tr:nth-child(3) td.col2.value.number","ret":"text","as":"single","x":590,"y":140,"wires":[["dd4fbafb.021548"]]},{"id":"5120fffe.dd8b1","type":"html","z":"4f7a68fe.44caa8","name":"1 Week","property":"payload","outproperty":"payload","tag":"#returnsTrailingDiv tr:nth-child(4) td.col2.value.number","ret":"text","as":"single","x":600,"y":180,"wires":[["71d3f223.2727cc"]]},{"id":"9cd280d1.6cc38","type":"html","z":"4f7a68fe.44caa8","name":"1 Month","property":"payload","outproperty":"payload","tag":"#returnsTrailingDiv tr:nth-child(5) td.col2.value.number","ret":"text","as":"single","x":600,"y":220,"wires":[["3f2c0d28.20fb42"]]},{"id":"9f3328a5.e32f78","type":"html","z":"4f7a68fe.44caa8","name":"1 Year","property":"payload","outproperty":"payload","tag":"#returnsTrailingDiv tr:nth-child(9) td.col2.value.number","ret":"text","as":"single","x":590,"y":260,"wires":[["38032aee.c1e206"]]},{"id":"37e1e02.3bf122","type":"function","z":"4f7a68fe.44caa8","name":"String>Float","func":"msg.payload = /(?:[^A-z]+)/.exec(msg.payload)[0].replace(\",\",\".\");\nmsg.payload = parseFloat(msg.payload);\nreturn msg;","outputs":1,"noerr":0,"x":930,"y":200,"wires":[["bdd00546.fec1f8"]]},{"id":"1df05d04.5c5aa3","type":"debug","z":"4f7a68fe.44caa8","name":"","active":true,"tosidebar":true,"console":true,"tostatus":false,"complete":"payload","targetType":"msg","x":940,"y":80,"wires":[]},{"id":"bdd00546.fec1f8","type":"join","z":"4f7a68fe.44caa8","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"7","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":910,"y":140,"wires":[["1df05d04.5c5aa3","e5900f64.3399c"]]},{"id":"dd4fbafb.021548","type":"change","z":"4f7a68fe.44caa8","name":"day","rules":[{"t":"set","p":"topic","pt":"msg","to":"day","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":140,"wires":[["37e1e02.3bf122"]]},{"id":"71d3f223.2727cc","type":"change","z":"4f7a68fe.44caa8","name":"week","rules":[{"t":"set","p":"topic","pt":"msg","to":"week","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":180,"wires":[["37e1e02.3bf122"]]},{"id":"3f2c0d28.20fb42","type":"change","z":"4f7a68fe.44caa8","name":"month","rules":[{"t":"set","p":"topic","pt":"msg","to":"month","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":220,"wires":[["37e1e02.3bf122"]]},{"id":"38032aee.c1e206","type":"change","z":"4f7a68fe.44caa8","name":"year","rules":[{"t":"set","p":"topic","pt":"msg","to":"year","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":260,"wires":[["37e1e02.3bf122"]]},{"id":"7bac34f9.8a751c","type":"change","z":"4f7a68fe.44caa8","name":"title","rules":[{"t":"set","p":"topic","pt":"msg","to":"title","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":20,"wires":[["bdd00546.fec1f8"]]},{"id":"4a8bac12.fe9be4","type":"change","z":"4f7a68fe.44caa8","name":"date","rules":[{"t":"set","p":"topic","pt":"msg","to":"date","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":60,"wires":[["bdd00546.fec1f8"]]},{"id":"32496c07.ab8f64","type":"change","z":"4f7a68fe.44caa8","name":"price","rules":[{"t":"set","p":"topic","pt":"msg","to":"price","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":100,"wires":[["37e1e02.3bf122"]]},{"id":"e5900f64.3399c","type":"ha-entity","z":"4f7a68fe.44caa8","name":"","server":"e707d943.9c83a8","version":1,"debugenabled":false,"outputs":1,"entityType":"sensor","config":[{"property":"name","value":"payload.title"},{"property":"device_class","value":""},{"property":"icon","value":""},{"property":"unit_of_measurement","value":"kr"}],"state":"payload.price","stateType":"msg","attributes":[{"property":"date","value":"payload.date","valueType":"msg"},{"property":"day","value":"payload.day","valueType":"msg"},{"property":"week","value":"payload.week","valueType":"msg"},{"property":"month","value":"payload.month","valueType":"msg"},{"property":"year","value":"payload.year","valueType":"msg"}],"resend":true,"outputLocation":"","outputLocationType":"none","x":1090,"y":140,"wires":[[]]},{"id":"e707d943.9c83a8","type":"server","z":"","name":"Home Assistant","legacy":false,"hassio":true,"rejectUnauthorizedCerts":true,"ha_boolean":"y|yes|true|on|home|open","connectionDelay":true,"cacheJson":true}]

bakman2 23 December 2019 22:27 2

Please export your flow again, read this post and edit your post.

Topic		Replies	Views
Using a variable for Home Assistant sensor name General	10	4440	12 February 2021
Collect https address General	16	1043	28 May 2020
Web scraping issue General	21	3303	16 January 2021
Sending a curl request and parse the the output of that request General	12	269	14 February 2024
How create HA entities from a HTTP request General	8	2056	18 May 2021

Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.

Scraping HTML for Home Assistant, need help with loop

Related topics