Hello
i have a basic question about getting the Data from the Net/Homepage with 2 example
this page
the second station
if somebody could tell/teache me how to dig the easy way would be happy
have a nice day
vinc
Hi Vikozo,
There is an example of how to get values from html pages, like in your case, in the Node-red website under the Cookbook section. What you need is a Http-request node and an Html node to filter out the data you need.
Test flow based on your link :
[{"id":"1f2076a1.2a52d1","type":"debug","z":"3a64a7e.a4f0ed8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":710,"y":360,"wires":[]},{"id":"2a09aa0f.6844be","type":"inject","z":"3a64a7e.a4f0ed8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":170,"y":360,"wires":[["a79dc749.492c48"]]},{"id":"a79dc749.492c48","type":"http request","z":"3a64a7e.a4f0ed8","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://www.ag.ch/app/hydrometrie/station/?id=11566","tls":"","persist":false,"proxy":"","authType":"","x":360,"y":360,"wires":[["6d58ef2.03b9e9"]]},{"id":"6d58ef2.03b9e9","type":"html","z":"3a64a7e.a4f0ed8","name":"","property":"payload","outproperty":"payload","tag":"td","ret":"text","as":"single","x":500,"y":360,"wires":[["1f2076a1.2a52d1","c2a56dcc.2f77b"]]},{"id":"c2a56dcc.2f77b","type":"debug","z":"3a64a7e.a4f0ed8","name":"value","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"payload[5]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":720,"y":440,"wires":[]}]
@UnborN thanks a lot - your example is better then the document.
i have done something but also some questions
payload[4] show a number and what it is in my case Meter how to get only the Number and doing this in the Debug i can't use it anymore to do a chart
to use something like this
payload[22,20] (not working) and hold it as a titel for the Chart is this to much?
[{"id":"1f2076a1.2a52d1","type":"debug","z":"484a0edd.1e80f","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":790,"y":80,"wires":[]},{"id":"2a09aa0f.6844be","type":"inject","z":"484a0edd.1e80f","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"1800","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":170,"y":80,"wires":[["a79dc749.492c48"]]},{"id":"a79dc749.492c48","type":"http request","z":"484a0edd.1e80f","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://www.ag.ch/app/hydrometrie/station/?id=11642","tls":"","persist":false,"proxy":"","authType":"","x":390,"y":80,"wires":[["6d58ef2.03b9e9"]]},{"id":"6d58ef2.03b9e9","type":"html","z":"484a0edd.1e80f","name":"","property":"payload","outproperty":"payload","tag":"td","ret":"text","as":"single","x":580,"y":80,"wires":[["1f2076a1.2a52d1","c2a56dcc.2f77b","1cf35991.f08456","f8bdec66.fbb4b","a121aca5.327e1"]]},{"id":"c2a56dcc.2f77b","type":"debug","z":"484a0edd.1e80f","name":"value","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[3]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":810,"y":160,"wires":[]},{"id":"1cf35991.f08456","type":"debug","z":"484a0edd.1e80f","name":"value","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[4]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":820,"y":220,"wires":[]},{"id":"f8bdec66.fbb4b","type":"ui_chart","z":"484a0edd.1e80f","name":"","order":0,"width":0,"height":0,"label":"chart","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"3600","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1F77B4","#AEC7E8","#FF7F0E","#2CA02C","#98DF8A","#D62728","#FF9896","#9467BD","#C5B0D5"],"outputs":1,"x":750,"y":380,"wires":[[]]},{"id":"a121aca5.327e1","type":"debug","z":"484a0edd.1e80f","name":"value","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[22,20]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":820,"y":280,"wires":[]}]
Indeed what comes back from the website td selector is a string that has the unit of the measurement in, that is not needed.
In order to filter that out you can use the power of JavaScript in a Function Node.
Here is an example flow :
[{"id":"1f2076a1.2a52d1","type":"debug","z":"19c90a4.6533276","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":1030,"y":200,"wires":[]},{"id":"2a09aa0f.6844be","type":"inject","z":"19c90a4.6533276","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"1800","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":250,"y":220,"wires":[["a79dc749.492c48"]]},{"id":"a79dc749.492c48","type":"http request","z":"19c90a4.6533276","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://www.ag.ch/app/hydrometrie/station/?id=11642","tls":"","persist":false,"proxy":"","authType":"","x":470,"y":220,"wires":[["6d58ef2.03b9e9"]]},{"id":"6d58ef2.03b9e9","type":"html","z":"19c90a4.6533276","name":"","property":"payload","outproperty":"payload","tag":"td","ret":"text","as":"single","x":660,"y":220,"wires":[["c2a56dcc.2f77b","1cf35991.f08456","14d46a76.4f3cee"]]},{"id":"c2a56dcc.2f77b","type":"debug","z":"19c90a4.6533276","name":"value","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[3]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":820,"y":300,"wires":[]},{"id":"1cf35991.f08456","type":"debug","z":"19c90a4.6533276","name":"value","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[4]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":800,"y":360,"wires":[]},{"id":"f8bdec66.fbb4b","type":"ui_chart","z":"19c90a4.6533276","name":"","group":"9e1250e8.866878","order":0,"width":0,"height":0,"label":"chart","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"3600","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"x":1030,"y":140,"wires":[[]]},{"id":"14d46a76.4f3cee","type":"function","z":"19c90a4.6533276","name":"","func":"// split \"426.31 m\" text into an array based on space\n// and pick only the first element that is the number in text\n// finally convert the string to actual Number\n\nlet abfluss = Number(msg.payload[3].split(\" \")[0])\nlet pegel = Number(msg.payload[4].split(\" \")[0])\n\nnode.warn(abfluss)\nnode.warn(pegel)\n\nmsg.payload = { abfluss, pegel }\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":860,"y":200,"wires":[["1f2076a1.2a52d1"]]},{"id":"9e1250e8.866878","type":"ui_group","name":"Default","tab":"69f3c1f1.278078","order":1,"disp":true,"width":"6","collapse":false},{"id":"69f3c1f1.278078","type":"ui_tab","name":"Home","icon":"dashboard","disabled":false,"hidden":false}]
Notes: String.split()
ps. the chart part i leave to you to experiment following the guide of how multi line data need to be structured 
@UnborN thanks for your help
the guide is impressiv
i have it like this, the function itself put numbers out, in the Debugview. But at the end it shows only empty graf.
[{"id":"2a09aa0f.6844be","type":"inject","z":"ac8d9bb9.eb8a58","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":200,"y":240,"wires":[["a79dc749.492c48"]]},{"id":"a79dc749.492c48","type":"http request","z":"ac8d9bb9.eb8a58","name":"","method":"GET","ret":"txt","paytoqs":"ignore","url":"https://www.ag.ch/app/hydrometrie/station/?id=11642","tls":"","persist":false,"proxy":"","authType":"","x":430,"y":240,"wires":[["6d58ef2.03b9e9"]]},{"id":"6d58ef2.03b9e9","type":"html","z":"ac8d9bb9.eb8a58","name":"","property":"payload","outproperty":"payload","tag":"td","ret":"text","as":"single","x":620,"y":240,"wires":[["c2a56dcc.2f77b","1cf35991.f08456","9057b2ab.e923c","8d1e6c27.b33e9"]]},{"id":"c2a56dcc.2f77b","type":"debug","z":"ac8d9bb9.eb8a58","name":"value","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[3]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":780,"y":140,"wires":[]},{"id":"1cf35991.f08456","type":"debug","z":"ac8d9bb9.eb8a58","name":"value","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload[4]","targetType":"msg","statusVal":"payload[5]","statusType":"auto","x":780,"y":280,"wires":[]},{"id":"9057b2ab.e923c","type":"function","z":"ac8d9bb9.eb8a58","name":"functionAbfluss","func":"let abfluss = Number(msg.payload[3].split(\" \")[0])\nnode.warn(abfluss)\nmsg.payload = {abfluss}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":800,"y":200,"wires":[["4f25a8e4.6c17c8","14e5279d.f1c088","7e1022d0.efe35c"]]},{"id":"7e1022d0.efe35c","type":"debug","z":"ac8d9bb9.eb8a58","name":"msg.abfluss","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1040,"y":220,"wires":[]},{"id":"8d1e6c27.b33e9","type":"function","z":"ac8d9bb9.eb8a58","name":"FunctionPegel","func":"// split \"426.31 m\" text into an array based on space\n// and pick only the first element that is the number in text\n// finally convert the string to actual Number\n\nlet pegel = Number(msg.payload[4].split(\" \")[0])\nnode.warn(pegel)\nmsg.payload = {pegel}\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":800,"y":340,"wires":[["b6eb80fb.e3e63","1330f358.811ead"]]},{"id":"b6eb80fb.e3e63","type":"debug","z":"ac8d9bb9.eb8a58","name":"","active":false,"tosidebar":true,"console":true,"tostatus":true,"complete":"payload","targetType":"msg","statusVal":"payload","statusType":"auto","x":1040,"y":360,"wires":[]},{"id":"1330f358.811ead","type":"ui_chart","z":"ac8d9bb9.eb8a58","name":"","group":"a17fab35.733d58","order":1,"width":0,"height":0,"label":"Höhe m.ü.M.","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"86400","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"x":1030,"y":320,"wires":[[]]},{"id":"14e5279d.f1c088","type":"ui_chart","z":"ac8d9bb9.eb8a58","name":"","group":"a17fab35.733d58","order":0,"width":0,"height":0,"label":"m³/s","chartType":"line","legend":"false","xformat":"HH:mm:ss","interpolate":"linear","nodata":"--","dot":false,"ymin":"","ymax":"","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"3600","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"x":1010,"y":180,"wires":[[]]},{"id":"4f25a8e4.6c17c8","type":"ui_gauge","z":"ac8d9bb9.eb8a58","name":"","group":"9e1250e8.866878","order":0,"width":0,"height":0,"gtype":"gage","title":"","label":"m³/s","format":"{{value}}","min":0,"max":"20","colors":["#00b500","#e6e600","#ca3838"],"seg1":"6","seg2":"8","x":1010,"y":140,"wires":[]},{"id":"a17fab35.733d58","type":"ui_group","name":"Wigger Zofingen","tab":"69f3c1f1.278078","order":1,"disp":true,"width":"8","collapse":false},{"id":"9e1250e8.866878","type":"ui_group","name":"Default","tab":"69f3c1f1.278078","order":2,"disp":true,"width":"6","collapse":false},{"id":"69f3c1f1.278078","type":"ui_tab","name":"Home","icon":"dashboard","order":1,"disabled":false,"hidden":false}]
Hi Vikozo,
All you have to do is remove the surrounding curly braces { } around the values passed in msg.payload in both functions.
msg.payload = abfluss
and
msg.payload = pegel
they have been there { } because first they have added together
yes it all depends how you want to structure the msg .. but in your example, the graph needs the payload to be a number instead of an object { }
The Second Link just to get the water Temperature works! just have been another station!
this one i just don't know how to find the live data!
there is a graph - but i only find the place to download files
Hi .. yea .. the second link you post was a little more difficult to get the data
i found that it has an api that sends you json if you select from the dropdown menu
I dont have time to expand but experiment with this
[{"id":"217fc432.f4946c","type":"inject","z":"c2caec2f.d90ff","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":180,"y":300,"wires":[["8c3774eb.32ab18"]]},{"id":"87a55d14.b03088","type":"http request","z":"c2caec2f.d90ff","name":"","method":"GET","ret":"obj","paytoqs":"ignore","url":"","tls":"","persist":false,"proxy":"","authType":"","x":530,"y":300,"wires":[["c3ae1a88.00d878"]]},{"id":"3b544f6b.3fbd58","type":"debug","z":"c2caec2f.d90ff","name":"value","active":true,"tosidebar":true,"console":true,"tostatus":true,"complete":"true","targetType":"full","statusVal":"payload[5]","statusType":"auto","x":880,"y":320,"wires":[]},{"id":"8c3774eb.32ab18","type":"function","z":"c2caec2f.d90ff","name":"JSON URL","func":"msg.url = `https://www.ag.ch/app/hydrometrie/kiwis/KiWIS?service=kisters&type=queryServices&request=getTimeseriesValues&datasource=0&metadata=true&md_returnfields=station_no,station_name,parametertype_name,ts_unitsymbol&format=JSON&from=2020-12-09&ts_id=63652042&to=2020-12-16`\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":350,"y":300,"wires":[["87a55d14.b03088"]]},{"id":"c1253192.698f3","type":"comment","z":"c2caec2f.d90ff","name":"parsed Json","info":"","x":530,"y":240,"wires":[]},{"id":"c3ae1a88.00d878","type":"function","z":"c2caec2f.d90ff","name":"Chart ","func":"/* [{\n \"series\": [\"Zofingen\"],\n \"data\": [ [5,6,9,10] ],\n \"labels\": [ \"Jan\", \"Feb\", \"Mar\", \"Apr\" ]\n}] */ \n\n// prepare chart data\nlet arr = msg.payload[0].data\n\nlet labels = [] \narr.forEach( el => labels.push(el[0])) // new Date(el[0]).getTime()\n\nlet data = []\narr.forEach( el => data.push(el[1]))\n\nmsg.payload = [{\n \"series\": [msg.payload[0].station_name],\n \"data\": [ data ],\n \"labels\": labels,\n \n}] \n\n//chart options\nmsg.ui_control = {}\n msg.ui_control.options = {\n scales: {\n xAxes: [ {\n type: \"time\",\n // distribution: \"series\",\n distribution: \"series\",\n time: {\n // parser: \"YYYY-MM-DD HH:mm:ss\",\n unit: \"day\",\n displayFormats: { year: \"DD MMM YYYY HH:mm\", month: \"DD MMM YYYY HH:mm\", day: \"DD MMM YYYY HH:mm\", hour: \"DD MMM YYYY HH:mm\" },\n },\n offset: true,\n ticks: { stepSize: 5, maxTicksLimit: 48, fontStyle: \"bold\" },\n },]\n }\n } \n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":690,"y":300,"wires":[["3b544f6b.3fbd58","c7904601.bb5e18","25289e41.5ee8ea"]]},{"id":"c7904601.bb5e18","type":"ui_chart","z":"c2caec2f.d90ff","name":"","group":"bd35e7f0.a3547","order":1,"width":0,"height":0,"label":"All Data","chartType":"bar","legend":"true","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"86400","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"x":880,"y":240,"wires":[[]]},{"id":"25289e41.5ee8ea","type":"ui_ui_control","z":"c2caec2f.d90ff","name":"","events":"all","x":880,"y":400,"wires":[[]]},{"id":"bd35e7f0.a3547","type":"ui_group","name":"Wigger Zofingen","tab":"c489782e.2d5f4","order":1,"disp":true,"width":"16","collapse":false},{"id":"c489782e.2d5f4","type":"ui_tab","name":"Home","icon":"dashboard","disabled":false,"hidden":false}]
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
Copyright OpenJS Foundation and Node-RED contributors. All rights reserved. The OpenJS Foundation has registered trademarks and uses trademarks. For a list of trademarks of the OpenJS Foundation, please see our Trademark Policy and Trademark List. Trademarks and logos not indicated on the list of OpenJS Foundation trademarks are trademarks™ or registered® trademarks of their respective holders. Use of them does not imply any affiliation with or endorsement by them.
The OpenJS Foundation | Terms of Use | Privacy Policy | OpenJS Foundation Bylaws | Trademark Policy | Trademark List | Cookie Policy