Get a value from website

General
1

Hi,

I want to use a value from a website to write it in my Influx database. The HTTP-request URL I use in Node-RED is https://waterinfo.rws.nl/#!/details/publiek/waterhoogte-t-o-v-nap/Tiel-Waal(TIEW)/Waterhoogte___20Oppervlaktewater___20t.o.v.___20Normaal___20Amsterdams___20Peil___20in___20cm and I intend to use the value behind "Laatste meting:" in a payload to feed the database. Can someone help me how to setup the HTML node to extract this value?

Thanks, Hans.

Node-RED
Node-RED.png1007Ă—438 21.8 KB

2

The nice thing about this website, is that the graph is actually loaded via an XHR object, the json can be read directly in node-red.

Try this flow:

[{"id":"e422b0bb.0be0d8","type":"inject","z":"7ff1daca.703a14","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":148,"y":176,"wires":[["e2d79a7e.da52c"]]},{"id":"e2d79a7e.da52c","type":"http request","z":"7ff1daca.703a14","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://waterinfo.rws.nl/api/detail?expertParameter=Waterhoogte___20Oppervlaktewater___20t.o.v.___20Normaal___20Amsterdams___20Peil___20in___20cm&locationSlug=Tiel-Waal%28TIEW%29&user=publiek","tls":"","proxy":"","authType":"basic","x":334,"y":176,"wires":[["e3363609.37dbe","b95c765e.65ff"]]},{"id":"e3363609.37dbe","type":"debug","z":"7ff1daca.703a14","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":642,"y":88,"wires":[]},{"id":"b95c765e.65ff","type":"debug","z":"7ff1daca.703a14","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload.latest.data","targetType":"msg","x":682,"y":220,"wires":[]}]
1 Like
3

Hi,

As a Node-RED newbie I have spend a few frustrating hours to do this, without succes. Your flow works without and further editing so you make me happy, thanks for helping!

Best regards, Hans.

1 Like
4

The problem is (I believe) the graph is shown/loaded via javascript, so when you get it via node-red, the panel is not loaded and you cannot extract the data.

Use the web inspector in a browser, look at the "resources" tab and reload the page, you will see the documents it is loading, a couple of XHR files as well, which contain the data you are looking for (many websites work like this, can come in helpful).

5

My garden borders on a lake where I sail, so I measure the water level with a TTN LoRa node. This works well with NodeRED, InfluxDB and Grafana. Because my house is situated in the river area of the Netherlands, between the two large rivers Waal and Rijn, I want to know the influence of the water level in those rivers on the water level in the lake. The water level of the two rivers are publicly available and I can now request and process them with your flow. I appreciate your help and explanation!

2 Likes
6

We willen geen overstromingen! :wink: