# Reading webserver data

**URL:** <https://discourse.nodered.org/t/reading-webserver-data/10621>\
**Category:** General\
**Created:** [28 April 2019 18:49 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621 "2019-04-28T18:49:50Z")\
**Posts on this page:** 18\
**Page:** 1

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [28 April 2019 18:49 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/1 "2019-04-28T18:49:50Z")

</div>

Hi there Node-Red fans,

I have been doing pretty amazing things with node-red the last two years. I am no programmer but i have a fair understanding of the basics.  
This challenge is a bit out of my league therefore I need a little guidance.

I am trying to read energy measurements from a logging server:

If I enter the following url into a web browser :

[http://192.168.1.10:8080/user1?16&1=GetLogHour&19&100&20&10&](http://192.168.1.10:8080/user1?16&1=GetLogHour&19&100&20&10&)

I get the following data:

{"header":["HourLog","","CH1 verbruik 01E00002","Year : 2019","Day Start : 100","Day's Qty : 20","Interval : 10 min","Units : TYPE 0"],  
"footer":["Import","Export"],  
"body":[["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],

how to I go about reading this into node-red so I can actually use the data for something useful, like display on a dashboard?  
your help will be much appreciated.

Duncan

---

<div class="post-metadata">

**Author:** ![ukmoose](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ukmoose/32/13_2.png) [@ukmoose](https://discourse.nodered.org/u/ukmoose)\
**Post date:** [28 April 2019 18:59 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/2 "2019-04-28T18:59:10Z")

</div>

Try an inject node followed by a http request node and then a debug node

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [28 April 2019 19:07 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/3 "2019-04-28T19:07:04Z")

</div>

What do you want to display on the dashboard specifically ?

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [28 April 2019 20:04 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/4 "2019-04-28T20:04:30Z")

</div>

I have tried that and I get this error?

4/28/2019, 10:02:37 PM[node: 3e768908.1db9b6](https://192.168.1.245:1880/#)

msg : error

"Error: Parse Error"

4/28/2019, 10:02:37 PM[node: 91fb943f.54cff](https://192.168.1.245:1880/#)msg : Object

object

\_msgid: "b1d4eec3.68f8e"

topic: ""

payload: "Error: Parse Error : [http://192.168.1.10:8080/user1?16&1=GetLogHour&19&100&20&10&](http://192.168.1.10:8080/user1?16&1=GetLogHour&19&100&20&10&)"

statusCode: "HPE\_INVALID\_CONSTANT"

---

<div class="post-metadata">

**Author:** ![ukmoose](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ukmoose/32/13_2.png) [@ukmoose](https://discourse.nodered.org/u/ukmoose)\
**Post date:** [28 April 2019 20:08 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/5 "2019-04-28T20:08:14Z")

</div>

how have you configured the http request node?

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [28 April 2019 20:28 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/7 "2019-04-28T20:28:40Z")

</div>

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/a/a5a72de47c98eb377c330b65f12b2814dea52691.png)

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [28 April 2019 22:51 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/8 "2019-04-28T22:51:23Z")

</div>

> [@Duncan](#):
>
> HPE\_INVALID\_CONSTANT

This indicates a different content-type header than expected.  
Open the url in the browser, use the web inspector to verify the content-header and match it in node-red with the `msg.header` option.

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [30 April 2019 20:44 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/9 "2019-04-30T20:44:49Z")

</div>

Hi bakman2, I don't know exactly what you mean, but this is the reply in the browser?

1. Request URL:

[http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&](http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&)

1. Request Method:

GET

1. Status Code:

200 OK

1. Remote Address:

192.168.1.10:8080

1. Referrer Policy:

no-referrer-when-downgrade

1. Request Headersview parsed

2. GET /user1?64&1=GetLogHour&19&100&20&10& HTTP/1.1 Host: 192.168.1.10:8080 Connection: keep-alive Cache-Control: max-age=0 Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,_/_;q=0.8,application/signed-exchange;v=b3 Accept-Encoding: gzip, deflate Accept-Language: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7 Cookie: username=adminIP

3. Query String Parametersview parsed

4. 64&1=GetLogHour&19&100&20&10&

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [30 April 2019 21:15 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/10 "2019-04-30T21:15:32Z")

</div>

What is the application you are contacting?

It may be that it's an old application with a HTTP/0.9 interface. That wouldn't send back HTTP headers, only the raw body content and therefore the Header Parse Error.

Can you run curl -v to that URL in a command line to see what comes back?

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [1 May 2019 17:39 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/11 "2019-05-01T17:39:13Z")

</div>

Hi Steve, can you be a bit more specific? Where should I try the curl command?

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [1 May 2019 17:51 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/12 "2019-05-01T17:51:01Z")

</div>

> [@Duncan](#):
>
> Request Headersview parsed

This part is essential. Use a different browser to show these headers, or as @Steve-Mcl says, use

`curl -v "http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&"`

on the commandline (if you are on linux, windows does not have curl installed i believe).

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [1 May 2019 18:22 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/13 "2019-05-01T18:22:56Z")

</div>

Great, at least we know that, that works. Now how to get it working in node-red?  
I appreciate your help.

HTTP/1.1 200 OK

{"header":["HourLog","","CH1 verbruik 01E00002","Year : 2019","Day Start : 100","Day's Qty : 20","Interval : 10 min","Units : TYPE 0"],  
"footer":["Import","Export"],  
"body":[["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],  
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [1 May 2019 19:10 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/14 "2019-05-01T19:10:20Z")

</div>

I dont think you can use the HTTP Request node as the repsonse from that web server is not valid & therefore you get an HPE (Header Parse Error).

You could use the TCP node to & generate a GET command to IP 192.168.1.10, PORT 8080 & the response you get back would be the JSON as expected.

See this for an idea of how to generate a HTTP GET using TCP (or in this case telnet - same thing)...  
[https://www.the-art-of-web.com/system/telnet-http11/](https://www.the-art-of-web.com/system/telnet-http11/)

---

<div class="post-metadata">

**Author:** ![bakman2](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/bakman2/32/6207_2.png) [@bakman2](https://discourse.nodered.org/u/bakman2)\
**Post date:** [1 May 2019 20:54 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/15 "2019-05-01T20:54:50Z")

</div>

Try this flow:

```auto
[{"id":"a92cf1.8c434b1","type":"inject","z":"cf07e2a5.1d217","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":264,"y":418,"wires":[["df1b46c2.dc0748"]]},{"id":"df1b46c2.dc0748","type":"exec","z":"cf07e2a5.1d217","command":"curl -v \"http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&\"","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"curl","x":424,"y":418,"wires":[["1f5ac981.6ccdde"],["f17d89c8.de8bf"],["cf10d274.0242e"]]},{"id":"cf10d274.0242e","type":"debug","z":"cf07e2a5.1d217","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":622,"y":462,"wires":[]},{"id":"1f5ac981.6ccdde","type":"debug","z":"cf07e2a5.1d217","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":622,"y":374,"wires":[]},{"id":"f17d89c8.de8bf","type":"debug","z":"cf07e2a5.1d217","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":622,"y":418,"wires":[]}]

```

note that your output is not a valid header, nor is it valid json as there the ending `}` are missing.

---

<div class="post-metadata">

**Author:** ![Duncan](https://avatars.discourse-cdn.com/v4/letter/d/ecc23a/32.png) [@Duncan](https://discourse.nodered.org/u/Duncan)\
**Post date:** [3 May 2019 20:14 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/16 "2019-05-03T20:14:15Z")

</div>

Hi Bakman2,

Great stuff that works!!! Now I will try to figure out how to get the usable data.  
Thanks!!

---

<div class="post-metadata">

**Author:** ![hiteshpopli](https://avatars.discourse-cdn.com/v4/letter/h/ccd318/32.png) [@hiteshpopli](https://discourse.nodered.org/u/hiteshpopli)\
**Post date:** [29 May 2021 20:09 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/17 "2021-05-29T20:09:16Z")

</div>

Hi can somebody help ...how to read the data of device status from webserver which has username and password

---

<div class="post-metadata">

**Author:** ![hiteshpopli](https://avatars.discourse-cdn.com/v4/letter/h/ccd318/32.png) [@hiteshpopli](https://discourse.nodered.org/u/hiteshpopli)\
**Post date:** [29 May 2021 20:10 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/18 "2021-05-29T20:10:35Z")

</div>

Hi Duncan,

Pls let me know how reading data from webserver worked for you.

---

<div class="post-metadata">

**Author:** ![brian](https://avatars.discourse-cdn.com/v4/letter/b/97f17d/32.png) [@brian](https://discourse.nodered.org/u/brian)\
**Post date:** [4 February 2022 08:18 UTC](https://discourse.nodered.org/t/reading-webserver-data/10621/19 "2022-02-04T08:18:34Z")

</div>

thank you for this flow.

This helped me alot with another issue.