Reading webserver data

General
1

Hi there Node-Red fans,

I have been doing pretty amazing things with node-red the last two years. I am no programmer but i have a fair understanding of the basics.
This challenge is a bit out of my league therefore I need a little guidance.

I am trying to read energy measurements from a logging server:

If I enter the following url into a web browser :

http://192.168.1.10:8080/user1?16&1=GetLogHour&19&100&20&10&

I get the following data:

{"header":["HourLog","","CH1 verbruik 01E00002","Year : 2019","Day Start : 100","Day's Qty : 20","Interval : 10 min","Units : TYPE 0"],
"footer":["Import","Export"],
"body":[["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],

how to I go about reading this into node-red so I can actually use the data for something useful, like display on a dashboard?
your help will be much appreciated.

Duncan

2

Try an inject node followed by a http request node and then a debug node

3

What do you want to display on the dashboard specifically ?

4

I have tried that and I get this error?

4/28/2019, 10:02:37 PMnode: 3e768908.1db9b6

msg : error

"Error: Parse Error"

4/28/2019, 10:02:37 PMnode: 91fb943f.54cffmsg : Object

object

_msgid: "b1d4eec3.68f8e"

topic: ""

payload: "Error: Parse Error : http://192.168.1.10:8080/user1?16&1=GetLogHour&19&100&20&10&"

statusCode: "HPE_INVALID_CONSTANT"

5

how have you configured the http request node?

7

image
image.png979×643 16.3 KB

8

This indicates a different content-type header than expected.
Open the url in the browser, use the web inspector to verify the content-header and match it in node-red with the msg.header option.

9

Hi bakman2, I don't know exactly what you mean, but this is the reply in the browser?

  1. Request URL:

http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&

  1. Request Method:

GET

  1. Status Code:

200 OK

  1. Remote Address:

192.168.1.10:8080

  1. Referrer Policy:

no-referrer-when-downgrade

  1. Request Headersview parsed

  2. GET /user1?64&1=GetLogHour&19&100&20&10& HTTP/1.1 Host: 192.168.1.10:8080 Connection: keep-alive Cache-Control: max-age=0 Upgrade-Insecure-Requests: 1 User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/73.0.3683.103 Safari/537.36 Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,/;q=0.8,application/signed-exchange;v=b3 Accept-Encoding: gzip, deflate Accept-Language: nl-NL,nl;q=0.9,en-US;q=0.8,en;q=0.7 Cookie: username=adminIP

  3. Query String Parametersview parsed

  4. 64&1=GetLogHour&19&100&20&10&

10

What is the application you are contacting?

It may be that it's an old application with a HTTP/0.9 interface. That wouldn't send back HTTP headers, only the raw body content and therefore the Header Parse Error.

Can you run curl -v to that URL in a command line to see what comes back?

11

Hi Steve, can you be a bit more specific? Where should I try the curl command?

12

This part is essential. Use a different browser to show these headers, or as @Steve-Mcl says, use

curl -v "http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&"

on the commandline (if you are on linux, windows does not have curl installed i believe).

13

Great, at least we know that, that works. Now how to get it working in node-red?
I appreciate your help.

HTTP/1.1 200 OK

{"header":["HourLog","","CH1 verbruik 01E00002","Year : 2019","Day Start : 100","Day's Qty : 20","Interval : 10 min","Units : TYPE 0"],
"footer":["Import","Export"],
"body":[["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],
["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],["3","0"],

14

I dont think you can use the HTTP Request node as the repsonse from that web server is not valid & therefore you get an HPE (Header Parse Error).

You could use the TCP node to & generate a GET command to IP 192.168.1.10, PORT 8080 & the response you get back would be the JSON as expected.

See this for an idea of how to generate a HTTP GET using TCP (or in this case telnet - same thing)...
https://www.the-art-of-web.com/system/telnet-http11/

15

Try this flow:

[{"id":"a92cf1.8c434b1","type":"inject","z":"cf07e2a5.1d217","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":264,"y":418,"wires":[["df1b46c2.dc0748"]]},{"id":"df1b46c2.dc0748","type":"exec","z":"cf07e2a5.1d217","command":"curl -v \"http://192.168.1.10:8080/user1?64&1=GetLogHour&19&100&20&10&\"","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"curl","x":424,"y":418,"wires":[["1f5ac981.6ccdde"],["f17d89c8.de8bf"],["cf10d274.0242e"]]},{"id":"cf10d274.0242e","type":"debug","z":"cf07e2a5.1d217","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":622,"y":462,"wires":[]},{"id":"1f5ac981.6ccdde","type":"debug","z":"cf07e2a5.1d217","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":622,"y":374,"wires":[]},{"id":"f17d89c8.de8bf","type":"debug","z":"cf07e2a5.1d217","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":622,"y":418,"wires":[]}]

note that your output is not a valid header, nor is it valid json as there the ending } are missing.

16

Hi Bakman2,

Great stuff that works!!! Now I will try to figure out how to get the usable data.
Thanks!!

17

Hi can somebody help ...how to read the data of device status from webserver which has username and password

18

Hi Duncan,

Pls let me know how reading data from webserver worked for you.

19

thank you for this flow.

This helped me alot with another issue.