Grafana Dashboard is less accurate than Node-Red Dashboard

General
1

I am new to Node-Red and to this Forum.
I am collecting data from a ESPCAM32 via MQTT. The data comes in two parts one just the Value and the other with object info: value, error, rate.

I make use of a small function and then pass the data to influxdb.
I graph this data then in a node red graph as well as in grafana but here is the difference.

Node-Red ignores the bad data (sensor error or connection etc) but grafana / influxdb records this and my graph is not consistent.

image

I was hoping to use the 'difference function' within grafana to show 'today' vs 'yesterday' or 'this week' vs 'last week' etc. But i think because of my bad data this is not working.

Here is my Flow:

image
image1025Ă—371 21.8 KB 

[{"id":"85b96703.91b2e8","type":"tab","label":"Get Water Data","disabled":false,"info":""},{"id":"e4b4bcc7.75523","type":"comment","z":"85b96703.91b2e8","name":"Get Water Meter info and Write into DB","info":"","x":190,"y":60,"wires":[]},{"id":"b038946c.786728","type":"influxdb out","z":"85b96703.91b2e8","influxdb":"22e779f5.7dee06","name":"to InfluxDB","measurement":"Data","precision":"","retentionPolicy":"","database":"database","precisionV18FluxV20":"ms","retentionPolicyV18Flux":"","org":"organisation","bucket":"bucket","x":930,"y":220,"wires":[]},{"id":"a4727018.fa0fc","type":"mqtt in","z":"85b96703.91b2e8","name":"","topic":"#","qos":"2","datatype":"auto","broker":"3b267256.a4deae","nl":false,"rap":true,"rh":0,"x":170,"y":220,"wires":[["e49b0f69.bac13","c62ff07c.d3b6b"]]},{"id":"e49b0f69.bac13","type":"json","z":"85b96703.91b2e8","name":"","property":"payload","action":"","pretty":false,"x":410,"y":220,"wires":[["5b68ed04.2297b4","a6ac87dd.11c9f8"]]},{"id":"5b68ed04.2297b4","type":"debug","z":"85b96703.91b2e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":650,"y":160,"wires":[]},{"id":"a6ac87dd.11c9f8","type":"function","z":"85b96703.91b2e8","name":"","func":"msg.payload = msg.payload.value;\n\n//Update the Status with current timestamp\n\nvar now = new Date();\nvar yyyy = now.getFullYear();\nvar mm = now.getMonth() < 9 ? \"0\" + (now.getMonth() +1) : (now.getMonth() +1); //getMonth() is zero-based\nvar dd = now.getDate() < 10 ? \"0\" + now.getDate() : now.getDate();\nvar hh = now.getHours() < 10 ?\"0\" + now.getHours() : now.getHours();\nvar mmm = now.getMinutes() < 10 ?\"0\" + now.getMinutes() : now.getMinutes();\nvar ss = now.getSeconds() < 10 ?\"0\" + now.getSeconds() : now.getSeconds();\nnode.status({fill:\"blue\",shape:\"ring\", text:\"Last update: \"+ dd + \".\" + mm + \".\" + yyyy + \" \" + hh + \":\" + mmm + \":\" + ss});\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":660,"y":220,"wires":[["adf7974.5b25c68","b038946c.786728","bac5d02c.682b9","324e8646.0c567a"]]},{"id":"adf7974.5b25c68","type":"debug","z":"85b96703.91b2e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":930,"y":160,"wires":[]},{"id":"c62ff07c.d3b6b","type":"debug","z":"85b96703.91b2e8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":430,"y":160,"wires":[]},{"id":"bac5d02c.682b9","type":"ui_text","z":"85b96703.91b2e8","group":"27c3d951.18a4e6","order":0,"width":"4","height":"1","name":"","label":"OCR Read","format":"{{msg.payload}}","layout":"row-spread","x":940,"y":300,"wires":[]},{"id":"324e8646.0c567a","type":"ui_chart","z":"85b96703.91b2e8","name":"Water Meter","group":"27c3d951.18a4e6","order":1,"width":"25","height":"8","label":"Water usage ","chartType":"line","legend":"true","xformat":"HH:mm:ss","interpolate":"linear","nodata":"Value","dot":false,"ymin":"","ymax":"","removeOlder":"4","removeOlderPoints":"","removeOlderUnit":"604800","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"useDifferentColor":false,"x":940,"y":360,"wires":[[]]},{"id":"22e779f5.7dee06","type":"influxdb","hostname":"influxdb","port":"8086","protocol":"http","database":"data","name":"","usetls":false,"tls":"","influxdbVersion":"1.x","url":"","rejectUnauthorized":false},{"id":"3b267256.a4deae","type":"mqtt-broker","name":"","broker":"192.168.178.154","port":"1883","clientid":"","usetls":false,"protocolVersion":"4","keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","birthMsg":{},"closeTopic":"","closeQos":"0","closePayload":"","closeMsg":{},"willTopic":"","willQos":"0","willPayload":"","willMsg":{},"sessionExpiry":""},{"id":"27c3d951.18a4e6","type":"ui_group","name":"Water Meter","tab":"156af96f.f8fd27","order":2,"disp":true,"width":"25","collapse":false},{"id":"156af96f.f8fd27","type":"ui_tab","name":"Home","icon":"home","order":"1"}]

Any suggestions would be very welcome.
Thank you
regards
Dieter

2

Why note validate the data just after the mqtt-in node and ignore the bad data before it is passed to the rest of the flow?

1 Like
3

You must make sure that in the function you ignore any missing values rather than passing them on as undefined.

4

Hi zenofmud,

I am not sure how to do that, what I build was from other flows on this Forum and by trial and error.
If you you would not mid how would you have done it in the flow?

The error screen shot look like this:

image
image588Ă—670 96.6 KB

Thank you for your help

5

Here is an example

[{"id":"487294af779a4ba7","type":"tab","label":"Flow 1","disabled":false,"info":"","env":[]},{"id":"05da7d48f51e0a64","type":"inject","z":"487294af779a4ba7","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payloadType":"date","x":160,"y":160,"wires":[["1b725ddd63497290"]]},{"id":"1b725ddd63497290","type":"switch","z":"487294af779a4ba7","name":"","property":"payload","propertyType":"msg","rules":[{"t":"null"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":390,"y":220,"wires":[["c157d1c81de26725"],["4e952eead9ce43e4"]]},{"id":"c157d1c81de26725","type":"debug","z":"487294af779a4ba7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":620,"y":140,"wires":[]},{"id":"4e952eead9ce43e4","type":"debug","z":"487294af779a4ba7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":660,"y":280,"wires":[]},{"id":"a18bbf7b06ead290","type":"inject","z":"487294af779a4ba7","name":"No payload","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":170,"y":260,"wires":[["1b725ddd63497290"]]}]
6

I see you are subscribing to #, that means that it will try to write the data from every single MQTT topic to the database. Normally you should only subscribe to the topics that you need.

You can remove the JSON node if you set the output of the MQTT node to Parsed JSON.

The reason for the undefined is that in your function you have
msg.payload = msg.payload.value
but sometimes msg.payload.value does not exist. If you separately subscribe to the two topics with the value and the object then you can handle them each as you want.

7

Hi zenofmud,
Thank you for your tip, I applied this to my flow, and it is only passing the good data on.
Thanks again.

8

Hi Colin,
I left the # in the Topic but changed the Output to parsed JSON and took out the Jason node, this simplified my Flow.
Thank you

9

Why? That means that if you extend the use of MQTT in your flows then it will try to write that data to influx, also it means that you are getting the two sets of data down the same wires, which complicates the logic.

10

Hi Colin, I removed the # but then I was not getting any data from the sensor. I checked the Top on the sensor as well as Node-Red to make sure the there was not a spelling or case issue but there were not.
Is there another way to check if this?

11

What did you change it to? You need to change it to whatever topic you are publishing to.

I wonder if you are not familiar with the basics of MQTT. I suggest you read through MQTT Essentials - All Core Concepts Explained

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.