Convert InfluxDB v2 data to ui_chart

Hi Chris
Yes am also looking at that and started with that.
It is great for visualization but then I ended up wanting to write and execute some rules to control my inverter (ie if batteries low at night switch of [some devices], etc) and that is not available in Grafana. Soo back to node red.
I think I will still use Grafana for Display and Node Red for control...:slight_smile:

1 Like

Many of us do that. It is possible to put Grafana in an iframe inside the node red dashboard though personally I prefer to keep them separate.

Sounds very sensible

As a side project in my non existent spare time I am building a energy management machine learning set of rules that can be used to predict /select and manage appliances to meet solar production. Have a basic but effective version working to optimise my pool solar heating based on forecast , actual weather conditions and pool temp.

If I get the ml stuff in a position to share I will ping you and see if you want to excitement with it

That looks very interesting... I installed a full solar system in Feb and since been looking at anything that can optimise the system. Some days to much energy available and other too little and no hot water. I am very in what you do. Let me know when you got something going.

Hi Tj!

Yes, you're right. The order is sometimes scrambled. But as I saw (after my offline-weekend :smile:) you solved the problem.

I came across this problem before and solved it for me but didn't update my post. So here is my currently used flow, just to complete your solution with mine:

[{"id":"9a04a24e.ee0a08","type":"influxdb in","z":"5919a1cc.3c9298","influxdb":"c88eceda.a6f37","name":"Get System Stats","query":"","rawOutput":false,"precision":"","retentionPolicy":"","org":"privat","x":390,"y":280,"wires":[["26643d5.9d86742","99e54818.08b9"]]},{"id":"22d20cf6.0b37f4","type":"function","z":"5919a1cc.3c9298","name":"InfluxDB v2 Query","func":"\nnode.log(\"start query preparation\")\n\n// fields to query\nvar queryfields;\n// add query string to fields\nqueryfields = msg.fluxdata.fields.map(x => 'r._field == \"' + x + '\"');\n// join all fields\nqueryfields = queryfields.join(' or ');\n\nmsg.query = \n'from(bucket: \"' + msg.fluxdata.bucket + '\")' +\n' |> range(start: -' + flow.get(\"timeframe\") + 'h)' +\n' |> filter(fn: (r) => r._measurement == \"' + msg.fluxdata.measurement + '\"' + \n' and (' + queryfields + '))' +\n' ' + flow.get(\"querymean\") + \n'';\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":370,"y":220,"wires":[["26643d5.9d86742","9a04a24e.ee0a08"]]},{"id":"4956864f.515978","type":"ui_button","z":"5919a1cc.3c9298","name":"","group":"fc5fded6.8684e","order":0,"width":0,"height":0,"passthru":false,"label":"Start Query","tooltip":"","color":"","bgcolor":"","icon":"","payload":"","payloadType":"str","topic":"topic","topicType":"msg","x":130,"y":120,"wires":[[]]},{"id":"aa866515.db1b48","type":"ui_chart","z":"5919a1cc.3c9298","name":"","group":"fc5fded6.8684e","order":1,"width":"10","height":"8","label":"chart","chartType":"line","legend":"true","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"","ymax":"","removeOlder":1,"removeOlderPoints":"","removeOlderUnit":"86400","cutout":0,"useOneColor":false,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"outputs":1,"useDifferentColor":false,"x":670,"y":400,"wires":[[]]},{"id":"99e54818.08b9","type":"function","z":"5919a1cc.3c9298","name":"Conversion","func":"// helper\n// used to get all distinct values from an array\nconst distinct = (value, index, self) => {\n    return self.indexOf(value) === index;\n}\n\n\n// main code\n\n// first, get all fields of the query result\nconst fields = msg.payload.map(x => x._field);\n// get all distinct fields\n//const distinctfields = fields.filter(distinct);\n\nconst distinctfields = msg.fluxdata.fields;\nnode.log(distinctfields);\nnode.log(distinctfields.length);\n\n// get all timestamps\nconst timestamps = msg.payload.map(x => Date.parse(x._time));\n// get all values\nconst values = msg.payload.map(x => x._value);\n\n\n// loop over all distinct fields\nvar ind;\nvar data = [];\nvar arr = [];\nvar minvalues = [];\nvar maxvalues = [];\n\nfor (var n = 0; n < distinctfields.length; n++) { \n    node.log(n);\n    // get indices of matching entries for current field\n    ind = [];\n    for (var m = 0; m < fields.length; m++) {\n        if (fields[m] === distinctfields[n]) {\n            ind.push(m);\n        }\n    }\n\n    // get matching timestamps\n    var matchingtimestamps = ind.map(x => timestamps[x]);\n    // get matching values\n    var matchingvalues = ind.map(x => values[x]);\n    // use only 2 digit floats\n    matchingvalues = matchingvalues.map(function(each_element){\n        return Number(each_element.toFixed(2));\n    });\n\n    // compose array\n    arr = [];\n    for (m = 0; m < matchingtimestamps.length; m++) { \n        arr.push({'x': matchingtimestamps[m], 'y': matchingvalues[m]});\n    }\n    // save array\n    data.push(arr);\n    \n    // save min and max as well\n    minvalues.push(Math.min.apply(Math, matchingvalues));\n    maxvalues.push(Math.max.apply(Math, matchingvalues));\n\n}\n\nconst payload = [{\"series\": distinctfields, \"data\": data, \"labels\": distinctfields}];\n\nmsg.payload = payload;\nmsg.minvalues = minvalues;\nmsg.maxvalues = maxvalues;\n\nreturn msg;\n\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":390,"y":340,"wires":[["aabf63bc.97bfd"]]},{"id":"e7fd49e8.785688","type":"inject","z":"5919a1cc.3c9298","name":"","props":[{"p":"payload"}],"repeat":"300","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"str","x":150,"y":160,"wires":[["cc8675e2.deecb8"]]},{"id":"5f3d1446.40cc6c","type":"ui_multistate_switch","z":"5919a1cc.3c9298","name":"","group":"fc5fded6.8684e","order":2,"width":"10","height":0,"label":"Time","stateField":"payload","enableField":"enable","rounded":false,"useThemeColors":true,"hideSelectedLabel":false,"options":[{"label":"1 h","value":"1","valueType":"str","color":"#009933"},{"label":"3 h","value":"3","valueType":"str","color":"#999999"},{"label":"1d","value":"24","valueType":"str","color":"#ff6666"}],"x":110,"y":200,"wires":[["cc8675e2.deecb8"]]},{"id":"cc8675e2.deecb8","type":"function","z":"5919a1cc.3c9298","name":"","func":"// check payload\nif (msg.payload === \"\") {\n    // empty payload, imput came from inject node\n    // check that we have someting in context store\n    // if not, set default\n    if ( typeof flow.get(\"timeframe\") == 'undefined') {\n        flow.set(\"timeframe\", \"1\");\n        flow.set(\"unit\", \"minute\");\n        flow.set(\"unitstepsize\", 10);\n        flow.set(\"querymean\", \"\");\n    }\n} else {\n    // non-empty payload, input from multistate button    \n    switch (msg.payload) {\n        case \"1\":\n            unit = \"minute\";\n            unitstepsize = 10;\n            querymean = \"\";\n            break;\n        case \"3\":\n            unit = \"minute\";\n            unitstepsize = 15;\n            querymean = '|> timedMovingAverage(every: 1m, period: 1m)';\n            break;\n        case \"24\":\n            unit = \"hour\";\n            unitstepsize = 3;\n            querymean = '|> timedMovingAverage(every: 10m, period: 10m)';\n            break;\n        case \"168\":\n            unit = \"day\";\n            unitstepsize = 1;\n            querymean = '|> timedMovingAverage(every: 1h, period: 1h)';\n            break;\n        default:\n            // nothing in here, we don't expect something different from multistate switch!!!\n            break;\n    }\n    flow.set(\"timeframe\", msg.payload);\n    flow.set(\"unit\", unit);\n    flow.set(\"unitstepsize\", unitstepsize);\n    flow.set(\"querymean\", querymean);\n}\n\nnode.log(\"set fluxdata\")\nmsg.fluxdata = {\n    \"bucket\": \"testbucket\",\n    \"measurement\": \"system\",\n    \"fields\": [\n        \"load1\",\n        \"load5\",\n        \"load15\"\n    ]\n};\n\n\nreturn msg;\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":320,"y":160,"wires":[["22d20cf6.0b37f4"]]},{"id":"aabf63bc.97bfd","type":"function","z":"5919a1cc.3c9298","name":"Conversion add ui_control","func":"\nmsg.ui_control = {\n    options: {\n        scales: {\n            xAxes: [{\n                type: 'time',\n                time: {\n                    displayFormats: {\n                        minute: 'HH:mm',\n                        hour: 'HH',\n                        day: 'MMM D'\n                    },\n                    unit: flow.get(\"unit\"),\n                    unitStepSize: flow.get(\"unitstepsize\"),\n                },\n                ticks: {\n                    display: true,\n                    fontSize: 11,\n                    maxRotation: 0,\n                },\n                gridLines: {\n                    display: true,\n                    drawBorder : false,\n                    drawOnChartArea: true,\n                    drawTicks: true,\n                    tickMarkLength: 5\n                }\n            }]\n        },\n        legend: {\n            display: true\n        }\n    }\n};\n\nreturn msg;\n\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":470,"y":400,"wires":[["aa866515.db1b48"]]},{"id":"26643d5.9d86742","type":"debug","z":"5919a1cc.3c9298","name":"Influx Query","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":610,"y":220,"wires":[]},{"id":"c88eceda.a6f37","type":"influxdb","hostname":"127.0.0.1","port":"8086","protocol":"http","database":"database","name":"Influx DB v2","usetls":false,"tls":"","influxdbVersion":"2.0","url":"http://localhost:8086","rejectUnauthorized":true},{"id":"fc5fded6.8684e","type":"ui_group","name":"Influx 2 Test","tab":"f24c3d76.06c3e","order":2,"disp":true,"width":"10","collapse":false},{"id":"f24c3d76.06c3e","type":"ui_tab","name":"Influx","icon":"dashboard","order":6,"disabled":false,"hidden":false}]

Basically I did:

  • add a multistate switch to show different time ranges
  • add an object with the definitions which measurement/fields to query
  • modify my conversion function to use this object to get the same order I specified in there

Maybe you want to try it or maye it is helpful for someone else :grinning:

Thanks Simon

I will definitely try it as currently have 6 buttons each with its own query injecting into the influxDB node to extract the different time frames...

image

image

I then keep the graph updated with the new data points feeding into the graph and this work fine at the moment. Only down side is the order of series is alphabetically and that could be a challenge when naming series to fit the order you want.

Thanks for your help...

That's why I use the multistate button instead of 6 single buttons.

If you look at my flow where I set the parameters for the query (first function), there are of course a lot of possibilities to improve the handling. Adding custom titles for queries is only one such thing...

I updated my application and I used your new functions. Works well now.
In my application I find injecting realtime data faster than refreshing the graph data every few minutes.
Just having some examples of how others apply the nodes is a great asset for us newbies. Would never have thought about the multi switch.:slight_smile: :smile:
Thanks again for making your code available .. much appreciated!

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.