I have all my sensors data grouped in rooms and stored in context. Could be possible to iterate through the rooms and creating a group for each one and then create widgets for the sensors values on them?
I'm still learning on node-red and still don't know about it's limitations. Thanks.
Not with the standard dashboard. But there are other options like the node-red-contrib-uibuilder project that give you much greater freedom to build it as you wish albeit at the cost of you having to create it.
3 Likes
You can certainly do this with uibuilder.
The trick is to start with the simplest elements and work up from there. One step at a time and you will soon have it sorted.
Thanks. I'm still doing tests and trying to decide whats the best way to store any topology of sensors in context. At the same time reading all I can about Vue. I know Angular and React, but I still didn't met Vue.
Any advice on nodes/sensors storage would be welcomed.
Personally, I use simple object variables as I don't have enough sensors to worry about memory utilisation that much.
I have a devices variable object that is keyed on a devices ID which is the hardware ID plus the controller ID, e.g. WIFI/D1M01 which is one of my ESP8266 based sensor platforms. Each ID is an object that describes it in text, its location and capabilities (I use single character codes for that so that I can process in and outbound flows based on device capability later on).
Locations are also held in another object so that I can have extended metadata as needed but keep the actual location ID fairly short (though still human readable).
All inbound sensor data is "normalised" with naming convensions for standard data like temperature, humidity, etc. Timestamps are also added along with extended data such as dewpoint, sealevel pressure (I always use that not absolute pressure as that is what weather forecasts use), etc.
In terms of actual sensor data, I send the latest data to MQTT in one flow (as a serialised JSON object). A second flow listens for updates and splits the data into individual readings which are more useful for some processing.
Another flow listens for updates and updates the in-memory data then sends to InfluxDB for long-term tracking.
Other flows also listen to sensor changes and process warnings, notifications, etc.
Any devices accepting outbound commands go through an opposite process in that I use standardised commands - e.g. COMMAND/SWITCH01 On - which are translated (using another global variable object) to actual device and controller ID's and sent accordingly.
2 Likes
Thnaks for sharing. It could be interesting to have a simple-and-empty example of what you're talking about so I could better understand. I fear that i'm loosing something.
I mainly use mysensors.org for my nodes. So I can shape them for the specific purpose. So each node(sensor for the project) have it's sensors. And I receive and send them through a lan-to-rf gateway. So I first have a flow that converts lan traffic to mqtt for mysensors.
Then a second flow stores all the incoming sensor data into influxDb almost as arrives, separating topology info (in topic) into tags.
Then another flow separates the true sensor readings data from utilities and actuators and stores them in a context object.
That context flow also serves as trigger point for the interface flows, like dashboards and now google integration. It has several input and output links (that aren't tied to a node or sensor), so when a value is processed, it triggers other flows through those links.
My context object is based on rooms or groups if you prefer. There's a group list and assignations object that it's queried when a reading arrives. Then it stores the value using naming conventions like you do for temp, hum.. etc.
So the object seems like:
rooms
L garage
L garage door
L temp (yes, it also has env sensors)
L hum
L doorsw
L lastseen
L lasttripped
I was now trying to solve some concerns, that are obvious by looking the object.
- How to correctly store lastseen for every sensor. Or seen/tripped duality.
- What if one node has several sensors of the same type? Like a furnace or fridge node.
I laid that structure because I only wanted it for dashboards and I thought that was better for enumeration or loop for every room.
I've seen that in your model the group is not a part of the hierarchy, but a qualifier. If I understand, you store things in a pattern like the mqtt topic. So your esp sensors are stored like "wifi/nodeXXsensorXX" and if you had mysensors could be stored "mys/nodeXXsensorXX". And then you have a separate collection of rooms to nodes/sensors? Or you store the group/room on every value, next to "lastseen"?
How do you re-integrate each "platform" into one for processing-storing?
Or do you only separate first sensors from actuators and then platform, like "sensors/wifi/xxx","sensors/mys/xxx" and "switches/mys/xxx"? Or platform stays as a property?
I wonder how you iterate your collection to get the data for your interface. And I think you store a temperature like "wifi/nodeXXsensorXX/temperature:value"
I guess it's possible using something like linq queries, but I'm so "relational" that I would never imagine that at the first option.
Undoubtedly it would - I have published stuff previously, you should be able to find references on this forum. The problem is that I created all of this over time so it has evolved. So not so simple that I could just extract some simple stuff I'm afraid.
I hadn't come across that before - interesting, shame it wasn't around when I started.
Your approach seems pretty logical & not too far off mine.
I do that in two ways, both are useful in different ways.
Firstly, each incoming device object includes (or has added) a timestamp. When split into individual elements in MQTT, there will be a matching timestamp entry. I use the "Homie" style topics for that.
Secondly, I keep a set of topics under "DEVICES" which tell me when things go on-/off-line, so the DEVICES/D1M01 topic, for example, will be set to "Online" or "Offline". The Online value is updated whenever a msg is received from that device and I pass that to my dashboard so I know exactly when each device last sent a message. The Offline value would be set by the MQTT broker itself using the LWT so again, the dashboard flow will know when it happened.
That is why all incoming devices use an ID that is unique. The location isn't unique but the ID is. The living room for example has two sensors and a controller. One sensor is my custom ESP8266 platform with temperature and humidity (soon to have light as well), the other is the TRV from the heating system which also measures temperature. The controller is a light switch (though the TRV is also a controller for the heating of course but that uses a very different API and I haven't integrated that as yet).
Here is an example from my dashboard that shows two switches in the same room as another example. You can find this code in the WIKI.
The data is grouped primarily by ID not by room. The room is metadata for most things. However, when sent to my custom dashboard, things are grouped differently because the output is primarily for display in a set of tables. In that case, everything is in an array of objects with the location encoded like "HOME/IN/02/BATHROOM" (building/in-out/floor#/name).
Here is a typical array entry:
{
"location":"HOME/IN/01/HALL",
"sensors":{
"Temperature":21,
"Humidity":47,
"Light":16.25
},
"switches":[
{
"id":"SWITCH02",
"status":"Off",
"lastUpdate":"2019-08-31T11:26:31.987Z",
"room":"Rear Hall"
}
],
"floor":0,
"outside":false,
"devices":[],
"id":"Rear_Hall",
"Name":"Rear Hall",
"CalculatedTemperature":210,
"CalculatedHumidity":47,
"CalculatedLight":16.25,
"_showDetails":false
}
Because this is combined from a number of sources, there is some overlap which I don't bother to eliminate. This particular entry has nothing in the devices array because that is specific to the heating and the rear-hall doesn't currently have a TRV or standalone thermostat.
I have both types of MQTT entries - a single device entry with all data as serialised JSON and a sub-tree set of entries in the Homie style. That's because, as you've realised, both styles of data are useful for different reasons. As I'm not dealing with industrial scale device loads, I can easily afford to have both. A single JSON entry is the most use for when you want to pass data onto a front-end dashboard. But the individual entries are most useful for alerting or for a simple, special dashboard (which I might do using NR's Dashboard rather than using uibuilder for a full custom job - depends on the need, Dashboard is still great for prototyping).
I don't because I store both 
I've already explained part. When a new tab connects or reloads, I send the current device status & the switch status objects which are held in memory.
The full homeDetails object is also resent and that contains everything necessary, other than the statuses, to build the full table. This is then resent about once a minute. Actually, this is something that I will likely refactor eventually as it is a bit heavy-handed. Really it would be better to split it up more and only send what is necessary. A job for this winter probably.
As mentioned, the main data is an array because that's how the bootstrap-vue table component wants it. You can see the layout above. Because I may have >1 sensor, I chose the most accurate of each sensor type and put that into the sensors object.
Actually, the creation of the homeDetails array is a mess right now and needs simplifying. That can be the downside of Node-RED - it is great for prototyping, not always so good for later refactoring (it works, why bother!). The creation/update of the array actually lives on a different Node-RED tab to the updating of the uibuilder interface.
1 Like
I need some time to chew on this. I realize that if I want that vue interface I have to think so well how to store things. You don't selective send data to the ui. You pass the json and the page gets what needs. I'm still on the .Net webforms era 
Mysensors is one of those projects too much interesting (aka never having enough time for it). To say, it has the advantages of espEasy or Tasmota, but killing the high power requirements. Nodes can be the bare minimal arduinos (even underclocked to 1mhz or without xtal) with a 2.4ghz radio attached. Some interrupt nodes, (garden door bell) stay alive for 2-3 yr with a button style battery.
A gateway is a must, and you have a variety of them. You can build one with an esp8266+2,4rf radio. And it can directly inject mqtt.
The advantage for you, could be that using the mysensors node-red module, you have "data splitter" nodes that bring all that info (node Id, sensor Id, type of command, and type of reading... temp,hum...) so that data collection can be populated without "copy&paste".
If you where to start again.
1 Like
Well that is always true though isn't it. Not really a Vue thing.
Yes, well I'm not saying that what I do is the best way to do it. Most likely not. Remember, I was learning Vue at the same time as well. Plus I had a load of evolved code/data and I'd added a smart heating system to the mix. So lots of complexity and I'm absolutely certain there are better ways to do it.
You make some really useful points about MySensors and I will have a further look - added my (long!) list of things to investigate for sure. I've several times thought about moving my data capture out of Node-RED into a separate gateway process so that fits in with my thinking. It's just that since it does actually work just fine in Node-RED, I've not had a lot of incentive to do anything. But I know that it all needs refactoring & I'm certainly tempted to start again & simplify everything.
:sigh: One day maybe 
