Need some opinion on smart home creation

Hi there,

I would like to set up some sensors around my home and get the necessary data on my android phone. In other words, make a hobby-type smart home.

I have a few basic sensors and PI3 as well as some Elegoo. I will use a node-red dashboard so data can be seen on the phone.

I am wondering about the node-red.. are there tutorials with flows available from other members? That's usually the fastest way how I could learn and then doing some adjustments.

Perhaps you could suggest something?

Regards

a lot of people use "home assistant". I don't.

I missed the memo when it was sent.

But basically you need to write some code which gathers all the sensor inputs and then start to work out how you want to show it.

That is a whole other thing to getting the data.

Have you looked here? https://flows.nodered.org/

And the docs also have examples here ... https://nodered.org/docs/

Part of the issue is that home automation setups in Node-RED tend to get complex and rather specific to the originator over time. So they are hard to share.

If you want a simple setup with less admin, use Home Assistant or something similar. If you want fun, control, adaptability and flexibility and to learn how things work - then you are in the right place here :smiley:

If you decide to join the fun, there are a few things to think about ...

How will you manage your sensor and controller devices and link them to locations around your house. The easy way to start is not to worry about it and jump straight in. Add some sensors and build a dashboard. And you may find that is all you ever need to do. But at some point you will probably reach the point where you no longer want to rejig flows every time you want to change a sensor or controller. At that point you will find yourself unpicking your spaghetti and disaggregating source sensors from target controllers and setting up small data structures for doing things like mapping your device ID's to real locations around the house.

I really would encourage you to set up a couple of sensors and maybe a couple of light switches so that you can have an initial play with things. Then start to think about how you want to manage the devices, what metadata you want to manage in relation to the devices. That is also the time to start thinking about historic data - do you want to track sensors and switches over time? If so, you will probably want to start learning about InfluxDB and Grafana.

Best advice I can give is to keep inputs and outputs on separate flows (disagreggated).

Capture inputs and normalise the data as far as possible so that inputs use common property names and value types and you add timestamps where the sensor doesn't provide them.

Feed those through to MQTT using a standard topic naming convention such as DEVICES/<deviceid> and maybe a copy to InfluxDB where you want the historic data. You could also accumulate that data into a global variable as well for later use though I'd probably hold off on that until you actually need it. That is one distinct set of flows. You now have the data available for whatever processing you want to do.

Next set up flows to output data to controllers (switches, etc). But don't connect them directly to any input flows even if you intend to use a sensor (e.g. light or heat) to control the output later. Drive them from the MQTT topics or CRON+/Inject nodes instead. That will let you more easily add other processing later when you need to.

You should now be able to see how easy it will be to add more sensors and controllers with minimal effort - sometimes no effort at all.

You might at this stage, also want to build a generic flow to output to other alerting tools such as Telegram, email, SMS or whatever is convenient. That way you can pass whatever data you like using standard msgs, let the flow reformat the data to the required output format. Reusable.

Then you can add more complex control flows - such as building some alerting/alarm features by listening for certain sensor events and outputting to both your alerting tools and to a controller (e.g. lights or a sounder).

Then you can start to build other flows that get data from other sources such as weather or news. Now you can weave that into your output flows as needed or simply send to your dashboard for information.

Node-RED is so flexible and has so many options for inputs, outputs and processing that there really isn't one way of doing things. You have the joy of building something that both works for you specifically and gives you the enjoyment of the design and build not just the using.

This kind of project is exactly what I'm working on, and I think @TotallyInformation has summed up most of the things you'll need to think about. Particularly his comments on keeping flows simple, separating input and output parts of the system, and using a standardised naming convention for messages, both over MQTT and internally.

My approach is using a simple software framework running on ESP8266 and Raspberry Pi Zero devices, and adding just the "module-specific" functionality. The same could be adapted to Arduino-based modules as well. I've kept each module to a single function, or a few closely related ones, such as room environment (temperature, humidity and light level), motion sensing or lighting control, which simplifies the design of each. Mixing multiple functions on a module could lead to complex and difficult to maintain software.

I've sort of "classified" the functionality into major areas including heating, lighting, security etc., although there's a bit of overlap in some areas. Again, you might want to have a think about how the different parts of your system are going to interact as well.

All communication is via MQTT with JSON payloads, which allows for future changes to message data without needing a lot of code or flow rewriting.

I've still got a lot to do even with the basic system, and have plans for a whole range of modules to add to it eventually, but the approach of keeping things small and simple makes the (never ending) project manageable.

That is also what I'm moving to. I've used Tasmota and EasyESP in the past but I'm slowly replacing those with my own code that does just what I want it to. Once set up, it never needs updating because it works. My own sensor framework uses my favourite sensors with temperature, humidity, light and sound level being fairly standard. The code evolves slowly over time but really I don't ever need to update it on a sensor platform unless one of the parts dies (never have yet). It really isn't that many libraries and bits of code to have all of those sensors on a single ESP8266 and I can use compiler switches to turn on/off sensor code for each device as I compile. There aren't so many that I have to worry about managing different versions and device types.

Things like PIR, and open door sensors I tend to buy as a package, up to now, they've generally been HomeEasy EU based as I have a RFXtrx433e so they are fairly cheap and they tend to look nicer than home-grown packages (I don't have a 3d-printer). The environment sensor platforms are ESP8266 based - so far uncased since they get tucked away. I've collected various cases though and I'll mount them one day. :slight_smile: They are also all USB powered since I can place them in reach of a power outlet.

Really, the Tasmota/ESPEasy/etc type framework isn't worth it for me, the updates that are needed because of their complexity I don't really want. Also, I find that their complexity leads them to crash far more often than my own code which tends to be simpler and more efficient.

Mains control plugs and switches I purchase because I'm not confident enough to do my own for mains power and by the time I'd put in thermal fuses and other protection and then put them in a reasonable looking case, it wouldn't really save any money. I do need to learn how to replace relays though as I've had quite a few go over the years - I've kept all the devices though so I can repair them eventually :smiley:

I've done that in the past but I'm really simplifying my MQTT now because I find I don't actually need that much. Really all I use it for is comms to/from WiFi devices and comms between my 2 Pi's (older code) and my home automation laptop (which is replacing the Pi's). So much of the logic is now inside Node-RED flows and with the use of link nodes and retained variables, my need for MQTT is now much less.

One day, I'll write a variable handler for Node-RED that includes event handling and then the need for MQTT will be even less since that is the one thing that is currently hard to do in Node-RED (link nodes are good but not really substitute for a variable handler that notifies flows when something changes and a variable listener node). If someone doesn't beat me to it that is.

1 Like

@TotallyInformation - we seem to be heading in the same direction for a lot of design choices. I've looked at Tasmota and ESPEasy as well, but like you I wanted something cleaner and simpler. They're good frameworks for getting started, but way too complex with all the variants and options.

Rather than compiler switches, I just have a pair of .cpp/.h files for each different device, and include the framework as libraries (well, currently compiling them in from source, but will move to libs once I'm 90% happy with them).

I've gone with the JSON over MQTT to keep communications simple and standard. It's easy to convert and process message fields by name, and if I need to add a field it's simple to do in both the device or flow.

It's always evolving though, and half the fun is in the development process :wink:

I'm not much of a C++ coder :blush: and I only use the Arduino IDE, everytime I try to use something else like PlatformIO, I just find it too much bother to set up and I give up and go back to the Arduino IDE again.

Yes, same here. It is only NR internal flows where I've removed most (maybe 80%) of the MQTT as link nodes and retained variables are generally more than enough.

Absolutely, that's why NR is so much fun. :smiley:

1 Like

I suppose I've been coding in various languages and using different development environments for <redacted*> years now, so I've learned to adapt to just about anything :slight_smile:

Currently I'm using Visual Studio with the Visual Micro plugin, although I've played with VS Code, Eclipse and the Arduino IDE as well. They all do pretty much the same things, just some have more bells and whistles, so go with what you're happy with. (One of these days I should write up a comparison of IDEs.)

[ * > 50... ]

1 Like

Hi,

So I managed to play around with node-red. I managed to set up different sensors and see them on the node-red dashboard.

I would like to move to the next step and make each sensor independent, meaning the use of MQTT and sensors to monitor different parts of the home.

For example, I would like to monitor different rooms (movement, temperature, humidity, plant moisture level, etc.) so to achieve that I will use MQTT and Pi3 as the main data collector via the node-red platform.

Is this even possible? MQTT would give me the option to save on cables and options to use in different places at home.

Later I want to connect data with Power Bi, to visualize the data (history).

What are your thoughts, would my concept event work with MQTT, Node-Red, and Pi3 ?
Perhaps there are better ways of what I am trying to achieve?

This is possibly the most common use of Node-RED (or was until the uptick in professional use over the last 18m or so).

I have multiple ESP8266 sensor platforms around the house measuring temperature, humidiy, sometimes light, sound and pressure too. A couple of other devices feed back power consumption. These all communicate direct over MQTT to a broker and Node-RED consumes the data also from the broker.

I also have other devices that have 433MHz radios and so can't talk MQTT directly (they are pretty dumb - cheap - devices), these talk direct to a transceiver device on my home server where Node-RED is running. I generally translate these into forms similar to the ESP device outputs and send those to MQTT as well though I don't really need to since NR is doing all of the actual processing and recording to a database, showing dashboards, etc.

PowerBI is an interesting (and certainly overkill) way to produce a dashboard for your home :slight_smile: Worth noting that Excel has the same processing engine as PowerBI in the form of PowerQuery. So you can use that instead if you don't need the web sharing of PowerBI. I assume you have a corporate license for PowerBI that you are using since the paid versions are quite expensive. I do have a PowerBI Pro license but I wouldn't bother to use if for home monitoring myself. Many of us do, however, use the free Grafana tool which is a more open dashboard builder. Not as powerful in some ways but certainly very good to use, especially when working with timeseries data from InfluxDB.

1 Like