General questions about home automation design in Node-RED

General
1

Hello

Thanks to @knolleary for prompting me on twitter to pop some ideas down here on the forum. And a request for help!

New Node-RED convert here, and I have a "there are many ways to skin a cat" kind of question. Getting overwhelmed with the possible number of ways of doing it, so hoped someone could give me some practical suggestions. I think this may be more of a generic programming question than specific to Node-RED, but hopeful that there might be a Node-RED flavoured solution. Or that you might be kind enough to help me regardless.

I have around 40 light circuits to automate*, some switched, some dimmed, some RGBW. My question is not about driving the actuators from within Node-RED but more about how to go about designing the flows.

I'm moving a system over from openHAB to Node-RED, and I'm basing the flows around controlling lighting zones from a home-brew panel (check out my panel featured on Automated Home website - pics of it installed and general concept)

The panel (Arduino Mega) deals only with MQTT. It has no hard-coded values or configurations, it uses my own simple ZeroConfig style of boot sequence. It's currently working in the state described below, using openHAB to pull the puppet-strings, but I want to implement it in Node-RED and make it better.

When it boots, here's what happens:

  1. Arduino sends its MAC address on an MQTT topic dedicated to ZeroConfig: home/megasetup

  2. Automation hub (i.e. Node-RED) sends back configuration info on a topic dedicated to that controller: home/megasetup/DEEDBAFEFEED
    ... this is done in sequence

  • payload = ZONENAME LG_Bedroom (name of zone)
  • payload = ZONEVECT bedroom (this is used to tell the Arduino on what topic to start publishing its commands to control lights, this translates to home/bedroom/light/xyz)
  • payload = ZONESENS 25 (some other stuff to customise this panel - this particular item sets the touch sensitivity for proximity sensing)
  1. After this, the Arduino then sends messages in sequence asking for two types of information: configuration info and current values. It does so on the topic it has just learned about:
  • topic home/bedroom/megacmd
    • circuit_count_please
    • circuit 1 name please
    • circuit 2 name please
      etc.
    • light preset 1 name please
    • light preset 2 name please
      etc.
    • circuit 1 level please
      etc.
    • album title please
    • album artist please
    • song title please
    • playpause status please
      ...
      and a whole LOAD of other things, moving into heating, weather, room temperature / humidity sensors etc.!

  1. The automation hub responds with answers to these requests. It doesn't matter when it responds - it will simply fill out the values then display them on the screen at any point from there onwards in the sketch. Each response is on a dedicated topic.

  2. Then when you press buttons etc., the Arduino sends commands on a single topic for that particular controller, home/bedroom/megacmd (the bedroom part it learnt about in ZeroConfig)

  • example, when the dimmer is turned, it fires off a load of light_1_up or light_2_down commands, whatever circuit is selected. The automation hub must then get the current light level, add 2 to it, set it back into a variable, and of course actuate the change by sending an RF message (or in the case of my new home, it will be sending another MQTT message to the DMX controller Arduino). This happens quickly so the user sees the light dimming up gradually.

In openHAB I was able - just about - to programatically query the items in a group to determine how many light circuits there were, and it determined their names by looking at the "item" name. Item name would be LG_Bedroom_Ceiling, and it split the string and therefore the circuit name would be sent out as "Ceiling".

Presets needed storing: the names of 5 presets (Sleep, Bedtime, Working, Reading, Full) and each preset stored arrays of values for each light. Obviously the array length is determined by the number of circuits in that zone.

Incidentally one main reason to move away form openHAB for rules processing is that I find OH slow and unreliable. Occasionally there seem to be timing issue in responding to requests, mainly because of the rule processing engine. This is running on a virtual server on a proper Dell server with fast SSDs in RAID 10, and with 128GB RAM ... so yeh. I tried.

My question is this. How should I approach this, generally. How do I define these items in Node-RED, so that I can e.g. dynamically query the number of curcuits in a zone? That should be easy. But what about this: how to dynamically assign values for stored light values within presets, given the number of circuits in a given zone can vary? Right now in openHAB I have a ridiculous amount of static variables like so:

OHscene-assign
OHscene-assign.png2297Ă—380 104 KB

I literally wouldn't know where to start in defining these dynamically. I'm not a great software designer, I just play and hack and make things work the best I can - mostly in an awkward way.

Should I be highly programmatic, which essentially takes away from the visual nature of Node-RED and hides the detail in functions?

Should I keep it visual and more static, but there will be a lot of repetition, and every time I need to add a circuit I need to remember to update the circuit number variable, as well as updating all the presets for that zone, etc.?

And are there any obvious things I should start with, that Node-RED is just screaming out to be used for in this context?

Sorry for such a high level question. It's definitely not "stack exchange" ready and I'm taking the name of this forum to heart ("discourse") :slight_smile: But I'd very much appreciate an exchange of ideas!

I'm most keen on setting this up so it's easy to change configuration later, and maybe even be able to document it for a less technical person to try and change later if necessary.

Finally - massive curveball. I may decide it's best to store as much of the definition of the lighting system (e.g. presets, circuit definitions, etc.) in an Arduino, and be able to somehow configure this from time to time using Node-RED as a configuration proxy. This strikes me as more reliable, and would mean that if a server went down, the lighting would still work. Better in the event that a server could fail at some stage. Should I push for that, or abandon the idea and make it depend on a server with an OS?

Thanks

Mat

(*) Incidentally I don't have all of these circuits yet, but I have the DMX dimmers and a shell of a house that is being built! Right now I'm using LightwaveRF actuators in my "test home" which is a smaller flat....

2

Hey Mat,

That was quite a bit to digest. I think that maybe if I share an architecture that I am using on a project, it may help you.

I am controlling a max of 26 devices. They exist as two different types of devices. One group of 20 and another group of 6.

The user controls all of these devices dynamically and I maintain state via 3 JSON files. The first tells tells me the metadata of the other two JSON files. Each of the other two JSON files can then grow and shrink based upon the actual quantity of devices that they are managing.

Doing it this way does require function code to parse through the JSON files but I believe it is minimized as the functions are mainly loops.

The nice thing about the JSON files is that the devices can be modified individually as well as looped through all of the devices easily to perform global commands like "all lights off" for example.

I hope this brief explanation helps.

Joe

3

If I've read this correctly, my own approach is to define the rooms and devices in their own JSON variables.

I maintain a number of global variables containing things like "switchLocations" (e.g. remote control wall switches - maps a switch to a location), "lightLevels" (default low/high light levels in different locations that let me turn on/off lights according to ambient light levels). I currently use a location naming convention that lets me recover a single floor, single building or single room via MQTT topics. Of course, anything I can do with topic wildcard subscriptions I can also do in JavaScript. So that lets me count things easily if I need to.

So I guess you would say that I maintain a model of locations and devices in Node-RED. The location and device ID's map through to MQTT for easier processing too.

I also push the model data to MQTT as well so that it can be consumed by other flows and devices though with NR's new ability to persist global/flow variables, this becomes slightly less important as I'm nearly always using that data in NR and not in the devices. I don't really have enough devices to warrant the level of abstraction you are doing though doubtless I should do.

4

My personal approach has been to genericize and abstract hardware from command input as much as possible so all similar devices can be issued the same commands even if there is a difference in configuration, model, or arrangement. This also requires some processing from command sources. This has one potential drawback (from your perspective) of requiring things like preset scenes to be stored within node-red rather than elsewhere so they can be invoked by any command input. End result is the other half and I can turn on any of our lighting in our home using echo, google assistant, node-red dashboard, wireless switches, desktop macros, email message, and so on and by the time the command data hits the lights they don't care and can't tell which source the command came from. I use flow/global variables as my source of truth for things like logging and dashboard metrics but I could see some very real advantages to moving this to some sort of database external to the compute power (eventual high availability and so on.)

That all said, my approach skews heavily toward prioritizing rapid and easy reconfiguration because we're still not 100% final on how we want things to behave.

5

Agree. I also disaggregate things between commands issued and the devices the commands control. So devices can easily be replaced or upgraded and only a single variable needs changing.

6

Not long ago there was the start of a discussion/development effort on a "Node Red Home Automation Hub," https://groups.google.com/forum/?nomobile=true#!msg/node-red/kZw64I9W_yo/901yUaKYBAAJ;context-place=forum/node-red. Unfortunately, that project faded away, and some of the discussion, which moved off of Google Groups, seems to have been lost. It might be useful to continue this conversation, perhaps in a sub-category of "Share Your Projects," where we can look at both architectural ideas that are not necessarily NR specific and implementation issues that are tied to NR and hardware.

1 Like