Google Home integration

General
1

I searched discourse in an attempt to find an answer but failed.

I have a KNX setup and want to link my devices to Google Home. I have no intention to buy realKNX for €1200 (which funny enough runs node-red) to accomplish this.

My KNX system is already bridged with my node-red installation. Is there some way I can add these to Google Home easily? I fail to find how to integrate these two.

Thanks

2

@DeanC has done a lot of work with Google Home, have a look at his node-red-contrib-google-action in the library.

3

I have a new version of node-red-contrib-google-action that I have been working on for the past few weeks. It should be ready by the end of next week

2 Likes
4

Can I use that library to provide devices to Google Home?
I can't find any references to this if I go I to Google Home app and choose to add a new service.

I'm not looking for voice control primarily but feeding the Home app on Android with my devices. It's so confusing when they have a product called Home and then an app in Android OS also called Home.

5

I have done this two different ways. It's non-trivial and takes a lot of setup. At a minimum, you need the following:

  1. A publically accessible IP and port 443 open
  2. A DDNS address pointing at said IP, if its not static.
  3. A Google Actions developer account that you setup an application in.
  4. A real SSL certificate (i.e., via LetsEncrypt most likely)
  5. Ideally, you are going to want to proxy node-red through nginx to provide the wrapping SSL, and to add actual security around your node-red instance endpoints so you aren't exposing the entire thing to the internet now.
  6. An OAuth server to provide authorization. Might be able to do this with Auth0, etc.
  7. An implementation of the Google Home API fullfillment.

Now, 1-5 are all just painful setup however you do this.

For 6 You COULD build an OAuth server in node-red itself (https://gist.github.com/i8beef/3f91d3a1ec543987fa2e590355246c69). Or you could try and integrate with some other OAuth provider you setup provided in the cloud like Auth0, etc. Or you use my rolled solution.

For 7, you could implement it in node-red itself (https://gist.github.com/i8beef/8f6252d80af34183f5ca554d1e0af8ab) but you will need to heavily modify that, and make sure you have some mechanism for capturing and retrieving state on demand of your exposed devices, etc. Again non-trivial. Or you use my rolled solution.

Next post will have the rest as I can only post two links at a time here for some odd reason...

6

So my rolled solution which I use NOW is here (https://hub.docker.com/r/i8beef/homeautio.mqtt.googlehome/). It is a Docker image that provides a configurable OAuth server, as well as a configurable MQTT to Google Home API implementation. It is STILL complicated to setup, and it assumes that ALL state and commands you need to expose use MQTT as a backplane. But I DO think its easier to setup than building it all in node-red like I did the first time. Note, that again, you'd need an nginx proxy sitting in front of this to provide the SSL wrapping at a minimum.

Note, its still got some rough edges, and it "works for me" so if you use it and want help, I will provide as I'm able, and I'm just letting you know about it in case you really want to explore. There's some big ugly JSON files you have to maintain to get it going for the mappings to MQTT topics and device metadata... I DID make a little web interface on it that MIGHT work to do SOME of that, but its... well its not really tested real well and isn't fully finished.

7

Wow, thanks for the thorough walkthrough. As a matter of fact I already have 1-5 setup using docker, nginx and let's encrypt.

What I still don't understand is how the devices will end up in Home since, it seems to me, that they only do so by linking a third party service from Home?

8

Correct. So if you want to expose your stuff to Home, you have to BECOME the third party service, i.e., write your own integration. Google doesn't do "private" integrations, so you actually have to basically write one, and put it in "test mode" which will make it show up as one of those services in the home app for your linked account. Actually FULFILLING the integration is the hard part (i.e., writing the HTTP endpoint that Google will call, and figuring out how to get your system to return all the configuration, metadata, state, etc. for your devices), and was 6 and 7 in my steps above.

I gave you the two solutions I've used successfully to do that.

My MQTT to Google Home bridge gives you an OAuth server that you can configure for Google for 6, and a big ugly configuration file for device configuration (names, metadata, etc.) and mapping to Google Home device TYPES, TRAITS, and COMMANDS. You basically set it up so that if you trigger the OnOff trait command with a value of "on" for the device called "family room light" (I.e., "turn on the family room light"), that it will take that payload of "on" and send it to a specified MQTT topic. Similarly, it listens to "state" topics and maintains an internal state cache of the system so if Google asks for something (i.e. "is the kitchen light off?") it can answer that question immediately.

The tricky part is that is a lot of configuration (example: https://github.com/i8beef/HomeAutio.Mqtt.GoogleHome/blob/master/src/HomeAutio.Mqtt.GoogleHome/config/googleDevices.sample.json), and all the details aren't documented in my project yet, so you kind of have to piece it together with the Google docs too.

I mean it works, and is pretty solid, and I even support some things like a poor man's "value map" (i.e., Google home sends "on" but you want that to translate to "100" because its a dimmer, etc.)...

But really my whole point is this:

  1. That amount of configuration is going to be necessary no matter what for any implementation.
  2. There is no standardized state management in node-red, and you NEED a state cache for this of all the devices in your system (MQTT just sort of works nicely for this with retained topics and such too), so any module that does this is going to have to figure out how to handle state storage (which my bridge thing does).
  3. OAuth is a pain even when you understand it, but that is one thing a module or my bridge can actually handle for you decently with a little configuration, and maybe some SSL certificate generation for signing if done right.

The existing Google Home module only allows you to build the intent fulfillment for voice intents, not the smart home intents... it could probably be done, but configuration complexity and allowing for arbitrary state management is gonna be a signficant hurdle. I ripped it all out of node-red and wrote this thing because it was complex enough to be its own standalone thing, and this actually made it easier.

1 Like
9

Hi, thanks for the write-up. However, I don't think the above is quite correct.

You can manage state in Node-RED in a number of ways. But with the latest releases, the most integrated is to use a file store for your state variables (that than the default memory store). Alternative ways include the use of MQTT and dumping state to a db or file.

You also have configuration nodes if you are going down a custom node route.

10

i8beef said there is no standardized state management. You are sort of verifying that by listing various options of doing state management. It's not impossible but node-red is not very opinionated.

11

Gotcha, thanks! Wonder if we could write a service for anyone to hook into but there will always be the additional cost of running it in the cloud.
I do see both Home Assistant and OpenHAB as available options in Home.

12

Not sure I really understand your response. There is a standard way but it was only implemented recently. There are plenty of alternatives that were used prior to that.

The point is that you have options to cache state.

Node-RED is certainly un-opinionated in some areas, not in others of course. That's because it is an application building tool.

13

Sorry let me clarify. Yes, you can store stuff in flow, and global variables. Yes there are dozens of storage backings you can use to store that state. But the "schema" of the data, the names you use, etc. are all completely up to you. Any module that attempted to provide this would have to handle that with either standard schema for the objects stored (more likely) or complex mappings.

You could do something like a "store device state" node for that module that you send in your values to with a specified structure, etc... but you need to put it in somehow that the node answering Google's QUERY intents, etc. can read, understand, map, etc. that state. There is no standard store for "device state" for such a node to leverage, you have to build a mechanism for users to dump state into something that is tied to your implementation. And thats just state, that's not even the other side of the equation, mapping COMMANDS to flows to execute those commands, though arguably thats probably a BIT easier...

My point was more about data "schema" not being standardized around device storage, than mechanism of storage.

And I think trying to CONFIGURE all that in a single node configuration is gonna be pretty difficult to pull of... I think you'd have to build a complex set of nodes, think node-red-dashboard levels of configuration. Probably doable, and I think you could implement some flows that would be harder to do with my solution, like if your MQTT messages were complex JSON objects instead of simple values, etc., and would make value mapping more robust.

And you'd still have to expose an OAuth server.

14

Yes, this is the other option, someone else handling the actual integration in the cloud for you, which is what HomeAssistant and OpenHAB did. For node-red you'd then have a set of nodes that would communicate with that implementation with some entirely different API in a simplified way. Of course someone's gotta build and pay for all that... :stuck_out_tongue:

It has some upsides. Definitely easier for most normal people. Don't have to manage a Google Actions account and application (Google's documentation leaves something to be desired...). Etc.

Some downsides too. You'll be dependent on another cloud provider for the support, and you'll get latency accordingly. Some things like "report state" and "SYNC" intents are gonna be a little trickier.

But it would definitely be better for a lot of people who aren't developers and have no interest in dealing with the complexity of a Google Home integration. A lot of the complexity around firewalls, OAuth, etc. would be eliminated this way.

15

Ah, a different point altogether, I see.

Well yes, there is no standard schema for IoT data full stop. Not a Node-RED issue as Node-RED facilitates rather than dictates. That's what makes it so useful.

If you look around, there are many suggestions for standard MQTT schema's for home automation as an example. But in the end, what you chose depends on what you want to achieve. In my own case, I actually use multiple schema's for device data. I use a modified version of the "homie" schema for a lot of stuff and I'm gradually moving more things into that format. But it is not very good if you want to process several pieces of information together across multiple devices. It is also a bit complex for some things. So I have 2 or 3 different, parallel schema's for other purposes.

But I'm sidetracking things.

Now, to work on a more standardised schema might be an interesting project actually - but not an easy one.

But back to your core point. Coming up with something standard for your Google Home node. I think that there would be ways to break down the problem and I don't think that you would need to solve all of the problem inside your node. The more device/service specific parts would be best kept separate anyway since, as you point out, you cannot predict what people want to do.

What part of the configuration could be pulled out into something more common?

The OAuth server would also be best as a separate node as this would be very useful for all sorts of purposes. But yes, you would have to expose it if that is what Google requires. You could always set up a second instance of Node-RED just for that purpose though having a dedicated NodeJS microservice might well be a better option anyway of course. There are plenty of other tools for doing OAuth authentication as well, perhaps something like Azure Single Sign-On for example.

16

Yeah, and that's exactly what I did for my standalone implementation for a lot of the same reasons. It was just pretty complex to try and force into node-red (I DID do that, see my first implementation, but it was just more easily managed as a real standalone service).

You raise a point though, my MQTT bridge FORCES you to MY MQTT standard... i.e., you have to have separate command and status topics for each device, must pass simple value payloads not complex message protocols (would be too hard to make that generic / configurable), really SHOULD use retain on all status topics to make sure things don't get out of sync on restarts, etc. You COULD still work with this with a layer of MQTT translation / republish node processes, if it really came to it though and still use mine.

Once you start looking at the details of the implementation, it really starts getting complicated enough that... well that I decided to write a real standalone because it just worked better. I think you probably COULD embed it in node-red, but you might end up doing some odd or heavy handed things to expose things like the OAuth, etc.

On thing to note if you look at my implementation, the device config file is really a 1 to 1 copy of Google's device schema for their API. Each of those is the metadata, per device you need to supply (a bit of a lie, there's some optional pieces there...) and thus configure for each distinct device, and then the traits element is representative of the MAPPING you'll also need to configure to get specific commands, for specific devices as node outputs or something like that, as well as supplying state.

If you wanted to implement the device config part, you could probably do something like this:

  1. A "set state" node that would take in one (or multiple somehow) states and stores it in the modules internal state representation. Configuration for the node would have those metadata pieces you need to define for representation as a "Google Device".
  2. Obviously, and OAuth server running somehow.
  3. The behavior of the QUERY intent is pretty static, and could just be exposed at some agreed upon URL. It's job would be to return that internal state to Google's API correctly.
  4. A COMMAND intent also could be static in behavior I guess. It's only real job is to take a command(s) (there can be multiple on a single request) and route or drop each one to a specific "command received" node.
  5. A "command received" node that would receive a single command for a single device intent. The output would let someone process that as need to any command process they need.

As I think about it, it's probably doable, and lets someone wire everything up themselves as they desire. Would be pretty flexible. I'm not sure how you would write any of that, but if someone wants to undertake it and wants to ask questions I'd be happy to help translate some of the requirements.

Note: You'd still have to setup all the Google Actions account stuff and deal with the testing mode management, etc. and supply some way to setup the OAuth server implementation.

17

How's it going @DeanC, taking a little longer than expected?
No worries though, look forward to your update.

Paul

18

To be honest, I use several layers of translation and republishing even within my own HA system. MQTT topic structures, as with any hierarchical structure, can never be a one-size-fits-all solution with a single hierarchy. One of the really useful features of Node-RED is how easy it is to republish things on a different hierarchy.

I start with a device centric topic structure. I republish to a location centric structure with all details together. I also republish to sensor/location structures (e.g. TEMPERATURE/Living-Room). Each is useful for different purposes.

This illustrates why it is very unlikely anyone will ever come up with a single, agreed schema. There are just too many ways you might want to consume the data.

19

Yeah, its not uncommon, I'm just musing that would be a way for someone to use my solution without being so tied to my specific MQTT rules.

And I'm not advocating for a standard around device state storage, just pointing out that an implementation has a bunch of metadata and current state information it must have access to to fulfill the Google Home requests. I imagine the easiest way to do that would be to store it all internally to the module itself though, not abstracted into some generic device storage standard. It'd be way too specific to Google Home.

I mentioned node-red-dashboard because I imagine it's approach of a suite of nodes and a separate interface exposed at a known URL under the site would be close in model. I mean you could actually do a node per supported Google Device type to handle some of the configuration of devices / the device storage stuff, and just takes inputs for specific state payloads for setting current state. Or you could read in the entire metadata config from a JSON file sort of like I do. Or a number of other options. I'm just trying to inspire someone else to take a stab at it since I (a) have a working solution I like and (b) don't really like node.js so am not interested in writing it for you guys, but would be happy to give ideas and thoughts on what would work if someone else picks it up.

20

Looks like a month passes, I just want to chime in a little bit about my setup without exposing the local Node-RED or MQTT or any compliance with Google daddy. It's a bit hacky, non standard I admit.

The main point was how to receive actions from Google side. For this I use Firebase real-time database as the shared memory for the local Node-RED behind NAT and the Google.

So the main setup is:

Google Home --> DialogFlow --> Firebase Cloud Function --> Update Firebase database.
Local Node RED --> Firebase database listen for changes --> device actuation

Was previously use websocket to hookup to an external fulfilment server, but found out the websocket node of Node-RED is not so reliable.

With the help of node-red-contrib-firebase-realtime-database of course.