Custom Node: collect & store runTime (mqtt) data in configuration note and access it from a node

Hello,
I*m new in developing my own nodes. My Idea is to develop a custom node to handle mqtt comunication according to the homie convention and use the auto discovery features for my sensor data. The initioal version should be able to do this

  • define a configuration node for a mqtt client subscribung to homie/#
  • collect all auto discovery data for devices, nodes and properties and store them in the configuration node
  • define a node where the user can select the device / node and property out of a drop down lists

So far the configuration node receives data and filters the relevant information (only devices to start with).
The problem is that it is not clear for me how to store the data (one object for each device) and access it later from the node for a drop down list.

By now I can store the information in the global context. The goal is to sore this inside the configuration note context.

I tried node.devices[] and this.devices[] within the this.messageArrived function but this never showed up in the node using the configuration node.

here is the configuration node:

var ip = require('internal-ip');
var mqtt = require('mqtt');

module.exports = function (RED) {
  function homieDeviceConfig (config) {
    RED.nodes.createNode(this, config);
    var node = this;

    this.state = 'disconnected';
    this.setState = function (state) {
      node.state = state;
      node.emit('state', state);
    };

    this.mqttHost = config['mqtt-host'];
    this.mqttPort = config['mqtt-port'];
    this.deviceId = config['device-id'];
    this.name = config.name;
    this.devices = {};
    this.devices[config.name] = {};
    this.baseTopic = 'homie/' + this.deviceId;

    this.client = mqtt.connect({ host: this.mqttHost, port: this.mqttPort, clientId: this.deviceId, will: {
      topic: this.baseTopic + '/$state', payload: 'away', qos: 2, retain: true
    }});
    this.sendProperty = function (nodeId, name, value) {
      node.client.publish(node.baseTopic + '/' + deviceId + '/' + name, value, { qos: 2, retain: true });
    };

    this.client.on('connect', function () {
      node.setState('connected');
      node.client.subscribe('homie/#', { qos: 2 });
    });

    this.messageArrived = function (topic, message) {
      var splitted = topic.split('/');
      var deviceName = splitted[1];
      var nodeId = splitted[2];
      var property = splitted[3];

      var context = node.context().global;
      var devices=context.get("homieDevices");
      if (devices===undefined) devices={};
      if (devices[node.name]===undefined) devices[node.name]={};
      if (devices[node.name][deviceName]===undefined) {
        console.log("Homie: Device found :"+ deviceName);
        devices[node.name][deviceName]={"name":deviceName};
      }
 
      switch (nodeId) {
        case "$homie": {
          devices[node.name][deviceName]={"$homie":message.toString()};
          break;
        }
        case "$name": {
          devices[node.name][deviceName]={"$name":message.toString()};
          break;
        }
        default:
          node.emit('message', nodeId, property, message);
      }
      context.set("homieDevices",devices);
    }

    this.client.on('message', this.messageArrived);

 }

  RED.nodes.registerType('homie-convention-broker-config', homieDeviceConfig);
};

Perhaps somebody can help. Thank you in advance.

So without studying your code fully, am I right in saying your node will subscribe to a specific MQTT topic (homie/#) and provide a kinda store to permit uses to access values whenever they want?

If so, users could achieve this with standard nodes but I do admire the efforts to simplify it for users.

However, you should be more generic. For example, hard coding homie/# will seriously limit usability for a lot of people. This should be a setting property on your nodes configuration.

Forgive me if I am reading this wrong.

Let me know and I will try to help as best I can

Steve
Thank you for your answer. The homie standard is a quite elaborate, open source and well developed convention for iot communication over mqtt. OpenHAB since 2.4 is offering this as their preferred option for mqtt data transfer and using it for convenient auto discovery of devices, sensors and actors. I developed a controller and receiver plugin for ESPEasy Firmware which is currently under beta testing.
It is quite nice to see your own device and all their parameters showing up as soon as your DIY esp based device is configured. (more on this below).
The homie/ topic you can compare with the two bytes PK in a zip files an indicator that everything under this topic tree is following the homie convention. I will make it configurable but this is not my real problem.
Here is what an ESP8266 sends out for a common BME280 temperature, pressure and humidity sensor:
image
(screenshot of mqtt explorer)
You can use homie devices with standard Node-RED: with only one 50 line function node


This will beside collecting the data, adding timestamps and other things filtering out all the "housekeeping" topics starting with the a $ letter.

Just recognized that I`m only allowed to post two images ...

My plan is to develop some nodes to do all of this less "exposed" and user friendly. In the prototype the user has to type in all their intensifiers for the switch nodes:
image
This is not only inconvenient it also is always a point where an error can be made. You have to remember all the names you defined during configuration of your device (or in this case you can look them up in the context tab. And in the end everything is "hard coded" "hidden" in some nodes.

  1. A (preferable) configuration node takes care of the broker specification and will collect all homie specification data.

  2. A node to represent a single value of an device. In this configuration you should be able to select by drop down lists the device / node and value

[...]

image

And here is where I'm stuck. The configuration page should fill the device list (later I will add a list for nodes and values) from the information collected by the configuration node. As there will be many of these nodes (one for each sensor value) I think the configuration node is the best place to provide these data because it is tied to one mqtt broker. In my case there are more than 500 topics in my current home automation system!

It is working now if I use the front end visible context store:
image
This is what the code above currently do. It would be easy to read this in the nodes to fill the drop down lists but I was asking myself if this is the right way to do it. I would be irritated if data is showing up in the front end as soon as i drag in a node.

I`m asking for help how to store data within the configuration node and read it from the node configuration page to fill the drop down lists. I think the problem is my lack of Java Script know how.

There will be other questions later but here I'm stuck now after some hours trial and error (and analyzing code from others)

Thank you for your help and excuse the German words in the screen shots. I´m developing this on my life system.

[..]

And here it is how it appears on openHAB 2.4:


no typing or coding requited (Only labeling the value because ESPEasy do not provide two names: identifier and a "friendly name")!
There are currently some issues with the mqtt implementation in openHAB 2.4 and 2.5dev and as I'm using Node-RED as my main "engine" anyway I thought it would be nice to implement the auto-discovery directly here.

Again thank you for your help and sorry for the spelling mistakes and typos.

Chris

(Now I flooded this thread, sorry)

I think I have to phrase the questions more generally:

I'm looking for a two way communication between a custom node and its config node.

  1. by reading values directly form the config node
  2. or calling a function defined inside the config node returning the data asked for. (preferred)

In my case the config node handles a mqtt client. When relevant data arrives it updates internal lists. The individual nodes should be able to get this list when necessary.

My Questions:

A) the best way to define the function in the config node?
B) how to call this function from the node using this config node? In my case the html part to present this data in the configuration dialog as a drop down list to the user. But also in the js part to in example filter messages.

The homie convention node example I'm working on is perhaps a little bit distracting.

Hi Chris,

Don't know if it helps, but here some stuff I wrote:

  • Server side: Here I have defined a getProfileTokenByName function in my config node, which is called by the other nodes. like here.
  • Client side: Here I get some attributes from the config node, after I found the config node via RED.nodes.node.
  • Client/Server: Here I publish on the server side a getProfiles function from my config node, which can then be called from the client side like here via getJson.

P.S. Hopefully Steve (I mean this one: @Steve-Mcl) doesn't read this, because then he is going to ask again why I don't have time to finish those Onvif node suite :rofl:

Good luck with it!
Bart

1 Like

Hi Bart,
Thank you for your reply,
I already figured out that you achieved some of the functionality I'm struggling with in your Onvif nodes but I was overwhelmed by the amount of code. Thank you for pointing me in the right direction. Will do some tests ASAP.

Chris

This post may also give you some more of the basics for how to get the edit dialog to communicate with the runtime: https://stackoverflow.com/questions/41567175/send-data-on-configuration/41567832#41567832

You do need to think carefully about the developer workflow in this scenario. Nothing is active in the runtime until you hit the deploy button. If you have one node that depends on runtime information from another node, then the user will need to have deployed that first node so it is running and gathering the required information before they can configure the second.

I seen nuffin, heard nuffin, and ain't not no way gonna ask nuffin about onvif :laughing:

1 Like

Thank you,
Precisely what I was looking for.
As all data the node needs to be configured by auto-discovery comes in via mqtt, either live or as retained messages I hope this will not be an issue.
Let’s see how far I get with this project. I really appreciate the work went into the homie convention that I spend quite a lot of time developing the homie plugins for ESPEasy. Due to ongoing issues in the mqtt/homie implementation in openHAB being unstable I thought to promote the homie convention with a hopefully stable Node-RED Node can be a nice challenge (perhaps to celebrate Node-RED 1.0;)

Success! After some try and error finally:
image
No I "only" have to do the "other things" :crazy_face:
Thank you for your help so far.
Chris

2 Likes