if you google “facebook and MQTT” it should keep you busy!
1 Like
Very interesting - ta 
It's worth saying that is quite an old reference, and they have rewritten their mobile app a few times since. So I'd be wary of rushing to cite them unless a more recent confirmation is found.
As Nick noted, the major messaging apps continually update their systems but given Facebook did use mqtt, at least as recently as 5 years ago that I can find cited, I think it's unlikely that they have changed their entire infrastructure to anything different.
From my knowledge, there are really only two practical choices for any large scale messaging apps and that would be XMPP or MQTT. I think in more recent years the shift has been to mqtt from a systems architecture point of view. Certainly, if I was making the decision on which way to go now in a new development it would be mqtt.
Robert Abramczyk over at netguru has an interesting read on implementing a messaging app use case using mqtt...
Brian
Not wanting to derail this topic (although I think this topic has pretty much run its course on the original question), but putting my messaging protocol nerd hat on, AMQP is widely used as well. For example, I believe, most of the Xbox online gaming systems us AMQP at a shockingly large scale.
1 Like
Hello gentlemen, I'm in a similar situation with a product we are attempting to introduce, please let me explain.
Remote diagnostics module (RDM) running node red attached via USB to underlying control system.
On boot the underlying control system sends a serial number to the RDM. The RDM creates a global variable in node red. From that point on all MQTT topics will subscribe and publish using that serial number as the primary high level topic. The RDM will then dynamically add lower level topics to the primary serial number topic based on data sent over USB from the control system.
It is easy to change the publish topic dynamically to the MQTT out node, however there is no method available to change the MQTT in node subscription dynamically. It would be nice if you could allow some code inside the subscription topic window of the "MQTT in" node like this:
Topic {msg.topic=global.get("system_serial") + "/remote_command"};
I hope the above scenario makes some sense. At present I'm stuck and searching for a solution. Any help here would be greatly appreciated.
You were to add a topic to the top level (like "RDM") so you publish would be "RMD/serial_number/other_topics", then you could subscribe to "RMD/#" and get everything tat was being published. You could parse the topic to get the serial number and do what ever processing you needed.
Thank you for the reply @ zenofmud, For me doing that would be a problem as there can be thousands of devices deployed. Amazon web services charges on a per transaction basis. Communications to each device will be done using a node red application running on a service PC. I was hoping for something a little more elegant that would reduce transaction traffic through AWS to a minimum.
I tried using the node dynamicMQTT but that node offers no method for SSL/TLS security. I really need a way to dynamically set the MQTT in node. I'm a bit new at this so please excuse ignorance, and thanks again for the reply.
Hi @stnwll. If you read back to my earlier reply, there is a design proposal on the table to allow this functionality in the core nodes. The next stage will be for that design to be implemented. That isn't a quick task and is on the backlog along with lots of other development items.
1 Like
My apology @knolleary, I scanned quickly through the posts. Am looking forward to this update, unfortunately I'm running out of time for this release and may have to way my options at taking another route. I appreciate your time and efforts, thank you.
Indeed, AMQP is a big contender in the messaging arena which I forgot about. My understanding, designed for rich messaging as opposed to telemetry which mqtt was built for, but just happens to be really, really good for messaging as well. True too, this topic has taken some twists and turns away from the original topic, which has been addressed with the road-map to implement the functionality into node-red, but, for me, that's what makes a discourse interesting when interacting with bright, creative, insightful individuals that can do lateral thinking 
On boot the underlying control system sends a serial number to the RDM. The RDM creates a global variable in node red. From that point on all MQTT topics will subscribe and publish using that serial number as the primary high level topic.
As it is set on boot you could possibly use the $(my_env_var) syntax for using an environment variable to set the topic. Not it is only red at start/deploy time , and has to be the complete variable - you can't concatenate text eg you can't do RMD/$(my_ser_number)
Good point @dceejay, I wonder if it would be possible to write py script for serial-port to obtain serial number and insert that serial number into the environment variable before node red starts. hmmm 
Depends where the serial number comes from and the OS. If Linux a simple bash script on boot may be enough.
Just an off-the-wall idea - run a separate external prog in python using paho lib that handles the dynamic subscription topic based on serial number and re-publishes them as as a fixed topic name that node-red can deal with
Not off-the-wall at all though I was going to suggest using a Node.JS app. Each to their own experience of course.
There is a tendency to forget that, because we have such an excellent tool at hand in Node-RED, that it has to be used for everything.
Especially in the professional/commercial world, it is helpful to give some thought to what alternative approaches there may be and whether they may be better. Node.JS apps can be pretty light-weight and focused and so retain easy maintenance.
1 Like
@cymplecy excellent suggestion. Pyserial has access to the connected device using pyserial but I'm a little confused as to how the environment variable of node red would set the topic of the MQTT input node. It is my understanding that the change node could access the environment variable but there is no input to the MQTT "in" node, what method would be used to accomplish injecting the new topic from the environment variable?
Sorry didn't mean to hijack thread. Just grasping at straws now.
As an alternative (until the new mqtt node will be available) see the dsm solution above https://github.com/cflurin/node-red-contrib-dsm/wiki/Mqtt-on-demand
The configuration can be modified/improved to fit your need.
rather than enter a value in a field (eg [ MY-TOPIC ] in the MQtt topic field you can use the (env_var) syntax e.g. [ (my_env_for_topic) ] and then outside of Node-RED set your operating system environment variable - eg for linux export my_env_for_topic=MY-TOPIC. When Node-RED starts (or redploys) it will then read that environment variable and use that for the topic.
and on a Pi - you can get the serial number eg by
cat /proc/cpuinfo | grep Serial | cut -d ' ' -f 2
so you could do
export MY-SER="RDM/( cat /proc/cpuinfo | grep Serial | cut -d ' ' -f 2 )"
to set the environment variable
then in Node-RED set the topic to (MY-SER)
This should work for just about any variable field that accepts text.
2 Likes
Wow @dceejay, very helpful indeed. Thank you all for helping me with this. I now have several options available for the short term.
