Change mqtt-input node topic name

#1

Hi,
Is there a way to modify the topic of mqtt input nodes from one script ? and then do a full deployment .
I did one script to modify topics in the flow file (from command line), but it’s necessary to stop node-red and then restart it for getting it working. (too long process). I am wondering if we can do it from the node-red editor and then do the full deployment

#2

You cannot change the topic of MQTT input nodes apart from editing them. Just use multiple nodes if possible. Show us what you are trying to do, there may be another way.

#3

I have several identical controllers. All of them reporting to the same broker. The topics are the same for every device, the only difference is the name of the topic first level path which is equal to the device serialNumber, so the topic for variable1 in the device 1 will be serialNumber1/variable1, in the device 2 will be serialNumber2/variable1. I have around 20 different variables in each device reported to the broker.
I did a node-red dashboard showing all the variables of one device in order to monitor the values and get the information on line of specific variables. Now, if I want to switch to another device to monitor it (using same dashboard) I need to resubscribe each node with the new topic path. (no way to edit them every time I want to switch, I would like to do it with a command)
I do it now going to a system console stoping node-red, running a script for changing the topics with the new one (i.e. changing serialNumber1 by serialNumber 2 in all nodes) in the flow file (.../.node-red/myFlow.json) and then restarting the node-red and reloading the flow.

Hope you can get a more clear picture of my project.
Thx for your replay
J.Z

#4

Use the MQTT wildcard + in the first level?

#5

As per my understanding using wild card subscribe you to all topics under the wild card and I only want to be subscribed to the topics of one device. I have al least 50 devices.

#6

The existing MQTT nodes do not provide a way to dynamically change their subscriptions.

This has recently been discussed here: Trigger a mqtt subscription

#7

I meant instead of serialNumber2/variable1 use +/variable1.
The whole topic will still get passed into the flow.

#8

thank you Nick for your comment.
I already tested the node-red-contrib-mqtt-dynamictopic but as per my tests, it does not unsubscribe the previous topic before subscribe the new one ( and I did not find a way to do it) so, after some topics changes I keep subscribed to a lot of topics creating an unnecessary traffic.
I was looking for another way to resolve my specific case.

kind regards

#9

I understand that if I do +/variable1 I will received the variable1 of the 50 devices, creating an unnecessary traffic in my lan. I only need the variable1 of one device.
am i wrong ?
thx

#10

That's true, but surely not a problem for the network. How often do you need to change the device?

#11

If it’s based off the serial number then maybe use the $(env_var) syntax instead for the topic. And set the appropriate environment variable outside of Node-RED.

#12

Sure -- the flow file is just a JSON string. And there is the /admin/flows api that lets you deploy a new version.

Sounds like you need an inject node to pass in the new topic -- wired to an http request node to GET /admin/flows as a js object -- wired to a json node to convert the flow to a string -- wired to a change node that uses a regex to find and change all the topics to the new value -- wired to a json node to rebuild the flow js object structure -- wired to another http request to POST the revised flow back to itself...

Easy-peasy, eh?

Alternately, you could skip the string/regex manipulations and use either a js function or JSONata expression to search/replace all the topic values... which might be safer since you don't run the risk of replacing matching strings inside non-topic parts of your flow.

#13

it is just to monitor, so it depends which one I need to monitor, let say every hour or may be more and some times five minutes ...

#14

did not know we can use env_var in the topic. I will check. :smile:
thx

#15

sounds good, but I we do the manipulation in the flow.json file we need to reboot the node-red right?

#16

Well, the POST is what happens during a Deploy -- so depending upon the mode (full, flow, or node) you can cause all the flows to restart (full) or just the modified nodes. (this should be faster than restarting node-red)

Check out the admin api page for more details...

#17

great. I will check your idea. also the one from dceejay using env_var.
thank you guys for your support. I have homework for the weekend :grinning:

#18

In that case the network load is less than trivial. If it was changing 1000 times a second then maybe you should worry about it.

#19

Hi,
I tried the two ideas :

  1. from dceejay about using the environment variables. We can do use env_var into the nodes with the limitation that the content of variables will be the one at the node-red start time. If we modify the environment variables (using the same console where we started the node red) those changes are not visible into the node red during run time. So, for my application it is not the solution. By the way when we declare the env_var in the console we must do: export variableName=myvalue.

  2. from shrickus, about using API references to access, modify and deploy flows. It works perfectly. The formula that I implemented is:
    Do a http request node with a GET/flows API method to obtain all the node-red flows in an object, then with a javascript node find the current path in the topics (in all the nodes) and replace them with the new path you want. Now wire another http request node but this time using the API POST/flows method to do the deployment (setting the deployment type that you want -full-flow-node) and that’s it.
    This page shows all available methods in Admin http API.
    Now we can change all the mqtt topics with a button without stopping/restarting the node-red, just doing a new deployment.

thank you for your help.
Cheers

PD: the application is running in a docker container on a Raspberry 3 platform.