# Best way to publish a list of MQTT topics in one shot?

**URL:** <https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101>\
**Category:** General\
**Created:** [7 August 2019 15:34 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101 "2019-08-07T15:34:43Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![thatcadguy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/thatcadguy/32/5273_2.png) [@thatcadguy](https://discourse.nodered.org/u/thatcadguy)\
**Post date:** [7 August 2019 15:34 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/1 "2019-08-07T15:34:43Z")

</div>

I need to publish a list (object really) of topics to an MQTT broker -- is there a way to publish multiple topics in one shot, or is the best bet to run my list through a split node coupled with a publish node?

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [7 August 2019 15:45 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/2 "2019-08-07T15:45:45Z")

</div>

Best to use split. Or, if that is proving hard, just do a small function.

---

<div class="post-metadata">

**Author:** ![thatcadguy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/thatcadguy/32/5273_2.png) [@thatcadguy](https://discourse.nodered.org/u/thatcadguy)\
**Post date:** [7 August 2019 15:57 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/3 "2019-08-07T15:57:50Z")

</div>

split works, just thinking of shooting 100 topics to the broker possibly several times a second -- don't want to deal with timeout issues or have to rely on delays and things to get everything to play nice -- that's why I was hoping to shoot everything over in one message.

---

<div class="post-metadata">

**Author:** ![cymplecy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/cymplecy/32/2773_2.png) [@cymplecy](https://discourse.nodered.org/u/cymplecy)\
**Post date:** [7 August 2019 16:06 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/4 "2019-08-07T16:06:58Z")

</div>

If your issue is that you don't want one topic to be out of sync with another topic, then you could bundle all the messages into one JSON string - send that on a single topic.

Disadvantage is that receiving clients would have to decode the string to find what they want

YMMV

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [7 August 2019 16:26 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/5 "2019-08-07T16:26:24Z")

</div>

You can deal with timeouts fairly readily but if you need tight timings and high volume, that could turn out to be an issue. You will want to test things and you might consider splitting workloads across host devices to make sure that you don't hit any processor, memory and disk I/O bottlenecks.

It is hard to predict if you will have issues as so much depends on what else is happening, how busy things are and how much data you are shifting.

With MQTT as with any other interface, you should always be thinking about how data will be processed so, as Simon says, you might consider aggregating some of the data if it makes sense.

I'm afraid that you will need to do some testing in order to work out the best approach for your situation.

---

<div class="post-metadata">

**Author:** ![thatcadguy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/thatcadguy/32/5273_2.png) [@thatcadguy](https://discourse.nodered.org/u/thatcadguy)\
**Post date:** [9 August 2019 14:59 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/6 "2019-08-09T14:59:00Z")

</div>

Posting results of my case for anyone who comes searching. Running NR on a 1 GHz Cortex-A8 (ARM).

Before adding MQTT publishes, NR was about 14-24% CPU.

After adding MQTT publishes, split up by topics, NR was 28-60% CPU 😬

Decided to send everything as one JSON object together in one-shot: 20-24% CPU with occasional spikes to 30-31%.

Mosquitto (MQTT broker) never went above 1% CPU.

One-shot JSON is better in many ways and requires much less to prepare, and I don't like the CPU %'s for split-by-topic, so going with one-shot. As Jiro would say, "Ultimate simplicity leads to purity."

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [9 August 2019 15:28 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/7 "2019-08-09T15:28:10Z")

</div>

I would be interested to knowing how many items were you publishing each time and how often?  
Is NR also subscribing to those topics?

---

<div class="post-metadata">

**Author:** ![thatcadguy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/thatcadguy/32/5273_2.png) [@thatcadguy](https://discourse.nodered.org/u/thatcadguy)\
**Post date:** [9 August 2019 15:33 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/8 "2019-08-09T15:33:07Z")

</div>

Here's the one-shot, didn't bother to count how many entries are inside:

```auto
{"Test.AvgPress":[0,"psig"],"Test.AvgTemp":[0,"F"],"Test.DO":[0.875,"g/cm3"],"Test.DW":[0.999,"g/cm3"],"Test.Step":[40,""],"Test.StepTime":[5441.219,"s"],"Test.StepVolume":[0,"bbl"],"Test.Well":["Default",""],"Test.GrossLiq":[0,"bbl"],"Test.ActOilNOC":[0,"bbl"],"Test.ActWaterNOC":[0,"bbl"],"Test.StdOilNOC":[0,"bbl"],"Test.StdWaterNOC":[0,"bbl"],"Test.ActOil":[0,"bbl"],"Test.ActWater":[0,"bbl"],"Test.StdOil":[0,"bbl"],"Test.StdWater":[0,"bbl"],"Test.StdGas":[-0.03768494234268098,"MSCF"],"Test.ActCutNOC":[0,"%"],"Test.StdCutNOC":[0,"%"],"Test.ActCut":[0,"%"],"Test.StdCut":[0,"%"],"Test.GrossLiqRate":[0,"BPD"],"Test.ActOilNOCRate":[0,"BPD"],"Test.ActWaterNOCRate":[0,"BPD"],"Test.StdOilNOCRate":[0,"BPD"],"Test.StdWaterNOCRate":[0,"BPD"],"Test.ActOilRate":[0,"BPD"],"Test.ActWaterRate":[0,"BPD"],"Test.StdOilRate":[0,"BPD"],"Test.StdWaterRate":[0,"BPD"],"Alarm.HighP":[0,""],"Alarm.HighT":[0,""],"Alarm.HighDriveLevel":[0,""],"Alarm.HighSoS":[0,""],"Alarm.HighGain":[0,""],"Alarm.LowSNR":[0,""],"UIT-AF01.VolTotal.GrossLiq":[0,"bbl"],"UIT-AF01.VolTotal.ActOilNOC":[0,"bbl"],"UIT-AF01.VolTotal.ActWaterNOC":[0,"bbl"],"UIT-AF01.VolTotal.StdOilNOC":[0,"bbl"],"UIT-AF01.VolTotal.StdWaterNOC":[0,"bbl"],"FIT-AF02.VolTotal.StdGas":[-0.03768494234268098,"MSCF"],"LIT-AF03.PV":[0,"%"],"PIT-AF04.PV":[0,"psig"],"TT-AF05.PV":[0,"F"],"LCV-AF06.PV":[100,"% closed"],"LCV-AF07.PV":[100,"% closed"],"UIT-AF01.VolFlow":[0,"BPD"],"UIT-AF01.MassFlow":[0,"lb/h"],"UIT-AF01.Temp":[77.31441772460943,"F"],"UIT-AF01.Density":[0.000050000000745058064,"g/cm3"],"UIT-AF01.DriveLevel":[5.735607147216797,"%"],"UIT-AF01.SensALevel":[79.96589660644531,"%"],"UIT-AF01.SensBLevel":[79.06885528564453,"%"],"UIT-AF01.TubeFreq":[264.48681640625,"Hz"],"UIT-AF01.2PhaseSig":[7.006492321624085e-42,""],"UIT-AF01.VolTotal":[0,"bbl"],"UIT-AF01.MassTotal":[0,"lb"],"FIT-AF02.VolFlow":[-0.4635500841199167,"MACFD"],"FIT-AF02.Velocity":[-0.3352603359328793,"ft/s"],"FIT-AF02.SoS":[1138.401381925648,"ft/s"],"FIT-AF02.Gain":[60,"dB"],"FIT-AF02.SNR":[42,"dB"],"FIT-AF02.VolTotal":[-0.006836107869609553,"MACF"],"UIT-AF01.InstWaterCut":[-675.1159108402285,"%"]}

```

So before, everything you see in there was split so that it was one item per publish. NR is not subscribing to anything (yet).

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [9 August 2019 15:33 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/9 "2019-08-09T15:33:53Z")

</div>

And how often was that being published?

---

<div class="post-metadata">

**Author:** ![thatcadguy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/thatcadguy/32/5273_2.png) [@thatcadguy](https://discourse.nodered.org/u/thatcadguy)\
**Post date:** [9 August 2019 15:37 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/10 "2019-08-09T15:37:42Z")

</div>

Oh right, sorry, once per second.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [9 August 2019 15:43 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/11 "2019-08-09T15:43:45Z")

</div>

That is only 66 publishes/second when split. I am surprised it consumed that much processor. Did you check by stopping just the publish that it was the publish that was causing the cpu usage?  
Is it a Pi you are running on? If so which?

---

<div class="post-metadata">

**Author:** ![thatcadguy](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/thatcadguy/32/5273_2.png) [@thatcadguy](https://discourse.nodered.org/u/thatcadguy)\
**Post date:** [9 August 2019 15:54 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/12 "2019-08-09T15:54:08Z")

</div>

Yes, it's the publish(es). No, this is a FreeWave ZumIQ board with a 1 GHz Cortex-A8, single core. It would probably be nothing on a Pi 3 or 4.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [9 August 2019 15:56 UTC](https://discourse.nodered.org/t/best-way-to-publish-a-list-of-mqtt-topics-in-one-shot/14101/13 "2019-08-09T15:56:01Z")

</div>

OK, that makes sense then.