Possible to listen to debug console and wait until no more messages?

[AleXSR700]

and

let lastTopic = context.get("lastTopic") ?? msg.setting
// topic unchanged => send to output 1
let messages = [msg, null]
if (msg.setting !== lastTopic) {
    // topic changed => send to output 2
    messages = [null, msg]
}
context.set("lastTopic", msg.setting)
return messages;

So where would you place the queue then?
Here it is after the start_all function which I changed as per your suggestion to output all 10 possible msg.setting payloads in a row.

The problem is that the msg.setting payload does not create multiple messages until after the split nodes which are located towards the end of the sub-flows.

flows.json (287 Bytes)

[Colin]

@AleXSR700

The problem is that the msg.setting payload does not create multiple messages until after the split nodes which are located towards the end of the sub-flows.

That does not agree with what you said earlier:

@AleXSR700

The "cell range" node you see in the gif can vary, so it can be one cell or 1000 cells. And each cell creates one message with the same topic.

I read that to mean that the cell range node sends one or 1000 messages. I see now that I misunderstood.

What is in the messages at the Out node?

[AleXSR700]

@Colin

That does not agree with what you said earlier:

Sorry, that was poorly phrased by me. The cell range is where I define which rows to read, so depending on the values there, the number of messages will vary. But the messages are not split from each other until more or less the end.

The OUT node see each individual message (so the split has been performed and it sees all x*y messages). So it sees all messages of the first batch, then all of the following batches. Hence, my earlier idea was to monitor the link-in node and only perform the flush after all the messages of the same msg.setting have been sent.

[Colin]

At the Out node is there any indication of what msg.settings was? Perhaps the topics are identifiable.

[AleXSR700]

Yes.
HEre is an example:

{"payload":"SO1 1; SO13 0; SO30 1; SO47 60; SO129 1; TelePeriod 10; SaveData 1; WattRes 0; PowerOnState 3; SwitchMode 0; SwitchMode2 0; Timezone 99; SerialLog 0; WebLog 2; LEDPower 0; Prefix3 tele; CurrentCal 10000; CurrentCal2 10000; PowerCal 1540; PowerCal2 1540; VoltageCal 26000; VoltageCal2 26000; savedata","topic":"cmnd/tasmota_235FAD/Backlog","setting":"settings","filename":"/data/IoT_Overview.xlsm","selectedSheetName":"Tasmota","selectedRange":"U2:U15","parts":{"id":"f564159a8c43d41d","type":"array","count":14,"len":1,"index":3},"_msgid":"cee9a27e2a2ccad9"}

or

{"payload":"Template {\"NAME\":\"Shelly 2.5\",\"GPIO\":[320,0,32,0,224,193,0,0,640,192,608,225,3456,4736],\"FLAG\":0,\"BASE\":18}; Module 0; DeviceName Light Dining Room Ambient; FriendlyName1 Right; FriendlyName2 Left","topic":"cmnd/tasmota_2306AE/Backlog","setting":"template","filename":"/data/IoT_Overview.xlsm","selectedSheetName":"Tasmota","selectedRange":"G2:G15","parts":{"id":"721cd8fc244cbd49","type":"array","count":14,"len":1,"index":4},"_msgid":"dfe0bba7bb35cd72"}

So the msg.setting is still there.

[Colin]

Put the queue in front of the MQTT node.

I don't think you need individual rate limit nodes on each path. Put one in output 1 of the function instead.

[AleXSR700]

The individual ones are to not flood the MQTT broker.

@Colin

Put the queue in front of the MQTT node.

Including the link-in, flush function etc.?

And keep the link-out in parallel to the the queue then? If debug OUT, MQTT and link-out are behind the queue, there would be no flushing. If I put the link-out in parallel, will I not still have the same problem that the flushing is performed after each message rather than after the batch?

[Colin]

The whole queuing flow. As originally posted, but in front of the MQTT node.

[Colin]

Like this

In fact the 1/sec rate limit has to go inside the group. It doesn't need to be a rate limit inside the group, it can just be a delay as it only sees one at a time anyway.

[AleXSR700]

@Colin , thank you, you solved it!

I apologize if I made this harder for you than necessary, given that the solution was posted earlier but I had the placement wrong. I would have always placed it at the beginning where the flow starts but placing it at the end is far simpler

[Colin]

I will leave it as an exercise to use the fact that the Delay node will accept msg.delay as the delay to impose on the message being sent in. Using that you can make the function just have one output and use one Delay node, with varying delay.