# \[ANNOUNCE\] node-red-contrib-multiple-queue

**URL:** <https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873>\
**Category:** Share Your Nodes\
**Created:** [10 January 2021 00:35 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873 "2021-01-10T00:35:09Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [10 January 2021 00:35 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/1 "2021-01-10T00:35:09Z")

</div>

This node provides multiple, independently controlled message queues. Each queue has the capabilities of the queueing mode of the [q-gate node](https://flows.nodered.org/node/node-red-contrib-queue-gate), with some additional features. The design goals were:

Flexible queue structure  
The user does not need to define in advance how many queues are required or how they are identified.

Economy of memory  
Queues are created as needed on a "per topic" basis and destroyed when empty. (Messages are routed using `msg.topic` by default, but any other message property can be used.) The size of each queue can be limited, and its behavior when full can be defined.

Complete command set  
Each queue is controlled using the command set of the `queueing` mode of the `q-gate` node: `trigger`, `status`,`flush`, `peek`, `drop`, and `reset`. In addition, commands are provided to allow each individual queue to be paused or to resume queueing and to modify the settings that control its size limit and behavior when full or empty. Any of these commands can also be addressed to all queues simultaneously.

Flexible command set  
None of the control message properties or payloads is "hard-wired" in the code. The edit dialog allows the user to define the message property and payload used for each command.

Enhanced status reporting  
An option is provided to create a second output that delivers a status object containing the requested message queue and all the queue-specific settings.

Persistence  
An option allows the queues and node settings to be restored from a non-volatile context store, if available, after a system or Node-RED restart.

Thanks to @nileio for [suggesting](https://github.com/drmibell/node-red-contrib-queue-gate/issues/16) this project. Please report bugs or leave suggestions here or on [GitHub](https://github.com/drmibell/node-red-contrib-multiple-queue/issues).

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [10 January 2021 10:53 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/2 "2021-01-10T10:53:12Z")

</div>

I don't want to sound unappreciative, but couldn't this of been an update to the existing q-gate node?  
We seem to have an increasing number of gate nodes with similar functionality, and users may find it difficult choosing!

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [10 January 2021 12:21 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/3 "2021-01-10T12:21:08Z")

</div>

Please see the discussion (issue) linked in my post. We thought about that approach, and others, but decided that a dedicated node was best in terms of design and useabiliy. For users who need only a gate or a single queue, learning the interface of the "one node to rule them all" could be more intimidating than deciding up front what functionality they need.

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [10 January 2021 13:07 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/4 "2021-01-10T13:07:35Z")

</div>

Thanks Mike, I did read the discussion earlier, but wondered if those obstacles could have been overcome.

I guess I'm also looking at it from the angle that I already use your q-gate node in a number of flows, but if I now want any of the new features I have to install yet another gate node (or update my existing flows to use this node).

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [10 January 2021 14:20 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/5 "2021-01-10T14:20:19Z")

</div>

> [@Paul-Reed](#):
>
> wondered if those obstacles could have been overcome

Frankly, I tried and failed. I'm always open to suggestions or PR's, though.

> [@Paul-Reed](#):
>
> install yet another gate node (or update my existing flows... )

The new features all relate to queueing, not gating, and really solve a different problem than the gate nodes. Think in terms of mqtt, where you have an arbitrary set of message streams (how many not known in advance), identified by `msg.topic` values, also not fully known in advance. If these things were known, you could use some fixed number of `q-gate` and perhaps `switch` nodes and have multiple, independent queues. If not, use `m-queue`. Either way, I don't see the need to replace existing nodes.

The main technical issue I encountered is that `gate` and and `q-gate` use `msg.topic` to distinguish between ordinary message traffic and control messages. For better or worse, that choice was made early on. To free up `msg.topic` to select among queues would break all existing flows that use the gate nodes. Requiring a different property for queue selection would make life difficult for users of mqtt or other pub/sub services, who are the ones most likely to be interested in `m-queue`. Ultimately, the value of [simplicity and doing one thing well](https://nodered.org/docs/creating-nodes/) seemed to outweigh the cost of adding a node to the pallete.

[PS] I do sympathize with the "cluttered palette" problem. I've complained about it myself. So when I considered packaging all three nodes together so the user wouldn't need to decide which one to download, I decided to do no harm.

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [10 January 2021 19:39 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/6 "2021-01-10T19:39:52Z")

</div>

Regardless of my feedback Mike, your nodes are very much appreciated, and there's always space for them in my palette 😃

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [10 January 2021 19:42 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/7 "2021-01-10T19:42:25Z")

</div>

You are too kind...

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [16 January 2021 11:46 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/8 "2021-01-16T11:46:12Z")

</div>

I think this will be a useful node for a particular set of problems. One issue I have is that I am trying to startup with a queue paused, but if I send a `pause` command to a queue that has not yet been created then the node errors out. I am wondering whether, if `Protect empty queues` is enabled, then pause and resume should create an empty queue if necessary. I have had to use a workaround of sending the node a dummy message, resetting it and then pausing it.  
I also see that other commands such as `maximum` cannot be applied before the queue is created, which could be a problem in some cases. One can imagine wanting setting that up in a startup flow.

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [16 January 2021 22:26 UTC](https://discourse.nodered.org/t/announce-node-red-contrib-multiple-queue/38873/9 "2021-01-16T22:26:58Z")

</div>

I think I understand what you want to accomplish. The original idea was to let the creation and destruction of queues be determined entirely by the flow of messages being queued. Clearly, the `protect` and `delete` commands break the rule, so it's just a question of what other violations might be useful. How about a `create` command? It would create an empty queue with its name taken from `msg.topic` or whatever other property was defined as the `Queue Selector`. It would be logical to have its `protect` property set, since the user probably wants his new empty queue to exist until he deliberately deletes it. Then `maxSize` and `keepNewest` can be set to the node's defaults, leaving only the value of `paused` as an issue. In your use case, `paused = true` would be convenient, but I would vote for `paused = false`, since it would allow the queue to begin queueing messages as soon as it is created. Any thoughts?