Node-red-contrib-queue-gate - how to add trigger identifier?

[bs-eng]

Hi there!
I am using node-red-contrib-queue-gate and am wondering how I can add trigger identifiers to the messages generated. For example I have a stream of data which is being queued. Then I have several triggers that are used to each release one item from the queue. In the resulting message from the queue I would like to add something like "trigger A" or "trigger B" to the message. This enables to know the source of the trigger.

Is there any way to achieve this directly with the queue node or do I need something else?

Thanks a lot!
JR

Example flow:

[
    {
        "id": "d94273488bc00d0c",
        "type": "tab",
        "label": "Flow 4",
        "disabled": false,
        "info": "",
        "env": []
    },
    {
        "id": "c54e919c.18d7f",
        "type": "q-gate",
        "z": "d94273488bc00d0c",
        "name": "q-gate demo",
        "controlTopic": "control",
        "defaultState": "queueing",
        "openCmd": "open",
        "closeCmd": "close",
        "toggleCmd": "toggle",
        "queueCmd": "queue",
        "defaultCmd": "default",
        "triggerCmd": "trigger",
        "flushCmd": "flush",
        "resetCmd": "reset",
        "peekCmd": "peek",
        "dropCmd": "drop",
        "statusCmd": "status",
        "maxQueueLength": "0",
        "keepNewest": false,
        "qToggle": false,
        "persist": false,
        "x": 630,
        "y": 360,
        "wires": [
            [
                "d2740316.7035c"
            ]
        ]
    },
    {
        "id": "c04db5ad.da76b8",
        "type": "inject",
        "z": "d94273488bc00d0c",
        "name": "input",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "myData",
        "payload": "",
        "payloadType": "date",
        "x": 350,
        "y": 360,
        "wires": [
            [
                "c54e919c.18d7f"
            ]
        ]
    },
    {
        "id": "fb8c7d0f.40eec",
        "type": "inject",
        "z": "d94273488bc00d0c",
        "name": "trigger A",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            },
            {
                "p": "source",
                "v": "A",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "control",
        "payload": "trigger",
        "payloadType": "str",
        "x": 360,
        "y": 480,
        "wires": [
            [
                "c54e919c.18d7f"
            ]
        ]
    },
    {
        "id": "d2740316.7035c",
        "type": "debug",
        "z": "d94273488bc00d0c",
        "name": "output",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "true",
        "targetType": "full",
        "statusVal": "",
        "statusType": "auto",
        "x": 790,
        "y": 360,
        "wires": []
    },
    {
        "id": "cab5863412e0d3eb",
        "type": "inject",
        "z": "d94273488bc00d0c",
        "name": "trigger B",
        "props": [
            {
                "p": "payload"
            },
            {
                "p": "topic",
                "vt": "str"
            },
            {
                "p": "source",
                "v": "B",
                "vt": "str"
            }
        ],
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "topic": "control",
        "payload": "trigger",
        "payloadType": "str",
        "x": 360,
        "y": 540,
        "wires": [
            [
                "c54e919c.18d7f"
            ]
        ]
    }
]

[cymplecy]

You could set the topic on the trigger A to triggerA and the topic on B to triggerB and that would identify them

[bs-eng]

Unfortunately that is what does not work. The gate node uses the "topic" to set the control intent.
Setting any other key in the message also gets swallowed by the gate node.

[marcus-j-davies]

Hi @bs-eng

I used q-gate for sometime, unless things have changed, the topic of control will have no affect on the resulting outgoing payload.

therefore, a topic of control and payload of trigger - will simply release the message at the front of the queue - and will not alter its shape

[cymplecy]

@bs-eng

The gate node uses the "topic" to set the control intent.

The default ctopic is control
(to clarify as the term "topic" is being overloaded here - msg.topic is sent thru untouched. To send control messages to the gate you need to set msg.control )

As you can see - the msg.topic is passed thru the gate untouched

[marcus-j-davies]

@cymplecy

(to clarify as the term "topic" is being overloaded here - msg.topic is sent thru untouched. To send control messages to the gate you need to set msg.control )

Mmm.

Unless things have changed (its been a while since I used it)

if msg.topic is set to the control value set by the user - the message is treated as an action to execute on the queue (denoted by its partner payload), if topic is NOT matched with the user defined control topic - its treated as a normal message (in its entirety) as a message that can be queued

[marcus-j-davies]

If I set control as the control topic in the node config

topic	payload	Result.
Apples	Are Green	Queued
Oranges	Are Orange	Queued
control	trigger	Release message at the front
control	close	Rejects (non control) message
control	flush	empties all messags

[cymplecy]

Hey - its Saturday evening here - I've had a few drinks - I think I'm right But I'll check in the morning

[edit} I am wrong I'll stop tonight and re-answer in the morming

[marcus-j-davies]

@cymplecy

[edit} I am wrong I'll stop tonight and re-answer in the morming

It's the Ale

[bs-eng]

Nope, I can only second marcus here.
I see exactly the behaviour he describes.

However the example of cymplecy kind of works, because the inject nodes are delivering data messages, not control messages. - that is not what I trying to achieve though.

[edit] that was a lot of simultaenuous typing I guess.

[cymplecy]

I know I said I wouldn't tonight but ... 10 sec test

[drmibell]

@bs-eng

Unfortunately that is what does not work.

Unfortunately, you are correct. The queue-gate node was deliberately designed to isolate the control messages from the message traffic being controlled, so that the two cannot affect one another. I would be happy to discuss the logic behind this decision, but it would take us away from your issue. I can think of several ways to meet your requirement. This is the simplest:

[{"id":"c54e919c.18d7f","type":"q-gate","z":"467a5e306d972f0d","name":"q-gate demo","controlTopic":"control","defaultState":"queueing","openCmd":"open","closeCmd":"close","toggleCmd":"toggle","queueCmd":"queue","defaultCmd":"default","triggerCmd":"trigger","flushCmd":"flush","resetCmd":"reset","peekCmd":"peek","dropCmd":"drop","statusCmd":"status","maxQueueLength":"0","keepNewest":false,"qToggle":false,"persist":false,"x":430,"y":280,"wires":[["dcdbc883723c0838"]]},{"id":"c04db5ad.da76b8","type":"inject","z":"467a5e306d972f0d","name":"input","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"myData","payload":"","payloadType":"date","x":210,"y":280,"wires":[["c54e919c.18d7f"]]},{"id":"fb8c7d0f.40eec","type":"inject","z":"467a5e306d972f0d","name":"trigger A","props":[{"p":"payload"},{"p":"topic","vt":"str"},{"p":"source","v":"A","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"control","payload":"trigger","payloadType":"str","x":260,"y":220,"wires":[["c54e919c.18d7f","dcdbc883723c0838"]]},{"id":"d2740316.7035c","type":"debug","z":"467a5e306d972f0d","name":"output","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":930,"y":280,"wires":[]},{"id":"cab5863412e0d3eb","type":"inject","z":"467a5e306d972f0d","name":"trigger B","props":[{"p":"payload"},{"p":"topic","vt":"str"},{"p":"source","v":"B","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"control","payload":"trigger","payloadType":"str","x":260,"y":340,"wires":[["c54e919c.18d7f","dcdbc883723c0838"]]},{"id":"dcdbc883723c0838","type":"join","z":"467a5e306d972f0d","name":"combine","mode":"custom","build":"object","property":"","propertyType":"full","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":660,"y":280,"wires":[["4a6695f9357cf5b5"]]},{"id":"4a6695f9357cf5b5","type":"change","z":"467a5e306d972f0d","name":"clean up","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload.myData.payload","tot":"msg"},{"t":"set","p":"topic","pt":"msg","to":"payload.myData.topic","tot":"msg"},{"t":"set","p":"source","pt":"msg","to":"source","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":800,"y":280,"wires":[["d2740316.7035c"]]}]

Only the join ("combine") node is really needed. The change ("clean up") node just puts the properties where I suspect you want them. I can't guarantee that the flow is bullet-proof, since I don't know the sources of your control messages. You might be able to create a race condition or other timing issue that would break it.

[bs-eng]

yes, that is definitley a possible way.
The problem I can see coming is when there is no message in the queue and a trigger is sent. but that I might handle with just another gate and some logic.

[marcus-j-davies]

Here you go. - this might work - its an extension to @drmibell approach
A subflow that adds the trigger to the output.

when issuing a trigger command - also include a msg.trigger value (of your choice)

Untested
You may want to add the trigger value ONLY if the topic is trigger

Subflow setup

[{"id":"84237a82f6bfa7da","type":"subflow","name":"Q-Gate with trigger value","info":"","category":"","in":[{"x":55,"y":75,"wires":[{"id":"f36b3dc06a60a6ca"}]}],"out":[{"x":815,"y":400,"wires":[{"id":"7d4dc61776cf7910","port":0}]}],"env":[],"meta":{},"color":"#DDAA99"},{"id":"c8881a40de3ee2f2","type":"q-gate","z":"84237a82f6bfa7da","name":"","controlTopic":"control","defaultState":"open","openCmd":"open","closeCmd":"close","toggleCmd":"toggle","queueCmd":"queue","defaultCmd":"default","triggerCmd":"trigger","flushCmd":"flush","resetCmd":"reset","peekCmd":"peek","dropCmd":"drop","statusCmd":"status","maxQueueLength":"100","keepNewest":false,"qToggle":false,"persist":false,"storeName":"default","x":440,"y":230,"wires":[["7d4dc61776cf7910"]]},{"id":"7d4dc61776cf7910","type":"change","z":"84237a82f6bfa7da","name":"","rules":[{"t":"set","p":"trigger","pt":"msg","to":"trigger","tot":"flow"},{"t":"delete","p":"trigger","pt":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":620,"y":300,"wires":[[]]},{"id":"f36b3dc06a60a6ca","type":"change","z":"84237a82f6bfa7da","name":"","rules":[{"t":"set","p":"trigger","pt":"flow","to":"trigger","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":260,"y":140,"wires":[["c8881a40de3ee2f2"]]},{"id":"395e5b7f73e9aac7","type":"subflow:84237a82f6bfa7da","z":"62c22fa051c1549f","name":"","x":535,"y":325,"wires":[[]]}]

[drmibell]

@bs-eng

when there is no message in the queue and a trigger is sent

I worried about that also, but by trial-and-error it doesn't seem to be an issue. I think the join node as configured waits for two different message topics before sending an output and uses the two most recent.

[Colin]

As a matter of interest, can you describe your use case?

[bs-eng]

A sensor connected by serial/USB is delivering at a fairly high rate data. The aplication needs the data only sporadic but for many different purposes. So to reduce the data rate while still being able to filter (smoothing) I am using a smooth-gate combined with several triggers. As the triggers come in by MQTT the timing can get tricky as the MQTT may deliver triggers that were some second appart at the same time.