Separating multiple executions of a flow

[ctSkennerton]

Does node red have the concept of an execution or instance of a flow. I have some previous experience with workflow orchestration tools like airflow where you can start multiple executions of a workflow simultaneously and all of the intermediate steps are separate.

To illustrate what I'm talking about...

I created the following test flow where a timestamp is injected, duplicated and one of the copies is delayed (to simulate a slow process). After this the messages are combined again and printed out. What I would like to happen is that the final result be an array where the same timestamp is found twice like this: [1643962351491,1643962351491]. However due to the slow path in the flow what ends up happening is the values are different [1643962360975,1643962361407] as two subsequent values from the fast path get combined in the join step.

Is there a solution in node red such that separate executions or initial inputs create distinct flows?

[{"id":"f6f2187d.f17ca8","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"d75fb16cf5282db7","type":"inject","z":"f6f2187d.f17ca8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":240,"wires":[["24aa14b5db6a8c4d"]]},{"id":"137bdaf1c11ca9fc","type":"delay","z":"f6f2187d.f17ca8","name":"","pauseType":"delay","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":420,"y":300,"wires":[["b50907d20858610a"]]},{"id":"b50907d20858610a","type":"join","z":"f6f2187d.f17ca8","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":570,"y":240,"wires":[["02863d76260a4b79"]]},{"id":"02863d76260a4b79","type":"debug","z":"f6f2187d.f17ca8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":730,"y":240,"wires":[]},{"id":"24aa14b5db6a8c4d","type":"function","z":"f6f2187d.f17ca8","name":"duplicate","func":"\nreturn [msg, msg];","outputs":2,"noerr":0,"initialize":"","finalize":"","libs":[],"x":260,"y":240,"wires":[["b50907d20858610a"],["137bdaf1c11ca9fc"]]}]

[Steve-Mcl]

@ctSkennerton

What I would like to happen is that the final result be an array where the same timestamp is found twice like this: [1643962351491,1643962351491] .

doesnt happen for me...

↑ The result from running your flow unmodified.

The timestamp inside payload inside the object msg (e.g. msg.payload) is sent twice by your function then the 2 are joined in an array

I suspect the delay node was added by you to simulate some long running task? If so, then that is what will be changing the timestamp.

What version of node-red are you running?

[ctSkennerton]

Sorry I wasn't clear. If I click the timestamp node multiple times in quick succession (to simulate multiple http requests) I get the problem. The delay node was added by me to simulate what would be another slow function. What I want to achieve is that the outputs are always joined with the same timestamp. I'm using node red 2.2

[Steve-Mcl]

@ctSkennerton

If I click the timestamp node multiple times in quick succession (to simulate multiple http requests) I get the problem

Ah, ok, that is the problem with your flow design and use of the join node.

The flow you wrote and its output are to be expected in those circumstances - i.e. it is doing what you asked it to do.

Another way to achieve that would be to key them and re-join them manually (by a key either self generated or the _msgid).

Here is a demo using a self generated key (change this to _msgid if you dont need a custom key) ...

[{"id":"202414650e8cf58b","type":"inject","z":"85e1791bc0cd6285","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1816,"y":128,"wires":[["a4d7e47491b477b5"]]},{"id":"30818c8f60e6f677","type":"function","z":"85e1791bc0cd6285","name":"split in 2","func":"\nreturn [msg, RED.util.cloneMessage(msg)];","outputs":2,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1816,"y":224,"wires":[["5df1fcbfb6ecb1c6"],["fb2d43c053cee36c"]]},{"id":"5df1fcbfb6ecb1c6","type":"function","z":"85e1791bc0cd6285","name":"join by key","func":"const msgStore = context.get(\"msgStore\") || {};\n\nif (msgStore[msg.key]) {\n    const newMsg = {\n        payload: {\n            msg1: msgStore[msg.key],\n            msg2: RED.util.cloneMessage(msg),\n        }\n    }\n    node.send(newMsg);\n    msgStore[msg.key] = undefined;\n} else {\n    msgStore[msg.key] = RED.util.cloneMessage(msg);\n}\n\ncontext.set(\"msgStore\", msgStore);\n\nreturn null;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":2106,"y":224,"wires":[["7bfd13cb839e68ec"]]},{"id":"a4d7e47491b477b5","type":"function","z":"85e1791bc0cd6285","name":"add a key","func":"msg.key = \"MSG-KEY-\" + Date.now() + \"-\" + Math.round((Math.random() * 999) + 1000)\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1976,"y":128,"wires":[["30818c8f60e6f677"]]},{"id":"fb2d43c053cee36c","type":"delay","z":"85e1791bc0cd6285","name":"","pauseType":"delay","timeout":"4","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":1944,"y":288,"wires":[["5df1fcbfb6ecb1c6"]]},{"id":"7bfd13cb839e68ec","type":"debug","z":"85e1791bc0cd6285","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":2158,"y":288,"wires":[]}]

[ctSkennerton]

Thank you!