Splitting Arrays

General
1

Lets say I have an array of items. I want to iterate over a subset of these item (using a split and switch) and do a couple of operations on each, the first of which takes some time to complete. So I add a stoptimer node to pause between the two operations:

image
image.png693×166 12 KB

  1. Will this work? Or will each qualifying split message reset the stoptimer?

  2. I now want to do a final operation after all the items have been processed. How can I do that?

  3. Let's say I only want to do the operation once for the first qualifying item. How can I stop processing after one filtered message? I tried joining after the switch, but I got an empty array (even though the debug node below confirmed at least one item made it through).

image
image.png800×243 16.5 KB

2

Can you post your flow ?

3

Do you mean the exported code? The above was a mock up (I'm using a web API to generate the array) but let me see if I can create a standalone version.

4

This should capture my questions:

image
image.png962×238 15.8 KB 

[{"id":"9a0c6b97.307988","type":"function","z":"87b395b1.644fe8","name":"generate array","func":"\nreturn {payload: [0,1,2,3]}","outputs":1,"noerr":0,"x":320,"y":200,"wires":[["505586ec.479828"]]},{"id":"bf920944.9c6128","type":"inject","z":"87b395b1.644fe8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":200,"wires":[["9a0c6b97.307988"]]},{"id":"505586ec.479828","type":"split","z":"87b395b1.644fe8","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":470,"y":200,"wires":[["8a851cf.d6aace"]]},{"id":"8a851cf.d6aace","type":"switch","z":"87b395b1.644fe8","name":"","property":"payload","propertyType":"msg","rules":[{"t":"gte","v":"2","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":590,"y":200,"wires":[["3d0c47ff.8e8b38","7caf7849.2856a8"]]},{"id":"7caf7849.2856a8","type":"function","z":"87b395b1.644fe8","name":"Slow op","func":"\nreturn msg;","outputs":1,"noerr":0,"x":360,"y":280,"wires":[["2509b0a2.f83cc"]]},{"id":"2509b0a2.f83cc","type":"stoptimer","z":"87b395b1.644fe8","duration":"5","units":"Second","payloadtype":"num","payloadval":"0","name":"","x":540,"y":280,"wires":[["4e3a3e2f.eb79f"],[]]},{"id":"4e3a3e2f.eb79f","type":"function","z":"87b395b1.644fe8","name":"Last op","func":"\nreturn msg;","outputs":1,"noerr":0,"x":720,"y":280,"wires":[["481d15d0.1d7a7c"]]},{"id":"3d0c47ff.8e8b38","type":"debug","z":"87b395b1.644fe8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":140,"wires":[]},{"id":"481d15d0.1d7a7c","type":"debug","z":"87b395b1.644fe8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":890,"y":260,"wires":[]}]

Some updates to my OP:

  1. While knocking the above up, I see that only "3" is output by the second debug, so it seems that the messages will in fact overwrite each other at the timer (as documented). How would I continue the flow (and pauses) for all qualifying items?

  2. I would still like to know how to do this correctly (my current workaround is to wait).

  3. In the meantime, I have solved question 3) by checking the "recreate message sequences" in the split.

5

More progress - I see there is a "delay" node which I should be using instead of the stoptimer as it specifically acts as a rate limiter.

That just leaves question 2.

6

Your idea for this flow is not entirely clear to me. Messages are processed sequentially, is the stoptimer not redudant in this case ?

Actually, you could create this flow with 1 function node.

7

For any item that passes the switch, I need Last Op to run after First Op is finished, however First Op is run asynchronously. Further I don't want First Op to be run multiple times for items are the same time. Here's the flow using delay, which I think does what I want wrt 1) and 3):

image
image.png1087×283 30.5 KB 

[{"id":"9a0c6b97.307988","type":"function","z":"87b395b1.644fe8","name":"Get list of archives to be created","func":"\nreturn {payload: [0,1,2,3]}","outputs":1,"noerr":0,"x":390,"y":200,"wires":[["505586ec.479828","d1061ae4.300828"]]},{"id":"bf920944.9c6128","type":"inject","z":"87b395b1.644fe8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":200,"wires":[["9a0c6b97.307988"]]},{"id":"505586ec.479828","type":"split","z":"87b395b1.644fe8","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":630,"y":200,"wires":[["8a851cf.d6aace"]]},{"id":"8a851cf.d6aace","type":"switch","z":"87b395b1.644fe8","name":"","property":"payload","propertyType":"msg","rules":[{"t":"gte","v":"2","vt":"num"}],"checkall":"true","repair":false,"outputs":1,"x":750,"y":200,"wires":[["3d0c47ff.8e8b38","2d0f351d.5321aa"]]},{"id":"7caf7849.2856a8","type":"function","z":"87b395b1.644fe8","name":"Slow Async Webservice","func":"\nreturn msg;","outputs":1,"noerr":0,"x":410,"y":280,"wires":[["76dde637.20c708"]]},{"id":"4e3a3e2f.eb79f","type":"function","z":"87b395b1.644fe8","name":"Transfer Created Archive ","func":"\nreturn msg;","outputs":1,"noerr":0,"x":910,"y":280,"wires":[["481d15d0.1d7a7c"]]},{"id":"3d0c47ff.8e8b38","type":"debug","z":"87b395b1.644fe8","name":"Print qualifying item","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":930,"y":140,"wires":[]},{"id":"481d15d0.1d7a7c","type":"debug","z":"87b395b1.644fe8","name":"Item Processed","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":960,"y":200,"wires":[]},{"id":"76dde637.20c708","type":"delay","z":"87b395b1.644fe8","name":"Wait 2s for service to finish","pauseType":"delay","timeout":"2","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":660,"y":280,"wires":[["4e3a3e2f.eb79f"]]},{"id":"2d0f351d.5321aa","type":"delay","z":"87b395b1.644fe8","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"minutes","rate":"1","nbRateUnits":"5","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":180,"y":280,"wires":[["7caf7849.2856a8"]]},{"id":"d1061ae4.300828","type":"delay","z":"87b395b1.644fe8","name":"Wait 1m for all items to process","pauseType":"delay","timeout":"2","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":330,"y":80,"wires":[["1fb5fcac.51cba3"]]},{"id":"1fb5fcac.51cba3","type":"function","z":"87b395b1.644fe8","name":"Finalise batch job","func":"\nreturn {payload: [0,1,2,3]}","outputs":1,"noerr":0,"x":590,"y":80,"wires":[["83915c40.9a34d"]]},{"id":"83915c40.9a34d","type":"debug","z":"87b395b1.644fe8","name":"Done","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":770,"y":80,"wires":[]}]

I don't like that I have to guess how long to wait before finalising the batch job though. Is there any way for the last processed item to trigger that flow?

In programming parlance I'm looking for the equivalent serial for each loop I suppose?

8

similar but not quite the same - but maybe enough to inspire you - https://flows.nodered.org/flow/cea8afa28b7a93ebdc0f