How to use batch node with concatenate sequences mode

General
1

Hello,

I am trying to understand the batch node with concatenate sequences mode. I am not able find any examples related to usage of this node in concatenate sequences mode.

Could anyone please share a sample or pointer to any documentation which shows how to use this particular mode?

Regards
Arun

1 Like
2

Hi.

Ok, sorry. You used the batch node.

I think it would be easier if you use the join node.

Like this:

Given you send message 4 last.

[{"id":"e3630952.9b58d","type":"inject","z":"184dc884.7aba5f","name":"","topic":"","payload":"Message 1","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":200,"y":1640,"wires":[["dd4306f7.660f48"]]},{"id":"4bb46cb3.3777ec","type":"inject","z":"184dc884.7aba5f","name":"","topic":"","payload":"Message 2","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":200,"y":1680,"wires":[["dd4306f7.660f48"]]},{"id":"b7093225.f3c128","type":"inject","z":"184dc884.7aba5f","name":"","topic":"","payload":"Message 3","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":200,"y":1720,"wires":[["dd4306f7.660f48"]]},{"id":"db7a2c52.ed5328","type":"inject","z":"184dc884.7aba5f","name":"","topic":"","payload":"Message 4","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":200,"y":1760,"wires":[["b69084cc.63b918"]]},{"id":"772c4a91.8c54dc","type":"debug","z":"184dc884.7aba5f","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":700,"y":1700,"wires":[]},{"id":"dd4306f7.660f48","type":"join","z":"184dc884.7aba5f","name":"","mode":"custom","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":560,"y":1700,"wires":[["772c4a91.8c54dc"]]},{"id":"b69084cc.63b918","type":"change","z":"184dc884.7aba5f","name":"","rules":[{"t":"set","p":"complete","pt":"msg","to":"4","tot":"num"}],"action":"","property":"","from":"","to":"","reg":false,"x":380,"y":1760,"wires":[["dd4306f7.660f48"]]}]
3

I am using the batch node to create bulk inserts for my InfluxDB. It aggregates points from multiple sources for 10s, re-joins them into an array sends it in one operation.

Here's an example:

[{"id":"10cf2688.297669","type":"batch","z":"af129d0b.bf6878","name":"","mode":"interval","count":10,"overlap":0,"interval":10,"allowEmptySequence":false,"topics":[],"x":450,"y":260,"wires":[["17aa7e85.166589"]]},{"id":"17aa7e85.166589","type":"join","z":"af129d0b.bf6878","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":630,"y":260,"wires":[["7bf361b8.9906e8"]]},{"id":"ec7d48bf.d27de8","type":"inject","z":"af129d0b.bf6878","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":280,"y":260,"wires":[["10cf2688.297669"]]},{"id":"7bf361b8.9906e8","type":"debug","z":"af129d0b.bf6878","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":810,"y":260,"wires":[]}]
4

Your flow does not appear to demonstrate what you are trying to do. As the batch node is configured it is just repeating the incoming messages. Can you provide a flow showing in more detail what you are trying to do.

5

As @Trying_to_learn has suggested, if you have data coming in from various sources with different topics and want to generate a message built over a period to pass to influx then this would be the usual way to do it.

image
image510Ɨ573 26.2 KB

If the topics contain the influx field names then you can feed the output direct to the influx node.

6

It was just a general example to demonstrate to the OP what the batch node is doing. It's not related to InfluxDB or my use-case.

Neither am I using msg.topic. In my specific use-case the batch node is collecting ready-to-insert Influx point objects coming from different sources over a timespan of 10s. The output of the batch node (all points within those 10s) is then joined into an array and given to the Influx batch insert node. :slightly_smiling_face:

I have been using this pattern for some time now to optimize inserts into my databases.

7

Oops... I just realized I misread the OP's post, he's using the concatenate sequences mode. :flushed:

It was before my morning coffee, a bit too early it seems. :see_no_evil: So please ignore everything I wrote above, sorry.

8

Sorry, I thought @kuema was the OP. Please disregard my posts also.

9

No problem.

But - like the OP - I have yet to come up with an actual, real-world use-case for the concatenate sequences mode. I wonder what the intention was, especially with the topic handling that is mentioned in the docs:

Concatenate Sequences

creates a message sequence by concatenating incoming sequences. Each message must have a msg.topic property and a msg.parts property identifying its sequence. The node is configured with a list of topic values to identify the order sequences are concatenated.

10

@kuema,

As you have stated an actual use case as well a technical example of how to use the batch node in concatenate sequence mode is the missing in node red documentation. The brief section in node red doc mentions to use msg.topic to concatenate . I attempted the flow which is having two inputs to batch node and each input having a specific msg.topic name but it was not concatenating the two messages .

So I came to a conclusion that my fundamental understanding of concatenate sequence mode is either wrong or my implementation is wrong.

@Colin , thanks for your response.

Regards
Arun

11

Here are a couple of examples
Note the second isn't just a sort (much too easy :-)... but takes the negative numbers into a group - (topic), likewise the positive - then rejoins the groups... in a sequence set by the topics...

[{"id":"5208ff70.44718","type":"inject","z":"e0631187.0034b","name":"","topic":"SEQ","payload":"[1,2,3,4,5]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":140,"wires":[["f631ec88.be849"]]},{"id":"f631ec88.be849","type":"split","z":"e0631187.0034b","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":310,"y":140,"wires":[["4c952bd2.0c3ef4"]]},{"id":"4c952bd2.0c3ef4","type":"batch","z":"e0631187.0034b","name":"","mode":"concat","count":10,"overlap":0,"interval":10,"allowEmptySequence":false,"topics":[{"topic":"SEQ"},{"topic":"SEQ"}],"x":450,"y":140,"wires":[["b0910814.ba9308"]]},{"id":"b0910814.ba9308","type":"join","z":"e0631187.0034b","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":590,"y":140,"wires":[["e22a7e3c.7a1d2"]]},{"id":"e22a7e3c.7a1d2","type":"debug","z":"e0631187.0034b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":750,"y":140,"wires":[]},{"id":"5879f216.2ce0bc","type":"comment","z":"e0631187.0034b","name":"Duplicate","info":"","x":80,"y":100,"wires":[]},{"id":"92182451.05d258","type":"inject","z":"e0631187.0034b","name":"","topic":"SEQ","payload":"[1,-6,-8,7,2,-3]","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":260,"wires":[["17bfb97.3315847"]]},{"id":"17bfb97.3315847","type":"split","z":"e0631187.0034b","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":310,"y":260,"wires":[["bcf68114.86dad"]]},{"id":"37ac894a.6691e6","type":"comment","z":"e0631187.0034b","name":"Filter & Concat","info":"","x":100,"y":220,"wires":[]},{"id":"bcf68114.86dad","type":"switch","z":"e0631187.0034b","name":"","property":"payload","propertyType":"msg","rules":[{"t":"gt","v":"0","vt":"num"},{"t":"else"}],"checkall":"true","repair":true,"outputs":2,"x":370,"y":320,"wires":[["524fb754.ec6008"],["15bf3a07.977816"]]},{"id":"44a61da0.b3fa64","type":"comment","z":"e0631187.0034b","name":"ā†‘ Duplicate SEQ","info":"","x":480,"y":180,"wires":[]},{"id":"524fb754.ec6008","type":"change","z":"e0631187.0034b","name":"Topicā†POS","rules":[{"t":"set","p":"topic","pt":"msg","to":"POS","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":530,"y":280,"wires":[["60d5ee4e.e9abd"]]},{"id":"15bf3a07.977816","type":"change","z":"e0631187.0034b","name":"Topicā†NEG","rules":[{"t":"set","p":"topic","pt":"msg","to":"NEG","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":530,"y":340,"wires":[["60d5ee4e.e9abd"]]},{"id":"60d5ee4e.e9abd","type":"batch","z":"e0631187.0034b","name":"","mode":"concat","count":10,"overlap":0,"interval":10,"allowEmptySequence":false,"topics":[{"topic":"NEG"},{"topic":"POS"}],"x":690,"y":320,"wires":[["3172cb41.204654"]]},{"id":"f18fb109.d9302","type":"debug","z":"e0631187.0034b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":910,"y":260,"wires":[]},{"id":"3172cb41.204654","type":"join","z":"e0631187.0034b","name":"","mode":"auto","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":"false","timeout":"","count":"","reduceRight":false,"x":750,"y":260,"wires":[["f18fb109.d9302"]]},{"id":"bd630efb.7ec67","type":"comment","z":"e0631187.0034b","name":"ā†‘ Order sequence: negativeā†’positive","info":"","x":790,"y":360,"wires":[]},{"id":"182c0d29.62f153","type":"comment","z":"e0631187.0034b","name":"Filter pos/neg and make separate sequenceā†‘ (but not a simple sort) ","info":"","x":300,"y":380,"wires":[]}]
3 Likes
12

Thanks a lot @dceejay. Perfect example.