Join node gives different values when run. Assumption was that it would be same?

[Soiski]

Hi, As many other I have barked my path to use nordpool data to find cheapest hours for electricity. The part getting the data, separating it and grouping it to 3hrs intervals (with 2 hrs overlap) works , but when it should group it back (join node) the system works correctly on the 1st time. i.e. 1st group data is from 23:00 / next 00:00 / next 01:00. Second is from 00:00 / next 01:00 / next 02:00. So perfect what I was looking for . But If I run the nodes again I get different values (even though the data until the join is identical). For unknown reason that join now picks 2 last values (21:00 and 22:00) and actually starts the join process like 21:00 /next 22:00 / next 23:00 , second batch 22:00 / next 23:00 / 00:00.
I thought that adding perhaps msg.reset / boolean TRUE would fix it but still the same.

Heres the code, any suggestion how to get it right?
Help much appreciated.

[{"id":"7828b08e04438531","type":"nordpool-api-plus","z":"5e45b8d5.4a6f28","name":"Nordpool_FI","area":"FI","currency":"EUR","x":333.0000190734863,"y":638.0000247955322,"wires":[["687e9a21fa879620"]]},{"id":"b2af7feb40b86c81","type":"inject","z":"5e45b8d5.4a6f28","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"00 20 * * *","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":128.0000114440918,"y":639.0000867843628,"wires":[["7828b08e04438531"]]},{"id":"687e9a21fa879620","type":"split","z":"5e45b8d5.4a6f28","name":"","splt":"","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"topic","x":515.2000350952148,"y":636.2000389099121,"wires":[["ab99cf1e39369a05"]]},{"id":"ab99cf1e39369a05","type":"batch","z":"5e45b8d5.4a6f28","name":"Batch 3 hrs periods","mode":"count","count":"3","overlap":"2","interval":10,"allowEmptySequence":false,"topics":[],"x":328.1999969482422,"y":717.2000389099121,"wires":[["6eb366ba958a5298"]]},{"id":"6eb366ba958a5298","type":"join","z":"5e45b8d5.4a6f28","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"parts.index","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"3","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":653.2000350952148,"y":695.2000389099121,"wires":[["6226d96cfdab07f9"]]},{"id":"6226d96cfdab07f9","type":"debug","z":"5e45b8d5.4a6f28","name":"debug 38","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":768.200065612793,"y":744.8000164031982,"wires":[]}]

[jbudd]

Do you mean that the same data split, batched and joined gives different results on different runs?

Can you put a debug node on the output of the nordpool api node and share the data with us?
ie copy the data from the debug pane and paste here like you did with your flow

[Soiski]

Yes exactly...cant get my head wrapped around though why...

[{"timestamp":"2023-02-09T23:00:00.000Z","price":18.32,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T00:00:00.000Z","price":17.87,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T01:00:00.000Z","price":17.84,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T02:00:00.000Z","price":18.88,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T03:00:00.000Z","price":23.86,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T04:00:00.000Z","price":34.99,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T05:00:00.000Z","price":48.14,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T06:00:00.000Z","price":91.62,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T07:00:00.000Z","price":116.42,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T08:00:00.000Z","price":109.71,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T09:00:00.000Z","price":85.9,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T10:00:00.000Z","price":66.93,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T11:00:00.000Z","price":59.9,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T12:00:00.000Z","price":50.79,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T13:00:00.000Z","price":37.29,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T14:00:00.000Z","price":37.43,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T15:00:00.000Z","price":38.45,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T16:00:00.000Z","price":41.26,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T17:00:00.000Z","price":31.3,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T18:00:00.000Z","price":29.98,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T19:00:00.000Z","price":29.82,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T20:00:00.000Z","price":30.84,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T21:00:00.000Z","price":30.06,"currency":"EUR","area":"FI"},{"timestamp":"2023-02-10T22:00:00.000Z","price":30.01,"currency":"EUR","area":"FI"}]

[Soiski]

1st run everything is OK.

2nd run for some reason 21:00 and 22:00 are inputted as start (see that date is also actually the two last data from the batch)

[E1cid]

The batch node will have the overlap stored from each run. so when second array object arrives and is split the last 2 objects from previous split will be in batch node cache. You will need to reset the batch node once last join occurs.
e.g.

[{"id":"b2af7feb40b86c81","type":"inject","z":"da8a6ef0b3c9a5c8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"timestamp\":\"2023-02-09T23:00:00.000Z\",\"price\":18.32,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T00:00:00.000Z\",\"price\":17.87,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T01:00:00.000Z\",\"price\":17.84,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T02:00:00.000Z\",\"price\":18.88,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T03:00:00.000Z\",\"price\":23.86,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T04:00:00.000Z\",\"price\":34.99,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T05:00:00.000Z\",\"price\":48.14,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T06:00:00.000Z\",\"price\":91.62,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T07:00:00.000Z\",\"price\":116.42,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T08:00:00.000Z\",\"price\":109.71,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T09:00:00.000Z\",\"price\":85.9,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T10:00:00.000Z\",\"price\":66.93,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T11:00:00.000Z\",\"price\":59.9,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T12:00:00.000Z\",\"price\":50.79,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T13:00:00.000Z\",\"price\":37.29,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T14:00:00.000Z\",\"price\":37.43,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T15:00:00.000Z\",\"price\":38.45,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T16:00:00.000Z\",\"price\":41.26,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T17:00:00.000Z\",\"price\":31.3,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T18:00:00.000Z\",\"price\":29.98,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T19:00:00.000Z\",\"price\":29.82,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T20:00:00.000Z\",\"price\":30.84,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T21:00:00.000Z\",\"price\":30.06,\"currency\":\"EUR\",\"area\":\"FI\"},{\"timestamp\":\"2023-02-10T22:00:00.000Z\",\"price\":30.01,\"currency\":\"EUR\",\"area\":\"FI\"}]","payloadType":"json","x":134.99999237060547,"y":4601.000061988831,"wires":[["687e9a21fa879620"]]},{"id":"687e9a21fa879620","type":"split","z":"da8a6ef0b3c9a5c8","name":"","splt":"","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"topic","x":542.2000160217285,"y":4598.20001411438,"wires":[["ab99cf1e39369a05"]]},{"id":"ab99cf1e39369a05","type":"batch","z":"da8a6ef0b3c9a5c8","name":"Batch 3 hrs periods","mode":"count","count":"3","overlap":"2","interval":10,"allowEmptySequence":false,"topics":[],"x":355.19997787475586,"y":4679.20001411438,"wires":[["6eb366ba958a5298"]]},{"id":"e4f3f1d63163f5a9","type":"function","z":"da8a6ef0b3c9a5c8","name":"function 20","func":"msg = {reset:true};\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":270,"y":4760,"wires":[["ab99cf1e39369a05"]]},{"id":"6eb366ba958a5298","type":"join","z":"da8a6ef0b3c9a5c8","name":"","mode":"custom","build":"array","property":"payload","propertyType":"msg","key":"parts.index","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"3","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":530,"y":4680,"wires":[["6226d96cfdab07f9","f085ae2dfb3da847"]]},{"id":"f085ae2dfb3da847","type":"trigger","z":"da8a6ef0b3c9a5c8","name":"","op1":"","op2":"reset","op1type":"nul","op2type":"str","duration":"1","extend":true,"overrideDelay":false,"units":"s","reset":"","bytopic":"all","topic":"topic","outputs":1,"x":640,"y":4760,"wires":[["e4f3f1d63163f5a9"]]},{"id":"6226d96cfdab07f9","type":"debug","z":"da8a6ef0b3c9a5c8","name":"debug 38","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":795.2000465393066,"y":4706.799991607666,"wires":[]}]

[Soiski]

Thank @E1cid, works perfectly. I was trying to send the msg.reset to the join, part but it seems that it was needed to be done before it (to the batch sequence). Or was it more that I tried to send it as inject node and not as function you do.