How to merge multiple csv into one

I am generating several csv that I want to merge into one so that I can send it by email,
The idea is that when pressing the timestamp I take the files I want from the path and merge them into 1 sooo, I have looked at how to do it with python and I could do it with the python node but I would like to be able to generate everything with Node Red , any suggestion Please

There are probably easier methods, but you could try this flow:

[{"id":"3a79ae2c.9908ea","type":"inject","z":"6bf9fb85.abde3c","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":132,"y":504,"wires":[["8fa40c3a.81cc4","4469c52c.6b30a4"]]},{"id":"8fa40c3a.81cc4","type":"file in","z":"6bf9fb85.abde3c","name":"","filename":".node-red/test.csv","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":306,"y":480,"wires":[["c5a4ce2b.a5eed8"]]},{"id":"b18add9e.c07308","type":"debug","z":"6bf9fb85.abde3c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1130,"y":504,"wires":[]},{"id":"c5a4ce2b.a5eed8","type":"csv","z":"6bf9fb85.abde3c","name":"","sep":",","hdrin":true,"hdrout":"none","multi":"mult","ret":"\\n","temp":"","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":506,"y":504,"wires":[["d5d50c16.bcd7e"]]},{"id":"4469c52c.6b30a4","type":"file in","z":"6bf9fb85.abde3c","name":"","filename":".node-red/test.csv","format":"utf8","chunk":false,"sendError":false,"encoding":"none","x":306,"y":528,"wires":[["c5a4ce2b.a5eed8"]]},{"id":"d5d50c16.bcd7e","type":"batch","z":"6bf9fb85.abde3c","name":"","mode":"count","count":"2","overlap":0,"interval":10,"allowEmptySequence":false,"topics":[],"x":626,"y":504,"wires":[["eaf46429.a6f038"]]},{"id":"eaf46429.a6f038","type":"join","z":"6bf9fb85.abde3c","name":"","mode":"auto","build":"array","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":false,"timeout":"","count":"","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"num","reduceFixup":"","x":746,"y":504,"wires":[["c670b1c5.8eee38"]]},{"id":"c670b1c5.8eee38","type":"function","z":"6bf9fb85.abde3c","name":"","func":"m = msg.payload\n\ncombined = m[0].concat(m[1])\n\nreturn {payload:combined}","outputs":1,"noerr":0,"initialize":"","finalize":"","x":876,"y":504,"wires":[["5f358519.1fec34"]]},{"id":"5f358519.1fec34","type":"csv","z":"6bf9fb85.abde3c","name":"","sep":",","hdrin":"","hdrout":"all","multi":"one","ret":"\\n","temp":"first,last,street,city,state,postcode","skip":"0","strings":true,"include_empty_strings":"","include_null_values":"","x":1010,"y":504,"wires":[["b18add9e.c07308"]]}]

The idea; read 2 csv files, use a batch + join node to create an array of objects, merge them and convert back to csv.

I think it might be worth your while explaining what you are doing with the csv files in this thread and the previous one. Are you using the csv files just for temporary storage? If so then there are probably much easier ways of solving your problems.

It works perfect, could you clarify what the join and batch node is used for?

Yes, it is to save temporary data, I have values ​​in the plc that are updated and crushed every 8 hours, the idea is to record the values ​​of these variables within those 8 hours through the pulses of the 2 Boolean inputs, which are the ones that tell me to record when the data has changed, when 8 hours pass, I merge the 2 csv into one and delete them, and once merged, I send the resulting csv by email to be able to see the data of the 2 variables registered, I don't know if I have explained myself clearly.

When I have everything working, I will upload a capture and the flow in case it is of interest to a colleague.

1 Like

You could use persistent context instead, almost certainly that would be simpler.
https://nodered.org/docs/user-guide/context

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.