How to make a split of a large CSV file to sub CSV files?

How to implement the partition of large CSV files and thus generate CSV subfiles?

What kind of partition? By number of lines? By a specific code in a column?

By number of lines. Thank you

You can probably use the split node assuming that you have already read in and parsed the CSV using file-in and csv nodes respectively.

Would it be better to use the exec node and use head, tail, awk or sed. If all you require is lines, exec may use less overheads. Depending on OS.

Do you need to repeat the first line at the start of each subfile?

Yes, it needs to be repeated

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.