How to implement the partition of large CSV files and thus generate CSV subfiles?
What kind of partition? By number of lines? By a specific code in a column?
By number of lines. Thank you
You can probably use the split node assuming that you have already read in and parsed the CSV using file-in and csv nodes respectively.
Would it be better to use the exec node and use head, tail, awk or sed. If all you require is lines, exec may use less overheads. Depending on OS.
Do you need to repeat the first line at the start of each subfile?
Yes, it needs to be repeated
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
