# How to make a split of a large CSV file to sub CSV files?

**URL:** <https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918>\
**Category:** General\
**Tags:** function-node\
**Created:** [25 August 2022 15:39 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918 "2022-08-25T15:39:40Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![ger94](https://avatars.discourse-cdn.com/v4/letter/g/a8b319/32.png) [@ger94](https://discourse.nodered.org/u/ger94)\
**Post date:** [25 August 2022 15:39 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/1 "2022-08-25T15:39:40Z")

</div>

How to implement the partition of large CSV files and thus generate CSV subfiles?

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [25 August 2022 16:01 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/2 "2022-08-25T16:01:06Z")

</div>

What kind of partition? By number of lines? By a specific code in a column?

---

<div class="post-metadata">

**Author:** ![ger94](https://avatars.discourse-cdn.com/v4/letter/g/a8b319/32.png) [@ger94](https://discourse.nodered.org/u/ger94)\
**Post date:** [25 August 2022 16:13 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/3 "2022-08-25T16:13:47Z")

</div>

By number of lines. Thank you

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [25 August 2022 16:19 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/4 "2022-08-25T16:19:41Z")

</div>

You can probably use the split node assuming that you have already read in and parsed the CSV using file-in and csv nodes respectively.

---

<div class="post-metadata">

**Author:** ![E1cid](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/e1cid/32/77971_2.png) [@E1cid](https://discourse.nodered.org/u/E1cid)\
**Post date:** [25 August 2022 16:25 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/5 "2022-08-25T16:25:34Z")

</div>

Would it be better to use the exec node and use head, tail, awk or sed. If all you require is lines, exec may use less overheads. Depending on OS.

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [25 August 2022 16:32 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/6 "2022-08-25T16:32:23Z")

</div>

Do you need to repeat the first line at the start of each subfile?

---

<div class="post-metadata">

**Author:** ![ger94](https://avatars.discourse-cdn.com/v4/letter/g/a8b319/32.png) [@ger94](https://discourse.nodered.org/u/ger94)\
**Post date:** [25 August 2022 17:00 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/7 "2022-08-25T17:00:19Z")

</div>

Yes, it needs to be repeated

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [24 October 2022 17:00 UTC](https://discourse.nodered.org/t/how-to-make-a-split-of-a-large-csv-file-to-sub-csv-files/66918/8 "2022-10-24T17:00:22Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.