I'm trying to traverse through a large CSV file. Each line, I do some data remodelling and convert it to XML format. Lastly, I save all of it into an XML file. In my actual implementation, the resulting remodelled XML data is 10x larger than the source CSV data. I have a use case where the sourc…

So I tried appending the XML data per iteration and delete the msg.payload before feeding the message to the Join node so it doesn't accumulate data. If you want to iterate, you need to use output msg per line in the file-in node. Still going to be heavy impact. I don't think node-red is the ri…

Memory issues processing and writing a large file

General

zenofmud Split this topic 5 September 2021 11:20 6

A post was split to a new topic: Converting CSV to mysql causes error with blank csv columns

Converting CSV to mysql causes error with blank csv columns

Topic		Replies	Views
Memory issue processing large csv file General	3	759	20 December 2021
Read large JSON file (to insert to MySQL) causes Node-RED crash due to JavaScript heap out of memory General	11	1792	5 October 2020
File transfers fail because of file size - buffer limit? General	23	2023	21 January 2020
Helpful criticism requested General	44	2504	17 July 2018
Memory leak, what am I doing wrong? General	28	11043	10 December 2018

Memory issues processing and writing a large file

Related topics