I'm trying to traverse through a large CSV file. Each line, I do some data remodelling and convert it to XML format. Lastly, I save all of it into an XML file. In my actual implementation, the resulting remodelled XML data is 10x larger than the source CSV data. I have a use case where the sourc…

So I tried appending the XML data per iteration and delete the msg.payload before feeding the message to the Join node so it doesn't accumulate data. If you want to iterate, you need to use output msg per line in the file-in node. Still going to be heavy impact. I don't think node-red is the ri…

Memory issues processing and writing a large file

General

zenofmud Split this topic 5 September 2021 11:20 6

A post was split to a new topic: Converting CSV to mysql causes error with blank csv columns

Converting CSV to mysql causes error with blank csv columns

Topic		Replies	Views
Memory issue processing large csv file General	3	746	20 December 2021
Converting CSV to mysql causes error with blank csv columns General	3	711	6 September 2021
Memory limits on Rpi processing chart data General	6	351	15 October 2021
Read large JSON file (to insert to MySQL) causes Node-RED crash due to JavaScript heap out of memory General	11	1744	5 October 2020
Reading csv file General	16	4889	4 September 2018

Memory issues processing and writing a large file

Related topics