I have created a Node-Red Flow that reads a huge XML file (Around 2.5GB) passes each line to a function which is looking about a specific tag and returns the value.
Everything is fine when I try to visualize the values using chart
Flow is like this timestamp-xml-function-chart
Now the problem presents when I try to store these exports to SQL Server using mssql-plus node.
Flow is like this timestamp-xml-function-mssql
Some of the values are passing through the SQL Server and then is stopping and I see errors in debug like Error Connection Reset or Connection Closed.
The file contains 5277363 values of type char (like 2019,2020) to be stored on the SQL Server Table with one column with type String nvarchar(4).
Both Node-Red and SQL Server are dockers which are communicating with each other inside the same host.
Using C# in the same system the parsing can be done in 10sec for storing those values into a variable.
So maybe is a frequency/latency error from SQL Server?
If that's the case why the MSSQL Node doesn't delay the whole flow and just keep accepting requests until it cannot handle them?