I have a question regarding the standard "csv node".
The node offers to input data in msg.columns. What about other parameters, e.g. the separator, the number of lines to skip, etc.? These are useful parameters and could make the node's usage more flexible. The node itself is able to use different separators, but I don't see how to set them in a message.
Are those parameters not available or not documented?
I have the situation, that I receive CSV files from different sources. Their content is logically similar, but the order of the columns and formats are different. An example could be lists of authorizations in different systems, where every system returns a slightly different format, but the data is logically similar.
Some files are separated by semicolon, others by comma, pipe, or whatever. The processing logic should handle that.
At the moment I have a switch node, that switches to different csv nodes, just to handle the different separators. Not a very nice solution (but works).
So, why not having such information optionally in a message. Same with the other options as these also could vary.
I think the solution you have is logically the cleanest... If they come from different sources then handle that first - route them to the individual "handlers" that are easy to visually see and debug - then recombine when normalised.