Read multiple csv files

[gioespo]

Hello everyone,

I need to read several csv files present in the same folder without having to change the path each time.
is it possible to do this? If yes, how? I have not found any solution at the moment.

I am new to node-red, sorry if the question may seem trivial.

Thank you to those who will answer.

[E1cid]

Yes
just posted example of how to use exec node to get files in directory, you could then use a split node to read each file and join them etc.

[TotallyInformation]

There are nodes to do that as well if you want to avoid an exec call.

[gioespo]

Thank you for these answers, they pointed me in the right direction.

I'll try to explain in more detail what I want to do because I haven't solved it yet but I think it's easier than expected.

I want to read all the .csv files in a directory and send them one at a time to an MQTT server.

I can easily do this if I work with just one file but what I would like to do is that upon completion of sending the first file the stream reads and sends the second then the third and so on.

Using an exec node (with command ls /path/*.csv) and then a split node I can individually get all the names I need but I don't really know how to continue.

What do you think is the best way ? I saw that there is a loop node or I am trying to use a trigger node but I still don't have a clear idea.

[E1cid]

After split use a file read node to read the file output as utf-8 then add a change node to set msg.topic to your mqtt topic or set it in the mqtt node, I would then add a delay node set to rate limit to allow x messages per second, as to not flood the mqtt node. finally add your mqtt node.

In the file read node set the path to msg. payload

P.s. You would be better off adding all relevant details, like what is the output of the exec node and split outputs, then helpers are not guessing.
Example

[{"id":"5673dabba0ad5b70","type":"inject","z":"b9860b4b9de8c8da","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":660,"y":360,"wires":[["3c5d089b98e91fc0"]]},{"id":"3c5d089b98e91fc0","type":"exec","z":"b9860b4b9de8c8da","command":"ls /path/*.csv","addpay":"","append":"","useSpawn":"false","timer":"","winHide":false,"oldrc":false,"name":"","x":840,"y":360,"wires":[["3152e6e82b3f2187","4ee6e74b70884bcd"],[],[]]},{"id":"3152e6e82b3f2187","type":"change","z":"b9860b4b9de8c8da","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"topic/to/send","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":720,"y":420,"wires":[["37a5d05801bac915"]]},{"id":"4ee6e74b70884bcd","type":"debug","z":"b9860b4b9de8c8da","name":"debug 332","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1030,"y":480,"wires":[]},{"id":"37a5d05801bac915","type":"split","z":"b9860b4b9de8c8da","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":890,"y":420,"wires":[["f222cdf3fdd2b509"]]},{"id":"16b91b168aedff38","type":"delay","z":"b9860b4b9de8c8da","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"4","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"allowrate":false,"outputs":1,"x":870,"y":480,"wires":[["4ee6e74b70884bcd"]]},{"id":"f222cdf3fdd2b509","type":"switch","z":"b9860b4b9de8c8da","name":"","property":"payload","propertyType":"msg","rules":[{"t":"neq","v":"","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":1010,"y":420,"wires":[["bdfac8957f86af42"]]},{"id":"bdfac8957f86af42","type":"file in","z":"b9860b4b9de8c8da","name":"","filename":"payload","filenameType":"msg","format":"utf8","chunk":false,"sendError":false,"encoding":"none","allProps":false,"x":700,"y":480,"wires":[["16b91b168aedff38"]]}]

[TotallyInformation]

Is each line of each CSV a separate MQTT topic or do you have big topic values?