File transfers with MQTT

I am trying to create an application that monitors a directory on a machine for a file or a set of files (say up to 1Mb size .json files) in the directory (i.e. directory contents) and transmits (publishes) these files via mqtt to a broker. Once transmitted, the files are moved out of the monitored directory to another "daily record" folder.

The broker or a subscriber then collects these files and updates a database with the recently received .json file data. The recently received files are moved to a "daily record folder" on the broker or subscriber.

We are currently using an old clunky and buggy Visual Basic set of applications (clients and hosts) to do this work. I works well, unattended, but it needs an upgrade.

I thought Node-Red would could be of service.

I am not finding the NODES that can do this , especially the MQTT nodes, do not have a way of reading a directory for a file to send. They seem to be are expecting the JSON data to be pasted in to the payload field.

I need the "BROKER FUNCTIONALITY" with TOPICS because we have many topics, which we have had to hard code over the years in the Visual Basic system, when it's time to add a new TOPIC it is not an easy task.

Any assistance would be most appreciated, otherwise we have to do hard coding again with one of the MQTT offerings.

Many thanks for any input.

Well it won't be a single node... but ones I think may be of use would be the watch node to watch a directory for changes, a file in node to read a file into a buffer - a change node to move some properties around as needed - an mqtt out node to point at your broker.

1 Like