I am doing a flow to store sensor data into a Sqlite database and require assistance at a concept level. The flow should read a CSV file and only insert a row if the data within the CSV file is unique. Uniqueness can be determined by searching within the db for an entry that looks the same as the data in the CSV file. Example: the CSV entry will have 3 measured values and a serial number for the source device that is unique. If any one of the three values in the CSV data is not the same as the last entry in the db for the same serial number then an entry should be made into the db.
The CSV file gets generated from an external application and will contain duplicate entries as it log a snapshot every 5 minutes.
Is the above method the best method to do it?
If it is, how can I query the db to search for the entry and if it fails, insert it. Note that the CSV file can easily have 100 entries in it.
Any assistance will be appreciated