Hello there!
I'm looking for some help on how to solve some specific challenges I'm facing in a current project. I'm a bit stuck, so I was hoping someone more experienced than myself could give me some hints.
Status:
I have 2 CSV files that each contains some columns with data and a timestamp. I have them imported into Node-Red and directed to a mySQL database- my current flow looks like this:
[{"id":"3992ae32.0275c2","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"6ab9153d.2b444c","type":"inject","z":"3992ae32.0275c2","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"true","payloadType":"bool","x":230,"y":320,"wires":[["fc6fa5a1.44d9d8","f9399507.2ca6d8"]]},{"id":"c7106530.531a18","type":"mysql","z":"3992ae32.0275c2","mydb":"60cb07a4.7ce3a8","name":"SQL_Server","x":1090,"y":360,"wires":[[]]},{"id":"fcba8c85.3901a","type":"function","z":"3992ae32.0275c2","name":"GPS","func":"var data = msg.payload.split(\",\");\nvar rosbagtimestamp_in_miliseconds = data[0];\nrosbagtimestamp_in_miliseconds = rosbagtimestamp_in_miliseconds/1000000;\nvar latitude = data[1];\nvar longitude = data[2];\nvar altitude = data[3];\nvar relative_altitude = data[4];\nvar out = \"INSERT INTO gps(rosbagtimestamp_in_miliseconds,latitude,longitude,altitude,relative_altitude) VALUES('\"+rosbagtimestamp_in_miliseconds+\"','\"+latitude+\"','\"+longitude+\"','\"+altitude+\"','\"+relative_altitude+\"')\";\nmsg.topic = out;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":770,"y":280,"wires":[["c7106530.531a18","97e312b4.43de2"]]},{"id":"e883cb4c.c3c608","type":"function","z":"3992ae32.0275c2","name":"Spectrum","func":"var data = msg.payload.split(\",\");\nvar sweep = data[0];\nvar timestamp = data[1];\nvar peak_freq_mhz = data[2];\nvar peak_ampl_dbm = data[3];\nvar out = \"INSERT INTO spectrum(sweep,timestamp,peak_freq_mhz,peak_ampl_dbm) VALUES('\"+sweep+\"','\"+timestamp+\"','\"+peak_freq_mhz+\"','\"+peak_ampl_dbm+\"')\";\nmsg.topic = out;\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":780,"y":360,"wires":[["c7106530.531a18","97e312b4.43de2"]]},{"id":"97e312b4.43de2","type":"debug","z":"3992ae32.0275c2","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1090,"y":280,"wires":[]},{"id":"f9399507.2ca6d8","type":"file in","z":"3992ae32.0275c2","name":"Spectrum","filename":"C:\\Users\\Oscar Gregers\\Desktop\\Dataopsamlings projekt\\UCL QuadSAT spectrum.csv","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":520,"y":360,"wires":[["e883cb4c.c3c608"]]},{"id":"fc6fa5a1.44d9d8","type":"file in","z":"3992ae32.0275c2","name":"GPS","filename":"C:\\Users\\Oscar Gregers\\Desktop\\Dataopsamlings projekt\\UCL QuadSAT GPS .csv","format":"lines","chunk":false,"sendError":false,"encoding":"none","x":510,"y":280,"wires":[["fcba8c85.3901a"]]},{"id":"60cb07a4.7ce3a8","type":"MySQLdatabase","name":"","host":"127.0.0.1","port":"3306","db":"drone_data","tz":"","charset":"UTF8"}]
Problem statement:
Now, my next goal is to merge these 2 files together by timestamps, but comparing the files according to the timestamps, looks something like this sketch:
So my challenges is:
1. How do I merge the files together, resulting in the structure marked with green in the figure above. (something about comparing column 1 in file 1 & column 2 in file 2, but from there I'm lost).
2. How do I discard the rows as shown in the figure above.
If in need of more details, please let me know.
This is my first post ever, so please go easy on me, in case my description is poorly composed.
Thanks in advance!