How to store & update NMEA-AIS data?

[Nice_to_have]

Hi Guys,

I've been on this problem for a while and I can't think of a way to make this work.
I am no programmer, I have learned what I know on this forum (Thank you all for that).

I am recieving NMEA AIS data. I calculate the distance and the bearing from me to the vessel.
Now besides displaying it on WorldMap, I'd like to have a table that I can sort on distance-from-me, so I can see which vessel is closest to me.

As AIS is transmitted in different time intervals (Sailing vessels more frequent then moored vessels) I need a way of storing the data.
Also, a vessel can transmit several sentences which hold different values/variables.
example:

2 different AIS messages can hold a similar MMSI number, but have different info about the vessel.
I'd like to store the data under the MMSI number, so when I call the latest, i get all the info I have, like MMSI, lon, lat, name_vessel, distance to me, etc etc.

I have tried this with InfluxDB, but I think because it is time based, I can't show it on the dashboard properly, because after a couple of minutes, I have several entries in that list from the same MMSI number.

What I'd like to achieve is store all data for a certain MMSI number and update some fields of it if need be.

I hope it all makes sence

[TotallyInformation]

Influx certainly not the tool to use for this I don't think.

The first thing to understand is how many vessels you need to hold data for?
Then, how long do you keep the data - or rather, how do you know when you no longer need the data?

If the number of vessels is a manageable number (no more than a few hundred maybe?) then you can hold all of this data in a Node-RED global variable as an object key'd on the MMSI number.

So when you get new data for that number, you are simply overwriting the previous data.

[E1cid]

There was a similar post, maybe this may help.

UI with selectable AIS ship target list

Hello, I am trying to create a Dashboard, where I can pick a ship/target from a drop-down list and display it's data... So far I set up a tcp:port where I receive AIS-NMEA data constantly. The "ais"-node transforms the data into readable json format (Objects?). The AIS data is sent every few seconds from several vessels (each has a specific time slot). If I want to select the data from just one special ship, l need to filter the incoming data, e.g. by ships name (or "mmsi")... ... additionall…

[jbudd]

I guess it depends if you need just the most recent position for each vessel or historical data. Either way, my instinct would be to use a "proper" SQL database such as Mariadb.
Create a VESSEL record whenever a new vessel enters your area of interest, and a VESSEL_LOCATION record whenever you receive a location message.

Once the data is in a database you can use it however you like: closest vessel, recorded movements for a vessel, time moored up, estimated arrival time at a particular location, etc.

[Nice_to_have]

It is a very clean and a smaller solution to what I had first.
It took me some time to figure out I needed a function node infront of the append, to convert my incoming messages. But I got it all working.

[{"id":"c7bd3578c3bc7062","type":"link in","z":"f3da98203cbc6de3","name":"To Temp Storage","links":["f1580da4afb64d3d"],"x":235,"y":140,"wires":[["2c36c60f2835559a","f9412fe1c31d7b7f"]]},{"id":"a7b7591d.dc3028","type":"json","z":"f3da98203cbc6de3","name":"convert to string","property":"payload","action":"str","pretty":false,"x":780,"y":140,"wires":[["4a0644c7.cdbd3c"]]},{"id":"4a0644c7.cdbd3c","type":"file","z":"f3da98203cbc6de3","name":"","filename":"shipData.json","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":980,"y":140,"wires":[[]]},{"id":"2eb876b6.f89f7a","type":"change","z":"f3da98203cbc6de3","name":"append...","rules":[{"t":"set","p":"payload","pt":"msg","to":"$append([$.payload],$flowContext(\"shipData\")[mmsi != $$.payload.mmsi])","tot":"jsonata"},{"t":"set","p":"shipData","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":580,"y":140,"wires":[["a7b7591d.dc3028"]]},{"id":"2c36c60f2835559a","type":"function","z":"f3da98203cbc6de3","name":"","func":"msg.payload = \n[\n    {\n        \"mmsi\": msg.payload.senderMmsi,\n        \"shipID\": msg.payload.shipId,\n        \"callsign\": msg.payload.callsign,\n        \"name\": msg.payload.name,\n        \"shiptype\": msg.payload.shipType,\n        \"dimensionToBow\":msg.payload.dimensionToBow,\n        \"dimensionToStern\":msg.payload.dimensionToStern,\n        \"dimensionToPort\":msg.payload.dimensionToPort,\n        \"dimensionToStartboard\":msg.payload.dimensionToStarboard,\n        \"navigationStatus\": msg.payload.navigationStatus,\n        \"navigationStatus_text\": msg.payload.navigationStatus_text,\n        \"fixType\": msg.payload.fixType,\n        \"draught\": msg.payload.draught,\n        \"destination\":msg.payload.destination,\n        \"dte\": msg.payload.dte,\n        \"speedOverGround\": msg.payload.speedOverGround,\n        \"longitude\": msg.payload.longitude,\n        \"latitude\": msg.payload.latitude,\n        \"courseOverGround\": msg.payload.courseOverGround,\n        \"timeStampSeconds\": msg.payload.timeStampSeconds,\n        \"source\": msg.originalAisMessage[0],\n        \"distanceToVesselNM\": msg.payload.DistanceToWaypointNM.toFixed(2),\n        \"distanceToVesselKM\": msg.payload.DistanceToWaypointKM.toFixed(2),\n        \"bearingToVessel\": msg.payload.BearingToWaypoint.toFixed(2)\n    }\n]\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":380,"y":140,"wires":[["2eb876b6.f89f7a"]]},{"id":"974f8ca35a81c9b0","type":"file in","z":"f3da98203cbc6de3","name":"","filename":"shipData.json","format":"utf8","chunk":false,"sendError":false,"encoding":"none","allProps":false,"x":580,"y":340,"wires":[["db29f6c9f36bb270"]]},{"id":"a5170b92f3a15809","type":"inject","z":"f3da98203cbc6de3","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"10","crontab":"","once":true,"onceDelay":0.1,"topic":"","payloadType":"date","x":370,"y":340,"wires":[["974f8ca35a81c9b0"]]},{"id":"db29f6c9f36bb270","type":"json","z":"f3da98203cbc6de3","name":"convert to string","property":"payload","action":"","pretty":false,"x":790,"y":340,"wires":[["59d5670179f38738","53e8d2e43a6005c0"]]},{"id":"59d5670179f38738","type":"debug","z":"f3da98203cbc6de3","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":990,"y":340,"wires":[]},{"id":"4586210adf7ec568","type":"ui_table","z":"f3da98203cbc6de3","group":"ccd54031.b4542","name":"","order":2,"width":13,"height":13,"columns":[],"outputs":0,"cts":false,"x":1030,"y":440,"wires":[]},{"id":"f9412fe1c31d7b7f","type":"debug","z":"f3da98203cbc6de3","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":370,"y":60,"wires":[]},{"id":"53e8d2e43a6005c0","type":"change","z":"f3da98203cbc6de3","name":"","rules":[{"t":"set","p":"ui_control.tabulator.layout","pt":"msg","to":"fitColomns","tot":"str"},{"t":"set","p":"ui_control.tabulator.persistentSort","pt":"msg","to":"true","tot":"str"},{"t":"set","p":"ui_control.tabulator.initialSort","pt":"msg","to":"[{\"column\":\"distanceToVesselNM\",\"dir\":\"asce\"}]","tot":"json"}],"action":"","property":"","from":"","to":"","reg":false,"x":860,"y":440,"wires":[["4586210adf7ec568"]]},{"id":"ccd54031.b4542","type":"ui_group","name":"Worldmap","tab":"8f81a49e.8fe7d8","order":2,"disp":false,"width":"23","collapse":false},{"id":"8f81a49e.8fe7d8","type":"ui_tab","name":"Worldmap","icon":"dashboard","order":2,"disabled":false,"hidden":true}]

Thanks

[E1cid]

Glad it helped.
If the data grows to big you could do similar in a db, just update existing entries rather than insert if the mmsi already exists

[Nice_to_have]

I got a question.
The AIS messages have "Message Types" 1 - 27.
If an array comes in with mmsi, lon, lat, sog
and later another comes in with the mmsi, lon, lat, cog, name
Is everything overwritten (example sog is removed), or only the ones that change or added?

[E1cid]

I think every thing is overwritten, The original AIS post all the info was delivered in one go.
You could merge the incoming object with the stored object and then append it, to preserve all the data.