UI with selectable AIS ship target list

Dashboard
1

Hello,
I am trying to create a Dashboard, where I can pick a ship/target from a drop-down list and display it's data...
So far I set up a tcp:port where I receive AIS-NMEA data constantly. The "ais"-node transforms the data into readable json format (Objects?). The AIS data is sent every few seconds from several vessels (each has a specific time slot). If I want to select the data from just one special ship, l need to filter the incoming data, e.g. by ships name (or "mmsi")...
... additionally there might be new ships coming into range and might be added to the list and ships leaving the range should be deleted from the list... (the date and time of the entry could be a trigger...)

I was thinking I store the incoming data into a file. e.g.:

{"mmsi":211667760,"navigationStatus":"Under way using engine","rateOfTurn":0,"speedOverGround":0,"longitude":9.920658333333334,"latitude":53.54305333333333,"courseOverGround":290,"trueHeading":268,"timeStampSeconds":14,"date":1621533253342,"isoDate":"2021-05-20 17:54:13","source":"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\r\n"}
{"mmsi":211667590,"navigationStatus":"Under way using engine","rateOfTurn":0,"speedOverGround":0.5,"longitude":9.921221666666666,"latitude":53.543015,"courseOverGround":254.7,"trueHeading":275,"timeStampSeconds":17,"date":1621533256452,"isoDate":"2021-05-20 17:54:16","source":"!AIVDM,1,1,,A,139o:QP0000eJVrN`kLItpVR0H9T,0*17\r\n"}
{"mmsi":211626350,"navigationStatus":"Under way using engine","rateOfTurn":0,"speedOverGround":0.1,"longitude":9.921696666666667,"latitude":53.543083333333335,"courseOverGround":293.2,"trueHeading":271,"timeStampSeconds":20,"date":1621533260219,"isoDate":"2021-05-20 17:54:20","source":"!AIVDM,1,1,,A,139laKP0010eJglN`kVcM8N`0<52,0*0E\r\n"}

Is there a node, which can update one entry if e.g. the mmsi already exists in the list and adds a new entry if there is none yet? Or do I have to write a code for this (I am unfortunately not yet familiar with programming in java language)...

... then I was trying to extract the data from the file and was looking for filter-nodes. For example I would like to get the speedOverGround- Value displayed of one and the same ship... I tried some scripting in the function node e.g. "for string in file...." but was not very successful...
... well... if the file would be updated constantly there would be on entry per mmsi only anyway...

2

Here is an example using JSONata using a change node.
I have put your storage values in an array

[{"mmsi":211667760,"navigationStatus":"Under way using engine","rateOfTurn":0,"speedOverGround":0,"longitude":9.920658333333334,"latitude":53.54305333333333,"courseOverGround":290,"trueHeading":268,"timeStampSeconds":14,"date":1621533253342,"isoDate":"2021-05-20 17:54:13","source":"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\r\n"},
{"mmsi":211667590,"navigationStatus":"Under way using engine","rateOfTurn":0,"speedOverGround":0.5,"longitude":9.921221666666666,"latitude":53.543015,"courseOverGround":254.7,"trueHeading":275,"timeStampSeconds":17,"date":1621533256452,"isoDate":"2021-05-20 17:54:16","source":"!AIVDM,1,1,,A,139o:QP0000eJVrN`kLItpVR0H9T,0*17\r\n"},
{"mmsi":211626350,"navigationStatus":"Under way using engine","rateOfTurn":0,"speedOverGround":0.1,"longitude":9.921696666666667,"latitude":53.543083333333335,"courseOverGround":293.2,"trueHeading":271,"timeStampSeconds":20,"date":1621533260219,"isoDate":"2021-05-20 17:54:20","source":"!AIVDM,1,1,,A,139laKP0010eJglN`kVcM8N`0<52,0*0E\r\n"}]

Eample flow (ctrl i in editor to import)

[{"id":"1bb7d78f.756dd","type":"inject","z":"c74669a0.6a34f8","name":"","props":[{"p":"hold","v":"[{\"mmsi\":211667760,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"},{\"mmsi\":211667590,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.5,\"longitude\":9.921221666666666,\"latitude\":53.543015,\"courseOverGround\":254.7,\"trueHeading\":275,\"timeStampSeconds\":17,\"date\":1621533256452,\"isoDate\":\"2021-05-20 17:54:16\",\"source\":\"!AIVDM,1,1,,A,139o:QP0000eJVrN`kLItpVR0H9T,0*17\\r\\n\"},{\"mmsi\":211626350,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.1,\"longitude\":9.921696666666667,\"latitude\":53.543083333333335,\"courseOverGround\":293.2,\"trueHeading\":271,\"timeStampSeconds\":20,\"date\":1621533260219,\"isoDate\":\"2021-05-20 17:54:20\",\"source\":\"!AIVDM,1,1,,A,139laKP0010eJglN`kVcM8N`0<52,0*0E\\r\\n\"}]","vt":"json"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"mmsi\":211667760,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":10,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"}","payloadType":"json","x":120,"y":1840,"wires":[["cf950648.c9b42"]]},{"id":"cf950648.c9b42","type":"change","z":"c74669a0.6a34f8","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$append([$.payload],$.hold[mmsi != $$.payload.mmsi])","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":300,"y":1840,"wires":[["1cd0bf9c.c79d38"]]},{"id":"1cd0bf9c.c79d38","type":"debug","z":"c74669a0.6a34f8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":530,"y":1840,"wires":[]}]

In the inject is the msg.hold which holds the storage array. msg.payload holds the new object to be updated.

$append([$.payload],$.hold[mmsi != $$.payload.mmsi])

[edit the second part of your question can also be solved with JSONata expressions predictive queries

e.g. To get all items who's speedOverGround = 0.5

hold[speedOverGround = 0.5]

]

3

Couldn't resist sticking them on a map :slight_smile:

[{"id":"972dd746.606458","type":"inject","z":"cc91a15c.faf28","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":130,"y":1200,"wires":[["40a15feb.92871"]]},{"id":"40a15feb.92871","type":"function","z":"cc91a15c.faf28","name":"array of data","func":"msg.payload = [\n{\"mmsi\":211667760,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"},\n{\"mmsi\":211667590,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.5,\"longitude\":9.921221666666666,\"latitude\":53.543015,\"courseOverGround\":254.7,\"trueHeading\":275,\"timeStampSeconds\":17,\"date\":1621533256452,\"isoDate\":\"2021-05-20 17:54:16\",\"source\":\"!AIVDM,1,1,,A,139o:QP0000eJVrN`kLItpVR0H9T,0*17\\r\\n\"},\n{\"mmsi\":211626350,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.1,\"longitude\":9.921696666666667,\"latitude\":53.543083333333335,\"courseOverGround\":293.2,\"trueHeading\":271,\"timeStampSeconds\":20,\"date\":1621533260219,\"isoDate\":\"2021-05-20 17:54:20\",\"source\":\"!AIVDM,1,1,,A,139laKP0010eJglN`kVcM8N`0<52,0*0E\\r\\n\"}\n]\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":1200,"wires":[["32b7ac91.1da364"]]},{"id":"32b7ac91.1da364","type":"function","z":"cc91a15c.faf28","name":"reformat for map","func":"msg.payload = msg.payload.map(function(x) {\nx.lat = x.latitude;\nx.lon = x.longitude;\nx.hdg = x.trueHeading;\nx.name = x.mmsi;\ndelete x.latitude;\ndelete x.longitude;\ndelete x.mmsi;\ndelete x.speedOverGround;\nx.icon = \"ship\";\nreturn x;\n});\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":510,"y":1200,"wires":[["5c89119b.5d9d3"]]},{"id":"5c89119b.5d9d3","type":"worldmap","z":"cc91a15c.faf28","name":"","lat":"","lon":"","zoom":"","layer":"OSM grey","cluster":"","maxage":"","usermenu":"show","layers":"show","panit":"false","panlock":"false","zoomlock":"false","hiderightclick":"false","coords":"none","showgrid":"false","allowFileDrop":"false","path":"/worldmap","x":700,"y":1200,"wires":[]}]

image
image846Ă—728 68.5 KB

image

4 Likes
4

@E1cid: thanks for the code and the link. I'll study the JSONata and .hold some more. As I understand the .hold already is the file I was going to create to store the data...

As I understand you trigger the msg.hold [{...}] and msg.payload {...} with your inject node and use the $append()-command to add the msg.payload{} to the msg.hold[{...}]

I tried now to use two inject nodes, doing the same... I hoped so... the first one creates the empty msg.hold [{}] and the other inject node creates the msg.payload {"mmsi":211...etc}, which then shall be added to the empty .hold... but my debug-node msg.hold says undefined. Is it an array-problem or debug-node error?

[{"id":"129a88e7.1187ef","type":"tab","label":"Flow 2","disabled":false,"info":""},{"id":"7d53955a.c1917c","type":"inject","z":"129a88e7.1187ef","name":"newArray msg.hold","props":[{"p":"hold","v":"[{\"mmsi\":0}]","vt":"json"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":210,"y":440,"wires":[["a83b645b.e0946"]]},{"id":"a83b645b.e0946","type":"debug","z":"129a88e7.1187ef","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"hold","targetType":"msg","statusVal":"","statusType":"auto","x":720,"y":440,"wires":[]},{"id":"4887d836.e6ed8","type":"inject","z":"129a88e7.1187ef","name":"newShip211667760","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"mmsi\":211667760,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"}","payloadType":"json","x":210,"y":500,"wires":[["2eb876b6.f89f7a"]]},{"id":"2eb876b6.f89f7a","type":"change","z":"129a88e7.1187ef","name":"append...","rules":[{"t":"set","p":"payload","pt":"msg","to":"$append([$.payload],$.hold[mmsi != $$.payload.mmsi])","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":490,"y":500,"wires":[["58ea85fd.648084"]]},{"id":"58ea85fd.648084","type":"debug","z":"129a88e7.1187ef","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"hold","targetType":"msg","statusVal":"","statusType":"auto","x":720,"y":500,"wires":[]}]

@dceejay: Jep, that's right. Working on a tug boat in the port of Hamburg (on the "Bugsier 2" to be precise). I want to install a "head up display" using a RaspberryPi 3B+ mirroring in the rear Window. When we approach a vessel to establish or release the line connection, the vsl should not be faster than 10kn. We don't have much power left to manoeuver and rescue from a possible danger situation. Attached you can find my Dashboard so far. The Inject node just triggers a sample AIS signal, which unfortunately contains the speed of 79.3, but that's just for testing purposes.

[{"id":"e45173f3.bfc09","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"68d4e1d1.cc628","type":"tcp in","z":"e45173f3.bfc09","name":"","server":"client","host":"localhost","port":"10110","datamode":"stream","datatype":"utf8","newline":"","topic":"","base64":false,"x":130,"y":180,"wires":[["a66029ed.f9fd88","7d3e5f4.508bea"]]},{"id":"a66029ed.f9fd88","type":"ais","z":"e45173f3.bfc09","name":"","x":430,"y":180,"wires":[["4f61f2eb.d0ce7c","46ddc68a.d0a188","2e0fa682.322dfa"]]},{"id":"4f61f2eb.d0ce7c","type":"debug","z":"e45173f3.bfc09","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":630,"y":20,"wires":[]},{"id":"7d3e5f4.508bea","type":"debug","z":"e45173f3.bfc09","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":350,"y":80,"wires":[]},{"id":"2e0fa682.322dfa","type":"ui_gauge","z":"e45173f3.bfc09","name":"","group":"5a1af3bc.7cd31c","order":1,"width":0,"height":0,"gtype":"gage","title":"","label":"kn","format":"{{msg.payload.speedOverGround}}","min":0,"max":"160","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":870,"y":180,"wires":[]},{"id":"46ddc68a.d0a188","type":"debug","z":"e45173f3.bfc09","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload.speedOverGround","targetType":"msg","statusVal":"","statusType":"auto","x":690,"y":80,"wires":[]},{"id":"83bb7be6.8ea92","type":"inject","z":"e45173f3.bfc09","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"!AIVDM,2,1,8,A,53P7AsD2<Iu=a4u7:21<B0tl4LR22222222222150pD2574g061PDk0C,0*3D !AIVDM,1,1,,B,13MARih000wbAbJP0kr23aSV0<0g,0*73","payloadType":"str","x":170,"y":240,"wires":[["a66029ed.f9fd88"]]},{"id":"928d686.a2e5598","type":"change","z":"e45173f3.bfc09","name":"","rules":[{"t":"set","p":"ui_control.options.minLabelMinFontSize","pt":"msg","to":"0","tot":"num"},{"t":"set","p":"ui_control.options.maxLabelMinFontSize","pt":"msg","to":"0","tot":"num"},{"t":"set","p":"ui_control.options.labelMinFontSize","pt":"msg","to":"8","tot":"str"},{"t":"set","p":"ui_control.options.valueMinFontSize","pt":"msg","to":"150","tot":"str"},{"t":"delete","p":"payload","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":730,"y":320,"wires":[["2e0fa682.322dfa"]]},{"id":"71267751.ade568","type":"ui_ui_control","z":"e45173f3.bfc09","name":"","x":260,"y":320,"wires":[["b9fe6ae1.35dc28"]]},{"id":"22f605b1.a034aa","type":"comment","z":"e45173f3.bfc09","name":"To send options when dashboard connects","info":"","x":160,"y":360,"wires":[]},{"id":"b9fe6ae1.35dc28","type":"delay","z":"e45173f3.bfc09","name":"","pauseType":"delay","timeout":"40","timeoutUnits":"milliseconds","rate":"1","nbRateUnits":"1","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":false,"x":470,"y":320,"wires":[["928d686.a2e5598"]]},{"id":"c0a3c82e.b5fae","type":"comment","z":"e45173f3.bfc09","name":"Let gauge to be initialized before change any options","info":"There will be browser errors otherwise ","x":470,"y":400,"wires":[]},{"id":"956b6db1.656f88","type":"comment","z":"e45173f3.bfc09","name":"Change the options","info":"gauge scales according to predefined card size\ndefined font sizes are not absolute values but minimum values\nFor example if you make your gauge smaller, the changed font sizes may happen to be too large and will overlap.\n\nminLabelMinFontSize - minimum size of min field // number\nmaxLabelMinFontSize - minimum size of max field // number\nlabelMinFontSize - minimum size of units field // numbert\n\ndelete msg.payload cos it does not contain proper value","x":750,"y":360,"wires":[]},{"id":"5a1af3bc.7cd31c","type":"ui_group","name":"Standard","tab":"24a7075b.b1d0b8","order":1,"disp":false,"width":"8","collapse":false},{"id":"24a7075b.b1d0b8","type":"ui_tab","name":"Home","icon":"dashboard","disabled":false,"hidden":false}]

PS: thank you very much for the hint of justgage options you postet in the thread "'gauge' options suggestion"!

5

The previous was an example of appending data to a var, both had to be in the same message. But you could make hold array a context storage array and call it any where in the flow. You can use context storage or a DB or a file.
I have given an example of context storage and a file back up, you could also use the file to reload data. The hold array has become flow.shipData , and this data can be seen in the context window on the right.

[{"id":"9baaf213.817658","type":"inject","z":"129a88e7.1187ef","name":"Initiate shipData or clear","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[]","payloadType":"json","x":220,"y":180,"wires":[["8e44735.f42b41"]]},{"id":"8e44735.f42b41","type":"change","z":"129a88e7.1187ef","name":"","rules":[{"t":"set","p":"shipData","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":180,"wires":[[]]},{"id":"7251eaf9.2f69a4","type":"inject","z":"129a88e7.1187ef","name":"overwrite data","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"mmsi\":211667760,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"}, {\"mmsi\":211667590,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.5,\"longitude\":9.921221666666666,\"latitude\":53.543015,\"courseOverGround\":254.7,\"trueHeading\":275,\"timeStampSeconds\":17,\"date\":1621533256452,\"isoDate\":\"2021-05-20 17:54:16\",\"source\":\"!AIVDM,1,1,,A,139o:QP0000eJVrN`kLItpVR0H9T,0*17\\r\\n\"}, {\"mmsi\":211626350,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.1,\"longitude\":9.921696666666667,\"latitude\":53.543083333333335,\"courseOverGround\":293.2,\"trueHeading\":271,\"timeStampSeconds\":20,\"date\":1621533260219,\"isoDate\":\"2021-05-20 17:54:20\",\"source\":\"!AIVDM,1,1,,A,139laKP0010eJglN`kVcM8N`0<52,0*0E\\r\\n\"}]","payloadType":"json","x":170,"y":220,"wires":[["8e44735.f42b41"]]},{"id":"c88f786f.ec7d18","type":"comment","z":"129a88e7.1187ef","name":"can be linked to save to file","info":"You will need to edit file node for your use","x":610,"y":300,"wires":[]},{"id":"a7b7591d.dc3028","type":"json","z":"129a88e7.1187ef","name":"convert to string","property":"payload","action":"str","pretty":false,"x":640,"y":340,"wires":[["4a0644c7.cdbd3c"]]},{"id":"4a0644c7.cdbd3c","type":"file","z":"129a88e7.1187ef","name":"","filename":"shipData.json","appendNewline":true,"createDir":false,"overwriteFile":"true","encoding":"none","x":850,"y":340,"wires":[[]]},{"id":"58ea85fd.648084","type":"debug","z":"129a88e7.1187ef","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":530,"y":400,"wires":[]},{"id":"2eb876b6.f89f7a","type":"change","z":"129a88e7.1187ef","name":"append...","rules":[{"t":"set","p":"payload","pt":"msg","to":"$append([$.payload],$flowContext(\"shipData\")[mmsi != $$.payload.mmsi])","tot":"jsonata"},{"t":"set","p":"shipData","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":410,"y":340,"wires":[["58ea85fd.648084"]]},{"id":"4887d836.e6ed8","type":"inject","z":"129a88e7.1187ef","name":"newShip211667860","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"mmsi\":211667860,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"}","payloadType":"json","x":190,"y":340,"wires":[["2eb876b6.f89f7a"]]}]

Screenshot_20210521-221311
Screenshot_20210521-2213111920Ă—1080 209 KB

6

Thank you for the update. But I don't understand how you create the flow.shipData, when both of your inject-nodes create msg.payload? ... and the change-node sets flow.shipData to msg.payload... Does it create themself just from being used in this node?

Currently I am fighting on scripting the following into Java-language (? am I correct) used in JSONata? :

  • check if the incoming data (msg.payload from tcp->ais->) is in flow.shipData:
    --true: replace the object with matching "mmsi" (-> full object) "like update all values"
    --false: $append msg.payload to flow.shipData
    ~> find all objects in array with object.isoDate older, than 1 day and delete the object from flow.shipData

all I got for now is $contains(msg.payload.*.mmsi, msg.ship.*.mmsi) which I hope is the correct typing and gives me the expected boolean... I couldn't figure out how to show the boolean, my debug-node says "_msg.id: "121b8603.c06ada" " ... it might not be correct using the $contains-code in a change-node with JSONata, like you did? does the function-node work instead? ... well, I keep on searching, reading and trying meanwhile. thanks alot so far

7

I am goin to use simple objects to explain

msg.payload = [ ] ( an empt array).
if I set flow.shipData to the value of msg.payload.
flow.ShpData now has a value of [ ].
So if msg.payload = [{""mmsi":1},{"mmsi":2}] and I set that to shipData
shipData now has value of [{""mmsi":1},{"mmsi":2}]

so if I
set payload to {"mmsi":3}
then set payload to
$append([$.payload],$flowContext("shipData")[mmsi != $$.payload.mmsi])
payload will be [{""mmsi":1},{"mmsi":2},{"mmsi"3}]
then that is stored back to shipData by setting ShipData to the value of msg.payload.

This has no javascript involved or java

Now if you want to append and then remove objects where date(which is a timestamp) is older than a day you could use
$append([$.payload],$flowContext("shipData")[mmsi != $$.payload.mmsi and date > ($millis() - 24*60*60*1000)])
and after set to shipData

you could just remove old data by setting shipdata to
$flowContext("shipData")[date > ($millis() - 24*60*60*1000)]

example of deleting old data

[{"id":"8e44735.f42b41","type":"change","z":"129a88e7.1187ef","name":"","rules":[{"t":"set","p":"shipData","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":180,"wires":[[]]},{"id":"9baaf213.817658","type":"inject","z":"129a88e7.1187ef","name":"Initiate shipData or clear","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[]","payloadType":"json","x":220,"y":180,"wires":[["8e44735.f42b41"]]},{"id":"7251eaf9.2f69a4","type":"inject","z":"129a88e7.1187ef","name":"overwrite data","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"mmsi\":211667760,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621533253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"}, {\"mmsi\":211667590,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.5,\"longitude\":9.921221666666666,\"latitude\":53.543015,\"courseOverGround\":254.7,\"trueHeading\":275,\"timeStampSeconds\":17,\"date\":1621533256452,\"isoDate\":\"2021-05-20 17:54:16\",\"source\":\"!AIVDM,1,1,,A,139o:QP0000eJVrN`kLItpVR0H9T,0*17\\r\\n\"}, {\"mmsi\":211626350,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0.1,\"longitude\":9.921696666666667,\"latitude\":53.543083333333335,\"courseOverGround\":293.2,\"trueHeading\":271,\"timeStampSeconds\":20,\"date\":1621533260219,\"isoDate\":\"2021-05-20 17:54:20\",\"source\":\"!AIVDM,1,1,,A,139laKP0010eJglN`kVcM8N`0<52,0*0E\\r\\n\"}]","payloadType":"json","x":170,"y":220,"wires":[["8e44735.f42b41"]]},{"id":"478c38c5.4ed94","type":"change","z":"129a88e7.1187ef","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$flowContext(\"shipData\")[date > ($millis() - 24*60*60*1000)]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":480,"y":260,"wires":[["58ea85fd.648084"]]},{"id":"58ea85fd.648084","type":"debug","z":"129a88e7.1187ef","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":650,"y":240,"wires":[]},{"id":"2eb876b6.f89f7a","type":"change","z":"129a88e7.1187ef","name":"append item","rules":[{"t":"set","p":"payload","pt":"msg","to":"$append([$.payload],$flowContext(\"shipData\")[mmsi != $$.payload.mmsi ])","tot":"jsonata"},{"t":"set","p":"shipData","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":420,"y":340,"wires":[["58ea85fd.648084"]]},{"id":"4cfe920e.a531f4","type":"inject","z":"129a88e7.1187ef","name":"remove items older than a day","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":210,"y":260,"wires":[["478c38c5.4ed94"]]},{"id":"4887d836.e6ed8","type":"inject","z":"129a88e7.1187ef","name":"newShip211667860","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"mmsi\":211667860,\"navigationStatus\":\"Under way using engine\",\"rateOfTurn\":0,\"speedOverGround\":0,\"longitude\":9.920658333333334,\"latitude\":53.54305333333333,\"courseOverGround\":290,\"trueHeading\":268,\"timeStampSeconds\":14,\"date\":1621999253342,\"isoDate\":\"2021-05-20 17:54:13\",\"source\":\"!AIVDM,1,1,,A,139o;<0000PeJLFN`kR;E8HL0@7h,0*7A\\r\\n\"}","payloadType":"json","x":190,"y":340,"wires":[["2eb876b6.f89f7a"]]}]
8

Well, this was confusing to me: when you set flow.ShipData -> to -> payload (which is the empty array) it takes over the content of payload, whereas when you msg.payload = [{"mmsi":1},{"mmsi":2}] and set this -> to -> shipData, it's stored into shipData. But I'll take it just like this.

$append() I understood. Question to this: what does the "$" stand for in the position of $.payload or $$.payload.mmsi? for $append() and $flowContext() it looks like a marker for a function. Is it a wildcard? I thought that's the '*' ...

... so what kind of programming language is it then?

9

copy payload value to shipData. may be less confusing.

Jsonata is a JSON query and transformation language
https://jsonata.org/

All built in functions start with a $, also $ and $$ have json context hierarchy meaning

10

slowly, slowly, I start to understand... thanks a lot for your clarifications and explanations...

another question about "conditional logic". It's said " The expression predicate is evaluated. If its effective boolean value (see definition) is true then expr1 is evaluated and returned, ..."

is the following term correct?
(msg.payload[0].mmsi in msg.payload.mmsi) ? msg.payload[3].mmsi
because I get "undefined"
would it even be "more correct" (^^) to write
($$.payload[0].mmsi in $$.payload.mmsi) ? $$.payload[3].mmsi ?

[{"id":"7b2e02b8.38fd3c","type":"tab","label":"Flow 5","disabled":false,"info":""},{"id":"dee26336.6e9c4","type":"inject","z":"7b2e02b8.38fd3c","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"mmsi\":2311},{\"mmsi\":2312},{\"mmsi\":2313}]","payloadType":"json","x":150,"y":140,"wires":[["3da251b7.75530e","3151517e.7ca6fe","89e0748e.6ba028","89ddc99.dddddb8","c3535123.ee2578","de52fda5.f76f9"]]},{"id":"3da251b7.75530e","type":"debug","z":"7b2e02b8.38fd3c","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload.mmsi","targetType":"jsonata","statusVal":"","statusType":"auto","x":920,"y":140,"wires":[]},{"id":"3151517e.7ca6fe","type":"debug","z":"7b2e02b8.38fd3c","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"msg.payload[0].mmsi","targetType":"jsonata","statusVal":"","statusType":"auto","x":920,"y":200,"wires":[]},{"id":"5d4a5f27.303898","type":"debug","z":"7b2e02b8.38fd3c","name":"2311","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":910,"y":260,"wires":[]},{"id":"89e0748e.6ba028","type":"change","z":"7b2e02b8.38fd3c","name":"msg.payload[0].mmsi","rules":[{"t":"set","p":"payload","pt":"msg","to":"msg.payload[0].mmsi","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":620,"y":260,"wires":[["5d4a5f27.303898"]]},{"id":"89ddc99.dddddb8","type":"change","z":"7b2e02b8.38fd3c","name":"msg.payload.mmsi","rules":[{"t":"set","p":"payload","pt":"msg","to":"msg.payload.mmsi","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":610,"y":320,"wires":[["bda30143.0fcbd8"]]},{"id":"bda30143.0fcbd8","type":"debug","z":"7b2e02b8.38fd3c","name":"[2311,2312,2313]","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":950,"y":320,"wires":[]},{"id":"c3535123.ee2578","type":"change","z":"7b2e02b8.38fd3c","name":"msg.payload[0].mmsi in msg.payload.mmsi","rules":[{"t":"set","p":"payload","pt":"msg","to":"msg.payload[0].mmsi in msg.payload.mmsi","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":400,"wires":[["42f82205.5c0324"]]},{"id":"42f82205.5c0324","type":"debug","z":"7b2e02b8.38fd3c","name":"true","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":950,"y":400,"wires":[]},{"id":"de52fda5.f76f9","type":"change","z":"7b2e02b8.38fd3c","name":"msg.payload[0].mmsi in msg.payload.mmsi","rules":[{"t":"set","p":"payload","pt":"msg","to":"(msg.payload[0].mmsi in msg.payload.mmsi) ? msg.payload[3].mmsi","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":460,"wires":[["b85537ed.6211"]]},{"id":"b85537ed.6211","type":"debug","z":"7b2e02b8.38fd3c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":970,"y":460,"wires":[]}]
11

There is no msg.payload[3].mmsi that should be 2

($.payload[0].mmsi in $.payload.mmsi) ? $.payload[2].mmsi

12

Ah! Yes! Counting starts from 0 - missed that... sry

13

Do you think I can update the flow.shipData with the Transform Operator?

| flow.shipData[0].mmsi | {"mmsi":2113} |

My intention is to change the position [0] using a variable from a "find position of "mmsi":2133"
might it be the right direction of thinking?

14

Not sure what you are intending to do.
If you want to move index[2] to index[0] I would just append() it again.
I would use append() again
$append($.payload[mmsi = 2313], $.payload[mmsi != 2313])

also check
order by

15

The flow.shipData contains objects (ships), which can be identified by their "mmsi". The incoming AIS data form the tcp -> msg.payload is one ship as object {"mmsi: ... } It might not yet be in the flow.shipData (-> $append()) or it might already be in the flow.shipData (-> update the values .speedOverGround, etc.)

but I could try the following:
-> search the msg.payload.mmsi (the one ship object from tcp has only one entry) in the flow.shipData.mmsi (all entries from the past)
-> delete the object in the flow.shipData
-> $append() the new data (msg.payload)

I'll come back with my try... :slight_smile:

another question meanwhile: What does $flowContext() do exactly?

16

My original append() will replace the object with the same mmsi number.
the shipData array will only hold objects with unique mmsi by using [mmsi!=$$.payload.mmsi] so it calls the shipData and removes matching mmsi then appends new.
$flowContext() reads the the context storage i.e. flow.shipData

so $flowwContext("shipData")[mmsi!=$$.payload.mmsi]reads all the array minus the matching mmsi from the incoming new ship payload .

p.s. if the new ship is not in the shipData the append() will add it.
Also I am assuming that the new ASI object contains all the fields not just the fields that have updated.

17

Yep, yep, got it now! I had to look at it a veeeeery long time to understand it fully... :smiley:
It thought it just appends the payload to the shipData, if it's not in the list, but it's like merging both minus the entry of the matching mmsi.... got it now. Thanks alot!

Coming up with the full program later and maybe you can have a look at it...?

18

would love to.

19

I have to ask is this AIS api a local harbour thing or is there an online public api

1 Like
20

I am running a RaspberryPI with a DVB-T Stick (nooelec NESDR Mini 2+) which scans the signal on the AIS frequency (around 162.000MHz). Also using SDR VHF, SignalK and Node-RED to process the data. So it's public (in the air) and local (on my machine)... but I saw a video, how you can get such data form marinetraffic.com

Well, second last goal is to extract the .speedOverGround from only one ship (specific mmsi). I'll try the &lookup() function.

The final goal is to have a selectable dropdown list on the dashboard, with all mmsi form the shipData to input the mmsi, that shall be looked up (?? this sounds grammatically weird to me...)
This looks difficult to me, because the entries of mmsi in the shipData changes frequently and I haven't studied the nodes yet... maybe I just work with a text-input . I can get the mmsi from another monitor on the bridge...
My thinking is to store it in another flow. to use it in the $lookup() function...

flows.json (34.4 KB)