Lookup JSON response in another JSON response

#1

I'm extremely new to all of this and have a small issue to do with JSON

I am trying to build up a small dashboard for a satellite project. Data is held in a couple of JSON URL's. The first API is one called Observations, this holds information relating to ground stations and the observations it has scheduled (https://network.satnogs.org/api/observations/?id=&ground_station=33&satellite__norad_cat_id=&transmitter=&vetted_status=&vetted_user=&start=&end= for example is my ground station).The second API is called db (as in database) where specific data about the satellite is held, for example transmitter data (https://db.satnogs.org/api/transmitters/). The two are linked by a unique id reference, below is the flow as it is. I have highlighted the transmitter as it is in the first API, the same data is called uuid in the second API

And here in the Database

image

The crux of the question is how do I go about looking up the response from the flow from the first instance in the second API and so allowing me to return more useful information such as it's description. I can then use the same principle to use the NORAD id to return a satellite name?

It all sounds fairly straightforward, i.e. look something up somewhere and cross reference it against some other information. Any help would be appreciated.

#2

Hi @g7kse,

I can imagine the following solution.

A flow to read the JSON from the transmitters API and store the data (as JSON) in a global context.

A second flow that will read the JSON data from the observations API and will split each observation in a separate message. A change node with a JSONata expression will use the transmitter property from the split message to look up the stored JSON. The full transmitter data will be added to the message as the property "payload.transmitter_full".

This way you will have data from both JSON APIs together (under msg.payload) making easier to manipulate (extract) the relevant data.

Sounds, complicated right ? Try running the flow and explore the result in the debug tab.

r-02

[{"id":"87eb59d5.c9fa88","type":"tab","label":"SatNOGS","disabled":false,"info":""},{"id":"ae8c24c1.db8728","type":"inject","z":"87eb59d5.c9fa88","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":280,"wires":[["44ded748.9c4888"]]},{"id":"44ded748.9c4888","type":"http request","z":"87eb59d5.c9fa88","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://network.satnogs.org/api/observations/?id=&ground_station=33&satellite__norad_cat_id=&transmitter=&vetted_status=&vetted_user=&start=&end=","tls":"","proxy":"","authType":"basic","x":310,"y":280,"wires":[["eff5d47d.bbca48"]]},{"id":"6f1ecae6.6cc1c4","type":"inject","z":"87eb59d5.c9fa88","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":200,"wires":[["20a28e41.4b5d02"]]},{"id":"6f89bb3e.9eaa44","type":"debug","z":"87eb59d5.c9fa88","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":710,"y":200,"wires":[]},{"id":"20a28e41.4b5d02","type":"http request","z":"87eb59d5.c9fa88","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://db.satnogs.org/api/transmitters/","tls":"","proxy":"","authType":"basic","x":300,"y":200,"wires":[["3f2641f1.bab68e"]]},{"id":"3f2641f1.bab68e","type":"change","z":"87eb59d5.c9fa88","name":"","rules":[{"t":"set","p":"transmitters","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":500,"y":200,"wires":[["6f89bb3e.9eaa44"]]},{"id":"eff5d47d.bbca48","type":"split","z":"87eb59d5.c9fa88","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":480,"y":280,"wires":[["fec5ee3f.3f139"]]},{"id":"fec5ee3f.3f139","type":"change","z":"87eb59d5.c9fa88","name":"","rules":[{"t":"set","p":"payload.transmitter_full","pt":"msg","to":"$globalContext(\"transmitters\")[uuid= $$.payload.transmitter]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":280,"wires":[["eb014cf.21eb7b"]]},{"id":"eb014cf.21eb7b","type":"debug","z":"87eb59d5.c9fa88","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":890,"y":280,"wires":[]}]
1 Like
#3

Thanks @Andrei A really quick response. I think I follow you, I'll have a play a bit later and see where that takes me. I had myself really stuck with this and was thinking about tables, variables and all sorts but this looks easier

Alex

#4

This works really well for transmitters so thanks for that, it now makes it easy to set up a dshaboard and is exactly what I was after. I had a play with the satellite names but didn't manage to replicate this with the different source. Debug shows that the JSON data is coming in correctly but the information is not appended as before.

I've had a play but can't seem to spot the error. Once I have this sorted I will try and get both sources into a simple flow.

Really enjoying the Node-RED way of doing stuff, quite easy to follow after a bit of a play around

[{"id":"8f21245d.58fd7","type":"tab","label":"SatNOGS","disabled":false,"info":""},{"id":"947c3a88.05ef58","type":"inject","z":"8f21245d.58fd7","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":280,"wires":[["e8f83218.c8b258"]]},{"id":"e8f83218.c8b258","type":"http request","z":"8f21245d.58fd7","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://network.satnogs.org/api/observations/?id=&ground_station=33&satellite__norad_cat_id=&transmitter=&vetted_status=&vetted_user=&start=&end=","tls":"","proxy":"","authType":"basic","x":310,"y":280,"wires":[["81a8d789.6f4fa8"]]},{"id":"c3436597.ce4758","type":"inject","z":"8f21245d.58fd7","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":200,"wires":[["540ea906.36a11"]]},{"id":"4d330d34.03a324","type":"debug","z":"8f21245d.58fd7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":710,"y":200,"wires":[]},{"id":"81a8d789.6f4fa8","type":"split","z":"8f21245d.58fd7","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":480,"y":280,"wires":[["fbcc3f21.8aeea8"]]},{"id":"1b907627.e47952","type":"debug","z":"8f21245d.58fd7","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":890,"y":280,"wires":[]},{"id":"540ea906.36a11","type":"http request","z":"8f21245d.58fd7","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://db.satnogs.org/api/satellites/","tls":"","proxy":"","authType":"basic","x":310,"y":200,"wires":[["b3f7a3a7.b0ac5"]]},{"id":"b3f7a3a7.b0ac5","type":"change","z":"8f21245d.58fd7","name":"","rules":[{"t":"set","p":"satellites","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":200,"wires":[["4d330d34.03a324"]]},{"id":"fbcc3f21.8aeea8","type":"change","z":"8f21245d.58fd7","name":"","rules":[{"t":"set","p":"payload.satellite_full","pt":"msg","to":"$globalContext(\"satellites\")[norad_cat_id= $$.payload.satellites]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":280,"wires":[["1b907627.e47952"]]}]

#5

You were very close. Looks like observations and satellites data structures use a coincident key name: norad_cat_id. Try using below config in the flow you created.

1 Like
#6

Brilliant...Tnx again @Andrei. It had me stumped for a couple of hours trying this and that. Now for a swanky dashboard :wink:

1 Like
#7

and this flow add both: transmitter and satellites to each observation.

[{"id":"d8d2b8b1.4b6988","type":"tab","label":"SatNOGS-3","disabled":false,"info":""},{"id":"ab23037b.cde82","type":"inject","z":"d8d2b8b1.4b6988","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":280,"wires":[["99f77fd4.74041"]]},{"id":"99f77fd4.74041","type":"http request","z":"d8d2b8b1.4b6988","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://network.satnogs.org/api/observations/?id=&ground_station=33&satellite__norad_cat_id=&transmitter=&vetted_status=&vetted_user=&start=&end=","tls":"","proxy":"","authType":"basic","x":330,"y":280,"wires":[["642bf6ce.2f0c38"]]},{"id":"75b54a8f.4ca984","type":"inject","z":"d8d2b8b1.4b6988","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":"0.5","x":150,"y":200,"wires":[["92584431.b72438"]]},{"id":"77572136.b36d8","type":"debug","z":"d8d2b8b1.4b6988","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":730,"y":200,"wires":[]},{"id":"642bf6ce.2f0c38","type":"split","z":"d8d2b8b1.4b6988","name":"","splt":"\\n","spltType":"str","arraySplt":1,"arraySpltType":"len","stream":false,"addname":"","x":500,"y":280,"wires":[["6756dd9e.2914e4"]]},{"id":"eed749fb.6857f8","type":"debug","z":"d8d2b8b1.4b6988","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","x":930,"y":280,"wires":[]},{"id":"92584431.b72438","type":"http request","z":"d8d2b8b1.4b6988","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://db.satnogs.org/api/satellites/","tls":"","proxy":"","authType":"basic","x":330,"y":200,"wires":[["3bc6f88f.57cf48"]]},{"id":"3bc6f88f.57cf48","type":"change","z":"d8d2b8b1.4b6988","name":"","rules":[{"t":"set","p":"satellites","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":540,"y":200,"wires":[["77572136.b36d8"]]},{"id":"6756dd9e.2914e4","type":"change","z":"d8d2b8b1.4b6988","name":"","rules":[{"t":"set","p":"payload.satellite_full","pt":"msg","to":"$globalContext(\"satellites\")[norad_cat_id= $$.payload.norad_cat_id]","tot":"jsonata"},{"t":"set","p":"payload.transmitter_full","pt":"msg","to":"$globalContext(\"transmitters\")[uuid= $$.payload.transmitter]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":670,"y":280,"wires":[["eed749fb.6857f8"]]},{"id":"737f1a11.320a14","type":"inject","z":"d8d2b8b1.4b6988","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":0.1,"x":150,"y":160,"wires":[["d614c56e.3bc548"]]},{"id":"126302fb.bbc0dd","type":"debug","z":"d8d2b8b1.4b6988","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":730,"y":160,"wires":[]},{"id":"d614c56e.3bc548","type":"http request","z":"d8d2b8b1.4b6988","name":"","method":"GET","ret":"obj","paytoqs":false,"url":"https://db.satnogs.org/api/transmitters/","tls":"","proxy":"","authType":"basic","x":320,"y":160,"wires":[["731ca76f.62e938"]]},{"id":"731ca76f.62e938","type":"change","z":"d8d2b8b1.4b6988","name":"","rules":[{"t":"set","p":"transmitters","pt":"global","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":160,"wires":[["126302fb.bbc0dd"]]}]

It is a pitty that the satellite image is not always available. That would make the dashboard awesome.

This project seems really nice. I feel tempted to build my own ground station. If I can use a discone antena that is lying around then it is missing only to find a RTL SDR dongle.

#9

It is a great project. I have had a ground station for a couple of years now using a NooELEC SMArt RTL-SDR. It cost around $20 from eBay. It is really just a $5 device with a TXCO and an sma coax connector in a metal box. The performance is what you would expect from such a device.

I can't see a discone giving you much though. Is it not vertically polarised? If you are really up for it then I have a spare turnstile antenna that I originally used. PM me your details and I'll get it sent. This will give you better results overhead. I changed it for a Wimo TA-1 about a year ago and that gives a slight reduction overhead but improved low angle reception.

A long time ago I built the 3d printed rotator but found the control side very complex and prone to failure. Recently I've moved it from the custom board to a Arduino Uno and CNCshield which is far easier and cheaper. I wrote the wiki page to show that it was much easier to do it that way and also it gives independence. (I can only post 2 links so search wiki.satnogs)

After all of that the static antenna, Nooelec SMArt and a turnstile gives great results. It can be 100% utilised without any danger of it breaking down. The UHF station is giving very poor results though.

2 Likes
#10

Hello @g7kse,

Indeed, I read somewhere that the discone antenna would not be a good choice for this case. Mine is an ICOM AH7000 (VHF-UHF , 25 ~ 1300 MHz, omnidirectional ). This antenna and an old Hoka code 3 decoder was everything left from the time I was active scanning RF. I will take some time to study the SatNOGs project before taking the decision to build something. In the meantime let me know if I can help with the dashboard challenge. What you plan to develop is something similar to https://dashboard.satnogs.org ?

#11

I was really aiming a bit lower (sort of matched my skills :wink:). I was going to aim to re use a discarded kindle as an epaper display for the local groud station, perhaps build on that.

The SatNOGS dashboard is way more complex as ground stations have, up until quite recently, only provided demodulated data from the sdr. there has been a load of effort to be able to reliably decode that data and then provide telemetry dashboards for some of the newer cubesats. The data comes in loads of formats like CW (morse code), FSK, AFSK etc and it is quite complex.

As a project it is moving very quickly and t starting to become a useful tool. It is a bit like Piaware that is used for aircraft ADSB tracking but a whole load fewer stations at the moment