Json Data Mapping

Ryanshafer · 25 August 2022 15:10

Hi,

I currently have a function node that maps this nasa data to an object for use later in the flow. I'm not totally happy with it because it occasionally throws an error. I could add some better error handling, but I was wondering if there was a more node-red way than a function node to convert this JSON data into an object, like so:

{
   "des":"2022 OU1",
   "orbit_id":"6",
   "jd":"2459817.400619856",
   "cd":"2022-Aug-25 21:37",
   "dist":"0.138855530107527",
   "dist_KM":"20772520",
   "dist_min":"0.13885499498311",
   "dist_max":"0.13885606523237",
   "v_rel":"7.62854662977278",
   "speed":"8km/s",
   "v_inf":"7.62603081310743",
   "t_sigma_f":"< 00:01",
   "h":"22.03",
   "size_min":"104",
   "size_max":"233"
}

Thanks for any help!

ScheepersJohan · 25 August 2022 15:25

Try change node.

TotallyInformation · 25 August 2022 15:41

Realistically, there are two methods to transform one JSON structure into another:

Use a change node with JSONata
Use a function node.

Occasionally, you might find a (2.a) which would be to use a node.js module with a function node if you can find a module that does the bulk of the processing you want.

JSONata is great if you can work out the transform. I generally recommend giving it 10 minutes, if you haven't worked it out by then, you should go back to a function.

Oddly enough, a function node is often less processing overhead anyway.

Sometimes though, you get lucky and one of the JSONata guru's takes an interest and posts an answer

Other times, not so much.

hardillb · 25 August 2022 15:46

It would probably help if you showed us what the error is

Colin · 25 August 2022 15:48

Did you start by sending it through a JSON node to convert it to a JavaScript object?

Ryanshafer · 25 August 2022 16:00

Thanks, I can sort that out myself, I'm not looking for help with the function node. I'm looking to see if there is a more node-red way of doing it.

E1cid · 25 August 2022 16:01

Why? The http request node can be set to output a json object.

@Ryanshafer You don't say if you parse all the arrays or if you after a specific one.
here is an example of using Javascript and JSONata to map all the values to an array of objects.

[{"id":"88aeec8c.8e386","type":"inject","z":"9a86d59e88442f2c","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"<?xml version=\"1.0\" encoding=\"utf-8\" ?> <!DOCTYPE data [  <!ELEMENT data       (parameter+)>  <!ELEMENT parameter  (name,value)>  <!ELEMENT name       (#PCDATA)>  <!ELEMENT value      (#PCDATA)> ]> <data>    <parameter>    \t<name>DB_L2174_ALARMS</name>    \t<value><![CDATA[0000000000000111000000000000000]]></value>    </parameter>    <parameter>    \t<name>DB_DEMOD_LICENCE_OPTIONS</name>    \t<value><![CDATA[1110010000000000000001000000000000000000]]></value>    </parameter>    <parameter>    \t<name>DB_DECOD_LICENCE_OPTIONS</name>    \t<value><![CDATA[0000000000000000000000000000000000000000]]></value>    </parameter>    <parameter>    \t<name>DB_DEMOD_SELFTEST_RESULTS</name>    \t<value><![CDATA[100000000000000000000000000000000]]></value>    </parameter>    <parameter>    \t<name>DB_L2174_RX_MODE</name>    \t<value><![CDATA[DVBT]]></value>    </parameter>    <parameter>    \t<name>DB_L2174_FREQ1</name>    \t<value><![CDATA[2.030000]]></value>    </parameter>    </data>","payloadType":"str","x":70,"y":300,"wires":[["e129cb3.1181238"]]},{"id":"e129cb3.1181238","type":"http request","z":"9a86d59e88442f2c","name":"","method":"GET","ret":"obj","paytoqs":"ignore","url":"https://ssd-api.jpl.nasa.gov/cad.api?date-min=now&dist-max=0.2","tls":"","persist":false,"proxy":"","authType":"","x":240,"y":300,"wires":[["72662c0b.42bec4","df71d57e.fe9248"]]},{"id":"72662c0b.42bec4","type":"change","z":"9a86d59e88442f2c","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$$.payload.data@$a.($$.payload.fields#$i.${$: $a[$i]})","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":420,"y":400,"wires":[["995d7e86.aea2f"]]},{"id":"df71d57e.fe9248","type":"function","z":"9a86d59e88442f2c","name":"","func":"msg.payload = msg.payload.data.map(arr => {\n    let obj = {};\n    msg.payload.fields.forEach((key, index) => {\n        obj[key] = arr[index];\n    })\n    return obj;\n})\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","libs":[],"x":450,"y":280,"wires":[["9e47108c.90d0f8"]]},{"id":"995d7e86.aea2f","type":"debug","z":"9a86d59e88442f2c","name":"jsonata","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":650,"y":400,"wires":[]},{"id":"9e47108c.90d0f8","type":"debug","z":"9a86d59e88442f2c","name":"javascript","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","targetType":"msg","statusVal":"","statusType":"auto","x":630,"y":280,"wires":[]}]

Javascript

msg.payload = msg.payload.data.map(arr => {
    let obj = {};
    msg.payload.fields.forEach((key, index) => {
        obj[key] = arr[index];
    })
    return obj;
})
return msg;

JSONata

$$.payload.data@$a.($$.payload.fields#$i.${$: $a[$i]})

Ryanshafer · 25 August 2022 16:05

Yea, but the response from the API is designed to save space, so you get a JSON response with all the fields list once and then all the data as separate items without key/value pairs. So I have to construct the final object.

{
   "signature":{
      "source":"NASA/JPL SBDB Close Approach Data API",
      "version":"1.4"
   },
   "count":"181",
   "fields":[
      "des",
      "orbit_id",
      "jd",
      "cd",
      "dist",
      "dist_min",
      "dist_max",
      "v_rel",
      "v_inf",
      "t_sigma_f",
      "h"
   ],
   "data":[
      [
         "2022 OU1",
         "6",
         "2459817.400619856",
         "2022-Aug-25 21:37",
         "0.138855530107527",
         "0.13885499498311",
         "0.13885606523237",
         "7.62854662977278",
         "7.62603081310743",
         "< 00:01",
         "22.03"
      ]

Ryanshafer · 25 August 2022 16:06

Thanks, I'll take a look. But can the change node+JSONata iterate over hundreds of items to make one object? I haven't ever used it that way.

Ryanshafer · 25 August 2022 16:08

I'm after all of them. Thanks I'll take a look!

TotallyInformation · 25 August 2022 16:09

Yes, it can. If you have a PhD in 10 dimensional quantum physics!

In other words, it is capable of it but working it out is likely to give you a splitting headache and a desire to retreat to a darkened room.

Doing it with JS is simple and almost certainly faster to write, faster to run and much easier to grok in 6 months time should you need to change it.

Colin · 25 August 2022 16:17

Even better. I asked because the data was shown as JSON not a JavaScript object, it was not clear that the OP knew that he should start by converting it.

hardillb · 25 August 2022 20:23

Yes, but knowing what the error is may be a result of variations in the data that would have an impact on what other approaches to take.

system · 24 October 2022 20:23

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Mapping a specific data of an array of properties in JSON General	10	1329	24 April 2021
Trying to use jsonata to reduce array of objects. Test data works OK but fails when deployed General	3	279	2 October 2022
Prepare array of JSON-Objects with jsonata through change-node for node-red-dashboard multiline-chart General node-red-dashboard , chart	7	991	4 January 2022
Need a jsonata magician! General	15	442	31 May 2022
Parsing Payload General function-node	15	643	17 February 2023

Json Data Mapping

Related topics