Apply transformation on nested object fields (based on dot notation)

Hi folks,

I am searching for a way to transform easily nested object properties, for one of my new nodes.

For example the input message of my node could be like this:

{
   "payload": [
    {
      "someProperty": "A",
      "someOtherProperty": "B"
    },
    {
      "someProperty": "C",
      "someOtherProperty": "D"
    }
  ]
}

On the config screen, the user should be able to specify which fields need to be transformed. For example (but other syntax might also do the job...):

  • "payload[1].someProperty means that the transformation only needs to be applied to the second element of the array. Which means that value "A" needs to be transformed to "E":
    {
       "payload": [
        {
          "someProperty": "E",
          "someOtherProperty": "B"
        },
        {
          "someProperty": "C",
          "someOtherProperty": "D"
        }
      ]
    }
    
  • "payload[].someOtherProperty means the transformation needs to be applied to all the elements in the array. Which means value "B" and "D" both need to be transformed to "G" and "H":
    {
       "payload": [
        {
          "someProperty": "A",
          "someOtherProperty": "G"
        },
        {
          "someProperty": "C",
          "someOtherProperty": "H"
        }
      ]
    }
    

I need to support multiple levels of nesting, because the property (that needs to be transformed) can be anywhere inside the complex input message object. The input can by any kind of object ...

Have been looking at many libraries on NPM, but can't find what I need. I just want to specify on which nested fields of an object my transformation function needs to be executed.

Has anybody seen a library that can accomplish like that?

Thanks !!!!
Bart

I'd start with jsonata given that is built-in to Node-RED and is a library for transforming JSON structured data.

Hey Nick,
Yes that indeed has popped up in my head already. So I had a quick look at jsonata yesterday. But I have to be honest that it is ages ago that I used jsonata, and now it has become one big cryptogram for me ...

I have in my config screen:

  1. A dropdown list containing 4 transformations, from which the user has to choose.
  2. A input field where the user can specify on which fields (in the input message) the selected transformation has to be executed.

But it is not clear to me how to accomplish something like that with jsonata...

Ah I now see that Jsonata has a ~> (transform) operator. So I assume I need to use that one ...

I'm very interested in how you solve this. In one of my nodes I needed to find a particular key-value pair buried in an object of arbitrary structure. I was able to do it (barely) using the JS Object.getOwnPropertyNames() function and the NR RED.util.compareObjects() function, but I am sure that there has to be a better way using JSONata.

[EDIT] It just dawned on me that @BartButenaers and I are talking about custom nodes. @knolleary, is it possible to use JASONata in the JS file of such a node?

Hey @drmibell,

Well you can use jsonata fairly easily in a custom node (see here).

However I am not sure whether I can use it to transform an object via a custom transformation call back function. I would like to have Jsonata to call my transformation callback function, for all the object properties that match the dot pattern. Because I want to have the transformation logic in my Javascript cod, and not inside the Jsonata expression.

P.S. Long long time ago I have asked Andrew Coleman whether he could add json manipulation to jsonata. In that discussion you will see that a.o. @shrickus were discussing such a callback function (for transform/replace). But it is not clear to me at the moment whether it has been implemented or not...

Thanks for that. My code is working now, but I'll give JSONata a try just to see if I can expand my toolbox. If I can keep track of what is a JS object and what is a JSON string, it might be fun.

Then you might also have a quick look at their Javascript API. I see that they also offer a registerFunction, so that might be what I am looking for. Will try it...

That seems to do the job. My callback function is nicely called with the required input, and my output is used by Jsonata. Case closed :+1:

2 Likes