The use of JSONata

Hi everyone,
I am keen to know about your experience using Jsonata with Node-RED.
I read the documentation back to front many times and since JSON manipulations is a must, I learned it only to use with Node-RED.. it is very powerful and can do lots. The issue is the learning curve is way too high, the language in my humble opinion is not anywhere from intuitive, too vague and can be even annoying ! For the life of me , no where I can read my query and remember what I am doing ! :slight_smile:
This could just be me but I looked elsewhere, and discovered JMESPath https://jmespath.org/
The experience is much better, does the same things but way more intuitive and friendly - easier to remember, etc.
So my question is really what is everyone thinking ? do you make much use of the Jsonata expressions ? and did you arrive at any conclusions or use a different approach for JSON manipulation in Node-RED.
Is there any possible plans to incorporate a different library such as JMESPath with Node-RED , perhaps even as a choice? I know I can simply install and import the library but I am asking whether there could be tight integration , similar to Jsonata

I suppose it all depends on your current and earllier experiences. Have you done any programming at all and in that case, what languages have you programmed in? SInce you ask "everyone" I can answer for my myself; I do sometimes use JSONata in rare occasions, but most of the time, if I need to write some code, I use javascript in function nodes. Or if I really need or want to go outside the box, I write separate Python code modules and interface them with Node-RED using MQTT

I can agree, I think JSONata is maybe not as intuitive as other languages, I also felt it is a bit limited in comparison but it might be me not grasping the full potential

My personal favourite is still Python as nbr one and then Javascript is okey

1 Like

I am using JSONata a lot in Node-RED. It is very powerful extension of Node-RED. Thanks to JSONata expressions I am hardly making any use of function nodes. I am a big fan of declarative/descriptive programming versus the procedural (= javascript in function node) programming. I think JSONata stands to JSON as RegEx stands to strings or XSLT / XPath stands to XML. It are all very powerful query/expression languages that require some investigation which according to me are worth investing it.

I agree that there is some learning curve. I have spend several hours on the site https://jsonata.org/ . Understanding the JSONata processing model (https://docs.jsonata.org/processing) helped me a lot : JSONata is mainly about transforming sequences/arrays of JSON objects.

What is very nice about JSONata is that you can immediately validate your JSONata expression in the test tab of the node-red change node or at https://try.jsonata.org/. Maybe this is one of the things I most loved about it.

The main drawback of JSONata is its performance on very big JSON objects (arrays with 1000 of elements). Those things most likely could be done much faster by a function node. That said in nearly all case (if not all cases) it was OK for me to be a bit more patient compared to rewriting the logic in a function node.

I don't know JMESPath https://jmespath.org/. So I can not speak for that. I wonder if it is equally powerful as JSONata ?

2 Likes

I only use JSONata for simple cases as I also do not find it intuitive in use. I have never got on well with declarative languages and much prefer procedural. I think it is down to how one's brain works. Some seem to get on well with languages like JSONata and others don't. So I stick to languages that seem to slot easily into my brain pathways and don't lose any sleep over not being able to easily grasp those that don't. Luckily, being retired now, I don't find myself having to use tools that I don't enjoy.

2 Likes

One of the most important statements made for us enjoying the senior level in life

5 Likes

100% agree.

That goes for home stuff too - I do enjoy developing for node-red and using node-red, I just dont enjoy JSONata :man_shrugging:


@nileio to answer "I am keen to know about your experience using Jsonata" - my experience is this...

  • JSONata is more difficult to grasp than JavaScript (for me & seemingly others too)
  • Is definitely slower (literally hundreds of times slower than JS in some cases) - but as @janvda correctly points out, this is usually not an issue unless processing lots of data.
  • JS has something like 12million+ active developers to call upon when you get stuck. I have no idea of the popularity of JSONata but to try an put some perspective on it, I would argue it is less than Python (9m active devs) or C/C++ (6m active devs) possibly even RUST (0.8 million active devs). (ref)

My point is...

If I am to invest my limited time learning something, I am going to evaluate things like support, popularity, speed, capability, ease of use.

Final thoughts...

All that said, I do use JSONata - often - it comes in VERY handy in MANY places in node-red (I am glad we have the option & would not want to lose it). But if the task is more difficult than making an object out of some fields in msg - or - concatenating some strings - or - some simple maths, I jump to JS (others have to support my work).

Hope that helps in some way?

1 Like

Regarding the support.

If you want some help with some JSON object conversions then I would recommend to post an issue in this forum specifying:

  • description of the conversion you want
  • actual input example (in text format)
  • corresponding expected output

then my experience is that you will very quickly receive a response (either in JSONata or javascript).

I love to do simple and complex conversions in JSONata (using https://try.jsonata.org/ ) similarly as other people love to do sudoku's or crossword puzzles. So you can always tag my name @janvda in such posts ! :slight_smile:

7 Likes

@janvda thanks for sharing your thoughts. I totally agree it is a very powerful language, I use it a lot too. The only issue for me is result structure, operators , etc.
JMESPath is pretty much the same but it feels more easier to grasp , specially very easy to define how you want the result structure to look like

It does help thank you for sharing. I actually dont like to have so many function nodes in my flow so I use it often too, primarily for some nice higher order functions (sift, map , reduce , etc.). I will possibly start using JMESPath just so I can see how much of difference in experience I would get and perhaps feel more at home

great to know thanks .. will try the forum here when I get stuck with those expressions

Just to add my two cents/pence - I've used JSONata for a few very simple cases, but for anything reasonably complex a function node is much quicker to write and easier to maintain.

I've been coding for over 50 years in almost every commonly-used language, and in a variety of application environments, but like others I find JSONata quite hard to get to grips with. I'm sure if I put in a bit of time really learning it I could master the intricacies, but as I don't need much in the way of data manipulation in my project I'll likely stick to JS functions.

1 Like

I can agree with all that has been said.

JSONata is very much like XLST in that it is a transformative language so it can be fantastic for translating an object from one structure to another with minimal code.

So I reach for it when I need to do that. However, I tend to timebox myself because beyond a certain point, even if you figure out the transformation, you will struggle mightily if you ever need to come back and amend it in the future.

I also reach for it when I need to do a quick manipulation of data (e.g. translate a topic, adding in a value from the payload) when It is much quicker to add a change node with JSONata than using a full function node. Here is an example, using the random number generator to quickly create dynamic test data:

I think that JMESPath is designed more as a query tool, more like XPath (a subset of XLST)? It doesn't seem to have the higher-order transformations that JSONata does.