Unexpected behaviour when using jsonata expressions like payload[..][] in change node

I have faced some unexpected behaviour when using jsonata in change node.

The following flow demonstrates the problem I am having.
Note that the actual flows I am using are using more complex jsonata expression but I believe that the essence of my problem is already demonstrated in this simple flow.

[{"id":"bfd23e84.79e77","type":"tab","label":"problem scenario","disabled":false,"info":""},{"id":"8424eb57.b8ac88","type":"inject","z":"bfd23e84.79e77","name":"msg.payload = [\"one\"]","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[\"one\"]","payloadType":"json","x":160,"y":340,"wires":[["2b87b72f.485d98","67db77ad.286958","3fc7f139.b687de"]]},{"id":"2b87b72f.485d98","type":"change","z":"bfd23e84.79e77","name":"(payload)[]","rules":[{"t":"set","p":"output","pt":"msg","to":"(payload)[]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":340,"wires":[["db08c8a7.2315a8"]]},{"id":"db08c8a7.2315a8","type":"debug","z":"bfd23e84.79e77","name":"output test 2 (OK)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":930,"y":340,"wires":[]},{"id":"67db77ad.286958","type":"change","z":"bfd23e84.79e77","name":"payload[true][]","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload[true][]\t/* simple json expression retrieving all elements of array (condition is true)\t   and assuring that the output is an array */","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":420,"y":300,"wires":[["ed307b32.12e1a8","278a2192.ee6bfe"]]},{"id":"ed307b32.12e1a8","type":"change","z":"bfd23e84.79e77","name":"(payload)[]","rules":[{"t":"set","p":"output","pt":"msg","to":"(payload)[]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":300,"wires":[["894cf205.51753"]]},{"id":"894cf205.51753","type":"debug","z":"bfd23e84.79e77","name":"output test 1 (NOT OK)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":940,"y":300,"wires":[]},{"id":"278a2192.ee6bfe","type":"debug","z":"bfd23e84.79e77","name":"jsonata output","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":480,"y":260,"wires":[]},{"id":"3fc7f139.b687de","type":"debug","z":"bfd23e84.79e77","name":"test input","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":140,"y":280,"wires":[]}]

In the screenshot below you can see that the 2 nodes (payload)[] (both are using exactly the same jsonata expression) are getting exactly the same payload as input according to the debug tab but the output is different. So it seems to indicate that the change node payload[true][] is changing the payload in a way that is not visible in the debug tab.

What version of NR and node.js are you running?
When I run your flow this is what I see:

as you can see, both outputs are the same. I'm running NR v1.1.3 node.js v12.13.0

Thanks for testing.

I am using the latest docker version: nodered/node-red-dev:test-12
it is a more recent Node.js version than you:

> node-red-docker@1.1.3 start /usr/src/node-red
> node $NODE_OPTIONS node_modules/node-red/red.js $FLOWS "--userDir" "/data"
17 Sep 12:51:55 - [info] 
Welcome to Node-RED
===================
17 Sep 12:51:55 - [info] Node-RED version: v1.1.3
17 Sep 12:51:55 - [info] Node.js  version: v12.18.3
17 Sep 12:51:55 - [info] Linux 4.18.0-193.6.3.el8_2.x86_64 x64 LE
17 Sep 12:51:55 - [info] Loading palette nodes

So I imported this on one of my pi's running NR v1.1.3 and node.js vv12.18.2 and here is the debug:

same results as on my Mac. Both outputs are the same. A docker issue maybe??

... I have some difficulties believing that.

For the record - I have tried with older docker version and I am having exactly same outcome:

docker run -p 1882:1880 nodered/node-red:1.0.0-12
Unable to find image 'nodered/node-red:1.0.0-12' locally
1.0.0-12: Pulling from nodered/node-red
e7c96db7181b: Pull complete 
...
Digest: sha256:c1d79e80bfc4d914bc92b0dc1ef12a3f9afdacc3d8eca05064d0b8910364fa61
Status: Downloaded newer image for nodered/node-red:1.0.0-12
> node-red-docker@1.0.0 start /usr/src/node-red
> node $NODE_OPTIONS node_modules/node-red/red.js -v $FLOWS "--userDir" "/data"
17 Sep 13:51:34 - [info] 
Welcome to Node-RED
===================
17 Sep 13:51:34 - [info] Node-RED version: v1.0.0
17 Sep 13:51:34 - [info] Node.js version: v12.10.0
17 Sep 13:51:34 - [info] Linux 4.19.76-linuxkit x64 LE

Just tried it on my raspberry pi running node-red version v1.0.4 and I am getting the same issue.
So it is not a docker issue.

It is very strange that you get a different outcome while following configurations for me give all the same output:

  • macbook using multiple docker versions
  • intel nuc machine running docker on top of centos
  • raspberry pi running raspbian if I remember it well.

Wow this is very odd!!! I copied your flow again and see the issue (I think I played with the change node before I tested it originally)

So I added another change node in and put the same jsonata in and rant it - look at the results - one works the other doen't!!


Here is the flow. see f it works for you.

[{"id":"395fb251.fbbb8e","type":"tab","label":"problem scenario","disabled":false,"info":""},{"id":"f5d377d6.217758","type":"inject","z":"395fb251.fbbb8e","name":"msg.payload = [\"one\"]","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[\"one\"]","payloadType":"json","x":160,"y":340,"wires":[["feff8b53.647e08","2b60829d.22ec86","c52253f9.b9de"]]},{"id":"c52253f9.b9de","type":"change","z":"395fb251.fbbb8e","name":"(payload)[]","rules":[{"t":"set","p":"output","pt":"msg","to":"(payload)[]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":340,"wires":[["85ae7452.c10a5"]]},{"id":"85ae7452.c10a5","type":"debug","z":"395fb251.fbbb8e","name":"output test 2 (OK)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":930,"y":340,"wires":[]},{"id":"2b60829d.22ec86","type":"change","z":"395fb251.fbbb8e","name":"payload[true][]","rules":[{"t":"set","p":"payload","pt":"msg","to":"payload[true][]\t/* simple json expression retrieving all elements of array (condition is true)\t   and assuring that the output is an array */","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":420,"y":300,"wires":[["f4c28ddf.32c04","ab8a1079.c06ae","2aa4e4a9.08a014"]]},{"id":"f4c28ddf.32c04","type":"change","z":"395fb251.fbbb8e","name":"(payload)[]","rules":[{"t":"set","p":"output","pt":"msg","to":"(payload)[]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":690,"y":300,"wires":[["65c9c2ef.11d774"]]},{"id":"65c9c2ef.11d774","type":"debug","z":"395fb251.fbbb8e","name":"output test 1 (NOT OK)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":940,"y":300,"wires":[]},{"id":"ab8a1079.c06ae","type":"debug","z":"395fb251.fbbb8e","name":"jsonata output","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":540,"y":200,"wires":[]},{"id":"feff8b53.647e08","type":"debug","z":"395fb251.fbbb8e","name":"test input","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":140,"y":280,"wires":[]},{"id":"2aa4e4a9.08a014","type":"change","z":"395fb251.fbbb8e","name":"(payload)[]","rules":[{"t":"set","p":"output","pt":"msg","to":"(payload)[]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":700,"y":260,"wires":[["6c594c94.e2af3c"]]},{"id":"6c594c94.e2af3c","type":"debug","z":"395fb251.fbbb8e","name":"output test 1B (NOT OK)","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":950,"y":220,"wires":[]}]

I've expereinced cases where a node acts strange but a new version of the node works fine.

I am getting exactly same results as you when testing your flow.
Which is indeed very weird.

I even looked at the flows and I can't see anything different. Maybe @dceejay will chime in on why this might happen

I have also reported the following issue for this:

which has further been propagated to jsonata-js :