Contrib JSON node returns nothing when using jq engine

General
1

Hi,

Bit of a NR newbie...I simply want to use the jq engine but it returns nothing. I've setup the contrib-json node using the example in the node description. My simple flow uses an inject not to pass JSON in msg.payload. If I run the jq command in a shell, it works perfectly. Nothing logged even if log level is set to trace. My flow is as follows:

[
    {
        "id": "e8fb13e0.4b16f",
        "type": "tab",
        "label": "Flow 2",
        "disabled": false,
        "info": ""
    },
    {
        "id": "ebe1cdf6.44d1a",
        "type": "debug",
        "z": "e8fb13e0.4b16f",
        "name": "",
        "active": true,
        "tosidebar": true,
        "console": false,
        "tostatus": false,
        "complete": "false",
        "x": 880,
        "y": 440,
        "wires": []
    },
    {
        "id": "ab8a8d71.2802b",
        "type": "contrib-json",
        "z": "e8fb13e0.4b16f",
        "engine": "jq",
        "command": "/usr/bin/jq",
        "expr": ".foo[-1:] | {last: .[]}",
        "complete": "property",
        "prop": "payload",
        "name": "Run JQ",
        "x": 660,
        "y": 380,
        "wires": [
            [
                "ebe1cdf6.44d1a"
            ]
        ]
    },
    {
        "id": "613cc843.4e14e8",
        "type": "inject",
        "z": "e8fb13e0.4b16f",
        "name": "",
        "topic": "",
        "payload": "{\"foo\":[\"a\",\"b\",\"c\"]}",
        "payloadType": "json",
        "repeat": "",
        "crontab": "",
        "once": false,
        "onceDelay": 0.1,
        "x": 450,
        "y": 340,
        "wires": [
            [
                "ab8a8d71.2802b"
            ]
        ]
    }
]

Any ideas?

Thanks.

1 Like
2

Hi Peter,

I didnt know what jq engine is so i looked it up on the net.
I seems that it does filtering and processing on json ?
You should look into jsonata that is already included in Node-RED by default in a Change node by choosing expression.
This way you dont even need to install a node that hasnt been updated in the last 5 years.

Screenshot_2020-08-27 Node-RED 192 168 1 160
Screenshot_2020-08-27 Node-RED 192 168 1 160519Ă—690 12.8 KB

More information on jsonata https://jsonata.org/

Example in Flow that filters all phone 'numbers' from a complicated json (at any level **) and take the last one :

[{"id":"59a44afc.500bcc","type":"debug","z":"a9ccf79.74c3888","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":850,"y":280,"wires":[]},{"id":"2ac52666.ce1b9a","type":"inject","z":"a9ccf79.74c3888","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"{\"FirstName\":\"Fred\",\"Surname\":\"Smith\",\"Age\":28,\"Address\":{\"Street\":\"Hursley Park\",\"City\":\"Winchester\",\"Postcode\":\"SO21 2JN\"},\"Phone\":[{\"type\":\"home\",\"number\":\"0203 544 1234\"},{\"type\":\"office\",\"number\":\"01962 001234\"},{\"type\":\"office\",\"number\":\"01962 001235\"},{\"type\":\"mobile\",\"number\":\"077 7700 1234\"}],\"Email\":[{\"type\":\"office\",\"address\":[\"fred.smith@my-work.com\",\"fsmith@my-work.com\"]},{\"type\":\"home\",\"address\":[\"freddy@my-social.com\",\"frederic.smith@very-serious.com\"]}],\"Other\":{\"Over 18 ?\":true,\"Misc\":null,\"Alternative.Address\":{\"Street\":\"Brick Lane\",\"City\":\"London\",\"Postcode\":\"E1 6RF\"}}}","payloadType":"json","x":230,"y":280,"wires":[["8b51afda.ae8608"]]},{"id":"8b51afda.ae8608","type":"change","z":"a9ccf79.74c3888","name":"Filter last number","rules":[{"t":"set","p":"payload","pt":"msg","to":"[**.number][-1]","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":550,"y":280,"wires":[["59a44afc.500bcc"]]}]
2 Likes
3

Thanks for the info. I have looked at jsonata, but I need to be able to convert the JSON to LDJSON and there doesn't seem to be way to do that. jq does do that easily. For example
[
{ "identifier": "ABC",
"enabled":true},
{"identifier":"XYZ",
"enabled":true}
]

Would need to be converted to:

{"identifier":"ABC","enabled":true}
{"identifier":"XYZ","enabled":true}

4

Could something like this work ?

[{"id":"ff03a7b8.ce4bb","type":"function","z":"a9ccf79.74c3888","name":"","func":"let ldjson = \"\";\nlet arr = msg.payload;\n\narr.forEach(el => {\n    ldjson += JSON.stringify(el);\n})\n\nmsg.payload = ldjson;\n//ldjson = JSON.stringify(ldjson)\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":520,"y":240,"wires":[["f22b4ac3.e66b8"]]},{"id":"73d52dd6.24d33c","type":"inject","z":"a9ccf79.74c3888","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"identifier\":\"ABC\",\"enabled\":true},{\"identifier\":\"XYZ\",\"enabled\":true}]","payloadType":"json","x":200,"y":240,"wires":[["ff03a7b8.ce4bb"]]},{"id":"f22b4ac3.e66b8","type":"debug","z":"a9ccf79.74c3888","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":810,"y":240,"wires":[]}]

Function node:

let ldjson = "";
let arr = msg.payload;

arr.forEach(el => {
    ldjson += JSON.stringify(el);
})

msg.payload = ldjson;
//ldjson = JSON.stringify(ldjson)
return msg;
5

[
{ "identifier": "ABC",
"enabled":true},
{"identifier":"XYZ",
"enabled":true}
]
Would need to be converted to:
{"identifier":"ABC","enabled":true}
{"identifier":"XYZ","enabled":true}

The input appears to be an array, but the output cannot be a single message, as that wouldn't be json.
What should the output look like, 2 single messages, or a single string ?

6

Thanks all.

For line delimited JSON or JSON lines, multiple JSON objects are separated by newline characters. So whilst the input is a JSON array, the output must be the elements of the array as separate JSON objects on separate lines.

7

Hi Peter. A slight modification in the function node could do it.
I kept both examples

[{"id":"4030cede.8b7108","type":"function","z":"b48b096.a2f83f8","name":"","func":"let ldjson = \"\";\nlet arr = msg.payload;\n\narr.forEach(el => {\n    ldjson += JSON.stringify(el);\n})\n\nmsg.payload = ldjson;\n//ldjson = JSON.stringify(ldjson)\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":560,"y":280,"wires":[["ccb090d2.e67d18"]]},{"id":"9b2bc13e.72b8f8","type":"inject","z":"b48b096.a2f83f8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"identifier\":\"ABC\",\"enabled\":true},{\"identifier\":\"XYZ\",\"enabled\":true}]","payloadType":"json","x":240,"y":280,"wires":[["4030cede.8b7108"]]},{"id":"ccb090d2.e67d18","type":"debug","z":"b48b096.a2f83f8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":850,"y":280,"wires":[]},{"id":"c10ca6b0.e68eb","type":"comment","z":"b48b096.a2f83f8","name":"Array to String ","info":"","x":560,"y":220,"wires":[]},{"id":"7928a10f.9060a","type":"function","z":"b48b096.a2f83f8","name":"","func":"\nlet arr = msg.payload;\n\narr.forEach(ldjson => {\n// send seperate msg for each element in arr\n    node.send({payload: ldjson})  \n    \n})\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":580,"y":480,"wires":[["4f9296bc.621bd8"]]},{"id":"30c54655.8979f2","type":"inject","z":"b48b096.a2f83f8","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"[{\"identifier\":\"ABC\",\"enabled\":true},{\"identifier\":\"XYZ\",\"enabled\":true}]","payloadType":"json","x":260,"y":480,"wires":[["7928a10f.9060a"]]},{"id":"4f9296bc.621bd8","type":"debug","z":"b48b096.a2f83f8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":870,"y":480,"wires":[]},{"id":"956017e8.bb5e48","type":"comment","z":"b48b096.a2f83f8","name":"JSON objects on separate lines","info":"","x":630,"y":420,"wires":[]}]

ps. maybe you'll need a delay node after the function node if the msgs are being sent to fast.

8

Hi,

Slight modification to your script to add newline character seems to do the trick:

let ldjson = "";

payload.forEach(el => {
ldjson += JSON.stringify(el)+"\r\n";
})

msg.payload = ldjson;

Thanks for your help.

9

cool .. its always trial and error in these kind of situations .. unless its a nuclear power plant :wink:

10

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.