I am trying to use a software-defined radio receiver to get data from a wireless thermometer. The program that controls the radio sends a stream of JSON strings separated by newline characters to its standard output. (All of this is running on a Pi with Raspbian.) If I start the program with the exec node (in spawn mode) or use the daemon node, I almost always get two messages for each JSON string. The place where the string is broken is unpredictable. I have managed to use a join node to put the strings back together so that a json node can handle them, but I still have to trap occasional errors (from the json node) and restart the program if they persist. Is there something I should know about how exec and daemon work (specifically what triggers an output message) that would help me get a more reliable flow?
Show us the config of the exec node and an example of split messages
The exec node configuration is:
and typical output looks like this:
The transmitter sends a pair of messages every 30 seconds, and the program sends output like this to stdout (captured in a terminal session).
{"time" : "2019-04-30 14:02:53", "brand" : "OS", "model" : "OSv1 Temperature Sensor", "sid" : 11, "channel" : 1, "battery" : "OK", "temperature_C" : 23.100}
{"time" : "2019-04-30 14:02:53", "brand" : "OS", "model" : "OSv1 Temperature Sensor", "sid" : 11, "channel" : 1, "battery" : "OK", "temperature_C" : 23.100}
These strings parse properly when injected into a json node. I also tried the "Use old style output" checkbox, although I couldn't find documentation of what it does, and got similar results.
Any thoughts?
Show us the debug output for the final part of a split message
Sorry if my explanation wasn't as clear as it should be. You may have to click on or download the second image to see that it shows the output from two JSON strings split into four messages. Is this what you want?
Post your flow and use the inject node with the json/strings as input. (to emulate your device)
This is the guts of my flow, dashboard and error trapping nodes removed and an inject node added.
[{"id":"7db2a4ac.a0d8a4","type":"inject","z":"3705653e.c16b9a","name":"start","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":230,"y":220,"wires":[["c95a681c.9782e"]]},{"id":"c95a681c.9782e","type":"exec","z":"3705653e.c16b9a","command":"rtl_433 -F json","addpay":false,"append":"","useSpawn":"true","timer":"","oldrc":false,"name":"","x":400,"y":240,"wires":[["2f93e801.45b19","7973400d.e3bb08"],["7866c46e.e44bcc"],["6e907ec5.8cacd8"]]},{"id":"2f93e801.45b19","type":"debug","z":"3705653e.c16b9a","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":570,"y":220,"wires":[]},{"id":"7866c46e.e44bcc","type":"debug","z":"3705653e.c16b9a","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":570,"y":260,"wires":[]},{"id":"6e907ec5.8cacd8","type":"debug","z":"3705653e.c16b9a","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":570,"y":300,"wires":[]},{"id":"4ee12036.6d8bd8","type":"change","z":"3705653e.c16b9a","name":"","rules":[{"t":"set","p":"kill","pt":"msg","to":"","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":230,"y":260,"wires":[["c95a681c.9782e"]]},{"id":"384d2671.a3a73a","type":"inject","z":"3705653e.c16b9a","name":"stop","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":90,"y":260,"wires":[["4ee12036.6d8bd8"]]},{"id":"7973400d.e3bb08","type":"join","z":"3705653e.c16b9a","name":"","mode":"custom","build":"string","property":"payload","propertyType":"msg","key":"topic","joiner":"","joinerType":"str","accumulate":false,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":570,"y":180,"wires":[["5574a127.bbcbc","178e7d67.e33b8b"]]},{"id":"5574a127.bbcbc","type":"json","z":"3705653e.c16b9a","name":"","property":"payload","action":"","pretty":false,"x":730,"y":160,"wires":[["2fa0cd01.d029ca"]]},{"id":"2fa0cd01.d029ca","type":"debug","z":"3705653e.c16b9a","name":"object","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":850,"y":160,"wires":[]},{"id":"178e7d67.e33b8b","type":"debug","z":"3705653e.c16b9a","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":730,"y":200,"wires":[]},{"id":"ddb76cd6.bd8138","type":"inject","z":"3705653e.c16b9a","name":"string","topic":"","payload":"{\"time\" : \"2019-04-29 23:56:52\", \"brand\" : \"OS\", \"model\" : \"OSv1 Temperature Sensor\", \"sid\" : 11, \"channel\" : 1, \"battery\" : \"OK\", \"temperature_C\" : 12.800}","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":570,"y":140,"wires":[["5574a127.bbcbc"]]}]
As I said in my original post, the json node is happy to parse a string from the rtl_433 program.
With rtl-sdr you have to check the output data on the stream as it can be flaky, so before it goes into the json node, first check if it is valid json (the json node won't validate, but will generate an error that can be captured with the 'catch' node)
The exec node just outputs "chunks" of data as given to it by the operating system. We don't have any control over it - it depends how the app and OS decide to flush their buffers... often it is on a line break, or after a certain time, or number of characters, or if the app calls a flush.
In this case it does look like a deliberate flush as the break seems to be in the same place each time.
Are there any command line parameters you could give (eg to remove the timestamp) to see if it can be manipulated ? (you could re-add a timestamp locally if required).
Otherwise it may need some digging in the c source of the rtl_433 code.
Or you could maybe wrap it (ideally as a companion node to the rtl_power node - PR please ! 
Thanks, Dave. I suspected something of the sort. The break seems to come at almost but not exactly the same place each time -- I don't have the patience to count characters. I can redirect the output of rtl_433 to a file, close the file, and read hundreds of messages from it into a json node with no errors, so I think the hardware and software are pretty solid. I'm scratching my head for a way to pipe the data into NR without having to involve stdout in the exec node. There is also an option in rtl_433 to use mqtt, but I haven't been able to get it to work. The last thing I want to do is a contributed node, but... 
The solution may be a function node that buffers and concatenates payloads till it sees a CR then sends the message on.
Your topic sparked my interest (had a rtl-sdr dongle laying around), I can confirm the same behaviour. Especially when the reception is bad, when I move the antenna closer to my devices i get more consistent data.
Another option may be to use another output style that isn’t json. Then parse it yourself.
Thanks, all, for the helpful comments.
Thanks for confirming. I'm actually surprised at how well this stuff works, at least for me. I think in radio you get at most what you pay for, and these are really just toys.
I started with the default, plain text output, and it also has the "chunking" issue. Using JSON made it easier to see what is happening.
Doh! 
(why isn't there a head-slap emoji?) I was messing with finding the { } delimiters and sort of parsing JSON myself. This is dead easy and should work well enough. Thanks.
Another head slap. Would the split node do this for you ? (Not near computer to check)
Maybe not but would make a good PR to add it. 
Possibly, though at first glance I doubt it, because the payload I want is already split between two or more messages. I'll check. And thanks for the emoji -- I know it will come in handy.
@Colin's suggestion solves my immediate problem. I expect to play a bit more with sdr, so if I find a need to wrap rtl_sdr in a contributed node or modify the rtl_power node, I will start a new thread. Thanks again for all the help.
I'm having a similar issue with ffmpeg in an exec node only sending partial frames when converting rtsp streams into jpeg imags. Looking for CR or any specific character won't work for binary data buffers.
Do you mean you receive the data as multiple partial frames? So it is all there but split up?
