Feature request: mqtt-in node converts JSON to JS object

[drmibell]

I would be surprised if this has not been suggested before, but I have not been able to find anything. I would like to see an option in the mqtt in node to convert an incoming JSON string to a JavaScript object. My most frequent use of mqtt is to transfer objects, so the mqtt in node has to be followed by a JSON node. The mqtt out necessarily translates ojects to JSON automatically, and it makes sense to get that object back from the receiving node. There may cases where you want the JSON string, so the conversion should be optional. Does this make sense? Have I missed some earlier discussion?

[dceejay]

Another of the thoughts behind Node-RED is that nodes should do their thing well and not try to go everything. So the mqtt node shoul do mqtt and the json node will do json.

Otherwise nodes end up with loads of options to handle things that aren’t really to do with that nodes capability.

[hotNipi]

MQTT can carry any sort of data. It is not always JSON.

[drmibell]

That makes sense in principle, but by the same argument the mqtt out node should accept only strings and require a JSON node ahead of it. The relationship between JSON and mqtt is a bit stronger than you imply.

[drmibell]

MQTT can carry any sort of data. It is not always JSON.

True. It is not even always UTF-8 encoded strings. My suggestion relates to JavaScript objects. When the mqtt out node receives one, it automatically encodes it as a JSON string. If you know at the receiving end that you expect an object, I don't see why you can't ask to have it decoded.

[dceejay]

welll the mqtt out node can only send buffers so it has to convert whatever it gets into a set of bytes to put on the wire. There is nothing to say what they are.

[drmibell]

the mqtt out node can only send buffers

Understood. So the decision was made to encode objects as JSON for transmission. That makes sense, but basically it is an arbitrary design choice. Some other encoding COULD have been used, although JSON is the obvious one, built for the purpose.

Obviously, I can go on using lots of JSON nodes, but it feels like undoing self-inflicted damage. As far as I know, none of the other communication nodes (serial, tcp, etc.) receive anything different than what was transmitted.

[Andrei]

I can't test now so have to ask if this node perhaps is designed to do what you proposed: node-red-contrib-mqtt-json

[drmibell]

Thanks, @Andrei. Could be. Not much documentation , and no discussion that I can find. I will try it out and see.

[dceejay]

nothing is encoded as json for transmission - MQTT just handle bytes - so if anything it's the other way round - hence why they come out again as strings on receive.

However - (having now actually looked at the MQTT in node code for v0.20).... we do now specify some specific decodes - so actually adding JSON is not the worst

Add output type to mqtt node

[cflurin]

I would be surprised if this has not been suggested before, but I have not been able to find anything. I would like to see an option in the mqtt in node to convert an incoming JSON string to a JavaScript object.

Hue Magic - Dashboard slider for brightness ?!

BTW a similar setting for the mqtt-in node would be helpful.

[drmibell]

Thanks for looking into this. Obviously, I think it would be good to add a parsed JSON object option to the output choices in v0.20.

BTW, it's interesting how your thinking can be affected by the level of abstraction you usually deal with in a system like NR. I "know" in the back of my head that under the covers everything in the core of NR is just bytes. Still, a casual user, or even an author of custom nodes, can find it a lot easier to deal with higher-level constructs, especially when the infrastructure makes it easy to move between levels by giving you the sort of thing you need when you need it.

[dceejay]

Coming soon to an MQTT node near you

[drmibell]

[Colin]

nothing is encoded as json for transmission - MQTT just handle bytes

Is that true for a number? Does it send the byte pattern of the number or convert it to a string?

[knolleary]

Is that true for a number?

You can see the code here: https://github.com/node-red/node-red/blob/4de1056d826d7b8b83c332e17969af56adf67670/packages/node_modules/%40node-red/nodes/core/io/10-mqtt.js#L339-L345

• If payload is Buffer, it gets sent as-is.
• Otherwise, if typeof msg.payload is 'object', then it JSON encodes the payload (which covers Objects and Arrays).
• Otherwise, if msg.payload is not a String, it gets converts to a string.

So if msg.payload is a number, it will be converted to a string 123 -> "123" before being sent. If you want to send the byte value 123 then you'd need to pass it in as a buffer - Buffer.from([123]);

[Colin]

Thanks @knolleary, it isn't that I want to do that, I just wanted to understand what is going on and reconcile that with Dave's comment.