# Asking about automatic message \*parsing\* / construction
**URL:** https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493
**Category:** Developing Nodes
**Created:** [23 March 2020 10:17 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493 "2020-03-23T10:17:41Z")
**Posts on this page:** 17
**Page:** 1
### Author:  [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)
#### Post date: [23 March 2020 10:17 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/1 "2020-03-23T10:17:41Z")
(Sorry if this isn't exactly the right place, but I think it is just a bit more than `general discussion` and could be a **developmental/Feature request** thing.)
In light of recent events for me with MQTT, this is a bit of _out loud thinking_:
I get that NR with its message and MQTT are two different animals and speak different languages (if you will).
I have just been through a bit of a painful ordeal (well, ok: I guess I deserved it.) with sending messages over MQTT.
Thought:
A node that will look at an incoming message and _construct_ a MQTT compatible message (and/or) deconstruct (why is spell check not liking that word?) an incoming message and break it back to Node-Red format?
Yes, most people wouldn't want/need it. But, now and then you are needing to send a _more complex_ message than simply `msg.payload`.
When you have such a case, you could use this node at both ends and save a lot of messing around.
Example:
At the **sending** end of the message.
```auto
msg.payload = "Something important"
msg.device = "The device's name"
msg.*something_else* = "Other information needed"
*and so on*
```
(and I'm not sure what happened there with formatting)
Can NR see there are _other_ items than the usual extra parts of the `msg` and build a MQTT compatible payload?
like:
`msg.payload.payload = msg.payload, msg.payload.device = msg.device, msg.payload.something_else = msg.something_else`
And then at the other end extract that _structure_ to its original?
I'd love to give it a go, but I don't think I have enough knowledge / confidence yet to do it.
But I am discussing it for consideration.
---
### Author:  [@knolleary](https://discourse.nodered.org/u/knolleary)
#### Post date: [23 March 2020 10:30 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/2 "2020-03-23T10:30:30Z")
I think you are suggesting a node that automatically copies all top-level properties (`msg.device`, `msg.*something_else*` ) under `msg.payload`.
That means you have to make sure the message _only_ contains properties you want to publish. For example, you'd get the `_msgid` property published as well, plus anything else that happened to still be there. In that case, you'd end up having to use a Change node to _delete_ the things you didn't want to publish.
If you are having to use a Change node to tidy up the message before passing to this new node anyway, you may as well cut out the middle-man and use the Change node to get `msg.payload` into exactly the format you want.
---
### Author:  [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)
#### Post date: [23 March 2020 10:38 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/3 "2020-03-23T10:38:27Z")
Thanks Nic.
Yes, I do know of these parts, but am not sure if they are/could/would be helpful for people with more knowledge than me to use for debugging if there are problems.
Given this node would be used either _in to_ or _out of_ a `MQTT` node, it would have to be a given that the message (in the bigger picture) only had things wanting to be sent.
Yes, I get that it would mean (possibly) a clean up of other things not wanted.
But if the flow is written _correctly_ (?) the message would be _constructed_ by things from a clean slate.
So - all things being good - there shouldn't be any extraneous parts.
If there are, well they would need cleaning.
And yes I agree with the last paragraph. It was only me thinking aloud.
But on the _other end_ of the MQTT trail, a node which _automagically_ de-constructs the message back to Node-Red format would be nice.
Yes?
---
### Author:  [@knolleary](https://discourse.nodered.org/u/knolleary)
#### Post date: [23 March 2020 10:47 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/4 "2020-03-23T10:47:47Z")
> [@Trying\_to\_learn](#):
>
> message back to Node-Red format would be nice.
The message is _already_ in "Node-RED format".
You can access all of the properties as `msg.payload.whatever`. That's a given. If you build a flow that expects the value to be somewhere else, you can use a Change node to move it to where you want it to be.
---
### Author:  [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)
#### Post date: [23 March 2020 10:58 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/5 "2020-03-23T10:58:57Z")
Yeah, ok.
I was more thinking about making the MQTT _pipe_ transparent to the message.
So it goes in with (eg:)
`msg.payload = "foo", msg.topic = "Topic", msg.device = "Devicename" (and so on)`
and at the other end of the MQTT tunnel, it comes out the same. (Of course with the extra node on the receiving end.)
Again: It is just a thought.
Or! And I just thought of it now......
Although there is the `a parsed JSON object` option, add something else to do what I was suggesting in that node.
But that may be stepping on MQTT's toes. I don't know where the _demarcation line_ is for what you can do in what node.
---
### Author:  [@cymplecy](https://discourse.nodered.org/u/cymplecy)
#### Post date: [23 March 2020 11:03 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/6 "2020-03-23T11:03:51Z")
So I'm translating your idea as to being an "MQTT link node" to go between different Node-RED instances?
---
### Author:  [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)
#### Post date: [23 March 2020 11:09 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/7 "2020-03-23T11:09:46Z")
Well, (after my little fiasco) where I forgot the MQTT tricks, and thinking at a different _level_:
A machine/flow/what ever has a message which needs to be sent over MQTT.
It has some (I can't remember the terms - sorry) _other sub parts_ which are part of the message.
Ok, I have to _construct_ the message with a `change` node (or `function`) to put it all in the `msg.payload` part. Then send it over/via `MQTT`.
At the other end it would be nice/handy to either have the node go that extra step (as I only just suggested) and put it all back to the original format, or have an extra node which _de-constructs_ the `msg.payload.(blah)` back to what it was before going into `MQTT`.
So sorry if that doesn't answer your question clearly. But I think I have.
(P.S. After a few more brain cycles: Yes. It would be like a _MQTT link node_.)
But be it that is an extra feature added to the `MQTT in` node, or a whole new node.....
---
### Author:  [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)
#### Post date: [23 March 2020 11:24 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/8 "2020-03-23T11:24:48Z")
You would need some kind of special flag in the msg content to indicate to the -in node that it needed to do something different. Otherwise it would be a breaking change for the mqtt nodes.
This is, in fact, very much an edge-case I think. The reason I think that is that MQTT is a lightweight msg format and most brokers have limits on the msg size they allow as do all of the clients. So sending serialised JSON to MQTT is itself an edge-case, just a very common one! The truth is that the content of an MQTT topic is a **string** , not a structure. In my view, if you send it anything other than a simple string, you should be thinking about how you are processing it anyway.
As such, I would use a subflow or a pair of subflows to achieve this without any need to change the mqtt node.
---
### Author:  [@cymplecy](https://discourse.nodered.org/u/cymplecy)
#### Post date: [23 March 2020 11:25 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/9 "2020-03-23T11:25:07Z")
> [@Trying\_to\_learn](#):
>
> Yes. It would be like a _MQTT link node_
🙂
> [@Trying\_to\_learn](#):
>
> But be it that is an extra feature added to the `MQTT in` node, or a whole new node.....
I think it would be best to have one node that packages the Node-RED message parts and feeds that into a standard MQTT out node
And then have another node on the output of the receiving MQTT in node to decode it back into its original Node-RED message format
---
### Author:  [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [23 March 2020 11:31 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/10 "2020-03-23T11:31:32Z")
I have never that I can remember had to do this, I wonder whether you might like to consider changing how you write this data to mqtt. Looking at your previous thread you appear to have `msg.device` with presumably some device identifier there, msg.status, with presumably the status and msg.payload with something else, and you are sending this to a topic which, again presumably, is used for multiple devices. Instead you could have topics like
`something/deviceid/status` to send the status to and
`something/deviceid/data` or similar, to send whatever is in your payload. Then, for example you would for example be able to subscribe to all the data for a particular device by subscribing to `something/deviceid/#`.
---
### Author:  [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)
#### Post date: [23 March 2020 11:38 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/11 "2020-03-23T11:38:10Z")
Colin,
I do agree with you in some of things.
The honest truth is that a lot of this is/was written YEARS ago when I was just starting with Node-Red and I was a bit _trigger happy_ with old habits of doing things the _old_ way, rather than the new way.
That was only an example and just now I am up to my arms with updating a few machines with newer (better?) code, and the heart aches it is giving me.
I shall have to look at the structure of the MQTT messages: yes.
I think there is some duplication of information being sent.
Alas just now, I've been awake for ..... 16 hours, there about. I think I will have to hit the sack soon/shortly.
---
### Author:  [@cymplecy](https://discourse.nodered.org/u/cymplecy)
#### Post date: [23 March 2020 13:26 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/12 "2020-03-23T13:26:13Z")
I've had a bit of a play and this seems to to work
This code in a function node should be placed before an MQTT out node
It moves all the msg objects into msg.mqttPackagedMessage
It removes the \_msgid as that will be recreated by Node-RED when it is decoded at the receiving end
```auto
var mqttPackagedMessage;
mqttPackagedMessage = ({ "payload": msg });
delete mqttPackagedMessage['payload']['_msgid'];
node.send([mqttPackagedMessage]);
```
Then this code should be placed in a function node after the receiving MQTT in node
```auto
var mqttPackagedMessage;
mqttPackagedMessage = (JSON.parse((msg['payload'])));
node.send([mqttPackagedMessage]);
```
---
### Author:  [@Colin](https://discourse.nodered.org/u/Colin)
#### Post date: [23 March 2020 13:36 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/13 "2020-03-23T13:36:51Z")
> [@cymplecy](#):
>
> mqttPackagedMessage = (JSON.parse((msg['payload'])));
Is that line not required if Parsed JSON is selected in the output dropdown in the MQTT In node?
---
### Author:  [@cymplecy](https://discourse.nodered.org/u/cymplecy)
#### Post date: [23 March 2020 13:38 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/14 "2020-03-23T13:38:59Z")
I'm keeping things simple (but explicit) - just works when using default MQTT in node settings 🙂
---
### Author:  [@zenofmud](https://discourse.nodered.org/u/zenofmud)
#### Post date: [23 March 2020 14:36 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/15 "2020-03-23T14:36:16Z")
You can also use this as the function
```auto
var nmsg = {}
nmsg.payload = msg
return nmsg;
```
and this as the out function
```auto
var nmsg = {}
nmsg = msg.payload
return nmsg;
```
```auto
[{"id":"ee19fba1.af0b3","type":"inject","z":"ad474a87.836418","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":100,"wires":[["fc10739f.051218"]]},{"id":"fc10739f.051218","type":"change","z":"ad474a87.836418","name":"","rules":[{"t":"set","p":"topic","pt":"msg","to":"test_stuff","tot":"str"},{"t":"set","p":"other","pt":"msg","to":"Look I'm other","tot":"str"},{"t":"set","p":"payload.animal","pt":"msg","to":"cat","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":160,"y":160,"wires":[["427154b.48e682c","baccbe3a.9ab05"]]},{"id":"427154b.48e682c","type":"function","z":"ad474a87.836418","name":"","func":"var nmsg = {}\nnmsg.payload = msg\nreturn nmsg;","outputs":1,"noerr":0,"x":150,"y":220,"wires":[["115bd288.d5810d","ee96b9f6.7a40e8"]]},{"id":"baccbe3a.9ab05","type":"debug","z":"ad474a87.836418","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":370,"y":160,"wires":[]},{"id":"ee96b9f6.7a40e8","type":"debug","z":"ad474a87.836418","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":370,"y":220,"wires":[]},{"id":"115bd288.d5810d","type":"mqtt out","z":"ad474a87.836418","name":"","topic":"test_stuff","qos":"","retain":"","broker":"70122dd0.b2a414","x":160,"y":280,"wires":[]},{"id":"d34fea20.43778","type":"mqtt in","z":"ad474a87.836418","name":"","topic":"test_stuff","qos":"2","datatype":"json","broker":"70122dd0.b2a414","x":590,"y":100,"wires":[["3b62f881.b9f0f8","733b200b.0b9ad"]]},{"id":"3b62f881.b9f0f8","type":"debug","z":"ad474a87.836418","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":790,"y":100,"wires":[]},{"id":"733b200b.0b9ad","type":"function","z":"ad474a87.836418","name":"","func":"var nmsg = {}\nnmsg = msg.payload\nreturn nmsg;","outputs":1,"noerr":0,"x":590,"y":160,"wires":[["788dcbcc.506e0c"]]},{"id":"788dcbcc.506e0c","type":"debug","z":"ad474a87.836418","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":790,"y":160,"wires":[]},{"id":"70122dd0.b2a414","type":"mqtt-broker","z":"","name":"mqttpizw","broker":"192.168.48.92","port":"1883","clientid":"","usetls":false,"compatmode":false,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closeQos":"0","closeRetain":"false","closePayload":"","willTopic":"","willQos":"0","willRetain":"false","willPayload":""}]
```
---
### Author:  [@Trying\_to\_learn](https://discourse.nodered.org/u/Trying_to_learn)
#### Post date: [23 March 2020 19:49 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/16 "2020-03-23T19:49:12Z")
Hi folks.
_New day_ here. Just got up.
Wow. Ok thanks to all. @cymplecy and @zenofmud for the ideas.
Just now I need to continue with the _mess_ from last night fixing up old mistakes from years gone by.
I shall seriously take a look at the two ideas.
~~Paul. in your example you use `nmsg`....... Ah! Got it.~~
---
### Author:  [@system](https://discourse.nodered.org/u/system)
#### Post date: [22 May 2020 19:49 UTC](https://discourse.nodered.org/t/asking-about-automatic-message-parsing-construction/23493/17 "2020-05-22T19:49:12Z")
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.