`_msgid` - what's going on here?

[Trying_to_learn]

Folks, I am seeming to be getting nowhere fast.

Here's the picture first.

Why do two different messages have the same _msgid?

TL;DR
I am splitting big message into smaller ones at strategic points, storing other message attributes to flow context, and re-applying them when they come out of the TTS node.

Some/most of the time it works.

But - sod's law - now and then it doesn't.

And it is driving me crazy.

Could this be a cause?

[Steve-Mcl]

@Trying_to_learn

I am splitting big message into smaller ones at strategic points, storing other message attributes to flow context, and re-applying them when they come out of the TTS node

Show us how you do that.

[Trying_to_learn]

(Weird, I didn't seem to get a notification of your reply)

I hope this is not too much of a cut down.

But the nodes - basically - are:

[{"id":"379961d654c16959","type":"function","z":"024291b92792cb06","name":"set alerts","func":"const messageLog = flow.get('messageLog') || {}\n\nconst id = msg._msgid\n\nmessageLog[id] = {\n    payload: msg.payload,\n    alert: msg.alert,\n    postalert: msg.postalert,\n    mute: msg.mute,\n    parts: msg.parts  // required and always expected\n}\n\nflow.set('messageLog', messageLog)\n\nreturn msg\n","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":2940,"y":710,"wires":[["ba53904f83eae319"]]},{"id":"ba53904f83eae319","type":"ttsultimate","z":"024291b92792cb06","name":"Translator","voice":"en-AU","ssml":false,"sonosipaddress":"192.168.1.109","sonosvolume":"30","sonoshailing":"0","config":"557d8082.eb5a8","property":"payload","propertyType":{},"rules":[],"playertype":"noplayer","speakingrate":"","speakingpitch":"","unmuteIfMuted":false,"elevenlabsStability":"","elevenlabsSimilarity_boost":"","x":3090,"y":710,"wires":[["e59215b9ea2f8d59"],[]]},{"id":"e59215b9ea2f8d59","type":"switch","z":"024291b92792cb06","name":"Only arrays","property":"filesArray","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":3185,"y":700,"wires":[["078c291e7d380288"],[]],"l":false},{"id":"078c291e7d380288","type":"function","z":"024291b92792cb06","name":"Get alerts","func":"const messageLog = flow.get('messageLog') || {}\n\nconst passthrough = msg.passThroughMessage\nconst id = passthrough?._msgid\n\nif (id && messageLog[id]) {\n    const stored = messageLog[id]\n\n    msg.alert = stored.alert ?? null\n    msg.postalert = stored.postalert ?? null\n    msg.mute = stored.mute ?? null  // ⬅️ Add this line to restore 'mute'\n    msg.parts = stored.parts\n\n    // Clean up after restoring\n    delete messageLog[id]\n    flow.set('messageLog', messageLog)\n}\n\nreturn msg\n","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":3290,"y":710,"wires":[[]]},{"id":"557d8082.eb5a8","type":"ttsultimate-config","name":"googletranslate","noderedipaddress":"127.0.0.1","noderedport":"1980","purgediratrestart":"leave","ttsservice":"googletranslate","TTSRootFolderPath":""}]

The two function nodes do all the magic.

Do you want an example message?

[dceejay]

If you are using the split node it will add the msg.parts property and keep the same _msgid - so that it can possibly rejoin them later. If you don’t want them you can delete them and a new *_*msgid will get generated.

[Trying_to_learn]

Um.... the split node here is NOT the standard one.
This one gets a text string and strategically splits it.

Sorry.

My concern is (at the top) I sent two messages through the TTS node and they both came out with the same _msgid.

Yeah, I maybe need to look a bit more at what's going on.

But it makes tracking things really annoying.

[Colin]

It is best not to use _msgid for anything. Leave it for node-red to worry about.

[Trying_to_learn]

Yeah, but I don't think you understand what I'm wanting to do.

(I think the terminology is TL/DR for this bit)

Ok, I have a message.
It is for TTS processing. (And I don't have an API key for google TTS - or any TTS)
There are .....4 (?) extra attributes to the message.
1 - MUTE
2 - ALERT
3 - split (I'll get back to this soon)
4 - VOLUME
(and I hope no more that have slipped off my memory.)
5 - PAUSE (forgot this one. Maybe because I don't use it much.)

What's going on and why?

Well, the messages can vary from basic ones which are just plain text, to muting, pausing, being split..... and so on.

These are given to the message at the start of their journey.
So there are a lot of options - yeah: option overload.

(See screen shot)

And that is only one machine.
Other machines have a whole lot of options for TTS and alarms.

(and - alas - there are some things there that don't need to be. I just like to keep an eye of what's going on under the hood.)

So these were working all well and good until the messages started to become LONG.
The TTs node would just spit the dummy and not work. (annoying)

So what I did was did a simple SPLIT of the message and send the subsequent messages after the one before them was complete.

This is all good in theory, but things aren't perfect and problems started to happen.

Ideally with no attributes the message would go through, be played and nothing discernible happens.

But when you start adding a tone to be played BEFORE the message, it would be like:
ding..... wait....... message heard.
Not too bad, but noticeable.

That was not too bad.
But when you had the MUTE it became a problem
(be it with or without a DING.)

mute (TV) ding.... wait.... message heard.... mute off
And again, all good.
But the delays started to become noticeable.

Then - sorry, but to show the evolution of where I'm at now and HOW I got here - BIG messages started to happen

So I split them with a simpler SPLIT node.
ding.... wait.....message1.....message 2... and so on.
Yeah, not too bad.

Here's the kicker!

Mute:
mute..... message 1....mute off... message 2.... and so on.

Suffice to say words were said.

So I had to work out a better way.

So I had to completely restructure the whole thing.

Queue the message, send to TTS node, get attributes back - alas they are lost going through the TTS node. (kinda - I'm open to alternatives) THEN process any MUTE, DING, VOLUME, etc. ....
THEN send it to the player.
Of course all that also has to be queued and clocked so the split messages sound fluid.

The SPLIT node saves all the extra stuff (attributes, as I've been calling them) to flow.context and when the message comes out the other side of the TTS node, it goes through the second node and gets all it's attributes back.

OR SO I THOUGHT!

Now and then messages would..... vanish.
Poof!
Gone!
And I don't know where/how.

Though when I start tracing them, weird things happen.

All the queues are empty yet when I send another message through, the lost one appears in front of the newer message.

WHERE WAS IT HIDING? (Rhetorical)
Anyway, while digging, I saw two messages come through with the same _msgid and it got me wondering.....

The flow I have is very complicated and I am sure it is over doing a lot of things.
But I've only added this extra stuff because messages get stuck or lost and when I add this extra stuff, they seem to work.

There, that's the John Dory.

Does it make sense?

[Colin]

@Trying_to_learn

Anyway, while digging, I saw two messages come through with the same _msgid and it got me wondering.....

That is not likely to be the problem. Ignore msg. _msgid.

[Trying_to_learn]

I was told that it is a good thing to use to track the message.

The _msgid follows the message on its journey through nodes.

So, it the message goes into the first node, and the _msgid is used to set the context.
When the message comes out of the TTS node, the _msgid is matched to the message and the stuff is put back.

That's what I was told and seems to work most of the time.

[Trying_to_learn]

So - sorry folks to keep going on with this.

I am still stuck.

Ok, the _msgid is working and I overlooked a very serious problem.

I'm storing messages which have been split into many smaller ones in flow.context using the _msgid as an index.

This all falls over if the message is broken into more than 1 part.
(Subsequent parts will have the same ID)

Help please.

Bit more information.

After a lot of messing around, I se this problem

The message is split into 4 parts.
You can see part 1 (and 2 - implied) going INTO the TTS node. (green lines)
But the only one you see coming out is part 2. (Blue lines)
The first message IS heard.
Why isn't it shown? (Bit academic, but still a good mention)

What I'll do is change the Get alerts node code to get the information back from the passThroughMessage part, rather than flow.context.

SHOULD work, but if I am not seeing message 1 coming out of the TTS node......

There's another Gremlin in here, somewhere.

[Trying_to_learn]

AFTER EVEN MORE DIGGING AND STUFF.

This is weird.

I send in two messages - and as suggested: we'll ignore the _msgid part - to the TTS node.

SOMETIMES it sends out TWO messages.

SOMETIMES it sens out ONE message with two parts.

eg:

(Two parts)
(and gee it would be nicer if it came out structured.)

{"passThroughMessage":{"_msgid":"33144809f7e01833","payload":"Inside temperature 21 degrees","topic":"weather","_event":"node:b93ca2a3e5a8db14","location":{"lon":151.1314,"lat":-33.8886,"city":"Ashfield","country":"AU"},"data":{"coord":{"lon":151.1314,"lat":-33.8886},"weather":[{"id":800,"main":"Clear","description":"clear sky","icon":"01d"}],"base":"stations","main":{"temp":297.82,"feels_like":29,"temp_min":296.9,"temp_max":299.1,"pressure":1015,"humidity":44,"sea_level":1015,"grnd_level":1007},"visibility":10000,"wind":{"speed":9.26,"deg":350},"clouds":{"all":0},"dt":1757986613,"sys":{"type":2,"id":2010638,"country":"AU","sunrise":1757966036,"sunset":1758008834},"timezone":36000,"id":7839683,"name":"Ashfield","cod":200},"time":"11:36","title":"Current Weather Information","description":"Current weather information at coordinates: -33.8886, 151.1314","it":21,"windspeed":9,"settings":{"input":"2025-09-16T01:36:54.020Z","input_format":"","input_tz":"Australia/Sydney","output_format":"HH:mm","output_locale":"en-AU","output_tz":"Australia/Sydney"},"parts":{"index":2,"total":2}},"payload":true,"filesArray":[{"file":"/home/pi/.node-red/sonospollyttsstorage/ttsfiles/210d77a1976da9236fca96c4f1d37086.mp3"},{"file":"/home/pi/.node-red/sonospollyttsstorage/ttsfiles/a94cc63b3683a98297bd0e383bd11d39.mp3"}],"_msgid":"e9c5c07769aa0291","_event":"node:b39c5f7d5fddc27e","alert":null,"postalert":null,"mute":null,"parts":{"index":2,"total":2}}

For the sake of attempting to help here is a screen shot.

Note the fileArray part near the top.

Here is another example, same message sent, and it is in two parts.

I can't find part 1 of the two, but I fear that is because it is being filtered further back by a filter node.

Anyway, the screen shot.

Note how fileArrays has only 1 object in it.
The previous one had 2.

(Hurray, I did actually catch one of these rogue messages)

{"passThroughMessage":{"_msgid":"16b4b714f58e4983","payload":"Inside temperature 21 degrees","topic":"weather","_event":"node:b93ca2a3e5a8db14","location":{"lon":151.1314,"lat":-33.8886,"city":"Ashfield","country":"AU"},"data":{"coord":{"lon":151.1314,"lat":-33.8886},"weather":[{"id":800,"main":"Clear","description":"clear sky","icon":"01d"}],"base":"stations","main":{"temp":297.72,"feels_like":29,"temp_min":297.09,"temp_max":299.18,"pressure":1015,"humidity":44,"sea_level":1015,"grnd_level":1007},"visibility":10000,"wind":{"speed":9.77,"deg":340},"clouds":{"all":0},"dt":1757987563,"sys":{"type":2,"id":2010638,"country":"AU","sunrise":1757966036,"sunset":1758008834},"timezone":36000,"id":7839683,"name":"Ashfield","cod":200},"time":"12:01","title":"Current Weather Information","description":"Current weather information at coordinates: -33.8886, 151.1314","it":21,"windspeed":10,"settings":{"input":"2025-09-16T02:01:11.723Z","input_format":"","input_tz":"Australia/Sydney","output_format":"HH:mm","output_locale":"en-AU","output_tz":"Australia/Sydney"},"parts":{"index":2,"total":2}},"payload":false,"_msgid":"1bf32398099e7a50"}

(screen shot)

Although it is part 2 of 2 also.

Any thoughts / ideas on what's going on and how to work around it?