`_msgid` - what's going on here?

Folks, I am seeming to be getting nowhere fast.

Here's the picture first.

Why do two different messages have the same _msgid?

TL;DR
I am splitting big message into smaller ones at strategic points, storing other message attributes to flow context, and re-applying them when they come out of the TTS node.

Some/most of the time it works.

But - sod's law - now and then it doesn't.

And it is driving me crazy.

Could this be a cause?

Show us how you do that.

(Weird, I didn't seem to get a notification of your reply)

I hope this is not too much of a cut down.

But the nodes - basically - are:

[{"id":"379961d654c16959","type":"function","z":"024291b92792cb06","name":"set alerts","func":"const messageLog = flow.get('messageLog') || {}\n\nconst id = msg._msgid\n\nmessageLog[id] = {\n    payload: msg.payload,\n    alert: msg.alert,\n    postalert: msg.postalert,\n    mute: msg.mute,\n    parts: msg.parts  // required and always expected\n}\n\nflow.set('messageLog', messageLog)\n\nreturn msg\n","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":2940,"y":710,"wires":[["ba53904f83eae319"]]},{"id":"ba53904f83eae319","type":"ttsultimate","z":"024291b92792cb06","name":"Translator","voice":"en-AU","ssml":false,"sonosipaddress":"192.168.1.109","sonosvolume":"30","sonoshailing":"0","config":"557d8082.eb5a8","property":"payload","propertyType":{},"rules":[],"playertype":"noplayer","speakingrate":"","speakingpitch":"","unmuteIfMuted":false,"elevenlabsStability":"","elevenlabsSimilarity_boost":"","x":3090,"y":710,"wires":[["e59215b9ea2f8d59"],[]]},{"id":"e59215b9ea2f8d59","type":"switch","z":"024291b92792cb06","name":"Only arrays","property":"filesArray","propertyType":"msg","rules":[{"t":"nnull"},{"t":"else"}],"checkall":"true","repair":false,"outputs":2,"x":3185,"y":700,"wires":[["078c291e7d380288"],[]],"l":false},{"id":"078c291e7d380288","type":"function","z":"024291b92792cb06","name":"Get alerts","func":"const messageLog = flow.get('messageLog') || {}\n\nconst passthrough = msg.passThroughMessage\nconst id = passthrough?._msgid\n\nif (id && messageLog[id]) {\n    const stored = messageLog[id]\n\n    msg.alert = stored.alert ?? null\n    msg.postalert = stored.postalert ?? null\n    msg.mute = stored.mute ?? null  // ⬅️ Add this line to restore 'mute'\n    msg.parts = stored.parts\n\n    // Clean up after restoring\n    delete messageLog[id]\n    flow.set('messageLog', messageLog)\n}\n\nreturn msg\n","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":3290,"y":710,"wires":[[]]},{"id":"557d8082.eb5a8","type":"ttsultimate-config","name":"googletranslate","noderedipaddress":"127.0.0.1","noderedport":"1980","purgediratrestart":"leave","ttsservice":"googletranslate","TTSRootFolderPath":""}]

The two function nodes do all the magic.

Do you want an example message?

If you are using the split node it will add the msg.parts property and keep the same _msgid - so that it can possibly rejoin them later. If you don’t want them you can delete them and a new *_*msgid will get generated.

Um.... the split node here is NOT the standard one.
This one gets a text string and strategically splits it.

Sorry.

My concern is (at the top) I sent two messages through the TTS node and they both came out with the same _msgid.

Yeah, I maybe need to look a bit more at what's going on.

But it makes tracking things really annoying.

It is best not to use _msgid for anything. Leave it for node-red to worry about.

Yeah, but I don't think you understand what I'm wanting to do.

(I think the terminology is TL/DR for this bit)

Ok, I have a message.
It is for TTS processing. (And I don't have an API key for google TTS - or any TTS)
There are .....4 (?) extra attributes to the message.
1 - MUTE
2 - ALERT
3 - split (I'll get back to this soon)
4 - VOLUME
(and I hope no more that have slipped off my memory.)
5 - PAUSE (forgot this one. Maybe because I don't use it much.)

What's going on and why?

Well, the messages can vary from basic ones which are just plain text, to muting, pausing, being split..... and so on.

These are given to the message at the start of their journey.
So there are a lot of options - yeah: option overload. :wink:

(See screen shot)

And that is only one machine.
Other machines have a whole lot of options for TTS and alarms.

(and - alas - there are some things there that don't need to be. I just like to keep an eye of what's going on under the hood.)

So these were working all well and good until the messages started to become LONG.
The TTs node would just spit the dummy and not work. (annoying)

So what I did was did a simple SPLIT of the message and send the subsequent messages after the one before them was complete.

This is all good in theory, but things aren't perfect and problems started to happen.

Ideally with no attributes the message would go through, be played and nothing discernible happens.

But when you start adding a tone to be played BEFORE the message, it would be like:
ding..... wait....... message heard.
Not too bad, but noticeable.

That was not too bad.
But when you had the MUTE it became a problem
(be it with or without a DING.)

mute (TV) ding.... wait.... message heard.... mute off
And again, all good.
But the delays started to become noticeable.

Then - sorry, but to show the evolution of where I'm at now and HOW I got here - BIG messages started to happen

So I split them with a simpler SPLIT node.
ding.... wait.....message1.....message 2... and so on.
Yeah, not too bad.

Here's the kicker!

Mute:
mute..... message 1....mute off... message 2.... and so on.

Suffice to say words were said.

So I had to work out a better way.

So I had to completely restructure the whole thing.

Queue the message, send to TTS node, get attributes back - alas they are lost going through the TTS node. (kinda - I'm open to alternatives) THEN process any MUTE, DING, VOLUME, etc. ....
THEN send it to the player.
Of course all that also has to be queued and clocked so the split messages sound fluid.

The SPLIT node saves all the extra stuff (attributes, as I've been calling them) to flow.context and when the message comes out the other side of the TTS node, it goes through the second node and gets all it's attributes back.

OR SO I THOUGHT!

Now and then messages would..... vanish.
Poof!
Gone!
And I don't know where/how.

Though when I start tracing them, weird things happen.

All the queues are empty yet when I send another message through, the lost one appears in front of the newer message.

WHERE WAS IT HIDING? (Rhetorical)
Anyway, while digging, I saw two messages come through with the same _msgid and it got me wondering.....

The flow I have is very complicated and I am sure it is over doing a lot of things.
But I've only added this extra stuff because messages get stuck or lost and when I add this extra stuff, they seem to work.

There, that's the John Dory.

Does it make sense?

That is not likely to be the problem. Ignore msg. _msgid.

I was told that it is a good thing to use to track the message.

The _msgid follows the message on its journey through nodes.

So, it the message goes into the first node, and the _msgid is used to set the context.
When the message comes out of the TTS node, the _msgid is matched to the message and the stuff is put back.

That's what I was told and seems to work most of the time.