I have started playing with the Audio Out for Text-To-Speach (TTS) and it is working well but I have run into one item that confuses me. In the dropdown, it shows 20 different voice choices, but some of them if I select them and deploy, it actually switches to a different voice and uses that.
Here is what I see for choices:
The quickest example is that it will let me select 0, Microsoft David, but if I select 1, Microsoft Zira, it switches to 0, David. It will let me select 2, Google Deustch, or 4, Google UK English Female, but not 3, Google US English .
He is referring to the audio-out node which is shipped with the standard dashboard nodes.
I think the mechanism works like this:
When looking at the code of that node's config screen:
voices = window.speechSynthesis.getVoices()
The getVoices returns the voices that are supported on the current device, and those are added to the dropdown. However this is in the config screen of the audio-out node, which means on the pc where the flow editor is being displayed.
Then the selected voice will be send to the dashboard client:
In the dashboard client, all voices have also been loaded:
if ('speechSynthesis' in window) {
voices = window.speechSynthesis.getVoices();
window.speechSynthesis.onvoiceschanged = function() {
voices = window.speechSynthesis.getVoices();
}
}
These are the voices available on the device where the dashboard is running.
And finally when the message (from step 2) arrives on the dashboard client, the voice (from step 3) with the same language (as the voice selected on the config screen in step 1) will be used:
if (msg.hasOwnProperty("tts")) {
if (voices.length > 0) {
var words = new SpeechSynthesisUtterance(msg.tts);
words.onerror = function(err) { events.emit('ui-audio', 'error: '+err.error); }
words.onend = function() { events.emit('ui-audio', 'complete'); }
for (var v=0; v<voices.length; v++) {
if (voices[v].lang === msg.voice) {
words.voice = voices[v];
break;
}
}
events.emit('ui-audio', 'playing');
window.speechSynthesis.speak(words);
}
}
So you select a voice in the config screen, from the voices available on the pc where the flow editor is displayed. The selected voice is send to the dashboard and it tries to find the SAME voice on the device where the dashboard is displayed. If that voice is available on both devices then it will be ok. But if the pc where the dashboard is running supports different voices, then you won't get in the dashboard what you have selected in the config screen. Unless I haven't understood it well ...
Bart,
That sounds logical and this mirrors my early thinking on what was happening, but it differs a bit from the experience I am having. When I am editing on a Windows 10 PC, I am provided the list of the 20 voice options. When I pick some voices and deploy, this works. When I pick some of the other voices and deploy and then reopen the node for editing, it is switched to one of the voices in the first category.
This is without actually hearing the voice on a given device displaying the dashboard. I can imagine just as you described if the dashboard is on the same Windows 10 PC it will probably work, but on an iPAD any given voice might work or might not and maybe even it would vary based on Chrome vs Safari on that iPad.
I am just surprised it would give voice options at edit time that it wouldn't actually allow me to save and then deploy.
-Mike
IMHO it is not really correct to store the lang in the node.voice variable, since it refers to multiple voices. Instead I would store the dataURI, since that seems to contain unique values.
Perhaps it is useful to store BOTH:
Keep the lang in the existing node.voice variable
Store extra the dataURI in a new node.dataURI variable
Indeed when there is no matching in the client based on the dataURI, we could - as a fallback scenario - match on the lang. E.g. when the "Google UK English Male" (selected in the flow editor) is not available in the dashboard, we could fallback to "Google UK English Female" since it has the same language code ("en-GB").
Although I my grandfather always said: "Never listen to a female..."
