Dashboard Audio Out Voice Options

I have started playing with the Audio Out for Text-To-Speach (TTS) and it is working well but I have run into one item that confuses me. In the dropdown, it shows 20 different voice choices, but some of them if I select them and deploy, it actually switches to a different voice and uses that.

Here is what I see for choices:
image

The quickest example is that it will let me select 0, Microsoft David, but if I select 1, Microsoft Zira, it switches to 0, David. It will let me select 2, Google Deustch, or 4, Google UK English Female, but not 3, Google US English .

Thoughts or pointers?

I can't immediately find this node. Can you specify which node you are actually using ?

He is referring to the audio-out node which is shipped with the standard dashboard nodes.
I think the mechanism works like this:

  1. When looking at the code of that node's config screen:

    voices = window.speechSynthesis.getVoices()
    

    The getVoices returns the voices that are supported on the current device, and those are added to the dropdown. However this is in the config screen of the audio-out node, which means on the pc where the flow editor is being displayed.

  2. Then the selected voice will be send to the dashboard client:

    ui.emit('ui-audio', { tts:msg.payload, voice:(node.voice || msg.voice || 0), tabname:node.tabname, always:node.always });
    
  3. In the dashboard client, all voices have also been loaded:

    if ('speechSynthesis' in window) {
       voices = window.speechSynthesis.getVoices();
       window.speechSynthesis.onvoiceschanged = function() {
          voices = window.speechSynthesis.getVoices();
       }
    }
    

    These are the voices available on the device where the dashboard is running.

  4. And finally when the message (from step 2) arrives on the dashboard client, the voice (from step 3) with the same language (as the voice selected on the config screen in step 1) will be used:

    if (msg.hasOwnProperty("tts")) {
       if (voices.length > 0) {
          var words = new SpeechSynthesisUtterance(msg.tts);
          words.onerror = function(err) { events.emit('ui-audio', 'error: '+err.error); }
          words.onend = function() { events.emit('ui-audio', 'complete'); }
          for (var v=0; v<voices.length; v++) {
             if (voices[v].lang === msg.voice) {
                words.voice = voices[v];
                break;
             }
          }
          events.emit('ui-audio', 'playing');
          window.speechSynthesis.speak(words);
       }
    }
    

So you select a voice in the config screen, from the voices available on the pc where the flow editor is displayed. The selected voice is send to the dashboard and it tries to find the SAME voice on the device where the dashboard is displayed. If that voice is available on both devices then it will be ok. But if the pc where the dashboard is running supports different voices, then you won't get in the dashboard what you have selected in the config screen. Unless I haven't understood it well ...

Might this somehow explain Mike's issue??

Bart

1 Like

Bart,
That sounds logical and this mirrors my early thinking on what was happening, but it differs a bit from the experience I am having. When I am editing on a Windows 10 PC, I am provided the list of the 20 voice options. When I pick some voices and deploy, this works. When I pick some of the other voices and deploy and then reopen the node for editing, it is switched to one of the voices in the first category.

This is without actually hearing the voice on a given device displaying the dashboard. I can imagine just as you described if the dashboard is on the same Windows 10 PC it will probably work, but on an iPAD any given voice might work or might not and maybe even it would vary based on Chrome vs Safari on that iPad.

I am just surprised it would give voice options at edit time that it wouldn't actually allow me to save and then deploy.
-Mike

Seems there is a bug in this node...
I have reproduced your issue, but looks like the internal storage mechanism of the node works correctly:

  1. When you open the config screen the first time, the language will be an empty string "":

    image

  2. But immediately afterwards the change handler (that detects changes in the dropdown list) will set it the voice to null

    image

    Because there is voice selected in the dropdown by default:

    image

  3. When I select "en-US":

    image

    Then the dropdown change handler will see this correctly and store the "en-US" voice in the node:

    image

  4. I press the 'Done' button (deploy is not necessary).

  5. When I open the config screen again, the stored "en-US" value is loaded into the dropdown field:

    image

  6. And indeed the change handler notices that the stored voice has been selected, since it stores the same value again in the node:

    image

  7. But on the screen another language is displayed:

    image

The problem is caused since there are multiple voices with the same language code:

image

When you open the config screen again, it will show the FIRST language with the selected language code...

My time is up for this morning. Hopefully somebody else can join this discussion now.

1 Like

This is all the data that is being returned when getting the available voices on the device:

IMHO it is not really correct to store the lang in the node.voice variable, since it refers to multiple voices. Instead I would store the dataURI, since that seems to contain unique values.

Perhaps it is useful to store BOTH:

  • Keep the lang in the existing node.voice variable
  • Store extra the dataURI in a new node.dataURI variable

Indeed when there is no matching in the client based on the dataURI, we could - as a fallback scenario - match on the lang. E.g. when the "Google UK English Male" (selected in the flow editor) is not available in the dashboard, we could fallback to "Google UK English Female" since it has the same language code ("en-GB").

Although I my grandfather always said: "Never listen to a female..." :rofl:

Any thoughts about this??

I suspect your grandmother had other ideas...

2 Likes

Hi Dave,
Welcome back!
I ment whether you had any thoughts about my proposal, not about my grandparents :joy:

well - yes it needs to be fixed by creating some unique but repeatable reference.