Node red dashboard audio wish

@BartButenaers I'd say send a blank and no blob for end of playing - eg like the preceding http request - so it is cleared when finished. This also leaves the possibility free for "stopped" if there was ever to be a stop command, or maybe "paused" etc.... Also I'd prefer "playing" instead of started... :slight_smile:

If you send a blank, is it possible to catch that change then with the status node?
So do i have a trigger that tells me it has stopped?

actually can be an empty string ""...
image

1 Like

Nope. Seems that I will need to handle the speech synthesis differently:

  • It should be stopped this way.
  • And to detect the end of the speech playback, I think I should use this event.

But for some reason I get no sound at all using TTS, so a bit difficult to test :woozy_face: Don't know what is wrong, because I have it on all my Raspberry's (so not a result of my dashboard changes)...

Don't know if my demo was not clear, but when you send a msg.reset=true then the playback will already be stopped. So that functionality is also part of this pull request ... Do you prefer perhaps msg.stop instead of msg.reset?

In the current version I haven't implemented pause or resume:

  • Web audio can only be started or stopped. However when 'pause' or 'resume' is required, we would need to remember ourselves the time offset (where we have interrupted the fragment). Like in this Codepen example .
  • On the other hand, the speech synthesis seem to have pause and resume methods out of the box.

No, implement as you suggested with msg.reset as that is consistent with other nodes (like delay and trigger etc). I was introducing feature creep which I should not as no-one is asking for it.

1 Like

Do you have the web interface ( /ui ) open?
Thats all ways my mistake when testing

@RogierQ: Yes I also saw the text arriving in the dashboard (using Chrome debugger) and entering in the 'speak' function, but no sound... But on Android it worked fine, so I could implement the 'reset' and 'oncomplete' features last night also for speech synthesis. Looks to be working identical to normal audio now...

@dceejay: I have only one open issue. When I have N audio nodes on my flow, they all get the same status. However when nodeA plays A.mp3 and nodeB plays B.mp3 (at another time), I only want the status of nodeA to be updated when A.mp3 is playing.

I do it like this:

  • In the ui.js an event is emitted when the socket gets an audio status change from the browser:
    socket.on('ui-audio', function(audioStatus) {
         ev.emit("audiostatus", audioStatus, socket.client.id, socket.request.connection.remoteAddress);
    });
    
  • In the ui_audio.js this event is handled by setting the node status:
    var updateStatus = function(audioStatus) {
         ...
         node.status({shape:"dot",fill:"blue",text:audioStatus});
     };
    ui.ev.on('audiostatus', updateStatus);
    

Do you know how I can determine in this event handler whether the status update is required for THIS audio-out node. If I know that answer, I will try to submit a pull request this evening. Now I'm off ...
Thanks !!

Hmm - actually that raises a good point... I wonder if this is the way we want to do it...

Wondering if we need a more general approach for setting status rather than setting specific listeners as that won't be extendable. I think maybe that rather like we have updateValueEventName and it's associated handlers - if we need updateStatusEventName to do similar for status - in order to allow widgets to update status.

The thinking with things like the ui_control and ui_audio node was that as they are across the complete dashboard then you only need one per "project" - the fact that you have two and they both feed data to the same speaker is good fortune. As there is only one physical speaker (or two for stereo :slight_smile: - then msg.reset will/should stop all the output anyway so the status should reflect that ?

So I think proceed anyway - lets get the PR in and then play with it before release.

Yes that is the same that I was thinking, since there is only one audio destination... But in my examples with A.mp3 and B.mp3 it is rather confusing when both audio node statusses are updated. But 'in theory' it is correct behaviour.

Will prepare the PR as soon as possible...
Thanks!

Btw think this would be interesting functionality for the new widget API anyway!

Evening guys,

The pull request has been submitted.
Now we have to wait for an approval from the master (of disaster :wink:)
Keep faith and fingers crossed ...

Bart

1 Like

Merged to master if you wish to try it.

Thanks Bart - looks fine so far.

Nice Dave, thanks for the fast support!
@RogierQ @hugobox: Morning guys, would appreciate if you could test the new dashboard version. You will most probablly use the audio-out node in another way as I do. The TTS support was a good example of this, since I completely had forgotten to implement the new features for TTS also ... Thanks in advance!

O thats why it is not working with tts
Did a quick test this morning

No - the TTS support is in there - as with all dashboard upgrades you need to really flush the browser cache to ensure it loads the latest version. It on master - not npm so

cd ~/.node-red
npm i node-red/node-red-dashboard

Cool! Its running!!
Happy with it!
Thanks for fixing it so quickly!

1 Like

Why not have multiple outputs for the play action node. one for media start, one for media playing and one for media error and one for media stopped

I think we have 3 of those. Adding error would make sense.

Hey @RogierQ, @hugobox,

In case you haven't noticed already: Dave has published yesterday dashboard version 2.11.0 on NPM, which contains the requested audio-out node features:

  • Ability to stop playing audio with msg.reset=true
  • The node status will display whether the audio is 'playing', 'reset', 'error' or empty when playing has finished normally.

Have fun with it!
Bart

2 Likes

Thanks! Got it running :wink: