Play Sound from USB Microphone

Dashboard
39

Hi @GChapo, @dceejay,

I have been doing a lot of experiments, and finally I have now a solution that seems to be working. Since these were my first steps into the world of audio signals, don't hesitate to let me know if something is not correct !

image

[{"id":"2852077b.af8aa8","type":"microphone","z":"279b8956.27dfe6","name":"microphone","endian":"little","bitwidth":"16","encoding":"signed-integer","channels":"1","rate":"22050","silence":"60","debug":false,"active":true,"x":950,"y":1220,"wires":[["c61c107a.970a2"]]},{"id":"c61c107a.970a2","type":"wav-headers","z":"279b8956.27dfe6","name":"","channels":1,"samplerate":22050,"bitwidth":16,"x":1136,"y":1220,"wires":[["7597ba6e.5adbb4"]]},{"id":"7597ba6e.5adbb4","type":"ui_audio","z":"279b8956.27dfe6","name":"","group":"180d570c.93b059","voice":"0","always":false,"x":1320,"y":1220,"wires":[]},{"id":"5482134.a222dec","type":"inject","z":"279b8956.27dfe6","name":"record","topic":"","payload":"true","payloadType":"bool","repeat":"","crontab":"","once":false,"onceDelay":"","x":770,"y":1200,"wires":[["2852077b.af8aa8"]]},{"id":"363b5caf.bf44c4","type":"inject","z":"279b8956.27dfe6","name":"stop","topic":"","payload":"false","payloadType":"bool","repeat":"","crontab":"","once":false,"x":770,"y":1240,"wires":[["2852077b.af8aa8"]]},{"id":"180d570c.93b059","type":"ui_group","z":"","name":"Devices","tab":"493cf398.76af9c","order":2,"disp":false,"width":"6"},{"id":"493cf398.76af9c","type":"ui_tab","z":"","name":"Ratby Road","icon":"dashboard"}]

Have lost two evenings figuring out why I got no sound at all. At the end I read some review of my USB microphone: I had to scream loud at 2 inch distance from my USB microphone, to be able to hear something. So I wouldn't suggest anybody to buy such a microphone :rage:

  1. I use the 'microphone' node (from node-red-contrib-micropi) to get audio samples from the USB microphone on my Raspberry. That node gives a stream of data chunks containing raw audio samples (PCM). I don't use the Micropi node (from the same node set), since that node - like Glenn already discovered - only starts playing data when you send a stop message!

    Remark: those nodes keep on streaming audio when you (re)deploy your flow, which is annoying since you will end up with a mix of multiple streams. I created a pull request to stop the streams automatically at deploys.

  2. However when you send such raw audio chunks to audio-out node in the dashboard, the browser cannot decode the raw audio samples. I just doesn't know what all the bytes mean, so I had to develop the node-red-contrib-wav-headers, which adds WAV headers to the raw audio data.

    Remark: I haven't published this node on NPM yet, so you can install it currently like this:
    npm install bartbutenaers/node-red-contrib-wav-headers

    Remark: the readme file of this node contains an introduction to PCM, WAV, ... to get you started.

    Remark: the disadvantage is that you have to enter the same settings in both the microphone node and the wav node. Would have been better if the microphone node would be able to produce a WAV stream....

  3. We now have WAV audio chunks which can be decoded by the browser. But the audio-out node currently can only play a single audio file:

    try {
     audiocontext = audiocontext || new AudioContext();
     var source = audiocontext.createBufferSource();
     var buffer = new Uint8Array(msg.audio);
     audiocontext.decodeAudioData(buffer.buffer, function(buffer) {
          source.buffer = buffer;
          source.connect(audiocontext.destination);
          source.start(0);
     })
}
catch(e) { alert("Error playing audio: "+e); }

    Indeed the decodeAudioData seems not to be designed for chunked audio stream, i.e. you need to feed it with a complete audio file of finite length. But I found a workaround, and I changed the dashboard code like this:

    try {
     audiocontext = audiocontext || new AudioContext();
     var source = audiocontext.createBufferSource();
     var buffer = new Uint8Array(msg.audio);
     audiocontext.decodeAudioData(buffer.buffer, function(audioBuffer) {
           audioStack.push(audioBuffer);
           while ( audioStack.length) {
                var chunkBuffer = audioStack.shift();
                var source = audiocontext.createBufferSource();
                source.buffer = chunkBuffer;
                source.connect(audiocontext.destination);
                if (nextTime == 0) {
                      nextTime = audiocontext.currentTime + 0.01;  
                }
                source.start(nextTime);
                // Make the next buffer wait the length of the last buffer before being played
                nextTime += source.buffer.duration; 
           }
      }, function(e) {
           console.log("Error decoding audio: "+e);
      });
}
catch(e) { console.log("Error playing audio: "+e); }

    Remark: the audioStack variable is declared here:

    app.controller('MainController', ['$mdSidenav', '$window', 'UiEvents', '$location', '$document', '$mdToast', '$mdDialog', '$rootScope', '$sce', '$timeout', '$scope',
 function ($mdSidenav, $window, events, $location, $document, $mdToast, $mdDialog, $rootScope, $sce, $timeout, $scope) {
     var audioStack = [];

    Remark: I had to change the 'alert' in the catch to 'console.log', otherwise - in case the stream cannot be decoded - an error popup would be displayed for every chunk!

    Remark: the inventor of this mechanism added a 50ms latency 'to work well across systems'. I assume a constant value is fine, but otherwise a number input field should be added to the audio-out node's config screen?

This was only a proof of concept, but - due to a lack of time - there are still some TODO's:

  • Some extra tests would need to be done. Would be nice if somebody else could do this ... E.g. I did a basic test whether the audio-out node can still play an audio file like in the original version, but perhaps other tests are required.
  • I haven't done any performance tests yet.
  • Some extra nodes should be developed, e.g. to convert the PCM chunks to MP3 chunks. Because currently a lot of data is transferred to the browser !! I have already developed some stuff, but I have not enough time to finish it all in a short period of time ...
  • And the most difficult challenge: convince Dave to update the audio-out node. Still have to figure out how to accomplish that :thinking:

Shoot !!!
Bart

2 Likes