[Announce] node-red-node-ui-microphone

Share Your Nodes
1

I've just published a new Dashboard node for capturing audio from the microphone.

It gives you a simple button in the dashboard that when clicked starts recording until the button is clicked again or the configured max-length is reached. The audio is then published from the node as a Buffer containing the WAV data.

It uses the MediaRecorder API that is widely supported, however:

  • no support in IE at all - don't ask
  • it needs to be enabled in Safari (Develop -> Experimental Features -> MediaRecorder)

This is very much a minimum-viable-node to address an immediate need. There are some ideas for further features such as:

  • timeslicing the recording so it publishes at regular intervals whilst recording rather than just at the end
  • automatically detect (relative) silence to stop the recording - found some good blog posts on how to do this in principle
  • some more UI options such as adding a time count

And I have no doubt you will all throw back some ideas. Just be aware I'm not planning on spending a huge amount of more time on this right now having got it working for my immediate needs.

You can see it in action here: https://www.youtube.com/watch?v=CxiwO4nxSgI

The microphone node is wired straight to an HTTP Request node that posts the audio to the IBM Watson Speech To Text service (Puts on IBMer hat - free to sign up for without a credit card, the lite plan gives you 500 minutes of speech-to-text a month). The response is passed to a ui_text node.

It's pleasing when these things Just Work.

@BartButenaers I did do some due diligence before starting to write this node, and of course it was only just after I hit publish did I stumble over your https://www.npmjs.com/package/node-red-contrib-ui-media-source node. But I note its in beta and has no node-red keyword, so I don't feel too bad about not finding it.

7 Likes
2

Nice. I like the ideas there.

Being able to "stream" the recording back to Node-RED over websockets seems like a really useful feature. Doubtless much harder than it sounds :slight_smile:

3

Absolutely not! Good news :champagne:

During development of that node, I was doubting whether the amount of data wouldn't become too high. For example if I want to talk (full duplex) via the dashboard (running on my smartphone) to my video doorbell. So I tried to compress the raw audio (in the dashboard at client side) to mp3 chunks, but - due to that extra feature - I ran out of time to complete that node ...

4

Heh - I know what you mean. This is why I've got the node to MVP state and published it. Lots of other things it could do - see my list above, but also you're reminded me that picking audio format (webm/wav etc) could be on the list.

1 Like
5

Just wondered what is the difference/advantage of this node is compared with node-red-contrib-micropi or node-red-contrib-mic plus a dashboard button?

6

Those nodes record audio from the microphone connected to the device running node-red.

This node records audio from the browser displaying the dashboard - which may not be the same device.

If my Node-RED instance is hosted in the cloud, I don't have a USB cable long enough to plug a microphone in. :wink:

4 Likes
7

Nick,

the config screen says that the recording stops when the button is clicked again, but it keeps recording here (in Chrome on Windows 10):

demo_microphone

Do I need to do anything special to stop recording?

P.S. For anybody interested in testing this lovely node, you simply wire the output to a node-red-contrib-play-audio node, to play the recorded WAV buffers in the flow editor:

image

[{"id":"5531e06.3b9f32","type":"play audio","z":"1a2564e7.36573b","name":"","voice":"","x":830,"y":140,"wires":[]},{"id":"7685eaa.40ac114","type":"ui_microphone","z":"1a2564e7.36573b","group":"b0bf7afb.3fb1c8","order":0,"width":0,"height":0,"name":"","maxLength":"2","timeslice":0,"x":640,"y":140,"wires":[["5531e06.3b9f32"]]},{"id":"b0bf7afb.3fb1c8","type":"ui_group","z":"","name":"Default","tab":"2aef1413.c4ee3c","disp":false,"width":"6","collapse":false},{"id":"2aef1413.c4ee3c","type":"ui_tab","z":"","name":"Home","icon":"dashboard","disabled":false,"hidden":false}]
8

Nice! that's ingenious.

9

Yeah, just noticed that and currently googling to find out what API call I've missed off.... except I've followed the Chrome Developer guide precisely, so not sure at the moment.

Keep an eye out for 0.1.1 when I do figure it out.

1 Like
10

And Lo! 0.1.1 was published and it did close the streams properly and they rejoiced.

1 Like
11

Hey Nick,

That problem is indeed solved. Thanks!

But I have two new questions:

  • You had a good proposal above about audio compression via webm, ... (which I hadn't thought about so I started struggling with mp3 in the past). But I see this in the current code:

    mediaRecorder = new MediaRecorder(stream,  {mimeType: 'audio/webm'});

    Does this mean you already use webm?
    But then I wouldn't expect to get WAV chunks? I'm lost ...

  • And it is not clear to me how this field works:

    image

    I understand from the MediaRecorder documentation, that the timeslice is being used as the length of EACH chunk:

    image

    But when I specify N seconds, then I get the audio only after N seconds in my Node-RED flow. And after N seconds the recording stops automatically. I would have expected to get every N seconds an audio chunk (N seconds long chunks), until I press the button to stop recording. Or am I mistaken?

Thanks again!

12

From my research, audio/webm is seemingly the only format that is supported by all browsers that support MediaRecorder. So I capture the audio with that, then use code from the microphone.js library to re-encode as WAV before sending back to the runtime.

There are other libraries for re-encoding as mp3 and others, but they add quite a lot of bulk to the node and would also slow down the time taken to process the audio before its sent back.

The max length of recording is literally that - it is the maximum length of audio the node will capture after you click the button. In other words, if you set it to X seconds, then it will stop recording after X seconds if you haven't pressed the button already. It is intended as a safety net if the user clicks the button but never stops it.

It is not the same as the timeslice option in the underlying API - that is a different feature that I mentioned in the list of possible future enhancements. In fact, the node already has code in it to expose the timeslice option, but its commented out as I hit some snags with it and ran out of time to get it working properly. It will add another option to select when the node should send the audio - with options of "at the end of the recording" or "at regularly intervals of Y seconds".

1 Like
13

It doesnt work for me.
Tried in in Safari on the Iphone and Chrome and Firefox on the Mac.
It never asks to use the microphone and just spins doing nothing.
I see this error in the firefox console:

TypeError: "navigator.mediaDevices is undefined"
    toggleMicrophone http://wohnungoh2:1880/ui/js/app.min.js line 596 > eval:18
    fn http://wohnungoh2:1880/ui/js/app.min.js line 272 > Function:4
    e http://wohnungoh2:1880/ui/js/app.min.js:219
    $eval http://wohnungoh2:1880/ui/js/app.min.js:175
    $apply http://wohnungoh2:1880/ui/js/app.min.js:175
    compile http://wohnungoh2:1880/ui/js/app.min.js:219
    dispatch http://wohnungoh2:1880/ui/js/app.min.js:20
    handle http://wohnungoh2:1880/ui/js/app.min.js:20
app.min.js:148:303

and this in chrome:

TypeError: Cannot read property 'getUserMedia' of undefined
    at m.$scope.toggleMicrophone (eval at <anonymous> (app.min.js:596), <anonymous>:18:56)
    at fn (eval at compile (app.min.js:272), <anonymous>:4:168)
    at e (app.min.js:219)
    at m.$eval (app.min.js:175)
    at m.$apply (app.min.js:175)
    at HTMLButtonElement.<anonymous> (app.min.js:219)
    at HTMLButtonElement.dispatch (app.min.js:20)
    at HTMLButtonElement.v.handle (app.min.js:20)
(anonymous) @ app.min.js:148
(anonymous) @ app.min.js:120
$apply @ app.min.js:176
(anonymous) @ app.min.js:219
dispatch @ app.min.js:20
v.handle @ app.min.js:20

Any clue what could be the reason?
Best regards Johannes

14

Have you enabled MediaRecorder in Safari?

Not sure why you get that error in Chrome/FF on Mac as that's exactly what I developed and tested it against.

I will add some feature detection and disable the button if MediaRecorder isn't available. And add those notes about safari to the readme.

15

I had enabled the MediaRecorder in the experimental features on the iphone before testing.

16

From MediaDevices: getUserMedia() method - Web APIs | MDN :

Note: If the current document isn't loaded securely, navigator.mediaDevices will be undefined , and you cannot use getUserMedia() . See Security for more information on this and other security issues related to using getUserMedia() .

So the issue is you are serving the page over http not https.

17

Ah im using http because using the adding the dashboard to the homescreen gave me problems with self signed certificates on the iphone. Thanks

18

I've just published v0.1.2 that disables the button if we detect mediaDevices is not available.

I've also updated the readme to add browser support information - including the need to use https if you are accessing the dashboard remotely (ie not via localhost).

3 Likes
19

hey Nick,

I installed the application, but no microphone icon appears on the panel. the following message is reported in the debug: "TypeError: Cannot read property 'push' of undefined".

20

Hi @jneto,

What version of node.js, Node-RED and node-red-dashboard have you got installed?

Any errors in the browser console? What browser are you using?