# Streaming live audio data via MQTT
**URL:** https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602
**Category:** General
**Tags:** mqtt
**Created:** [18 August 2022 11:29 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602 "2022-08-18T11:29:08Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [18 August 2022 11:29 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/1 "2022-08-18T11:29:08Z")
Hello,
I am trying to stream live audio via MQTT in Node-red to Raspberry Pi.
For testing microphone and speaker, when I connected them through a "Add WAV headers" (like the below image), the output sound is quite clear.
I also tested to stream live audio via MQTT between 2 Raspberry Pi Zero. The output sound is quite clear too.
But when I try to recieve and send audio data via MQTT , between Node-Red and Raspberry Pi, like 2 pictures below, the audio output ( in both PC and Raspberry Pi) is full of noise.
and
Could anyone tell me the reason why and how I can solve this problem.
Best regards,
Nhien.
---
### Author:  [@Andrei](https://discourse.nodered.org/u/Andrei)
#### Post date: [18 August 2022 13:24 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/2 "2022-08-18T13:24:08Z")
Welcome to the forum. This is the first time I hear someone trying to use MQTT for streaming real time data. What is the reasoning behind using MQTT instead of a protocol designed specifically for real time data (like RTSP) ? Where the MQTT broker is located and how your MQTT is setup (qos) ?
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [18 August 2022 14:24 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/3 "2022-08-18T14:24:14Z")
Hi Andrei,
Thank you so much for your reply.
I'm very new in streaming real-time data, and I found MQTT helping me to construct the network conveniently. I have never try RTSP, but I will try soon.
I set another Raspberry Pi as MQTT broker, and also Node-Red installed machine.
QoS for both "MQTT in node" and "MQTT out node" is 0.
Best regards.
---
### Author:  [@Andrei](https://discourse.nodered.org/u/Andrei)
#### Post date: [18 August 2022 15:10 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/4 "2022-08-18T15:10:14Z")
Well, you could experiment with qos 2 instead of 0 to see if it makes any difference. I am not optimistic though. Your use case is somehow similar to voip in the sense that latency and jitter will have a great impact on the quality of the results. There is no mechanism built in MQTT to help with jitter (delay variations). I wish I had some spare time to read some papers and essays from the experts on this subject. Your testing may provide insightful feedback to the community so please keep on. One more question, which node is the one in purple color, named "Add WAV headers"? Perhaps something from Home Assistant ?
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [18 August 2022 15:28 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/5 "2022-08-18T15:28:42Z")
Thank for your reply!
I also tried with QoS 2, but nothing was changed.
"Add WAV headers" node adds WAV headers to raw audio data for playing with "audio out" node.
I tested "Add WAV headers" node , and it ran corectly.
Thank for your support and best regard!
---
### Author:  [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)
#### Post date: [18 August 2022 22:01 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/6 "2022-08-18T22:01:58Z")
> [@nhiennguyenhuy](#):
>
> "Add WAV headers" node adds WAV headers to raw audio data for playing with "audio out" node.
> I tested "Add WAV headers" node , and it ran corectly.
in which case the data or format of the data from MQTT may be incorrect.
What does the payload of a working audio clip look like vs the payload you get from MQTT?
---
### Author:  [@wb666greene](https://discourse.nodered.org/u/wb666greene)
#### Post date: [18 August 2022 23:42 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/7 "2022-08-18T23:42:57Z")
Check your MQTT input node settings. I've ran into an issue where it defaults to "String" instead of "Auto". Changing it to "Buffer" fixed the problem I was having.
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [19 August 2022 00:30 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/8 "2022-08-19T00:30:32Z")
Thank for your reply!
The data from MQTT is look like the below image. Data type is "byte"
.
It is audio raw data, so I think I have to add wav header for "audio out" node to play it.
Best regards.
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [19 August 2022 01:03 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/9 "2022-08-19T01:03:07Z")
Thank for your reply!
I tried to change MQTT input node settings as you recommened , but the quality of the audio is not changed.
I can hear what other say, but the sound is clipping. After checking the MQTT data which I send from Raspi Zero, I found that sometimes data is full of 0 byte. Maybe it is the reason of sound clipping.
I'm going to check the code in Raspi Zero.
Best regards.
---
### Author:  [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)
#### Post date: [19 August 2022 07:34 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/10 "2022-08-19T07:34:27Z")
Hi @nhiennguyenhuy,
I don't think your problem has anything to do with mqtt. _ **The dashboard's audio-out node simply can only play a single audio buffer, for example an mp3 file** _. You push an infinite stream of audio fragments to it, which it is not what this node is designed for...
I have tried in the past to implement this feature, but it was just to hard in my limited spare time.
- You can easily fool your eyes by sending separate images to it: your eyes will believe they see fluent video.
- It is very hard to fool your ears: they will hear every abnormal transition from one segment to the next.
At one moment the audio chunks will arrive too soon, so some of them will be skipped by the audio node. At another moment the audio chunks will arrive not fast enough, which means there will be a gap of silence. In both cases you get _ **steep edges** _, as transition between succesive audio fragments. And that is what your ears hear.
The audio-node immediately plays a fragment, as soon as you pass it one. I tried to add a buffer to it in the past (i.e. an array where N input fragments are being stored), and then it was already much better. Because if the buffer is large enough, the node will play all the segments nicely after each other. So no steep edges between the fragments.
However the buffer size needs to be small, otherwise the delay will become too noticable. And then you get again more distortions.
Perhaps meanwhile new javascript libraries have been implemented that can solve this problem. Don't know...
When you search on this forum, you will quickly find that others have tried this before you already (e.g. [here](https://discourse.nodered.org/t/node-allowing-for-live-chat-talk-and-live-sound/8739/5)) ...
Bart
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [19 August 2022 14:06 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/11 "2022-08-19T14:06:37Z")
Hi @BartButenaers,
Thank you so much!
Your explanation is very clear and intelligible. I learn so much from that.
As I said above, I also tested to stream live audio via MQTT between 2 Raspberry Pi Zero and the output sound is quite clear.
In the below picture, when I wanted to send raw audio from microphone (plugged to Node-red installed machine) to Raspi Zero. I got _ **steep edges** _ as you said.
I mean in the first case, Raspi Zero can reproduce audio nice, but in the second case,
it can't.
I think the way how the audio transfers is the same with which I am trying to do: Raw data from microphone is splited in separate buffer ( size of buffer can be set as you said) before sending by MQTT.
I can't understand what is the different thing here.
Best regards,
Nhien.
---
### Author:  [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)
#### Post date: [19 August 2022 19:36 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/12 "2022-08-19T19:36:05Z")
> [@nhiennguyenhuy](#):
>
> I also tested to stream live audio via MQTT between 2 Raspberry Pi Zero and the output sound is quite clear.
This is something I cannot really understand. Of course what do you mean with "quite clear"? During my experiments I had quite some irritating sounds. Based on my experiments there was no way to get a clear audio signal on my speakers this way.
The only solution (at the time being) was _ **smoothing** _ the transitions between (out of order) audio segments to get rid of the steep edges.
Don't know if it is possible to share a short audio sample here, so that we can hear the result of your test?
> [@nhiennguyenhuy](#):
>
> But when I try to recieve and send audio data via MQTT , between Node-Red and Raspberry Pi, like 2 pictures below, the audio output ( in both PC and Raspberry Pi) is full of noise.
Although your question is very well documented, I don't understand the difference between the experiments:
- A live audio stream via MQTT between 2 Raspberry zero's give good quality.
- A live audio stream between Node-RED and Raspberry Pi is full of Noice.
Isn't that the same? I assume you run Node-RED also on both Raspberry zero's?
A quick sketch of both setups would probably help, so that I can understand which nodes are running where...
> [@nhiennguyenhuy](#):
>
> the audio output ( in both PC and Raspberry Pi) is full of noise.
In both cases (pc and raspberry) you listen to the audio via your Node-RED dashboard? Do you mean that in the second case you run your browser installed on the Raspberry? Would also help if you explain this in your sketch...
> [@nhiennguyenhuy](#):
>
> Raw data from microphone is splited in separate buffer ( size of buffer can be set as you said) before sending by MQTT.
Can you explain a bit more why you think this could be the cause?
From your screenshots I assume that you use the microphone node in all your experiments to capture the voice? Or is that not correct? And is it is [this](https://flows.nodered.org/node/node-red-contrib-browser-utils/in/_pz2l9UcJUsc) microphone node?
Note that you could perhaps use my node-red-contrib-msg-speed and node-red-contrib-msg-size nodes to calculate statistics about how much segments are travelling through your wires, and the sizes of those segments. If you think data is chunked somewhere, you should different statistics in your two experiments...
To understand where chunking could get messy, it would be very useful to see the entire tranject (from microphone to the speakers) in a single (manually drawn) sketch...
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [20 August 2022 02:11 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/13 "2022-08-20T02:11:47Z")
Thank you so much for your reply!
> [@BartButenaers](#):
>
> In both cases (pc and raspberry) you listen to the audio via your Node-RED dashboard? Do you mean that in the second case you run your browser installed on the Raspberry? Would also help if you explain this in your sketch...
I'm so sorry for my unintelligible post.
The below image is the first case sketch, which I have quite clear audio.
and
The below image is the second case sketch.
In both case I use python for processing audio data on Raspi Zero.
> [@BartButenaers](#):
>
> From your screenshots I assume that you use the microphone node in all your experiments to capture the voice? Or is that not correct? And is it is [this](https://flows.nodered.org/node/node-red-contrib-browser-utils/in/_pz2l9UcJUsc) microphone node?
No, It isn't. I use this one for capturing audio
> **[node-red-contrib-micropi](https://flows.nodered.org/node/node-red-contrib-micropi)**
>
> MicroPi: A set of nodes for recording and streaming audio from an usb microphone connected to a Raspberry Pi in Node-Red.
[microphone.json](https://discourse.nodered.org/uploads/short-url/fJiZ2u1dhYG2IHgytYx8j5qWcqd.json) (518 Bytes)
> [@BartButenaers](#):
>
> Can you explain a bit more why you think this could be the cause?
Maybe the program read the previous buffer to fast and no buffer come ...
> [@BartButenaers](#):
>
> This is something I cannot really understand. Of course what do you mean with "quite clear"? During my experiments I had quite some irritating sounds. Based on my experiments there was no way to get a clear audio signal on my speakers this way.
This is first case's audio sample.
[https://drive.google.com/file/d/1HxfDCtEb15dCDfe6XmM-VNAmW16I-qHo/view?usp=sharing](https://drive.google.com/file/d/1HxfDCtEb15dCDfe6XmM-VNAmW16I-qHo/view?usp=sharing)
And this is second case's audio sample.
[https://drive.google.com/file/d/1HyIDSqBx8vLbaaLO3-1KJIlw9amWi6sh/view?usp=sharing](https://drive.google.com/file/d/1HyIDSqBx8vLbaaLO3-1KJIlw9amWi6sh/view?usp=sharing)
> [@BartButenaers](#):
>
> Note that you could perhaps use my node-red-contrib-msg-speed and node-red-contrib-msg-size nodes to calculate statistics about how much segments are travelling through your wires, and the sizes of those segments. If you think data is chunked somewhere, you should different statistics in your two experiments...
>
> To understand where chunking could get messy, it would be very useful to see the entire tranject (from microphone to the speakers) in a single (manually drawn) sketch...
Once again thank you for your advices.
Best regards,
Nhien.
---
### Author:  [@krambriw](https://discourse.nodered.org/u/krambriw)
#### Post date: [20 August 2022 04:35 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/14 "2022-08-20T04:35:45Z")
> [@nhiennguyenhuy](#):
>
> In both case I use python for processing audio data on Raspi Zero
OK, so on the second example, that's the difference. There you process the audio data with NR javascript
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [20 August 2022 06:06 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/15 "2022-08-20T06:06:02Z")
Hi @krambriw ,
Thank for your reply.
> [@krambriw](#):
>
> There you process the audio data with NR javascript
Exactly! And when I test like the below example ( in the first post), the result is OK.
> [@nhiennguyenhuy](#):
>
> For testing microphone and speaker, when I connected them through a "Add WAV headers" (like the below image), the output sound is quite clear.
That means if I do all in python or NR javascript, it is OK,
But for example, if I send audio by NR and receive audio by python, it is not OK.
And it confused me.
---
### Author:  [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)
#### Post date: [20 August 2022 06:57 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/16 "2022-08-20T06:57:19Z")
Thank you for explaining it so well!
## Experiment 1
From this we can conclude that Python does it very well, both for capturing the audio from the USB microphone and for playing that audio on your speaker. So Python does somehow reassemble the audio chunks fine. Is there a large delay perhaps: because that would indicate that they use a buffer that is large enough. When that is not the case they do some good processing of the chunks.
So we need to know which of those two part (capture or playing) fails in Node-RED, by two extra experiments.
## Experiment 2
Perhaps you did this experiment already:
By doing this, we only replace the python library (which captures the usb microphone audio) by the node-red-contrib-micropi node. If this works well, then we at least know that the micropi node is doing a good job.
## Experiment 3
By doing this we only replace the python library (which plays the audio on the speaker) by the ui-audio-out node. I _ **don't** _ expect this to give good audio quality, by the web-audio issue that I described above....
---
### Author:  [@nhiennguyenhuy](https://discourse.nodered.org/u/nhiennguyenhuy)
#### Post date: [20 August 2022 08:19 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/17 "2022-08-20T08:19:11Z")
Hi Bart,
Thank you for your time on my post!
> [@BartButenaers](#):
>
> Experiment 2
I didn't use Node "Add headers" for experiment 2. I send raw audio data directly to mqtt-out.
I tested all the experiment above.
The quality order is Experiment 1 \> Experiment 3 \> Experiment 2 .
I also tested for this case. The audio is very clear.
Does it have any meaning? Does it mean the NR library is good enough for capturing and playing audio?
---
### Author:  [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)
#### Post date: [20 August 2022 08:41 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/18 "2022-08-20T08:41:27Z")
> [@nhiennguyenhuy](#):
>
> I also tested for this case. The audio is very clear
From your experiment (let's call it _ **experiment 4** _) I conclude that the node-red-contrib-micropi node captures the audio very well, since it results in good audio after playback.
For some reason the dashboard audio-out ui node is able to play it rather good: I assume the chunks arrive in a pretty decently order, otherwise I cannot explain this. But this won't be the case always.
You could do also _ **experiment 5** _:
Because it seems that mqtt introduces timing issues, which is just too much for the dashboard audio-out node to be able to play it correctly.
---
### Author:  [@krambriw](https://discourse.nodered.org/u/krambriw)
#### Post date: [20 August 2022 09:19 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/19 "2022-08-20T09:19:39Z")
Experiment 1: only Python & MQTT involved, no distortion added, best quality nbr 1
Experiment 2: Input from mic captured by NR adding distortion, then transferred via MQTT, Python at receiving end not able to "repair" distorted sound, quality nbr 3
Experiment 3: Input from mic captured by Python, then transferred via MQTT, NR at receiving adding distortion, quality nbr 2
Maybe use Python on both ends _before_ transferring to NR -\> speaker
---
### Author:  [@BartButenaers](https://discourse.nodered.org/u/BartButenaers)
#### Post date: [20 August 2022 09:39 UTC](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602/20 "2022-08-20T09:39:42Z")
> [@krambriw](#):
>
> Maybe use Python on both ends _before_ transferring to NR -\> speaker
Hey Walter,
that could indeed be a quick workaround.
But if nobody diggs into this problem, we keep on getting the same issue until ethernity.
Would be nice if we could have a complete Node-RED solution...
Because there are enough use cases for this.
[Next page](https://discourse.nodered.org/t/streaming-live-audio-data-via-mqtt/66602.md?page=2)