Dashboard2 - seeking help formatting a msg for the ui-audio node, endpoints don't seem to work

Dashboard
,
1

Hello,

I am having success using Piper to generate TTS for voice notifications. I'm able to use a POST message to send text to an instance of Piper running in a Docker container, which returns the spoken text as a binary buffer.

I've tested it with node-red-contrib-play-audio, and the results are about as glorious as I could ever have hoped.

However, the next most obvious step is to wrap that buffer properly in a nice object and publish it to MQTT, and thus any device with a speaker that is subscribed to that topic will play it. I will be using this for spoken notifications in specific locations.

The node-red-contrib-play-audio node is great, but I'd also like to be able to play voice notifications on any device viewing the Dashboard, therefore ui-audio.

Where I need help is to format a ui_update.src message so a the ui-audio node is happy to accept that message as input. It wants the location of the audio as a URL ... but I have it right here in this buffer. Here. Have it. Please.

I need to somehow con ui-audio into believing that a local buffer is an URL, or ... uhhhh ... help.

Please, assuming a msg.payload of a raw buffer, please help me write a function or set up a change node such that ui-audio will play it. There has to be a field or a parameter I'm not seeing.

Simply feeding ui-audio the buffer in ui_update.src has not worked. This is a formatting thing. Halp. I stuck.

This is what I have so far:


[{"id":"6a137c41d3addec5","type":"ui-template","z":"3567e03e18126502","group":"d3197b4c4dd7b057","page":"","ui":"","name":"","order":2,"width":0,"height":0,"head":"","format":"<template>\n    <!-- if you remove the `controls` attribute, you MUST click soomething on the page before initiating play -->\n    <audio ref=\"audio\" controls autoplay src=\"\"> </audio>\n</template>\n\n<script>\n    export default {\n        watch: {\n            msg: function () {\n                if (this.msg?.topic === 'play') {\n                    if (this.$refs.audio.src !== this.msg.payload ) {\n                        this.$refs.audio.src = this.msg.payload\n                    }\n                    this.$refs.audio.play()\n                } else if (this.msg?.topic === 'stop') {\n                    this.$refs.audio.src = ''\n                } else if (this.msg?.topic === 'pause') {\n                    this.$refs.audio.pause()\n                }\n            }\n        }\n    }\n</script>\n\n<style>\n.hidden {\n    display: none !important;\n}\n</style>","storeOutMessages":true,"passthru":true,"resendOnRefresh":true,"templateScope":"local","className":"hidden","x":320,"y":340,"wires":[[]]},{"id":"caf334102ab7952d","type":"inject","z":"3567e03e18126502","name":"cantina","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"play","payload":"https://media.geeksforgeeks.org/wp-content/uploads/20240218213800/CantinaBand60.wav","payloadType":"str","x":170,"y":380,"wires":[["6a137c41d3addec5"]]},{"id":"8e135a0e9bfdccb7","type":"inject","z":"3567e03e18126502","name":"march","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"play","payload":"https://www2.cs.uic.edu/~i101/SoundFiles/ImperialMarch60.wav","payloadType":"str","x":170,"y":340,"wires":[["6a137c41d3addec5"]]},{"id":"3c290f1ea6cf8b36","type":"inject","z":"3567e03e18126502","name":"pause","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"pause","x":170,"y":300,"wires":[["6a137c41d3addec5"]]},{"id":"436efb1217534529","type":"inject","z":"3567e03e18126502","name":"stop","props":[{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"stop","x":170,"y":260,"wires":[["6a137c41d3addec5"]]},{"id":"d3197b4c4dd7b057","type":"ui-group","name":"P2 G1","page":"6685af11067a04cd","width":11,"height":"1","order":1,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"6685af11067a04cd","type":"ui-page","name":"grid page","ui":"22ea43815413e748","path":"/page2","icon":"home","layout":"grid","theme":"70e58855f40712e7","breakpoints":[{"name":"Default","px":"0","cols":"3"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":2,"className":"","visible":"true","disabled":"false"},{"id":"22ea43815413e748","type":"ui-base","name":"base","path":"/dashboard","appIcon":"","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":false,"showPageTitle":true,"navigationStyle":"icon","titleBarStyle":"default"},{"id":"70e58855f40712e7","type":"ui-theme","name":"pink Theme","colors":{"surface":"#ffffff","primary":"#d355a5","bgPage":"#e1bcbc","groupBg":"#fbe9e9","groupOutline":"#dc8f8f"},"sizes":{"density":"compact","pagePadding":"12px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"12px"}}]
2

You can serve the audio file like any other file via an endpoint. That endpoint can be served from node-red using either httpStatic settings in your settings.js or create and endpoint using http-in-> read file -> http-response

Update. I forgot to say, you can also send the buffer instead of loading a file. You'll likely have to store it in context or generate/grab it upon request, but the concept is the same.

3

Hi @Steve-Mcl

I'm so frustratingly close to understanding what you said. Like, I get in theory, but my brain needs to see an implementation.

Let me try and restate.

In Dashboard2, audio must be played with the ui-audio node. It won't accept a buffer as input, for reasons that confuse me - surely this is arbitrary? If it can pull the buffer from an URL, why can't it just accept a buffer that's handed to it?

ui-audio must have input as an URL to where the audio is. Even though I am standing there waving a buffer at it, it wants that audio hosted somewhere instead, is the totality of what my tiny brain has grasped so far.

So I have msg.payload containing a buffer of good audio that I can already play with other methods. I just can't play it in the dash because of ... whyyyyyyyyyyy?

All I am trying to do is turn that buffer into something ui-audio will play. I've spent a day trying to do this and failing. I'm an old farmer, trying to code, and am frustrated beyond belief.

4

An endpoint is a URL - you make it serve your buffer.

Here are 2 similar solutions (casting to a google home speaker is the same deal)

1 Like
5

Hi @Steve-Mcl,

Thank you, I learned a lot, and have got it working. Your help was invaluable.

For anyone else who finds this thread via search, here is my understanding of what I learned; please correct as necessary.

If you have audio in a buffer (such as the output of a TTS ai like Piper), you cannot pass it directly to the ui-audio node, because that node needs an URL as input.

We therefore create one, using a thing called an HTTP Endpoint. The name was confusing to me, but essentially you're creating a mini-webserver that any other flow can punt a request at.

So you set up an http-in node, and you give it a URL that you like. For example, it might be /voice. That means, this node will respond on http://localhost:1880/voice, if you send a request there. You plug it directly into an HTTP-out node (which is the node that will respond when called), and forget about it.

Back to the ui-audio node. Pop it open, and in the URL field, put http://localhost:1880/voice. That's where it'll send a request when it receives a message.

Great, now back to the msg.payload that has a buffer in it.

You send that to an http-request node, configured to call the http-in endpoint we created earlier (http://localhost:1880/voice). Because the buffer is in msg.payload, it gets sent to the endpoint too, thus making that buffer the thing that will return when the URL is queried.

You connect the the output of the http-request node to the ui-audio node, which you configured to query http://localhost:1880/voice, which is now the URL of that buffer.

Tadaaaa! audio should play.

This is the flow that worked for me:

[{"id":"18236fdc807e7dd6","type":"function","z":"1b1d637ac0577302","name":"Format the message the way LocalAI likes","func":"var inputtext = msg.payload;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":400,"y":1120,"wires":[["55b6b2b54568c96f"]]},{"id":"55b6b2b54568c96f","type":"http request","z":"1b1d637ac0577302","name":"Send to Piper on LocalAI instance","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://[your instance]:8080/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"other","keyValue":"Content-Type","valueType":"other","valueValue":"application/json"}],"x":400,"y":1180,"wires":[["ce5946b404c49809"]]},{"id":"e6cc61786434fe8c","type":"inject","z":"1b1d637ac0577302","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"This is what voice notifications sound like when generated using Piper's alba model.","payloadType":"str","x":170,"y":1120,"wires":[["18236fdc807e7dd6"]]},{"id":"325349d714f921ac","type":"http in","z":"1b1d637ac0577302","name":"","url":"/voice","method":"post","upload":true,"swaggerDoc":"","x":170,"y":1300,"wires":[["454d2974a46b1b2a"]]},{"id":"454d2974a46b1b2a","type":"http response","z":"1b1d637ac0577302","name":"","statusCode":"","headers":{},"x":350,"y":1300,"wires":[]},{"id":"ce5946b404c49809","type":"http request","z":"1b1d637ac0577302","name":"","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://localhost:1880/lucy","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":730,"y":1120,"wires":[["369a44ec9d422aaa"]]},{"id":"369a44ec9d422aaa","type":"ui-audio","z":"1b1d637ac0577302","group":"0c76141b6b8548ea","name":"","order":2,"width":0,"height":0,"src":"http://localhost:1880/voice","autoplay":"on","loop":"off","muted":"off","x":940,"y":1120,"wires":[[]]},{"id":"d20b7f9e1a875f8b","type":"comment","z":"1b1d637ac0577302","name":"Injects a string. How you get your tts audio is up to you.","info":"","x":300,"y":1060,"wires":[]},{"id":"1e35b3c403beaf1f","type":"comment","z":"1b1d637ac0577302","name":"This calls the endpoint, and sends it the buffer","info":"","x":830,"y":1060,"wires":[]},{"id":"5430ea06f24919f3","type":"comment","z":"1b1d637ac0577302","name":"This is the endpoint thingy. It's what gives msg.payload a URL","info":"","x":320,"y":1260,"wires":[]},{"id":"0c76141b6b8548ea","type":"ui-group","name":"Zone arming","page":"a30cb6a10bfc3330","width":"4","height":"1","order":8,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"a30cb6a10bfc3330","type":"ui-page","name":"Newdash","ui":"cf9dc2ba60d1396c","path":"/page6","icon":"home","layout":"grid","theme":"7ba40ffa60b22682","breakpoints":[{"name":"Default","px":"0","cols":"4"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":1,"className":"","visible":"true","disabled":"false"},{"id":"cf9dc2ba60d1396c","type":"ui-base","name":"My Dashboard","path":"/dashboard","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":true,"navigationStyle":"default","titleBarStyle":"default"},{"id":"7ba40ffa60b22682","type":"ui-theme","name":"Default Theme","colors":{"surface":"#ffffff","primary":"#0080ff","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"pagePadding":"2px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"4px","density":"comfortable"}}]
1 Like
6

Nooooo I have broken something!

Please have a look at my flow; there must be something obvious I am not seeing.

[{"id":"b9765ce47c15b7ae","type":"debug","z":"1b1d637ac0577302","name":"First hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":710,"y":300,"wires":[]},{"id":"df41fe923050cae5","type":"debug","z":"1b1d637ac0577302","name":"Second hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":340,"y":520,"wires":[]},{"id":"203876051c371051","type":"debug","z":"1b1d637ac0577302","name":"Third hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1170,"y":300,"wires":[]},{"id":"99915e6780a51e6e","type":"inject","z":"1b1d637ac0577302","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"This is what voice notifications sound like when generated using Piper's alba model.","payloadType":"str","x":250,"y":180,"wires":[["61a1f51e26df5f13"]]},{"id":"e7ff0621bb935849","type":"http in","z":"1b1d637ac0577302","name":"","url":"/voice","method":"post","upload":true,"swaggerDoc":"","x":310,"y":400,"wires":[["df41fe923050cae5","5c90a491d9d9b7ff"]]},{"id":"5c90a491d9d9b7ff","type":"http response","z":"1b1d637ac0577302","name":"","statusCode":"","headers":{},"x":530,"y":400,"wires":[]},{"id":"301523ffa21de59d","type":"ui-audio","z":"1b1d637ac0577302","group":"0c76141b6b8548ea","name":"","order":0,"width":0,"height":0,"src":"http://localhost:1880/voice","autoplay":"on","loop":"off","muted":"off","x":1240,"y":260,"wires":[[]]},{"id":"61a1f51e26df5f13","type":"function","z":"1b1d637ac0577302","name":"Format the message","func":"var inputtext = msg.payload;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":300,"y":260,"wires":[["46b45af3b681a9d6"]]},{"id":"46b45af3b681a9d6","type":"http request","z":"1b1d637ac0577302","name":"Send to Piper","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://192.168.2.75:8080/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"other","keyValue":"Content-Type","valueType":"other","valueValue":"application/json"}],"x":520,"y":260,"wires":[["b9765ce47c15b7ae","a5ed50a96ffe5256"]]},{"id":"a5ed50a96ffe5256","type":"http request","z":"1b1d637ac0577302","name":"","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://localhost:1880/voice","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":950,"y":260,"wires":[["301523ffa21de59d","203876051c371051"]]},{"id":"e90eb6ec2ac6b923","type":"comment","z":"1b1d637ac0577302","name":"1. Everything is fine until here","info":"","x":700,"y":220,"wires":[]},{"id":"02c6732b4a8078ef","type":"comment","z":"1b1d637ac0577302","name":"2. when this is called, the payload arrives empty","info":"","x":560,"y":460,"wires":[]},{"id":"0c76141b6b8548ea","type":"ui-group","name":"Zone arming","page":"a30cb6a10bfc3330","width":"4","height":"1","order":8,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"a30cb6a10bfc3330","type":"ui-page","name":"Newdash","ui":"cf9dc2ba60d1396c","path":"/page6","icon":"home","layout":"grid","theme":"7ba40ffa60b22682","breakpoints":[{"name":"Default","px":"0","cols":"4"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":1,"className":"","visible":"true","disabled":"false"},{"id":"cf9dc2ba60d1396c","type":"ui-base","name":"My Dashboard","path":"/dashboard","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":true,"navigationStyle":"default","titleBarStyle":"default"},{"id":"7ba40ffa60b22682","type":"ui-theme","name":"Default Theme","colors":{"surface":"#ffffff","primary":"#0080ff","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"pagePadding":"2px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"4px","density":"comfortable"}}]

Right now, when I press Inject, I do get a valid buffer back. The debug at First Hurdle shows exactly what I expect: a msg.payload that's a nice buffer. If I point that to a Play Audio node, I can hear the voice; that part works.

It breaks when it goes through the HTTP endpoint, and I need to understand why: this is where my understanding gets weak.

The debug at Second Hurdle is in the middle of the HTTP endpoint. This debug shows that the payload arrives empty. I don't understand how.

What went wrong between those two points?

debug
debug301Ă—554 20.5 KB

This is the debug output. I am so confused right now.

7

Hello all, and @Steve-Mcl, who is nice.

I still haven't solved this and I'm close to tears. I don't know if it's a typo or what, but I know that the http endpoint is working because I can call it and feed it to node-red-contrib-play-audio and it plays just fine.

So the end point is working.

Why, then, will ui-audio not play it?

Oh, dearest @BartButenaers, how difficult would it be to alter ui-audio to accept msg.audio or somesuch as a binary buffer? Surely that's what it's grabbing from the URL anyway? This would totally remove the need for an http endpoint in the first place. The actual playing code would already be there, surely?

I apologise if I seem to be oversimplifying this; there's probably a trickiness I'm not seeing.

But at this point I'm so desperate I'll offer a cash bounty for anyone who can do that edit (I don't think I'm knowledgeable enough to edit the underlying code). I'm pretty much broke, but I think could find fifty euros for whoever contributes this code.

I'm sorry if it seems like a small amount. If you were in my country I would shower you with more avocados and honey than you would know what to do with.

Anyway, on the offchance that it's my shitty flow, please show me where I'm going wrong. Everything says it should work.

[{"id":"b9765ce47c15b7ae","type":"debug","z":"21e11e4ca2b93801","name":"First hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":910,"y":1160,"wires":[]},{"id":"df41fe923050cae5","type":"debug","z":"21e11e4ca2b93801","name":"Second hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":620,"y":1160,"wires":[]},{"id":"203876051c371051","type":"debug","z":"21e11e4ca2b93801","name":"Third hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1150,"y":1160,"wires":[]},{"id":"e7ff0621bb935849","type":"http in","z":"21e11e4ca2b93801","name":"","url":"/voice","method":"post","upload":false,"swaggerDoc":"","x":350,"y":1100,"wires":[["df41fe923050cae5","5c90a491d9d9b7ff"]]},{"id":"5c90a491d9d9b7ff","type":"http response","z":"21e11e4ca2b93801","name":"Serving the audio","statusCode":"","headers":{"content-type":"audio/x-wav"},"x":670,"y":1100,"wires":[]},{"id":"61a1f51e26df5f13","type":"function","z":"21e11e4ca2b93801","name":"Format the message for Piper","func":"var inputtext = msg.payload;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":410,"y":1060,"wires":[["46b45af3b681a9d6"]]},{"id":"46b45af3b681a9d6","type":"http request","z":"21e11e4ca2b93801","name":"Send to Piper","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://192.168.2.75:8080/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"application/json","valueValue":""}],"x":660,"y":1060,"wires":[["a5ed50a96ffe5256","b9765ce47c15b7ae"]]},{"id":"a5ed50a96ffe5256","type":"http request","z":"21e11e4ca2b93801","name":"Then ","method":"POST","ret":"bin","paytoqs":"body","url":"http://localhost:1880/voice","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"other","valueValue":"audio/x-wav"}],"x":890,"y":1060,"wires":[["d922edc260915e57","203876051c371051"]]},{"id":"a55d6d0955f88eac","type":"link in","z":"21e11e4ca2b93801","name":"Message for TTS","links":[],"x":160,"y":1060,"wires":[["61a1f51e26df5f13"]],"l":true},{"id":"d922edc260915e57","type":"ui-audio","z":"21e11e4ca2b93801","group":"0c76141b6b8548ea","name":"","order":1,"width":0,"height":0,"src":"http://localhost:1880/voice","autoplay":"on","loop":"off","muted":"off","x":1160,"y":1060,"wires":[[]]},{"id":"56f5b0fc664904a9","type":"inject","z":"21e11e4ca2b93801","name":"Example text","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Everyone should get a plate before anyone has seconds. It's not complicated.","payloadType":"str","x":150,"y":980,"wires":[["61a1f51e26df5f13"]]},{"id":"0c76141b6b8548ea","type":"ui-group","name":"Zone arming","page":"a30cb6a10bfc3330","width":"4","height":"1","order":9,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"a30cb6a10bfc3330","type":"ui-page","name":"Newdash","ui":"cf9dc2ba60d1396c","path":"/page6","icon":"home","layout":"grid","theme":"7ba40ffa60b22682","breakpoints":[{"name":"Default","px":"0","cols":"4"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":1,"className":"","visible":"true","disabled":"false"},{"id":"cf9dc2ba60d1396c","type":"ui-base","name":"My Dashboard","path":"/dashboard","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":true,"navigationStyle":"default","titleBarStyle":"default"},{"id":"7ba40ffa60b22682","type":"ui-theme","name":"Default Theme","colors":{"surface":"#ffffff","primary":"#0080ff","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"pagePadding":"2px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"4px","density":"comfortable"}}]

Setting the content-type is important. However, I have tried all kinds of formats, and while the play-audio node seems happy either way, ui-audio refuses all.

I wish I could figure this out. I'm so close to having a scottish lady warn me about bastards nicking in the tractor shed again.

8

The node you linked to ( node-red-contrib-play-audio) is for the old (original, depreciated) Dashboard (AKA Dashboard 1) node-red-dashboard

But you have tagged this post with dashboard-2 aka new dashboard-2
@flowfuse/node-red-dashboard

If you are using Dashboard 2, then it has built in a ui-audio node which you can try out for yourself: Audio ui-audio | Node-RED Dashboard 2.0

image
image1397Ă—431 50.3 KB

9

Hi @Steve-Mcl, my tags are correct, I'm talking about Dashboard 2.

I've not explained well. Let me try again.

I am attempting to use Dashboard 2's ui-audio, as per the title of this thread. I am failing.

The node you say is part of Dashboard 1, still works. Dashboard 2? No, hence this thread.

10

No, dashboard 1 nodes do not work with dashboard-2.
dashboard-2 has its own ui-audio node built-in:

If you visit the link i provided, you will see a working demo under the "Try Demo" button and it has an exportable flow you can import if you wish

1 Like
11

@no_mpimpi it is clear you are making an effort to achieve your goal (as opposed to just getting folk to do it for you) so please don't be afraid to speak up and let us know if you are still struggling. Everyone starts somewhere.

If necessary, tomorrow, I will attempt to design a flow for you that does what you need - let me know (and also, include a details like how/when you want this audio to play in the browser e.g., on click, on msg, when a user visits a page etc etc. Details are important)

If you have solved this, then it would be good if you can share that for future readers.

1 Like
12

I think there is still a slight disconnect in understanding. (Quite possibly mine). The play audio node initially mentioned plays audio on the server side and can take a buffer input. The ui nodes have to fetch their content from the server so they can play in the browser, so need a url. Obviously if browser and sever (Node-RED) are running on the same machine, the audio would come out of the speakers in both cases - but the path to get there is quite different.

2 Likes
13

@Steve-Mcl, no disrespect intended, but are you an AI? I am very much trying to use the ui-audio node already, and that is the subject of the thread. And the audio node that you said is part of Dashboard 1, is not. It's a server-side node I'm using to make sure that the buffer is good and the endpoint works.

But ui-audio will not play the audio even though the buffer is correct and the endpoint works.

At this stage, I'm not even sure ui-audio functions correctly. Has anyone successfully used it to play a buffer? If so please post an example flow.

Anyhoo, @dceejay, that's correct. I am using the node I initially linked to, server side, to check that the right kind of payload is arriving from the HTTP endpoint.

It is, and it plays beautifully.

But the ui-audio node; the one in the dashboard, the one that would work on whatever device you were using the dashboard on ... that one won't play.

14

Are we not all AI? No, I am not! Offence taken :wink:


Yes, your subject states "Dashboard 2 and ui-audio"

Ah, there is my mistake. There is a very similar named node that is designed for dashboard 1 and I missed that this is the one that plays audio server side. My bad. You are right.


Yes. By setting up an endpoint to serve the buffer.

Did you try the demo I linked to? Does it play for you?

1 Like
15

Hi @Steve-Mcl,

The demo you linked me to works.

I implemented an HTTP endpoint, as you suggested. Please have a look at what I did, maybe there is a typo I still have not found.

[{"id":"b9765ce47c15b7ae","type":"debug","z":"21e11e4ca2b93801","name":"First hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":910,"y":1160,"wires":[]},{"id":"df41fe923050cae5","type":"debug","z":"21e11e4ca2b93801","name":"Second hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":620,"y":1160,"wires":[]},{"id":"203876051c371051","type":"debug","z":"21e11e4ca2b93801","name":"Third hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1150,"y":1160,"wires":[]},{"id":"e7ff0621bb935849","type":"http in","z":"21e11e4ca2b93801","name":"","url":"/voice","method":"post","upload":false,"swaggerDoc":"","x":350,"y":1100,"wires":[["df41fe923050cae5","5c90a491d9d9b7ff"]]},{"id":"5c90a491d9d9b7ff","type":"http response","z":"21e11e4ca2b93801","name":"Serving the audio","statusCode":"","headers":{"content-type":"audio/x-wav"},"x":670,"y":1100,"wires":[]},{"id":"61a1f51e26df5f13","type":"function","z":"21e11e4ca2b93801","name":"Format the message for Piper","func":"var inputtext = msg.payload;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":410,"y":1060,"wires":[["46b45af3b681a9d6"]]},{"id":"46b45af3b681a9d6","type":"http request","z":"21e11e4ca2b93801","name":"Send to Piper","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://192.168.2.75:8080/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"application/json","valueValue":""}],"x":660,"y":1060,"wires":[["a5ed50a96ffe5256","b9765ce47c15b7ae"]]},{"id":"a5ed50a96ffe5256","type":"http request","z":"21e11e4ca2b93801","name":"Then ","method":"POST","ret":"bin","paytoqs":"body","url":"http://localhost:1880/voice","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"other","valueValue":"audio/x-wav"}],"x":890,"y":1060,"wires":[["d922edc260915e57","203876051c371051"]]},{"id":"a55d6d0955f88eac","type":"link in","z":"21e11e4ca2b93801","name":"Message for TTS","links":[],"x":160,"y":1060,"wires":[["61a1f51e26df5f13"]],"l":true},{"id":"d922edc260915e57","type":"ui-audio","z":"21e11e4ca2b93801","group":"0c76141b6b8548ea","name":"","order":1,"width":0,"height":0,"src":"http://localhost:1880/voice","autoplay":"on","loop":"off","muted":"off","x":1160,"y":1060,"wires":[[]]},{"id":"56f5b0fc664904a9","type":"inject","z":"21e11e4ca2b93801","name":"Example text","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Everyone should get a plate before anyone has seconds. It's not complicated.","payloadType":"str","x":150,"y":980,"wires":[["61a1f51e26df5f13"]]},{"id":"0c76141b6b8548ea","type":"ui-group","name":"Zone arming","page":"a30cb6a10bfc3330","width":"4","height":"1","order":9,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"a30cb6a10bfc3330","type":"ui-page","name":"Newdash","ui":"cf9dc2ba60d1396c","path":"/page6","icon":"home","layout":"grid","theme":"7ba40ffa60b22682","breakpoints":[{"name":"Default","px":"0","cols":"4"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":1,"className":"","visible":"true","disabled":"false"},{"id":"cf9dc2ba60d1396c","type":"ui-base","name":"My Dashboard","path":"/dashboard","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":true,"navigationStyle":"default","titleBarStyle":"default"},{"id":"7ba40ffa60b22682","type":"ui-theme","name":"Default Theme","colors":{"surface":"#ffffff","primary":"#0080ff","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"pagePadding":"2px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"4px","density":"comfortable"}}]

The endpoint in the flow above does appear to be providing the audio because, as mentioned, the play-audio node will play it.

I have indeed been attempting to do all of the things you have suggested. If you are willing, please look at my flow and tell me where the error lies. I've read everything you've linked me to, and I'm still failing.

EDIT: the example you linked to does not use an HTTP endpoint. However, I do note that all of the manually loaded media files have filenames. Is that relevant?

16

I think your flow needs to be more like this. untested as i do not have piper

[{"id":"26fd0a8d9d069582","type":"function","z":"d1395164b4eec73e","name":"function 1","func":"msg.payload = `http://192.168.1.10:1880/voice?message=${encodeURI(msg.payload)}`\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":420,"y":2240,"wires":[["d922edc260915e57","b9765ce47c15b7ae"]]},{"id":"e7ff0621bb935849","type":"http in","z":"d1395164b4eec73e","name":"","url":"/voice","method":"get","upload":false,"swaggerDoc":"","x":100,"y":2320,"wires":[["df41fe923050cae5","4644cd19a87ac699"]]},{"id":"df41fe923050cae5","type":"debug","z":"d1395164b4eec73e","name":"Second hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":600,"y":2360,"wires":[]},{"id":"4644cd19a87ac699","type":"function","z":"d1395164b4eec73e","name":"Format the message for Piper","func":"var inputtext = msg.payload.message;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":350,"y":2320,"wires":[["dbc81157d23dc64e"]]},{"id":"dbc81157d23dc64e","type":"http request","z":"d1395164b4eec73e","name":"Send to Piper","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://192.168.2.75:8080/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"application/json","valueValue":""}],"x":570,"y":2320,"wires":[["5c90a491d9d9b7ff"]]},{"id":"5c90a491d9d9b7ff","type":"http response","z":"d1395164b4eec73e","name":"Serving the audio","statusCode":"","headers":{"content-type":"audio/x-wav"},"x":750,"y":2320,"wires":[]},{"id":"56f5b0fc664904a9","type":"inject","z":"d1395164b4eec73e","name":"Example text","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Everyone should get a plate before anyone has seconds. It's not complicated.","payloadType":"str","x":230,"y":2240,"wires":[["26fd0a8d9d069582"]]},{"id":"d922edc260915e57","type":"ui-audio","z":"d1395164b4eec73e","group":"9d83cd8b3d5ee33c","name":"","order":1,"width":0,"height":0,"src":"","autoplay":"on","loop":"off","muted":"off","x":700,"y":2220,"wires":[["b9765ce47c15b7ae"]]},{"id":"b9765ce47c15b7ae","type":"debug","z":"d1395164b4eec73e","name":"First hurdle","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":890,"y":2260,"wires":[]},{"id":"9d83cd8b3d5ee33c","type":"ui-group","name":"Group1","page":"c694d0ebe0d2b702","width":"6","height":"1","order":1,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"c694d0ebe0d2b702","type":"ui-page","name":"Page1","ui":"1805777f90e92057","path":"/page1","icon":"home","layout":"grid","theme":"35ee7753b5b3599b","breakpoints":[{"name":"Default","px":"0","cols":"3"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":1,"className":"","visible":"true","disabled":"false"},{"id":"1805777f90e92057","type":"ui-base","name":"dashboard ","path":"/dashboard","appIcon":"","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":false,"headerContent":"page","titleBarStyle":"default","showReconnectNotification":false,"notificationDisplayTime":"5","showDisconnectNotification":false},{"id":"35ee7753b5b3599b","type":"ui-theme","name":"Theme Name","colors":{"surface":"#16234b","primary":"#1d44b9","bgPage":"#ecf2f8","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"density":"default","pagePadding":"12px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"12px"}}]
1 Like
17

They have URLs (which are in effect endpoints)

Ok, your flow needed some work.

image
image1090Ă—367 33.3 KB

  1. something needs to be clicked, so I added abutton to be that somethine.
  2. makes a request to you TTS (i faked it with my own endpoint that loads a buffer into context memory)
  3. prepare the URL to send in msg.payload to the ui-audio
  4. the ui-audio will fetch the URL - this hits your endpoint /voice
  5. load the buffer into msg.payload
  6. return the buffer (with headers)
  7. audio is played.

Where you got tripped up:

  1. "getting" the audio is a GET request (your endpoint was setup as a POST)
  2. you had nothing in between your http-in and http-response (so nothing was being served)
  3. using a http request (backend operation) to call your own /voice endpoint - the ui-audio node (frontend) does this.

Demo flow. Unfortunately, I cannot include sample audio (too large) and you will need to get your TTS buffer into the middle of the http-in ~ http-response (likely via content). I have left in my whole faking parts - you can try it out with a local MP3 first.

[{"id":"6d9705111ffafd35","type":"debug","z":"3eb62d10d1081306","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":850,"y":1380,"wires":[]},{"id":"655b25d85e77810c","type":"http in","z":"3eb62d10d1081306","name":"","url":"/voice","method":"get","upload":false,"swaggerDoc":"","x":680,"y":1320,"wires":[["6d9705111ffafd35","a010345165f427d2"]]},{"id":"5c068b0e58e3e8de","type":"http response","z":"3eb62d10d1081306","name":"Serving the audio","statusCode":"","headers":{"content-type":"audio/x-wav"},"x":1110,"y":1320,"wires":[]},{"id":"ac0eda0b8288c44d","type":"ui-button","z":"3eb62d10d1081306","group":"0c76141b6b8548ea","name":"","label":"button","order":1,"width":0,"height":0,"emulateClick":false,"tooltip":"","color":"","bgcolor":"","className":"","icon":"","iconPosition":"left","payload":"","payloadType":"str","topic":"topic","topicType":"msg","buttonColor":"","textColor":"","iconColor":"","enableClick":true,"enablePointerdown":false,"pointerdownPayload":"","pointerdownPayloadType":"str","enablePointerup":false,"pointerupPayload":"","pointerupPayloadType":"str","x":670,"y":1560,"wires":[["b8234b4054e02ff8"]]},{"id":"b8234b4054e02ff8","type":"function","z":"3eb62d10d1081306","name":"Format the message for Piper","func":"var inputtext = msg.payload;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":750,"y":1500,"wires":[["832dd8ac39130c82"]]},{"id":"832dd8ac39130c82","type":"http request","z":"3eb62d10d1081306","name":"Send to Piper (calls my piper fake endpoint)","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://localhost:1880/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"application/json","valueValue":""}],"x":1090,"y":1500,"wires":[["facce2bfe6e6a4bb","d55d336ab3bb764e"]]},{"id":"f8fe35268e36e4cd","type":"inject","z":"3eb62d10d1081306","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":980,"y":1000,"wires":[["80954a794e052692"]]},{"id":"80954a794e052692","type":"file in","z":"3eb62d10d1081306","name":"","filename":"bark.mp3","filenameType":"str","format":"","chunk":false,"sendError":false,"encoding":"none","allProps":false,"x":1140,"y":1000,"wires":[["93dc236442f15d13"]]},{"id":"93dc236442f15d13","type":"change","z":"3eb62d10d1081306","name":"","rules":[{"t":"set","p":"buffer","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1360,"y":1000,"wires":[[]]},{"id":"facce2bfe6e6a4bb","type":"function","z":"3eb62d10d1081306","name":"Send URL","func":"msg.payload = 'http://localhost:1880/voice?t=' + Date.now() // cache bust\n\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1370,"y":1500,"wires":[["0deab529610c0d67","d47703a9b1419e9e"]]},{"id":"a010345165f427d2","type":"change","z":"3eb62d10d1081306","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"buffer","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":880,"y":1320,"wires":[["5c068b0e58e3e8de","e034fae1eef6ad04"]]},{"id":"e034fae1eef6ad04","type":"debug","z":"3eb62d10d1081306","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1070,"y":1380,"wires":[]},{"id":"fe91dea955003518","type":"debug","z":"3eb62d10d1081306","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1130,"y":1140,"wires":[]},{"id":"bbc4bb1330254cca","type":"debug","z":"3eb62d10d1081306","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1330,"y":1140,"wires":[]},{"id":"d55d336ab3bb764e","type":"debug","z":"3eb62d10d1081306","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1350,"y":1440,"wires":[]},{"id":"0deab529610c0d67","type":"debug","z":"3eb62d10d1081306","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1530,"y":1560,"wires":[]},{"id":"d47703a9b1419e9e","type":"ui-audio","z":"3eb62d10d1081306","group":"0c76141b6b8548ea","name":"","order":2,"width":0,"height":0,"src":"","autoplay":"on","loop":"off","muted":"off","x":1540,"y":1500,"wires":[[]]},{"id":"05e1c44e588254a5","type":"ui-template","z":"3eb62d10d1081306","group":"","page":"","ui":"46de808d763294c2","name":"","order":0,"width":0,"height":0,"head":"","format":"/* hide the player */\n.nrdb-ui-widget.nrdb-ui-audio {\n    visibility:hidden;\n}","storeOutMessages":true,"passthru":true,"resendOnRefresh":true,"templateScope":"site:style","className":"","x":1540,"y":1460,"wires":[[]]},{"id":"8ce218e741475fe5","type":"group","z":"3eb62d10d1081306","style":{"stroke":"#999999","stroke-opacity":"1","fill":"none","fill-opacity":"1","label":true,"label-position":"nw","color":"#a4a4a4"},"nodes":["304af57ef72d4039","27bdf922e02eb4be","e469c33d99a90f1e"],"x":874,"y":1039,"w":532,"h":82},{"id":"304af57ef72d4039","type":"http in","z":"3eb62d10d1081306","g":"8ce218e741475fe5","name":"faking piper","url":"/tts","method":"post","upload":false,"swaggerDoc":"","x":970,"y":1080,"wires":[["27bdf922e02eb4be","fe91dea955003518"]]},{"id":"27bdf922e02eb4be","type":"change","z":"3eb62d10d1081306","g":"8ce218e741475fe5","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"buffer","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":1160,"y":1080,"wires":[["e469c33d99a90f1e","bbc4bb1330254cca"]]},{"id":"e469c33d99a90f1e","type":"http response","z":"3eb62d10d1081306","g":"8ce218e741475fe5","name":"","statusCode":"","headers":{"content-type":"audio/mp3"},"x":1330,"y":1080,"wires":[]},{"id":"0c76141b6b8548ea","type":"ui-group","name":"Zone arming","page":"a30cb6a10bfc3330","width":"4","height":"1","order":1,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"46de808d763294c2","type":"ui-base","name":"My Dashboard","path":"/dashboard","appIcon":"","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":false,"headerContent":"page","navigationStyle":"default","titleBarStyle":"default","showReconnectNotification":true,"notificationDisplayTime":1,"showDisconnectNotification":true,"allowInstall":true},{"id":"a30cb6a10bfc3330","type":"ui-page","name":"Newdash","ui":"46de808d763294c2","path":"/page6","icon":"home","layout":"grid","theme":"7ba40ffa60b22682","breakpoints":[{"name":"Default","px":"0","cols":"4"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":2,"className":"","visible":"true","disabled":"false"},{"id":"7ba40ffa60b22682","type":"ui-theme","name":"Default Theme","colors":{"surface":"#ffffff","primary":"#0080ff","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"pagePadding":"2px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"4px","density":"comfortable"}},{"id":"917e956f8d38af98","type":"global-config","env":[],"modules":{"@flowfuse/node-red-dashboard":"1.24.1"}}]
1 Like
18

Good morning @Steve-Mcl

First, thank you for going to all of this effort; it was very kind of you to build that flow. I still haven't succeeded, but I learned a lot from you, especially in terms of my understanding of endpoints.

For what it's worth, when/if this works, I intend to post a tutorial for others on how to do voice notifications with Piper in Node-RED. But I have to make it work first.

Using what you taught me, I adapted my flow to use flow variables - the Piper output is immediately shoved into a buffer.

However, the debug output shows that the HTTP endpoint is never called.

Is there a reason that the URL you pass to ui-audio contains an additional timestamp? I tried removing it but it made no difference.

My updated flow - can you see where a fresh mistake is made?

[{"id":"61a1f51e26df5f13","type":"function","z":"21e11e4ca2b93801","name":"Format the message for Piper","func":"var inputtext = msg.payload;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1130,"y":1080,"wires":[["46b45af3b681a9d6"]]},{"id":"46b45af3b681a9d6","type":"http request","z":"21e11e4ca2b93801","name":"Send to Piper","method":"POST","ret":"bin","paytoqs":"ignore","url":"http://192.168.2.75:8080/tts","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[{"keyType":"Content-Type","keyValue":"","valueType":"application/json","valueValue":""}],"x":1360,"y":1080,"wires":[["d38c36075f15260b"]]},{"id":"56f5b0fc664904a9","type":"inject","z":"21e11e4ca2b93801","name":"Example text","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"Everyone should get a plate before anyone has seconds. It's not complicated.","payloadType":"str","x":890,"y":1140,"wires":[["61a1f51e26df5f13"]]},{"id":"d38c36075f15260b","type":"change","z":"21e11e4ca2b93801","name":"","rules":[{"t":"set","p":"buffer","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":1540,"y":1080,"wires":[["077423dc569df3ad"]]},{"id":"122c461c8c1c8063","type":"http in","z":"21e11e4ca2b93801","name":"","url":"/voice","method":"get","upload":false,"swaggerDoc":"","x":1060,"y":1140,"wires":[["17069e5e037940d2"]]},{"id":"92f1c760d927279e","type":"http response","z":"21e11e4ca2b93801","name":"Serving the audio","statusCode":"","headers":{},"x":1450,"y":1140,"wires":[]},{"id":"17069e5e037940d2","type":"change","z":"21e11e4ca2b93801","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"buffer","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":1240,"y":1140,"wires":[["92f1c760d927279e","f611694f28f728c8"]]},{"id":"3cd0a0599f65464b","type":"ui-audio","z":"21e11e4ca2b93801","group":"0c76141b6b8548ea","name":"","order":6,"width":0,"height":0,"src":"","autoplay":"on","loop":"off","muted":"off","x":1900,"y":1080,"wires":[[]]},{"id":"82bf471e67bbf6b1","type":"ui-template","z":"21e11e4ca2b93801","group":"","page":"","ui":"cf9dc2ba60d1396c","name":"","order":0,"width":0,"height":0,"head":"","format":"/* hide the player */\n.nrdb-ui-widget.nrdb-ui-audio {\n    visibility:hidden;\n}","storeOutMessages":true,"passthru":true,"resendOnRefresh":true,"templateScope":"site:style","className":"","x":1900,"y":1040,"wires":[[]]},{"id":"077423dc569df3ad","type":"function","z":"21e11e4ca2b93801","name":"Send URL","func":"msg.payload = 'http://localhost:1880/voice?t=' + Date.now() // cache bust\nreturn msg;","outputs":1,"timeout":0,"noerr":0,"initialize":"","finalize":"","libs":[],"x":1730,"y":1080,"wires":[["3cd0a0599f65464b","7491efb8383aedbe"]]},{"id":"f611694f28f728c8","type":"debug","z":"21e11e4ca2b93801","name":"Inside the endpoint","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1450,"y":1180,"wires":[]},{"id":"7491efb8383aedbe","type":"debug","z":"21e11e4ca2b93801","name":"What arrives at ui-audio","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":1950,"y":1140,"wires":[]},{"id":"0b82146d6cdd467d","type":"comment","z":"21e11e4ca2b93801","name":"The output of Piper is put into flow.buffer","info":"","x":1610,"y":1040,"wires":[]},{"id":"b9771cea7017a73f","type":"comment","z":"21e11e4ca2b93801","name":"Inside the endpoint, flow.buffer is put into the payload","info":"","x":1350,"y":1220,"wires":[]},{"id":"1686b9640daa51cc","type":"comment","z":"21e11e4ca2b93801","name":"The debug shows us that this endpoint is never called","info":"","x":1360,"y":1260,"wires":[]},{"id":"e0048b1e825e8a74","type":"inject","z":"21e11e4ca2b93801","name":"Play","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":1600,"y":1360,"wires":[["077423dc569df3ad"]]},{"id":"0c76141b6b8548ea","type":"ui-group","name":"Zone arming","page":"a30cb6a10bfc3330","width":"4","height":"1","order":9,"showTitle":true,"className":"","visible":"true","disabled":"false","groupType":"default"},{"id":"cf9dc2ba60d1396c","type":"ui-base","name":"My Dashboard","path":"/dashboard","includeClientData":true,"acceptsClientConfig":["ui-notification","ui-control"],"showPathInSidebar":true,"navigationStyle":"default","titleBarStyle":"default"},{"id":"a30cb6a10bfc3330","type":"ui-page","name":"Newdash","ui":"cf9dc2ba60d1396c","path":"/page6","icon":"home","layout":"grid","theme":"7ba40ffa60b22682","breakpoints":[{"name":"Default","px":"0","cols":"4"},{"name":"Tablet","px":"576","cols":"6"},{"name":"Small Desktop","px":"768","cols":"9"},{"name":"Desktop","px":"1024","cols":"12"}],"order":1,"className":"","visible":"true","disabled":"false"},{"id":"7ba40ffa60b22682","type":"ui-theme","name":"Default Theme","colors":{"surface":"#ffffff","primary":"#0080ff","bgPage":"#eeeeee","groupBg":"#ffffff","groupOutline":"#cccccc"},"sizes":{"pagePadding":"2px","groupGap":"12px","groupBorderRadius":"4px","widgetGap":"4px","density":"comfortable"}}]
19

With a couple of tweaks (because I dont have piper) it works for me.

I have un-hid the ui-audio to provide visual proof it works (this is a gif not a video with sound)

watch the counters and watch the media player as I press inject.

chrome_kVOtdKrSps

Reasons it might not work for you.

  1. The URL in the Send URL function sets the ui-audio src to http:localhost:1880 - thats where my Node-RED server is. This is probably different. What URL is in your node-red editor?
  2. You forgot to click somewhere in the dashboard page.
  1. The dashboard page was not visible (in a hidden tab)
1 Like
20

It is probably not necessary but I added it "just in case"

Adding ?t=<timestamp> is what is known as a cache buster. I added it because the endpoint is ALWAYS /voice even if the buffer has changed. So to prevent the browser thinking "i've already got this audio cached, I am just gonna re-use it instead of making an actual HTTP GET", I bust the cache by adding a difference to the URL.

2 Likes