Could this be the future for NR too?
(I would like to delete applestrudel)
2 Likes
Yes indeed I also had seen arriving those devices recently. It is indeed very interesting technology, that has great potential.
I have completely zero experience with it, but a quick view learns me that there are 3 docker images involved:
- Whisper for speech-to-text
- Piper for text-to-speech
- OpenWakeWord for invoking the Assist Pipeline
Since I have no clue how it works, I cheated by asking ChatGpt how to call these services from a Node-RED flow. He claims that you can use a http-request node and send simply POST requests to these containers:
And he/she/it also provided me a sample flow:
[{"id":"inject-node","type":"inject","z":"flow-id","name":"Trigger Flow","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"audio-file-path","payloadType":"str","x":100,"y":100,"wires":[["openwakeword-http-node"]]},{"id":"openwakeword-http-node","type":"http request","z":"flow-id","name":"Check Wake Word","method":"POST","ret":"txt","url":"http://<openwakeword-container-ip>:8080/api/detect","headers":{},"x":300,"y":100,"wires":[["whisper-http-node"]]},{"id":"whisper-http-node","type":"http request","z":"flow-id","name":"Send to Whisper","method":"POST","ret":"txt","url":"http://<whisper-container-ip>:<whisper-container-port>/transcribe","headers":{},"x":500,"y":100,"wires":[["piper-http-node"]]},{"id":"piper-http-node","type":"http request","z":"flow-id","name":"Send to Piper","method":"POST","ret":"bin","url":"http://<piper-container-ip>:10200/api/tts","headers":{},"x":700,"y":100,"wires":[["debug-node"]]},{"id":"debug-node","type":"debug","z":"flow-id","name":"See Result","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":900,"y":100,"wires":[]}]
That flow should offer this functionality:
- Detect a wake word using OpenWakeWord.
- Convert spoken audio to text using Whisper.
- Convert the text back to speech using Piper.
But you already know that ChatGpt can tell a lot of very convincing stuff, but it can be complete bollocks! I am afraid that this time the proof of the pudding is in the eating.
Good luck with your journey!
And please share your experiences here, so others can learn from it (and assist you of course)...
Bart
3 Likes
Ok, thank you ....
A really nice project in this area on the hardware side is the onju project. It utilizes a custom PCB for the Gen2 of the Google speaker. There are currently also two different loadable firmware options to choose form. One of them is based on ESPhome. Fun to play. I spent quite a bit of time in this area about 6 months ago and things were just not up to snuff to be reliable at that time. But I see that there have been many updates as of late. I will be trying this again.
