Need help using HTTP Endpoint to serve a binary buffer

General
1

Hello Fellow Noderoids,

I initially asked this in Dashboard, but I’m not trying to make it work with Dashboard anymore, and I'm still failing at this.

I am trying to pull audio out of a buffer using an HTTP endpoint, but I am doing something wrong.

My use case is voice notifications played by Pi Zero satellite nodes in various rooms. Initially I had the nodes calling the local Piper AI server themselves, but Piper doesn't like it when it gets multiple concurrent requests, which it will do every time a broadcast goes to all speaker nodes.

Therefore, a more efficient way of doing it is for the main Node-RED to make one call to Piper, and publish the audio to the relevant node(s) via MQTT payload. I can’t get it to work.

I am trying to solve this with a test case. The flow is a modification of something @Steve-Mcl came up with, changed for context.

I have succeeded in testing an HTTP endpoint to send some text; it works. Now I’m trying to do it with a binary buffer. I figure that if I can get the buffer to work, then putting a payload into a buffer is trivial after that.

So, since we need a buffer, there’s an inject node to load a pre-generated WAV into flow.buffer; that's so the endpoint has something to pull.

When called, the HTTP endpoint pulls the audio from the buffer and sticks it in msg.payload, and tags it with a ContentType of audio/wav.

I make that call using the node-red-contrib-play-sound (node) - Node-RED PlaySound node, which shells out to aplay or aplayer of your choice. It is configured to call the HTTP endpoint.

Now, the playback node should work. I know this because there's a second one configured to play the wav file directly, and it does work. The one that doesn't work is the one calling the HTTP endpoint.

Somehow, somewhere along the line, I've misconfigured a thing. Please help.

Here is the flow:

[{"id":"da92c63e371ffa35","type":"file in","z":"287522cb42f89190","name":"","filename":"/home/pinode01/hello.wav","filenameType":"str","format":"","chunk":false,"sendError":false,"encoding":"none","allProps":false,"x":290,"y":80,"wires":[["ae842cd82137cd13"]]},{"id":"ce0e015b9c560588","type":"inject","z":"287522cb42f89190","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":180,"wires":[["137433bcd27c6f50"]]},{"id":"86bd43344523cd4c","type":"http in","z":"287522cb42f89190","name":"","url":"/why","method":"get","upload":false,"swaggerDoc":"","x":100,"y":360,"wires":[["7f76adae882b8835"]]},{"id":"7f76adae882b8835","type":"change","z":"287522cb42f89190","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"buffer","tot":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":260,"y":360,"wires":[["722eb454e42dc823","fd2f78d6299155b3"]]},{"id":"722eb454e42dc823","type":"http response","z":"287522cb42f89190","name":"","statusCode":"","headers":{"content-type":"audio/wav"},"x":490,"y":360,"wires":[]},{"id":"137433bcd27c6f50","type":"http request","z":"287522cb42f89190","name":"","method":"GET","ret":"bin","paytoqs":"ignore","url":"http://localhost:1880/why","tls":"","persist":false,"proxy":"","insecureHTTPParser":false,"authType":"","senderr":false,"headers":[],"x":250,"y":180,"wires":[["6f26b724287edd74","7e462c00451ddab3"]]},{"id":"d743425956788990","type":"inject","z":"287522cb42f89190","name":"","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":100,"y":80,"wires":[["da92c63e371ffa35"]]},{"id":"ae842cd82137cd13","type":"change","z":"287522cb42f89190","name":"","rules":[{"t":"set","p":"buffer","pt":"flow","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":520,"y":80,"wires":[[]]},{"id":"d72a969579578390","type":"comment","z":"287522cb42f89190","name":"1. Put something in a buffer","info":"","x":160,"y":40,"wires":[]},{"id":"958b3bf0f0af0d6a","type":"comment","z":"287522cb42f89190","name":"3. The HTTP endpoint, pulls the buffer when asked.","info":"","x":230,"y":320,"wires":[]},{"id":"6f26b724287edd74","type":"PlaySound","z":"287522cb42f89190","name":"Play the file as served by the endpoint","playerOptions":"{}","audioURI":"http://localhost:1880/why","options":"{}","x":590,"y":180,"wires":[[]]},{"id":"fd2f78d6299155b3","type":"debug","z":"287522cb42f89190","name":"Just so I can see it being called","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":570,"y":400,"wires":[]},{"id":"7e462c00451ddab3","type":"debug","z":"287522cb42f89190","name":"It looks like this when it comes back","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":580,"y":220,"wires":[]},{"id":"fb20e1880632fd94","type":"comment","z":"287522cb42f89190","name":"2. Call the endpoint and play","info":"","x":160,"y":140,"wires":[]},{"id":"1c71a86a670a0d45","type":"PlaySound","z":"287522cb42f89190","name":"Play the wav file directly","playerOptions":"{}","audioURI":"/home/pinode01/hello.wav","options":"{}","x":550,"y":260,"wires":[[]]}]
1 Like
2

Can you edit your post and repost the flow (copy it again from your editor) and use the option in the menu

image
image924×754 43.7 KB

  • what does not work exactly 'The one that doesn't work is the one calling the HTTP endpoint.'
  • did you try a curl to the endpoint ?
    • if yes, what does it return ?
    • does it return anything ?
1 Like
3

Hi @bakman2,

Thank you for responding, I have edited the flow as you suggested.

The specific thing that does not work is this: I can play audio directly from a file. I cannot play audio served by an HTTP node. I cannot figure out why and it’s killing me.

The debug nodes show that the payload is showing up where it’s supposed to: the endpoint gets called, and responds. However, the PlaySound node does not play anything. I had similar difficulty when trying to play audio through the Dashboard audio node.

Here’s why that’s weird.

Right now, in the version that is currently deployed in my house, I have this working by writing the audio to a file in the home folder. I then give the PlaySound node the path to the audio file, and it plays just fine.

But this approach will wreck the SD card in time, which is why I’m trying to do it differently.

So, with respect to the fact that the soundcard does work, and the PlaySound node is happy to play audio through it (because it just shells out to aplay), I cannot understand why this does not work.

My gut instinct is that it is to do with the ContentType headers. Maybe there is an additional one that I need other than audio/wav? I’ve been at this problem on and off for several weeks now, it’s super frustrating. It’s like it’s staring me in the face. BUT WHEEERRRRRRRE?

4

On the commandline, can you do file --mime-type /home/pinode01/hello.wav
It might indeed the mime/content type. audio/x-wav vs audio/wav, apparently this was not officially defined in w3c standards, browsers may interpret it differently too. When i use audio/x-wav for my test file it works directly in browser via the endpoint, but not in node-red. Node-red will use the local system to produce sound and i dont have a sound card there.

1 Like
6

Hi @bakman2, and tangentially, @Steve-Mcl

I have solved this insofar as I have got it working. I am posting in case someone else finds this via search; I hate threads without resolution.

So, I stopped using the SoundPlayer node, and instead reverted to an exec node - BigExec in this case. I found that I could successfully play a wav file that way, so the speaker was indeed working.

I couldn’t get it to play anything sent via MQTT payload; I don’t know why.

But, I had success by calling Piper from the playback device itself. I served the result from an endpoint. Then I used an exec node to run mplayer http://localhost:1880/tts. It worked perfectly.

Therefore, my main goal is now achieved: I will no longer trash the SD card with file writes.

Here is the flow in its current working state:

[
    {
        "id": "3a23e0778b405107",
        "type": "function",
        "z": "436b05d74282c879",
        "name": "check if we've played this",
        "func": "var data = global.get(\"messagedata\") || {};\nvar play = true;\n//flow.buffer = flow.set(\"buffer\", msg.payload.audio);\nvar missive = msg.payload.message;\nvar audio = msg.payload.audio;\nvar time = Date.now();\nvar lastpayload = data[msg.topic] || {};\nvar lastmessage = lastpayload.message;\nvar oldtime = lastpayload.nowtime;\n\nnode.warn(\"last message was : \" + lastmessage + \" and time \" + lastpayload.nowtime);\nnode.warn(\"this message is  : \" + missive);\n\nif (lastpayload = undefined){} else {\n    var elapsed = (time - oldtime) / 1000;\n    if (lastmessage == missive){\n        if (elapsed < 300){\n        node.warn(\"already played this message \" + elapsed + \"seconds ago\");\n        time = oldtime;\n        play = false;     \n        } else {\n            node.warn(elapsed + \" seconds since last identical message, repeating\");\n            var newmissive = \"I've surpressed repeats of this notification for \" + elapsed + \" seconds, but they keep coming in, so here it is again. \" + missive + \". I'm now surpressing this notification for another five minutes.\";\n            missive = newmissive;\n            play = true;\n        };\n    };\n};\nmsg.payload = {\n    \"message\" : missive,\n    \"nowtime\" : time,\n    \"play\": play\n};\ndata[msg.topic] = msg.payload;\n//msg.filename = \"hello.wav\"\nmsg.play = play;\n//msg.payload = audio;\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "// Code added here will be run once\n// whenever the node is started.\nconst last = global.set(\"messagedata\", {});",
        "finalize": "",
        "libs": [],
        "x": 370,
        "y": 140,
        "wires": [
            [
                "29cc0d104382863b"
            ]
        ]
    },
    {
        "id": "29cc0d104382863b",
        "type": "switch",
        "z": "436b05d74282c879",
        "name": "If it's new",
        "property": "play",
        "propertyType": "msg",
        "rules": [
            {
                "t": "true"
            }
        ],
        "checkall": "true",
        "repair": false,
        "outputs": 1,
        "x": 640,
        "y": 140,
        "wires": [
            [
                "c8fefc984d78c750"
            ]
        ]
    },
    {
        "id": "c8fefc984d78c750",
        "type": "function",
        "z": "436b05d74282c879",
        "name": "Format the message for Piper",
        "func": "var inputtext = msg.payload.message;\nmsg.payload = {\n    \"model\": \"en_GB-alba-medium.onnx\",\n    \"backend\": \"piper\",\n    \"input\": inputtext\n}\n\nreturn msg;",
        "outputs": 1,
        "timeout": 0,
        "noerr": 0,
        "initialize": "",
        "finalize": "",
        "libs": [],
        "x": 210,
        "y": 220,
        "wires": [
            [
                "840ab37137195564"
            ]
        ]
    },
    {
        "id": "840ab37137195564",
        "type": "http request",
        "z": "436b05d74282c879",
        "name": "Send to Piper",
        "method": "POST",
        "ret": "bin",
        "paytoqs": "ignore",
        "url": "http://192.168.2.75:8080/tts",
        "tls": "",
        "persist": false,
        "proxy": "",
        "insecureHTTPParser": false,
        "authType": "",
        "senderr": false,
        "headers": [
            {
                "keyType": "Content-Type",
                "keyValue": "",
                "valueType": "application/json",
                "valueValue": ""
            }
        ],
        "x": 460,
        "y": 220,
        "wires": [
            [
                "946d952a9ed1f9f0"
            ]
        ]
    },
    {
        "id": "c09a36a6e3787816",
        "type": "bigexec",
        "z": "436b05d74282c879",
        "name": "mplayer calls the endpoint",
        "command": "mplayer",
        "commandArgs": " http://localhost:1880/tts",
        "minError": 1,
        "minWarning": 1,
        "cwd": "",
        "shell": "",
        "extraArgumentProperty": "",
        "envProperty": "",
        "format": "",
        "limiter": true,
        "payloadIs": "trigger",
        "x": 370,
        "y": 300,
        "wires": [
            [],
            [],
            []
        ]
    },
    {
        "id": "946d952a9ed1f9f0",
        "type": "change",
        "z": "436b05d74282c879",
        "name": "Put the result in flow.buffer",
        "rules": [
            {
                "t": "set",
                "p": "buffer",
                "pt": "flow",
                "to": "payload",
                "tot": "msg"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 700,
        "y": 220,
        "wires": [
            [
                "c09a36a6e3787816"
            ]
        ]
    },
    {
        "id": "f8c91aac189963a2",
        "type": "http in",
        "z": "436b05d74282c879",
        "name": "",
        "url": "/tts",
        "method": "get",
        "upload": false,
        "swaggerDoc": "",
        "x": 130,
        "y": 380,
        "wires": [
            [
                "5cdd1ceea892e100"
            ]
        ]
    },
    {
        "id": "65e579fb32d2e065",
        "type": "http response",
        "z": "436b05d74282c879",
        "name": "Serves the audio",
        "statusCode": "",
        "headers": {
            "content-type": "audio/wav"
        },
        "x": 650,
        "y": 380,
        "wires": []
    },
    {
        "id": "5cdd1ceea892e100",
        "type": "change",
        "z": "436b05d74282c879",
        "name": "Grab the buffer, shove it in msg.payload",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "buffer",
                "tot": "flow"
            },
            {
                "t": "set",
                "p": "filename",
                "pt": "msg",
                "to": "hello.wav",
                "tot": "str"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 380,
        "y": 380,
        "wires": [
            [
                "65e579fb32d2e065"
            ]
        ]
    },
    {
        "id": "8e796952bdc87230",
        "type": "mqtt in",
        "z": "436b05d74282c879",
        "name": "",
        "topic": "/some/topic",
        "qos": "2",
        "datatype": "auto-detect",
        "broker": "2580135670619669",
        "nl": false,
        "rap": true,
        "rh": 0,
        "inputs": 0,
        "x": 150,
        "y": 140,
        "wires": [
            [
                "3a23e0778b405107"
            ]
        ]
    },
    {
        "id": "2580135670619669",
        "type": "mqtt-broker",
        "name": "MQTT SERVER",
        "broker": "localhost",
        "port": 1883,
        "clientid": "",
        "autoConnect": true,
        "usetls": false,
        "protocolVersion": 4,
        "keepalive": 60,
        "cleansession": true,
        "autoUnsubscribe": true,
        "birthTopic": "",
        "birthQos": "0",
        "birthRetain": "false",
        "birthPayload": "",
        "birthMsg": {},
        "closeTopic": "",
        "closeQos": "0",
        "closeRetain": "false",
        "closePayload": "",
        "closeMsg": {},
        "willTopic": "",
        "willQos": "0",
        "willRetain": "false",
        "willPayload": "",
        "willMsg": {},
        "userProps": "",
        "sessionExpiry": ""
    }
]

Of course, now I must solve the problem of why I am failing to transmit the buffer via MQTT.

The message I send from the main machine contains an object as the payload, of which one property is the buffer. I thought I would be able to put msg.payload.audio into flow.buffer and just have it work, but it doesn’t.

It’ll be something to do with encoding. I wish I understood that stuff better too, but at least I have a working solution in the meantime.

Thank you for your assistance.