Audio does not work in uibuilder

Hello,

I'm a novice JS/CSS/HTML programmer and I'm trying to use a uibuilder to create a multimedia display interface for an Escaper Room.

I'm having problems with HTML audio, I have put some code together starting from examples but it doesn't often work. In particular, when I refresh the uibuilder page, it does not work anymore: I can see metadata about the length of the audio loaded but it gets stuck at the start without reproducing. Something to to with preloading? The video work in a similiar way, but they work fine. I noted that even the videos, if I remove the "muted" flag, get stuck at the starting frame, even tough the "autoplay" is enabled.

The "src" of the audio is changed at execution time by some messages, switching from "file.mp3" to "" at command. I do that for the video and they play fine.

Do you have any suggestions?

Below, my uibuilder:

index.html:

<html lang="en">

<head>
    <!-- Put your own custom styles in here -->
    <link rel="stylesheet" href="./index.css" media="all">
    <!-- Include Webfont to style text in custom font -->
    <link href="https://fonts.googleapis.com/css?family=Staatliches&display=swap" rel="stylesheet">
</head>

<body>
    <!-- The "app" element contains any content that receives dynamic updates -->
    <div id="app">
        <!--  Sounds -->
        <audio autoplay controls id="audio-player" ref="audioRef">
            <source src="" type="audio/mpeg" />
        </audio>
        
        <!--  Music -->
        <audio autoplay controls style="top:600px" loop id="music-player" ref="musicRef">
            <source src="" type="audio/mpeg" />
        </audio>
        
        <!--  Video Overlay-->
        <video autoplay muted class="overlay" id="overlayVideo" ref="overlayRef">
            <source src="" type="video/mp4" />
        </video>

        <!--  Video Background -->
        <video autoplay muted loop class="background" id="backgroundVideo" ref="backgroundRef">
            <source src="" type="video/mp4" />
        </video>
        
        <!--  Time -->
        <div class="timerTextNormal" id="timerTextId">
            {{msg.formattedTimeRemaining}}
        </div>

        <!--  Help -->
        <div class="clueText">
            {{msg.clueText}}
        </div>
    </div>
    <!-- uibuilder script libraries -->
    <script src="../uibuilder/vendor/socket.io/socket.io.js"></script>
    <script src="../uibuilder/vendor/vue/dist/vue.min.js"></script>
    <script src="./uibuilderfe.min.js"></script>
    <!-- Put any additional custom code in here -->
    <script src="./index.js"></script>
</body>

</html>

index.js:

lastAudioSrc = "";
lastMusicSrc = "";
lastBackgroundSrc = "";
lastOverlaySrc = "";

var app = new Vue({
    // The HTML element to attach to
    el: '#app',
    // Variables defined here will be avalable and updated within the HTML
    data: {
        msg: '[No Message Received Yet]',
    },
    // Callback function when Vue library is fully loaded
    mounted: function () {
        // Start up uibuilder
        uibuilder.start();
        // Keep a reference to the Vue app
        var vueApp = this;
        // Callback triggered when node receives a (non-control) msg
        uibuilder.onChange('msg', function (msg) {
            vueApp.msg = msg; 

            /* Cambia stile del tempo rimasto, in base al tempo rimasto */
            var timer_classes = document.getElementById('timerTextId').classList
            
            if (msg.timeRemaining == 0)
            {
                /* Animazione quando scade! */
                timer_classes.replace('timerTextNormal','timerTextGameOver');
                timer_classes.replace('timerTextDanger', 'timerTextGameOver');
            }
            else
            {
                /* Torna normale quando il tempo rimasto è maggiore di 0 */
                timer_classes.replace('timerTextGameOver', 'timerTextNormal');

                if (msg.timeColour == "red")
                {
                    timer_classes.replace('timerTextNormal', 'timerTextDanger');
                }
                else
                {
                    timer_classes.replace('timerTextDanger', 'timerTextNormal');
                }
            }

            /* Audio */
            if (lastAudioSrc != msg.audioSrc) 
            {
                vueApp.$refs.audioRef.src = msg.audioSrc;
                lastAudioSrc = msg.audioSrc;
            }

            /* Musica */
            if (lastMusicSrc != msg.musicSrc) 
            {
                vueApp.$refs.musicRef.src = msg.musicSrc;
                lastMusicSrc = msg.musicSrc;
            }

            /* Aggiorna sorgente video con quello scelto */
            if (lastBackgroundSrc != msg.videoBackgroundSrc) 
            {
                vueApp.$refs.backgroundRef.src = msg.videoBackgroundSrc;
                lastBackgroundSrc = msg.videoBackgroundSrc;
            }
            
            /* Aggiorna sorgente video con quello scelto */
            if (lastOverlaySrc != msg.videoOverlaySrc) 
            {
                vueApp.$refs.overlayRef.src = msg.videoOverlaySrc;
                lastOverlaySrc = msg.videoOverlaySrc;
            }
        });
    },
});

Some complexity here but the first thing that strikes me is that you are doing the classic mistake of adding a front-end framework (VueJS) but then doing things via the HTML DOM. You should do one or the other, not both. At an initial guess, I'd say that is why you are having odd issues.

You should not, for example, be manipulating the DOM to change classes when using Vue. Use Vue to do that since it has the capability. Otherwise, ditch Vue and stick with just the DOM (which is increasingly the way I'm going).


Also, I need to know what version of uibuilder you are using because the template you are using is pretty old now.

Based on the latest dev templates (which should work fine with uibuilder v5.1.1 or v6.0.0) and the audio and video tag examples from MDN. Here is a trivial thing that loads a video on demand when you send a msg from Node-RED. It does not need Vue.

html

<!doctype html>
<html lang="en"><head>

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <title>Blank template - Node-RED uibuilder</title>
    <meta name="description" content="Node-RED uibuilder - Blank template">
    <link rel="icon" href="./images/node-blue.ico">

    <!-- Your own CSS (defaults to loading uibuilders css)-->
    <link type="text/css" rel="stylesheet" href="./index.css" media="all">

    <!-- #region Supporting Scripts. These MUST be in the right order. Note no leading / - socket.io no longer needed  -->
    <script defer src="../uibuilder/uibuilder.iife.min.js"></script>
    <script defer src="./index.js">/* OPTIONAL: Put your custom code in that */</script>
    <!-- #endregion -->

</head><body class="uib">
    
    <h1>uibuilder Blank Template - v6.1.0</h1>

    <div id="more"><!-- '#more' is used as a parent for dynamic HTML content in examples --></div>

    <div>
        <figure>
            <figcaption>Watch the video:</figcaption>
            <video id="myvideo" controls width="250">
                <!-- <source src="./flower.webm" type="video/webm">
                <source src="/.flower.mp4" type="video/mp4"> -->
            </video>
        </figure>

        <figure>
            <figcaption>Listen to the T-Rex:</figcaption>
            <audio id="myaudio" controls src="./t-rex-roar.mp3">
                <a href="./t-rex-roar.mp3">
                    Download audio
                </a>
            </audio>
        </figure>

    </div>


    <!-- Two different ways to send data back to Node-RED via buttons.
         fnSendToNR uses standard `uibuilder.send`.
         eventSend includes `data-*` attributes, keyb modifiers, etc. Works with any event. -->
    <button onclick="fnSendToNR('A message from the sharp end!')">Send a msg back to Node-RED</button>
    <button onclick="uibuilder.eventSend(event)" data-type="eventSend" data-foo="Bah">eventSend</button>

    <pre id="msg" class="syntax-highlight">Waiting for a message from Node-RED</pre>

</body></html>

JavaScript

// @ts-nocheck

/** The simplest use of uibuilder client library
 * Note that uibuilder.start() should no longer be needed.
 * See the Tech docs if the client doesn't start on its own.
 */
'use strict'

// logLevel 2+ shows more built-in logging. 0=error,1=warn,2=info,3=log,4=debug,5=trace.
// uibuilder.set('logLevel', 2)
// uibuilder.log('info', 'a prefix', 'some info', {any:'data',life:42})

// Helper function to send a message back to Node-RED using the standard send function - see the HTML file for use
window.fnSendToNR = function fnSendToNR(payload) {
    uibuilder.send({
        'topic': 'msg-from-uibuilder-front-end',
        'payload': payload,
    })
}

// Listen for incoming messages from Node-RED
uibuilder.onChange('msg', function(msg) {
    // Dump the msg as text to the "msg" html element
    // either the HTML way or via uibuilder's $ helper function
    // const eMsg = document.getElementById('msg')
    const eMsg = $('#msg')
    if (eMsg) eMsg.innerHTML = uibuilder.syntaxHighlight(msg)

    $('#myvideo').innerHTML = '<source src="./flower.webm" type="video/webm">'
})

Hi, thanks for the answer, I'm uing 5.1.1.

Yesterday I found the source of my problem: anti-autoplay policy inside browsers:

The reason why I had the problem sporadically is that sometimes I clicked the window by mistake and other times not.

I have to start chrome with the "--autoplay-policy=no-user-gesture-required" argumen, since there cannot be interaction with the page.

Regarding your example I completely agree, best not to use stuff without using it. I'll follow your example to "modernize" my code. Thanks a lot again.

1 Like

Glad you got it sorted. I stopped using Google Chrome ages ago as I don't like their (lack of) consistent privacy. I use Edge mostly. Microsoft are at least not dependent on advertising to make a profit and I need to use Edge for work anyway - I really like the separate profiles that I use to keep different personal and professional use separated.

One of the things that uibuilder development has been teaching me is just how easy it can now be to make highly reactive, data-driven web pages. So much so that I rarely use Vue any more (my previous go-to framework). Vue haven't helped themselves at all over the debacle of forcing the move from v2 to v3 too quickly. My recent couple of posts show how easy it is to work directly on the DOM - isn't the most efficient way so maybe not great for really complex SPA's but for many things it is certainly the easiest.

The uibuilder new client already has the $('my-css-selector') helper which can give you easy access to DOM elements without having to remember the more complex document.querySelectorAll('my-css-selector').

In uibuilder v6.1, there will be a new node called uib-update which will make it easy to update 1 or more elements also by CSS selector. It will have the same caching ability as the other nodes as well so that newly connected browser windows can get the last update if desired.

I'm also redoing all of the templates and example flows. Including the Svelte example. Svelte is a very interesting and light weight framework that provides lots of features without compromising the use of modern HTML standards like many of the other frameworks do. It does require a build step but the interesting thing is that it's dev server is fully compatible with uibuilder so you get the benefit of live reloading after changes without having to make any changes to your front-end code.

Very interesing, I'm a complete newbie in this sector, so it is nice knowledge for me :smile:
I'm mostly a Embedded SW engineer so web programming is quite far from my job. I'm developing this for a side/personal project.

Revisiting the code, now it looks like this (better?)

index.html:

<html lang="en">

<head>
    <!-- Put your own custom styles in here -->
    <link rel="stylesheet" href="./index.css" media="all">
    <!-- Include Webfont to style text in custom font -->
    <link href="https://fonts.googleapis.com/css?family=Staatliches&display=swap" rel="stylesheet">
</head>

<body>

    <!--  Effetti sonori -->
    <audio id="audioID" ref="audioRef"></audio>
    
    <!--  Musica -->
    <audio loop id="musicID" ref="musicRef"></audio>
    
    <!--  Video in primo piano -->
    <video muted class="overlay" id="overlayID" ref="overlayRef"></video>

    <!--  Video sullo sfondo -->
    <video muted loop class="background" id="backgroundID" ref="backgroundRef"></video>
    
    <!--  Testo tempo rimanente -->
    <div class="timerText textNormal" id="timerTextID"></div>

    <!--  Testo aiuto -->
    <div class="clueText textNormal", id="clueTextID"></div>
    
    <!-- uibuilder script libraries -->
    <script src="../uibuilder/vendor/socket.io/socket.io.js"></script>
    <script src="./uibuilderfe.min.js"></script>
    
    <!-- Put any additional custom code in here -->
    <script src="./index.js"></script>
</body>

</html>

index.js:

'use strict'

// logLevel 2+ shows more built-in logging. 0=error,1=warn,2=info,3=log,4=debug,5=trace.
// uibuilder.set('logLevel', 2)
// uibuilder.log('info', 'a prefix', 'some info', {any:'data',life:42})

var LastTimer = "";
var lastClue = "";
var LastBackground = "";

/* Riproduce o interrompe audio con ID "id", in base a "source" pieno o vuoto */
function update_audio(source, id) {
    /* Trova elemento dall'ID */
    var audio_obj = document.getElementById(id)

    if (source != "") {
        audio_obj.src = source;   /* Imposta sorgente */
        audio_obj.load();         /* Ricarica */
        audio_obj.play();         /* Riproduce */
        //console.log("Play audio")
    }
    else {
        audio_obj.pause();        /* Pausa */
        audio_obj.currentTime = 0;/* Rimette da capo */
        //console.log("Pause audio")
    }
}

/* Riproduce o rimuove video con ID "id", in base a "source" pieno o vuoto */
function update_video(source, id) {
    var video_obj = document.getElementById(id)

    /* Se la sorgente nel messaggio è impostata */
    if (source != "") {
        video_obj.src = source;             /* Imposta sorgente */
        video_obj.load();                   /* Ricarica */
        video_obj.play();                   /* Riproduce */
        //console.log("Play video")
    }
    else {
        video_obj.removeAttribute('src');   /* Svuota sorgente */
        video_obj.load();                   /* Ricarica, rimuovendo video */
        //console.log("Remove")
    }
}

// Needed
uibuilder.start()

// Listen for incoming messages from Node-RED
uibuilder.onChange('msg', function (msg) {

    var timer = document.getElementById('timerTextID');
    var clue = document.getElementById('clueTextID');
    var timer_classes = timer.classList;

    /* Aggiorna testo del tempo rimasto */
    if (msg.formattedTimeRemaining != undefined && LastTimer != msg.formattedTimeRemaining) {
        timer.innerHTML = msg.formattedTimeRemaining;
        LastTimer = msg.formattedTimeRemaining;
    }

    /* Aggiorna testo dell'aiuto */
    if (msg.clueText != undefined && lastClue != msg.clueText)
    {
        clue.innerHTML = msg.clueText;
        lastClue = msg.clueText;
    }

    /* Cambia stile del tempo rimasto, in base al tempo rimasto */
    if (msg.timeRemaining == 0) {
        /* Animazione quando scade! */
        timer_classes.replace('textNormal', 'textGameOver');
        timer_classes.replace('textDanger', 'textGameOver');
    }
    else {
        /* Torna normale quando il tempo rimasto è maggiore di 0 */
        timer_classes.replace('textGameOver', 'textNormal');

        if (msg.timeColour == "red") {
            timer_classes.replace('textNormal', 'textDanger');
        }
        else {
            timer_classes.replace('textDanger', 'textNormal');
        }
    }

    /* Audio */
    if (msg.audioSrc != undefined) {
        update_audio(msg.audioSrc, "audioID");
    }

    /* Musica */
    if (msg.musicSrc != undefined) {
        update_audio(msg.musicSrc, "musicID");
    }

    /* Aggiorna sorgente video con quello scelto */
    if (msg.videoOverlaySrc != undefined) {
        update_video(msg.videoOverlaySrc, "overlayID")
    }

    /* Aggiorna sorgente video con quello scelto */
    if (msg.videoBackgroundSrc != undefined && LastBackground != msg.videoBackgroundSrc) {
        update_video(msg.videoBackgroundSrc, "backgroundID")
        LastBackground = msg.videoBackgroundSrc;
    }
})

It seems I require "uibuilder.start()" or nothing works. How do I update my uibuilder version, eventually on Node-RED? The latest version is 5.1.1 there.

Try this for your HTML. Slightly cleaner and uses the new client. Your HTML was also not entirely valid and this corrects that. Also note that the new client no longer requires you to separately load socket.io as it does it for you.

It would be good for you to update uibuilder to v6.0.0 as well as long as you are using at least node.js v14 (which is strongly recommended and the minimum also for Node-RED v3). It has numerous bug fixes and enhancements.

<!doctype html>
<html lang="en"><head>

    <meta charset="utf-8">
    <meta name="viewport" content="width=device-width, initial-scale=1">

    <!-- Put your own custom styles in here -->
    <link rel="stylesheet" href="./index.css" media="all">
    <!-- Include Webfont to style text in custom font -->
    <link href="https://fonts.googleapis.com/css?family=Staatliches&display=swap" rel="stylesheet">

    <!-- #region Supporting Scripts. These MUST be in the right order. Note no leading / - socket.io no longer needed  -->
    <script defer src="../uibuilder/uibuilder.iife.min.js"></script>
    <script defer src="./index.js"></script>
    <!-- #endregion -->

</head><body>

    <!--  Effetti sonori -->
    <audio id="audioID" ref="audioRef"></audio>
    
    <!--  Musica -->
    <audio loop id="musicID" ref="musicRef"></audio>
    
    <!--  Video in primo piano -->
    <video muted class="overlay" id="overlayID" ref="overlayRef"></video>

    <!--  Video sullo sfondo -->
    <video muted loop class="background" id="backgroundID" ref="backgroundRef"></video>
    
    <!--  Testo tempo rimanente -->
    <div class="timerText textNormal" id="timerTextID"></div>

    <!--  Testo aiuto -->
    <div class="clueText textNormal", id="clueTextID"></div>
    
</body></html>

Also, if using the new client, you can remove the uibuilder.start() from your index.js file as that is also no longer needed.

Thanks, I reinstalled node-red and I was able to update uibuilder!

Your patch now works fine, and I could remove the start function :slight_smile:

Thanks again

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.