Ui_mp4frag for DB2

[edje11]

Made somebody already a working replacement for DB2 to use something like the node-red-ui-mp4frag node.
I want to have some live CCTV streams in my DB2 dashboard, but without the high cpu load.
Nodes from Kevin working perfect for me in DB1, however I'm now in process to migrate some stuff to DB2.

I have tried the Video player example, but it's not clear to me how to connect this to the node-red-mp4frag. Tried different url's in the source src section of the example, but without luck.

Somebody a hint to get me in the right direction.

[joepavitt]

This isn't an area I'm too familiar with, but maybe @kevinGodell could advise, we can combine mp4frag and D2.0 knowledge to come to a solution

[BartButenaers]

See similar question here

[edje11]

We got picture

Played around with the sample from Joe and there's now a live CCTV stream on my test DB2 in combination with the mp4frag node.
Reliable it's totally not. Sometimes it runs for longer time and then it stops every minute or something like that. Do not found another as to restart the browser to get the stream going again.

Cpu load is still around 1% on an Odroid C4, nice.

<template>
    <video ref="Voor" style="width: 100%" autoplay>
        <source src="http://192.168.12.127:1880/mp4frag/achter/video.mp4" type="video/mp4">
        Your browser does not support the video tag.
    </video>
</template>

[kevinGodell]

@edje11 nice job with that. But of course, that will not work in safari (or any other browser that expects byte range responses), which may not be an issue for you if using chrome or firefox. Since you are adept with using the template, you could also try your hand at importing the hls.js library and using it to play the hls.m3u8 served from node-red-mp4frag.

Also, I was working on a replacement for node-red-ui-mp4frag, but it wasn't going to be published as a node, but instead shared as a flow. The plan is for it to be able to be included in the template of db1 or db2 or used directly in a basic html page. I had the functionality working pretty good, but the appearance was tricky to get right. I will try to get something available soon...

[BartButenaers]

Hi @kevinGodell,

Could you please support my old brain and summarize shortly our different options in dashboard D2:

1. Use hls.js to play the hls.m3u8 served from node-red-mp4frag.
2. Use the video tag in edge or chrome, and that supports live streaming out of the box? I assume I have not interpreted this correctly...
3. Others?

Thanks!!

[kevinGodell]

@BartButenaers

Use hls.js to play the hls.m3u8 served from node-red-mp4frag.

i just found an interesting section on the hls.js readme showcasing multiple video players, might be a good starting point instead of building one from scratch -> hls.js build a custom ui

@BartButenaers

Use the video tag in edge or chrome, and that supports live streaming out of the box? I assume I have not interpreted this correctly...

The mp4 video source is almost like live streaming, but i cheat a little and trick the browser to make it work (safari not included). You can link directly to the video.mp4 generated by node-red-mp4frag, but based on your browser, you can run into a limit on the number of persistent http connections. Using nginx reverse proxy and upgrading the connection to http2 will work around that. Safari will not load this type of video source because it is strict on video streaming and prefers native hls, but kinda supports media source and now in later versions of ios 17 supports managed media source instead.

@BartButenaers

Others?

The main problem, as documented in db1 and db2, is the race condition when importing an external library and hoping it loads before trying to use it. It may fail the first load, get cached, and then later work reliably as it will be loaded from the browsers cache. I was trying to work around that issue elegantly, but... i can't remember where i left off on that.

[BartButenaers]

@kevinGodell

The main problem, as documented in db1 and db2, is the race condition when importing an external library and hoping it loads before trying to use it

I assume you mean loading the hls.js package from a public cdn, or an endpoint in Node-RED?

I "think" it needs to be done like this:

1. Add hls.js as a dependency to your ui node package json file.
2. Import hls.js in your vue file:

   <template>
     <video ref="video" width="100%" height="640" controls></video>
   </template>
   
   <script>
      import Hls from 'hls.js';
      ...
   

3. Run a Vite build of your ui node via npm run build. Due to the above import statement, Vue will automatically detect that you need the hls.js library in your frontend: it will automatically download hls.js from NPM, and add it (minimized) to your bundle file.
4. Since the code for hls.js is included in your bundle, it will be available in your frontend so you can use it directly:

   export default {
     mounted() {
       let hls = new Hls();
       let stream = "http://demo.unified-streaming.com/video/tears-of-steel/tears-of-steel.ism/.m3u8";
       let video = this.$refs["video"];
       hls.loadSource(stream);
       hls.attachMedia(video);
       hls.on(Hls.Events.MANIFEST_PARSED, function () {
         video.play();
       });
     },
   };
   

There is also a vue-hls-video-player but I have not enough Vue knowledge to know whether that is a better solution.

[joepavitt]

This is correct. If built as a third party node, the race condition isn't an issue anymore

[BartButenaers]

@kevinGodell

Also, I was working on a replacement for node-red-ui-mp4frag, but it wasn't going to be published as a node, but instead shared as a flow

Hey Kevin,
Since a third party ui node is required for bundling the hls.js package, I will try to create a custom ui node for D2 dedicated to hls. So no other video formats... Then I at least can see my ip cams again in the new dashboard. Hopefully you can share your knowledge as soon as I am stuck, because I have no in depth knowledge about the topic...

[Colin]

@BartButenaers

I will try to create a custom ui node for D2

Hi Bart, did you see my subflow for creating the basic code needed for a D2 ui node?

Subflow for generating D2 UI widget

Creating all the files for a basic D2 UI node is somewhat tedious, so I have developed a subflow to create the necessary files, with basic content, which will build to a functioning widget. If anyone would like to try it out that would be great. In particular I have no idea if it will work on Windows, with the file paths specified with backslashes. These instructions are from the subflow help text: A subflow that generates a basic UI widget for node-red Dashboard 2 (@flowfuse/node-red-dashbo…

[BartButenaers]

@Colin

did you see

Only now Thanks!!

[Colin]

If you are using Windows you might have to use forward slashes in paths, but let me know if that is necessary and I will update the flow to cope with it. Let me know in the other thread.

[BartButenaers]

BTW I quickly created a new ui node for D2. Did it for now without Colin's nice subflow, because I had some cyclic serialization problems in my older Node-RED version. So I did it with some old school manual replacing. Getting used to that after so many years

I have now a custom node-red-dashboard-2-ui-hls node (not on Github yet...), that shows in my VueJs dashboard a very smooth and very sharp m3u8 live stream using hls.js:

BTW it is not that sharp in my demo, since I recorded it with low quality to show here...

[joepavitt]

Curious, how are you finding the developer experience now you've done it once or twice?

Any obvious areas we can improve?

[BartButenaers]

Hi Joe,

Well overall - once you have a very basic Vue knowledge - it goes pretty well.
The following things are (imho) a bit difficult:

1. The example ui node project is very useful. But you need to go through every file manually and change everything required. However if you make a typo somewhere (or if you forget to update something) it won't work and it is difficult to track what you have done wrong. For that e.g. the ui node generator flow from @Colin could be useful (i.e. to generate a basic ui node skeleton based on a bunch of settings).

2. There is still a lot of basic stuff being refactored in the dashboard. For example the structure of input messages (ui-control, ...). I find it a bit hard to keep track, especially because I don't have enough free time to keep reading through the docs over and over again. But the number of these changes will reduce after a while...

3. To be honest I have no idea anymore how I need to store my state and input messages anymore to keep my clients up to date and after a refresh. That is why I initially started the server-side state discussion at the time being, because I hoped to avoid getting into this situation. Got a lot of useful info from you and Colin, but I completely lost overview. But since Colin already managed to get a grip on it for his node(s), it must be due to a twist in my brain. If anybody would be able to draw a scetch of how it all comes together (config screen properties, dynamic properties from input messages, server-side stores, client-side stores, vuex, refresh behaviour and so on ...) that would REALLY help. Normally I go through all discussions and documentation, and then I start drawing. But don't see that happening soon due to lack of free time...

Perhaps some of the moderators can split the comments about custom ui node developments (and Colin's subflow) in a separate discussion, because I would like to focus here on the ui_mp4frag question.