Node-ui-table: Inconsistent load behavior, leading to changing no. of columns on refresh

Hello together, thank you for providing a great user experience with node-red.
I am using node-red for a smart home dashboard.

For this I like also to embed a view of my upcoming calendar entries. However, after receiving and formatting of these I have a inconsistency problem while showing them in the node-ui-table (v0.4.3).

In roughly 50% of the cases the initial browser load leads to a messed up formatting of the calendar columns. I don't know, whether the node adds an additional column, or the overall ui-table size shrinks, leading to an unwanted compression of my table.
After hitting different times F5 at one point suddenly the formatting gets correct. Sometimes it is correct from the beginning.

I have no clues what causes this problem. I tried a lot of keyword injections, different column definitions, devices and so on. It doesn't get away.

The first picture shows a correct view of my ui-table, at the top of the dashboard.
The second picture shows the not correct view, happening sometimes without ovious resons.

Thank you very much for reading my problem.

Best regards

Moppel

I have had similar issues with dashboard-ui-table. This node is a wrapper of the JS Tabulator package, but uses a very outdated Tabulator version, which seems to collide sometimes (depending on load order) with other packages (and also offers very limited functionality).
Eventually I stopped using this node, and instead I am importing & instantiating the (latest) Tabulator package directly, which works like a charm, and offers much more functionality.

Hello Omrid,
thanks for your view of the problem. It is really bad news, however, it avoids fiddling around further without getting it stable.
However, it is quite strange. I don't use any complicated functions but encounter these problems. Because it is the only easy usable table for node-red, a lot more people should be affected by the problem.
I heard of other people, following your way. And are very satisfied now. I on the other hand don't need the extra functionality. For me that means a lot of more effort, just "to get it running".
I had this effect before, when I decided to use plane chart.js instead of the node-red plot wrapper. It was a lot of work, but at least I got the desired extra features.
In case of the table, the return of investment would be much worse ...
A workaround, even by hard-coded modifying the source of the node, would be very much appreciated ...
Best regards
Moppel

PS: If I would decide to go your way, is there any template/example you can recommend to use?

PS: If I would decide to go your way, is there any template/example you can recommend to use?

Yes, I have some examples & documentation

Hello Omrid,
if you like to share, I would be grateful. I know of at least on other implementation example online, but it looked too complicated with a lot of features implemented.

Best regards
Moppel

PS: Above problem by the way is due to a sneaked in column, not a layout size problem of the nide itself. It wasn't visible in above screenshot due to the only black addons used. The origin of the problem however still eludes me. I never defined more columns in the gui.

Can you send me your email address. I cannot upload Word documents in this forum.

hi Omrid, thank you very much: My mail adress is my username used plus "@posteo.de". Best regards, Moppel

Hello together,
hello Omri,

thank you, I got finally the Tabulator table working, the Random-Added-Column problem is solved. I followed Omris recommendation to directly access Tabulator, without the wrapper table-UI node. However, full story is, that I even with the Tabulator-in-template approach I had in the beginning the additional sporadic wrong column problem.
To get the native Tabulator implementation within a template running I used the tutorial and snipped Omri provided to me (except small variations). If Omri allows, I can submit some core snippets which show how to got it run.

But please let me tell you chronically:

  1. To get the library tabulator loaded within the template was very easy with the data provided by Omri. I thought it would be much more complicated.

  2. However, I suddenly run into my old problem known from "Table UI". The last column still gets sometimes squeezed while pressing F5 in the browser:
    Node Red Tabulator wrong usage

  3. The error appeared when I tried to implement an automatic resending of a cached table data message into Tabulator after a connection loss.

  4. I got the solution, after I found out that a manual resend by "Inject node" didn't lead to that squeezed row problem. So it was obviously a timing problem. The former automatisated resend was sometimes so fast that it probably hit sometimes in a not final initialzed Tabulator table.

  5. To avoid a race condition I just waited for the "Grid is ready" message of the Tabulator template and forwarded it backward to release the my cached table data message. It is no endless loop, because "Grid is ready" message is only send once after initialization, not after every data load in.

Maybe the UI-Table wrapper suffers under the same problem I observed above which leads to crashes during table refilling with the cached data.

@Omri:
Some points in your tutorial didn't work as expected though for me:

  • Your "ng-init" template snippet doesn't send a message with the content defined (data & user) in this template node. It was always an empty message consisting only of msg_Id and socketid.

@Other Tabulator experts:

  • Question: Does anyone know, how the avoid the grey ca. 6px border around the table? The UI-Table node was in this regards at least "full screen in group" and didn't have a border as in my above screenshot.

I am very happy now. Thanks again for your nice software and helpful forum!

Have a nice day -

Hi,
Glad to know you got it working, and indeed it is a good practice to wait for the 'Table ready' event.
(when working with the node-red table node, I sometimes had to set a delay of 500 msec to wait for the table to finish initializing).

The page-load (ng-init) template I provided is just an example (hello-world style). you need to set the init message yourself as you wish or pass it through nodes that will populate it.

You can experiment with visualization by either overriding the default Tabulator CSS, or adding styling in the Tabulator div, for example:

<div id="pGrid" style="background-color:#E3EFFB;margin:5px;border:2px solid black;"></div>

Last but not least: feel free to publish any part of my tutorial for the benefit of the Node-red community

Hello Omi, thanks again. Your CSS styling tip works like a charm. I even now added conditional formatting myself, which wouldn't be easy possible with ui-table wrapper. Real cool! Thank you.

Conserning the ng-init template: I got your point, however, the example you added in the template where you initated two fields in the message didn't work for me. I only got the empty message out of it with Socket-ID and msg.ID. However, if needed on can populate it using other technics as you told as well.

Thanks for your okay for source release. Here you go:

Template node: "Tabulator href & style"
It refers to the Tabulator library, which has to manually downloaded and extracted into the this folder "/home/pi/.node-red/node_modules/node-red-dashboard/dist". So you should see there the "tabulator-master" folder of the zip-file.

<link href="tabulator-master/dist/css/tabulator.min.css" rel="stylesheet">
<script type="text/javascript" src="tabulator-master/dist/js/tabulator.min.js"></script>
<!--
<link href="tabulator-master/dist/css/tabulator_midnight.min.css" rel="stylesheet">


<link href="https://unpkg.com/tabulator-tables/dist/css/tabulator_midnight.min.css" rel="stylesheet">
<script type="text/javascript" src="https://unpkg.com/tabulator-tables/dist/js/tabulator.min.js"></script>

-->
<style>
    .tabulator-table .tabulator-cell {
      //color: #CC3A82;
      font-weight: bold;
    }
</style>

The tabulator template node looks like this for me:

<form>
    <div id="gridExample" style="background-color:#000000;margin:0px;border:0px solid black"></div>
</form>

<script type="text/javascript">

var $scope = this.scope;

var grid = new Tabulator("#gridExample", {
      // height:"300",
      layout:"fitDataStretch",
      movableColumns: false,
      headerVisible: false,
      data:[
          {summary:"Lade Kalender ..."}
      ],
      columns:[
          {title:"Bezeichnung", field:"relativeDate",   resizable: false, width:60},
          {title:"Wochentag",   field:"eventStart",     resizable: false, width:70},
          {title:"",            field:"eventWeekday",   resizable: false, width:10},
          {title:"Ereignis",    field:"summary",        resizable: false, 
           formatter: function (cell) {
                let  el = cell.getElement();
                let val = cell.getValue();
                let lastThreeChars = val.substring(val.length - 3);
                let content        = val.substring(0, val.length - 3);
                
                if (lastThreeChars == "*S*") {
                    el.style.backgroundColor = "#000099";
                } else if (lastThreeChars == "*R*") {
                    el.style.backgroundColor = "#009900";
                } else if (lastThreeChars == "*M*") {
                    el.style.backgroundColor = "#990000";
                }

                el.style.color      = "#FFFFFF";
                //el.style.fontWeight = "bold";
                
                return content;
                }
            }
      ]
    });

    grid.on("tableBuilt", function(){
        $scope.send({payload: "Grid is ready"});
    });

(function(scope) {
    $scope.$watch('msg', function(msg)
    {
        if (msg)
        switch (msg.payload)
        {
            case "setdata":
                //grid.clearData();
                grid.setData(msg.data);
                $scope.send({payload: "Grid updated"});
                break;
        }
    });
})(scope);
</script>

A message to fill the template node looks like this with topic "setdata" and the array in msg.data.
I think that is analog to ui-table, only one used the msg.payload for the table data.

That is the final look:

So the mystery of the init message is now resolved.
You probably copy/pasted the code snippet as-is from the Word document:

...
newMsg.user = “Omri”;
...

Thanks to the (unrequested) help of Microsoft, Word automatically replaced the double quotes around 'Omri' to "Smart Quotes" (a right/left pair, “”). Replace them back to straight quotes and it will work.
BTW, the easiest way to catch such issues (e.g. syntax & runtime errors in the HTML or JavaScript code in the ui-template), is to open the console pane of the web browser and watch the error messages.

haha, wow, I would never have seen the difference!

Thanks for the tip with the console. I will try it out.

I often have such problems, but hadn't any chance to see the psotion of the ptoblrm. Only that there was a problem. That slows down debugging by a large margin.

So then every point here is adressed and solved ... for now :wink:

See you, it was a pleasure.

Best regards

1 Like

Hi,

very interesting discussion !

I started a project yesterday at first using ui-table, but quickly tried and adopted the standard tabulator way.
I still have a lot to figure out, but it seems ok. I'll have a deeper look later in all the advices from the topic.

Is there anything else useful from Omrid document which was not copied in the discussion ? If so, I'd happy to get a copy.

Thanks,

Jerome

Hi agin,

just made some more tried, but unfortunately I can't get it to work....

The header template is set-up correctly, with the tabulator master folder DL and copied in the right location.

<link href="tabulator-master/dist/css/tabulator.min.css" rel="stylesheet">
<script type="text/javascript" src="tabulator-master/dist/js/tabulator.min.js"></script>

<style>
    .tabulator-table .tabulator-cell {
        //color: #CC3A82;
        font-weight: bold;
    }
</style>

I have an array of objects to be sent to the table, which looks like this :

I modified the node to be able to display some of the properties, but nothing happens, while I correctly get an out message telling the grid was updated....
here is the template node code, showing some columns fields which should match object properties from the array ... :

<form>
    <div id="gridExample" style="background-color:#009900;margin:0px;border:0px solid black"></div>
</form>

<script type="text/javascript">
    var $scope = this.scope;

var grid = new Tabulator("#gridExample", {
      // height:"300",
      layout:"fitDataStretch",
      movableColumns: false,
      headerVisible: true,
      data:[
          {control:"Lade Kalender ..."}
      ],
      columns:[
          {title:"ID",          field:"#",   resizable: true, width:200},
          {title:"control",   field:"control",     resizable: true, width:200},
          {title:"sipsMode",            field:"sipsMode",   resizable: true, width:200},
          {title:"Ereignis",    field:"summary",        resizable: true, width:200,
           formatter: function (cell) {
                let  el = cell.getElement();
                let val = cell.getValue();
                let lastThreeChars = val.substring(val.length - 3);
                let content        = val.substring(0, val.length - 3);
                
                if (lastThreeChars == "*S*") {
                    el.style.backgroundColor = "#000099";
                } else if (lastThreeChars == "*R*") {
                    el.style.backgroundColor = "#009900";
                } else if (lastThreeChars == "*M*") {
                    el.style.backgroundColor = "#990000";
                }

                el.style.color      = "#FFFFFF";
                //el.style.fontWeight = "bold";
                
                return content;
                }
            }
      ]
    });

    grid.on("tableBuilt", function(){
        $scope.send({payload: "Grid is ready"});
    });

(function(scope) {
    $scope.$watch('msg', function(msg)
    {
        if (msg)
        switch (msg.payload)
        {
            case "setdata":
                //grid.clearData();
                grid.setData(msg.data);
                $scope.send({payload: "Grid updated"});
                break;
        }
    });
})(scope);
</script>

and result in DB :

I suspect something very obvious but which I can't find right now... any help appreciated !

Thanks !

Hello Jerome,
happy to see, there is some interest in the topice aside from my initial problem.
Omrid small documentation handled also the individual update or addition of single rows. Furthermore it also mentiones the meaning of the template sockedID, so that in his Tabulator implementation multiple user can add data and see the updates of others. That at least is what I understand. It also mentions the adress meaning in the browser adresse eg. : http://raspberry:1880/ui/#!/1?socketid=e9TubBBUikK91takAAAj
If you replace the "1?" with "2?" you cann directly in the brower go to the second tap without using the navigation hamburger menue. That information is very cool. So if Omrid could make his nice documentation available that would be a clear win for the community, on the other hand 90% of the features with Tabulator are already mentioned in this thread.

Concerning your problem. You listing of the message information looks good. msg.payload is given correctly, as msg.data. Any obvious mistake is not seen by me. However, I try to help you get it running.
First I recommend you strip down your example. e. g. remove the two last columns with the formatter. That makes it more concise. But the most important piece of imformation would be the answer, whether at least the static data load works. That means, is your static data at least shown in the control column?

data:[
          {control:"Lade Kalender ..."}
      ],

If this isn't the case, than the problem is probably not the message itself coming later with the desired update.
Did you by the way delete the "ui-table" library? Omid documentations mentions, that there might be a conflict. Futher check, that in your template node "Reload last data if updated" is not checked.

Hi,

thanks for your answer. Indeed I did not remove the regular ui-table node, that might be the cause of my troubles.

I'll investigate more this evening, and will come back to you .

Thanks !

Hi,

following your suggestion I removed the column which includes all the formatter, and now I can get data in the table.

So I'll investigate what happens here.

Thanks for your help !

hi, glad to here.
Yeah of course :stuck_out_tongue: , the formatter crashed because it got in your case unexpected output (it is tailored for my input). You van check the firefox console output, or the node-red log output whether we could have seen that maybe earlier.

Best regards

Hi, did not checked browser console, but NR console did not display anything....

For now I'm starting to play with rowformatter first, and get a few things to work. I'll go with standard formatter conditions later.
I also added some code to get msg out when a row is clicked, which was not implemented in your template code example.

Cheers !

A small comment:
In case you want to access specific rows, you must have a unique row identifier. The default Tabulator row identifier is a field called "id" (lowercase).
It seems you switched between field and title (also, "#" is not a valid field name unless it's quoted). I assume what you meant is:

     columns:[
          {title:"#", field:"id", resizable: true, width:200},

if you want to keep "ID" (uppercase) as you row index, you will need to specify it in a property called "index" while instantiating the grid.