Dashboard "Lost connection" when in background tab

#1

Hello,
I face a "Lost connection" issue in the dashboard when putting the dashboard tab in background task (hidden). The issue is easily reproducible in Firefox (much more difficult to reproduce in Chrome but still).

The easy way to reproduce the issue is to take an inject node (sending timestamp every 100ms) and link it to a ui_template node (with default template code). Here is the flow:

[
  {
      "id": "ead64dd9.a36088",
      "type": "inject",
      "z": "4aed941a.4518b4",
      "name": "",
      "topic": "",
      "payload": "",
      "payloadType": "date",
      "repeat": "0.1",
      "crontab": "",
      "once": false,
      "onceDelay": 0.1,
      "x": 270,
      "y": 220,
      "wires": [
          [
              "cfde5b78.259018"
          ]
      ]
  },
  {
      "id": "cfde5b78.259018",
      "type": "ui_template",
      "z": "4aed941a.4518b4",
      "group": "d610e123.12c21",
      "name": "",
      "order": 0,
      "width": "12",
      "height": "6",
      "format": "<div ng-bind-html=\"msg.payload\"></div>",
      "storeOutMessages": true,
      "fwdInMessages": true,
      "templateScope": "local",
      "x": 520,
      "y": 220,
      "wires": [
          []
      ]
  },
  {
      "id": "d610e123.12c21",
      "type": "ui_group",
      "z": "",
      "name": "Design",
      "tab": "5fd58689.008b98",
      "disp": true,
      "width": "12",
      "collapse": true
  },
  {
      "id": "5fd58689.008b98",
      "type": "ui_tab",
      "z": "",
      "name": "Acquisition",
      "icon": "memory",
      "order": 2,
      "disabled": false,
      "hidden": false
  }
]

Then open the dashboard in a new Firefox tab on the same machine, and put it in background. After a 20-30s bring the dashboard tab back in the front => the "connection lost" popup appears and the dashboard is completely refreshed !
The following logs appears in firefox dev console:

Rendered http://127.0.0.1:1880/ui/css/app.min.less successfully. app.min.js:514:12823
CSS for http://127.0.0.1:1880/ui/css/app.min.less generated in 298ms app.min.js:514:12823
Less has finished. CSS generated in 300ms app.min.js:514:12823

My test environment is :

  • Windows_NT 10.0.16299 x64 LE
  • Node.js version: v10.15.3
  • Node-RED version: v0.20.5
  • Dashboard version 2.14.0
#2

I think that lost connection issues have been knocking around for a long time - worse on mobile I think.

AFAIK, nobody has really been able to get to the bottom of it.

#3

I guess we can try to suppress the popup if the window is in the background. It is triggered by a websocket disconnect event - it may be that the browser is suspending or dropping the socket while in background (not sure why it would but...) and that may stack up the event so it then appears when visible again...
but if that happens for some reason when it comes back into the foreground (i.e. it bounces) then not sure what we can do as a loss when in foreground is correct behaviour. Will have a look but... may be out of our control.

#4

Thanks for the answer. Then do you know is there is a way to prevent the browser to suspend / drop the socket while in the background ?
Also not sure to understand what you mean by "a loss when in foreground is correct behaviour" ? I mean as long as the tab remains in foreground there is no disconnection ...

#5

I think that Dave means that you should get a pop-up if you loose connection when the tab is in the foreground as this indicates some other issue.

Really, browsers need some kind of per-tab flag that lets you control whether backgrounding triggers the normal power-/memory-save functions that are most likely triggering the behaviour you are seeing.

#6

Ok quite normal to get a pop-up if the connection is lost while in foreground => no problem.
The problem is that the pop-up only appears when I put the tab in background, wait few seconds, then put back in foreground !

#7

Hi - sorry - what I meant was - if is in background and something triggers a suspend/disconnect - (but doesn't actually do anything as it is suspended) - then switches back to foreground and the queue of pending events gets executed - and it finds the disconnect one there it could then popup as it was told to do so, and is now in the foreground...

Anyway I have added some code to master version of dashboard on github (not yet on npm) - that may help - but as I'm unable to get it to fail here anyway it may well not help... but fingers crossed.

#8

Hi, I just tried dashboard 2.15.1 from npm.... unfortunately, the problem is still present.
It's quite surprising you're unable to reproduce the issue. With the flow I provided in my first post it is systematic on my side when I use Firefox.

In fact, to me the main problem is not to get the disconnect popup when putting back the dashboard page in foreground, but more to get the whole page refreshed ...

#9

This implies that the issue is actually in your environment. So either something to do with the configuration of your PC or the browser. Possibly an extension - did you try with no extensions loaded? Given this is happening over multiple browsers though, most likely some kind of network configuration issue.

#10

Hi, I don't think it has something related to my PC or my network configuration. Indeed, i reproduce the issue on various environment (BTW running on localhost):

  • at my office : on my office laptop (environment described in the first post) behind a corporate proxy for network access.

  • at home: on my MacBookPro, on my personal network.

On both configurations it is quite easy to reproduce when using Firefox, but finally can't reproduce with Chrome browser. I will try on a Linux configuration later, but I would be surprised not to face the problem again.

So I guess it is related to Firefox suspending or dropping the socket while in background (as suggested by @dceejay in an earlier post) , but in that case you should also be able to reproduce on your side ?!
Also I don't know if some Firefox settings exist to prevent it from having such behavior for tab in background ?

#11

What device are you running on? That's the next place to look I guess.

Clearly something is happening. I don't see this either even when running Node-RED locally on a PC that goes into low-power mode.

#12

As mentioned, I run Node-RED locally on a PC (or a MacBookPro) and I reproduce the issue quite easily when using Firefox with the provided flow and with dashboard tab in background. I also put the PC (or the MBP) out of any network. Moreover, my colleagues also have the same behavior.

Are you using the flow I provided, and the latest version of Firefox ?

#13

OK - so adding some debug to the disconnection event I can see Firefox does indeed seem to fire the disconnect approx once per minute. (and Chrome does not, nor Safari) However with the extra detection I added into dashboard 2.15.0 I can see that when hidden it is not stacking up any toast /alert requests. Switching back after several disconnect events so far does not trigger an alert for me. I'll try leaving it over lunch and see what happens then.

As our response to the disconnect is to then reconnect and update the screen to ensure it has latest info. That is working as designed. Not sure what else we could do.

#14

Also please notice that the occurrence of the problem increases with the injection frequency.
For instance:

  • setting the injection node to repeat every 10ms will drastically increase the occurrence.
  • setting the injection node to repeat each 5s, and the problem disappears ..
#15

I don't know if you have time to test this with uibuilder? I wonder if the same is happening there as well?

The difference being that I don't alert on connection lost, that is left up to the ui author to handle should they want to. However, you would see lots of disconnect/reconnect messages from the second output of the uibuilder node.

If I get time, I will check myself.

Quick test of your flow does show a difference in the way that Chromium and Firefox process background tabs with websockets. (I'm using Vivaldi which is Chromium based).

Chromium seems to keep the socket active but Firefox doesn't and it then restarts the socket connection from scratch as you can see twice in this log:

This is all running Node-RED and the browsers on the same Windows 10 PC, latest everything, Node.js v10.x

#16

So I can now make it so the popup doesn't appear... BUT Firefox is still reporting a disconnect event and thus we do a reconnect which WILL do the redraw. So I think that is about as far as we can go. Anything else will be suggestions to swap your browser.

#17

I just tried uibuilder, and face no problem with it ... also I don't get any disconnect/reconnect message from the second output of the uibuilder node.

#18

Ok then still something weird here as even with latest dashboard version the popup still appears on my side (then reconnect, then redraw ...). Also I don't understand the link between the frequency of inject msg and the occurrence of disconnect/reconnect issue ?! Any guess on your side ?

#19

Well I haven't pushed the latest latest version so yes you may still get the occasional popup.
I would guess that the high frequency of updates may mean that you are managing to hit a small timing windows that causes the socket to be upset for some reason. Though what that window is I have no clue. I'll push out 2.15.2 later today.

#20

Just tried dashboard 2.15.2 and the popup disappears but unfortunately not the disconnect/reconnect/redraw behavior which is my main concern.

Also did you see my test with uibuilder ? I have no trouble with it (no disconnect/reconnect/redraw) meaning that the issue is only related to dashboard running in Firefox ?!