[uibuilderfe:socket-disconnect] Reason: transport close when receiving json from node-red

[fangfang94]

Currently I am using post to upload a excel file , and I will use ajax to get this response.
and the next step is reading the excel header into a smart-select such as

After select the table heading, and I will generate a table base on this flow

the configurator and custom_output I use for filtering excel table heading and the data from json
to append correctly.

the result output will be like ：

but the problem is if the excel have 100-200 rows (test data) it will work fine.
but for the real data usually have 1,000 - 5,000 rows and my dev console will return

[uibuilderfe:socket-disconnect] Reason: transport close

and nothing error in node-red debug option.

May I know how do I fix this or any suggestions to increase my flow?

ps: still new in all of these, keep learning

[fangfang94]

The problem happen while I send to uibuilder

here the console
and my configurator node:

[TotallyInformation]

@fangfang94

The problem happen while I send to uibuilder

The command you are showing is sending from uibuilderfe to Node-RED, is that where the error is?

Also, you are manually adding the _socketId - are you sure that the id you are adding matches the client's current id? The id does change if your browser has any network issues and so temporarily looses connection.

Other than that, I suspect that you are trying to send too large a message. I've never actually done any message size testing with Socket.IO (which is what uibuilder uses for its websocket connections).

My guess would be that either the network connection between your browser and the Node-RED Server is a little slow and so the request times out before it is finished or there is some inherent limit in a websocket message.

Possibly reviewing the websocket exchanges in your browser dev tools network tab might shed some light.

Maybe test the length of the payload before sending and split it into chunks if needed.

[UnborN]

@fangfang94

if the excel have 100-200 rows (test data) it will work fine.
but for the real data usually have 1,000 - 5,000 rows and my dev console will return

[uibuilderfe:socket-disconnect] Reason: transport close

Found this discussion on the socket.io github. Maybe its related

v3 and v4 close socket if more than 1MB is sent (but v2 is fine)

[TotallyInformation]

Thanks Andy, that's helpful. I didn't know that.

uibuilder now uses Socket.IO v4.

I see that there is a potential work-around but it requires a configuration option setting in Socket.IO on setup. That is one of the things on my backlog to allow in uibuilder.

I'll also add this limitation to the docs.

[fangfang94]

the websocket is fine, may I know how do I split the payload. (
Sorry for asking such a stupid question)

[TotallyInformation]

Not at all a stupid question - a perfectly reasonable one.

[fangfang94]

I set the socketId like this.
so I should add this

  maxHttpBufferSize: 1e8
});

inside to my code right?

[TotallyInformation]

When sending from the client, you don't need to worry about the socket id, uibuilderfe will add that for you

[fangfang94]

Understood, so I can remove

uibuilder.send({
  "topic": "get_socket_id"
});

from my code

just stay with

uibuilder.start();
uibuilder.onChange('msgsReceived', function (newVal) {
  if (msg.topic === "get_socket_id") {
    app_socket = msg._socketId;
  }
});

to receive the socketid?

[fangfang94]

Just notice that

CAUTION: request is not finished yet!

[TotallyInformation]

That is normal. What happens with websockets is that an http connection is made initially and once the handshake as completed, the connection is "upgraded" from http to ws. In the network list, you will then see a connection with a status of 101 which is the open connection. This will then hopefully stay open as long as that tab is connected.

If you click on that line, you should be able to see the ongoing comms between the client and the server.

[fangfang94]

Alright, so what I need to do right now is split the data before send to UI builder, and after process the data in node red I receive it back to append the table.

[UnborN]

@fangfang94

split the data before send to UI builder

this sounds like a tedious process

you could, as a test, add maxHttpBufferSize: 1e8 in socket server options
change .node-red\node_modules\node-red-contrib-uibuilder\nodes\libs\socket.js

Lines 130

let ioOptions = {
            'path': uib_socketPath,
            "maxHttpBufferSize": 1e8

save, restart NR and test it so @TotallyInformation can have some feedback whether its a feature worth adding in uibuilder.

[TotallyInformation]

@UnborN

this sounds like a tedious process

You do have a computer to hand - so it won't be tedious, that's what computers are for!

@UnborN

feedback whether its a feature worth adding in uibuilder

As mentioned, already was on the backlog to be able to change socket.io config. Not all that high up the backlog admitedly but now bumped up some, for sure. It will be up to people to change the config themselves though as, for most people, they wont need it.

If I can. I'll try to squeeze this into the v5 release currently being worked on.

[UnborN]

@TotallyInformation

so it won't be tedious, that's what computers are for!

hehe .. i know .. i was thinking about the small complexity of unnecessary code, to have to slice the array of data and send it piece by piece and then in the front end do the reverse to append it to whatever table its gonna be presented on. when it could be as simple as increasing the bufferSize (hard coded)

ps. i read you other post of the new uibuilder changes. excellent work!!

[fangfang94]

after add maxHttpBufferSize and it works like a charm

but I will also learn how to split to prepare for future

anyways thanks @UnborN and @TotallyInformation

you guys really kind and helpful.

thousand thanks!

[UnborN]

nice .. glad it worked for you because i tried to test it also by sending a 7mb string as a payload but i couldnt replicate the error.
anyway .. keep in mind that when you update uibuilder node to a newer version you'll lose that change

[UnborN]

@fangfang94

uibuilder.onChange('msgsReceived', function (newVal) {
  if (msg.topic === "get_socket_id") {
    app_socket = msg._socketId;
  }
});

i later noticed that in most of your code screenshots you use the onChange msgsReceived event that passes in newVal. Based on the documenation newVal here will be a counter / integer ?

In order to get the actual msg from NR, shouldn't that be :

uibuilder.onChange('msg', function(msg){
    console.info('msg received from Node-RED server:', msg)
})