Input file in uibuilder V7

[papka1974]

Hello.

I'm pretty new in Node-Red and uibuilder and i'm totally not a web guy. Learning by doing. I created a form via uib-element node. One of inputs is a file upload. When I'm submitting form I see message coming about file upload separately to _ui message with form fields content. After form submitting, I would like to import uploaded file (xlsx) into database (I need buffer only), including also other fields values from the form.
I didn't find any good way to kind of merge messages to process further. However, in _ui message under form->appropriate object->value, I found tempUrl key with value "blob:http://{ip of my server}:1880/ee205300-e317-4abc-8d4f-6b6e7381b21d". if I use this url in browser - i'm able to download uploaded file. Maybe I could get buffer from this url?
Or maybe I'm doing completely wrong?
Sorry for probably pretty stupid questions.

[TotallyInformation]

@papka1974

Sorry for probably pretty stupid questions.

No such thing Generally only stupid answers!

I can see, from a quick browse of the docs, that I have really failed to properly document file uploads.

There was something here in a release note.

When using a form on a page and using <input type="file">, if a file is selected by the client and the file is less than the size of the maximum message size, the file will be automatically uploaded to Node-RED when the form is submitted (assuming you use uibuilder.eventSend(event) to submit the form). The upload is a message with file details and the file itself as a buffer in msg.payload.

If a file input is sent back to Node-RED, you will get a 2nd message that contains the file:

Where the file content is in msg.payload as a buffer.

As you have noted, the blob URI works in the browser but does not work for an http-request node. So it could be passed to another client browser but if you want Node-RED to have the file, you need to process the 2nd message with the msg.topic "file-upload".

There is a full example demonstrating the uib-element node in the examples library under uibuilder. It includes a reasonably comprehensive form example with file-upload.

If you need to process a file upload outside of uib-element's construction, you have access both to the eventSend function that you can attach to an input tag or a form tag which is what uib-element uses. Or you can use the [uibuilder.uploadFile()](UIBUILDER Documentation v7) function which is what eventSend uses internally.

Please note the limitations on file sizes though. If you need to upload a file larger than 1MB, you will need to change some settings to allow larger message sizes. Details in that last link.

---

Incidentally, if you wanted to allow a user to upload a file and then have it served to all users, you could pass the upload message to a uib-save node.

[papka1974]

Thank you for quick reply. Most of the things I discovered myself already, as I wrote.

@TotallyInformation

you need to process the 2nd message with the msg.topic "file-upload".

This is where I stuck fund file buffer in a separate message. I thought to join messages, but it looks a bit ugly just to wait for another one message or by seconds... What if someone else will submit a form and second message will not be file upload? Especially that there going to be multiple forms and some of them will not have file upload. Of course I would look into form ID, which I suppose I could provide as HTML ID in uib_element node, but _ui message under form->id providing self generated "form X", which saying me nothing. Another thing I found is that file uploaded even if form is not valid. This a bit surprised me as well. I most probably could control submit button enable/disable by watching changes on the form, but as I wrote, I'm far with knowledge how to yet.

@TotallyInformation

There is a full example demonstrating the uib-element node in the examples library under uibuilder. It includes a reasonably comprehensive form example with file-upload.

Will try to find. Didn't found before. Might be would not asking.

@TotallyInformation

If you need to process a file upload outside of uib-element's construction, you have access both to the eventSend function that you can attach to an input tag or a form tag which is what uib-element uses.

You mean to do a post to another url and catch by http in?

@TotallyInformation

Or you can use the [uibuilder.uploadFile()]( UIBUILDER Documentation v7) function which is what eventSend uses internally.

Read documentation but not really understood how to use.

@TotallyInformation

Incidentally, if you wanted to allow a user to upload a file and then have it served to all users, you could pass the upload message to a uib-save node.

No. Files used only to extract information and insert into database. All users fetching information from there.

[TotallyInformation]

@papka1974

What if someone else will submit a form and second message will not be file upload?

Each will result in its own file-upload message and you can tell they are different and who they are from by enabling the advanced _uib extension:

@papka1974

Especially that there going to be multiple forms and some of them will not have file upload.

Hmm, so I've spotted a slight weakness in the upload message - it should include the input tag's HTML ID and name attributes. You get these in the form upload message but not in the file-upload. Added to the backlog.

You do get the filename in both messages so it is possible to tie them together if you need to, but not very friendly of me. Sorry about that.

I do have another fix to publish so this could go into the same release. I'm hoping to get some time later this week.

@papka1974

Another thing I found is that file uploaded even if form is not valid. This a bit surprised me as well.

Yes, that's a decision I took because people might want to do more validation and messaging from Node-RED and I didn't want to totally block that. The form message contains a flag to indicate if the form is valid and flags for each form element to show their validity.

The other reason for not auto-blocking is that the same process handles individual inputs - even without a wrapping form. You might choose to do everything in Node-RED and indeed, you might find that valuable if you lack front-end coding knowledge.

@papka1974

most probably could control submit button enable/disable by watching changes on the form, but as I wrote, I'm far with knowledge how to yet.

Just add a switch node on the uibuilder node's output if you want to block invalid forms.

@papka1974

Will try to find. Didn't found before

[papka1974]

Hi,

@TotallyInformation

Hmm, so I've spotted a slight weakness in the upload message - it should include the input tag's HTML ID and name attributes. You get these in the form upload message but not in the file-upload.

I wrote before that HTML_ID of the form not appearing in form submitting message also (V7.1).
This is what I would expect under _ui->form.

@papka1974

Another thing I found is that file uploaded even if form is not valid. This a bit surprised me as well.

I understand, different cases - different needs. This is a small problem.

@TotallyInformation

Just add a switch node on the uibuilder node's output if you want to block invalid forms.

Exactly how i'm doing now.

I found form example. Unfortunately I'm interested in such form processing part after submit. Exactly this is not really explained there.
Just an idea, maybe for such cases, when form submission happening and multiple messages to be generated, it's possible to add msg.parts for these connected messages? The it would be peanut to join them in one message.

[TotallyInformation]

@papka1974

possible to add msg.parts

Ah, interesting idea! I will look into that for sure.

[papka1974]

I suppose, by adding all parts

join of messages will be fully automated and as parts.id could be used form submission _msgid or maybe _socketId

[TotallyInformation]

Added to the backlog so I don't forget. I will need to check how feasible it is but I think I will be able to do something.

Assuming that my wife doesn't fill up my leave-time with house/family activities, I'm hoping to do some uibuilder work later this week.

[papka1974]

Great. Just remember, happy wife = happy life.
I just emulated msg.parts and did join in automated mode - works great.
Then I used join in manual mode in such config

where msg.parts.key was set in change node to either msg.topic (if exists) or simply to "form" (maybe form could also get topic in original message?).
Result was even better I think:

Even easier for further automation since each part of payload could be referred by name.
Just dropping you some ideas, maybe you will find something valued.

[TotallyInformation]

@papka1974

I would look into form ID, which I suppose I could provide as HTML ID in uib_element node, but _ui message under form->id providing self generated "form X", which saying me nothing.

Rechecked the code and the form's id attribute IS included in the data. However, it will auto-create an id if you haven't given one in the first place and the no-code uib-element form adds its ID to the parent div and not to the form itself. That means that the response will always have an id of form-x where x is a number, usually 1 unless you've added other forms first.

@papka1974

I wrote before that HTML_ID of the form not appearing in form submitting message also (V7.1).
This is what I would expect under _ui->form.

This will be fixed in the upcoming release. where the form id will be form-<elementId> since the element ID you set on uib-element should always be unique on the page.

@papka1974

Just an idea, maybe for such cases, when form submission happening and multiple messages to be generated, it's possible to add msg.parts for these connected messages? The it would be peanut to join them in one message.

This turns out to be simultaneously more complex and less useful than I first thought.

The issue is that, when processing a form, the system does not "know" how many uploads will be generated. It would have to process the whole form and hold everything in memory before sending any messages. This would be quite inefficient and might easily break things if uploading a load of really large files.

To further complicate things, a file upload input can allow multiple files to be selected. So you could have multiple inputs each with multiple files.

And, as a final complication, I have a backlog item to allow auto-splitting of messages for large messages and that would take priority for using the msg.parts feature

So instead, I am adding appropriate meta-data to each upload:

• A seq number and a seqCount for multi-file submissions.
• id is the HTML id of the source input element.
• formId is the HTML id of the form that the input element belongs to. The formId won't exist if the input was submitted without a form.
• The temporary file URL. This is probably the easiest property to use if you need to merge data with the main message.
• I've also now added the clientId, pageName and tabId to the message. This is duplicate data if you've turned on the _uib property in your uibuilder node but it should make it easier to filter/switch messages in your flows.
• Finally, I've added the data object which contains any data-* attributes from the input element.

So if you want to combine other form inputs with your file upload, Save the form message to a flow variable - referenced by the client ID (the socket ID changes too easily) to allow for multiple, overlapping, client submissions. Then, for each file upload, compare msg.tempUrl against msg.payload["input-id"][0].tempUrl (where the id and offset might be different of course).

If you want to make your life even easier and you are prepared to add some front-end logic, you could use change events on your inputs to add data attributes to your file input elements. I can show you how to do that if you like.

I'm also creating a new "how to" document explaining this stuff.

As a side-note, don't forget that, with uibuilder and the no-code nodes - what is created is standard HTML. This means that if your form doesn't change very much, you may want to consider using uib-element to do the initial page creation but then save the updated page back to the index.html file. By doing that, you can then go tweak your form and the page manually and you benefit of not needing to send the form to every newly connected or reloaded client - so the page load is faster and can be cached by the browser making it faster still. Check out the examples for the uib-save node to see how you can do this automatically without any coding.

@papka1974

join of messages will be fully automated

On further consideration, I don't think that joining the file upload messages would be sensible. All of the metadata for the files is in the form upload message and the file uploads contain large buffers which it would not make sense to merge. If you really have an overriding reason to do so and understand the potential consequences, then - with the new metadata - you should be able to add the msg.parts data yourself.

---

I've a few additional tests to run, some documentation to update and a few other things to finish off. But you can expect a new version of uibuilder - v7.2.0 - very soon with these and some other fixes and new features (such as the new uibuilder sidebar UI!).

[papka1974]

Hi,

Thank you for detailed reply. I clearly see complications. I suppose new version will solve my problems. Currently I'm creating msg.parts myself, but it looks ugly.
About joining file upload message and form submission message, it's needed because other form inputs used when importing file. Now thinking maybe I can simply enrich file upload message with form submission payload information....

@TotallyInformation

So if you want to combine other form inputs with your file upload, Save the form message to a flow variable - referenced by the client ID (the socket ID changes too easily) to allow for multiple, overlapping, client submissions. Then, for each file upload, compare msg.tempUrl against msg.payload["input-id"][0].tempUrl (where the id and offset might be different of course).

I need to read more about this. Not used flow variables before.

@TotallyInformation

If you want to make your life even easier and you are prepared to add some front-end logic, you could use change events on your inputs to add data attributes to your file input elements. I can show you how to do that if you like.

This would be great.

[papka1974]

Tested new version and it's definitely easier now to handle file uploads. form.id helping to a lot. Will try to dig into sidepanel. I implemented it myself via css before... Big thanx to you for the effort.
One thing seems stopped working for me is notifications. Using same approach as before, setting all parameters and no message on client side. Already checked everything....

[TotallyInformation]

@papka1974

One thing seems stopped working for me is notifications. Using same approach as before, setting all parameters and no message on client side. Already checked everything....

Apologies, a bit distracted right now as my daughter not well. Could I ask you for a simple example? I will then try to look at it over the weekend.

[papka1974]

Hope your daughter will be well soon!
Here is a message I'm sending to uibuilder node to notify client

And no message.
Tried without _socketId. Tried with full _ui. Nothing. A bit lost what to try else.

[papka1974]

I will try to investigate myself today. Most probably my not really straight hands fault. Question - if I do page reload, _socketId should not be changed. Am I right?
Also question - is it possible to have kind of "splash notification" while message processed and result not known? Kind of "Please wait" on blurred screen, but without option to click a mouse and it disappears?

[TotallyInformation]

@papka1974

Question - if I do page reload, _socketId should not be changed. Am I right?

No, it DOES change. Which is why I added a clientId which will be stable for the browser until it is completely restarted.

@papka1974

Also question - is it possible to have kind of "splash notification" while message processed and result not known? Kind of "Please wait" on blurred screen, but without option to click a mouse and it disappears?

A timed notification. I don't often use notifications though but the notify command should time out by default.

UIBUILDER Documentation v7

This msg content creates a popover that goes away automatically after 10 seconds:

    "_ui": [
        {
            "method": "notify",
            "variant": "info",
            "title": "A Manual Title",
            "content": "A message from low-code",
            "payload": "This is a message from the payload",
            "topic": "show-popup",
            "autoHideDelay": 10000,
            "autohide": true
        }
    ],

Here is a simple flow:

[{"id":"4349922788703674","type":"link out","z":"778eb3505a1c57cf","name":"link out 73","mode":"link","links":["0711e37d3a8911b0"],"x":955,"y":440,"wires":[]},{"id":"bf862bbd0cfe2ff0","type":"inject","z":"778eb3505a1c57cf","name":"Show an on-page notification (method: notify)","props":[{"p":"_ui","v":"{\"method\":\"notify\",\"variant\":\"info\",\"title\":\"A Manual Title\",\"content\":\"A message from low-code\"}","vt":"json"},{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"show-popup","payload":"This is a message from the payload","payloadType":"str","x":310,"y":440,"wires":[["4349922788703674"]],"info":"Note that this is different to the `notification` uib command,\r\nthat shows an OS toast notification, this shows an on-page\r\npop-over.\r\n\r\nThe difference between this and the \"alert\" method is that\r\nthis does autohides by default and it has no warning icon.\r\n\r\nOtherwise, it is the same and both use the `showDialog`\r\nmethod under the skin."},{"id":"415adea0247fbd47","type":"inject","z":"778eb3505a1c57cf","name":"Show an on-page notification (method: alert)","props":[{"p":"_ui","v":"{\"method\":\"alert\",\"variant\":\"info\",\"title\":\"A Manual Title\",\"content\":\"A message from low-code\"}","vt":"json"},{"p":"topic","vt":"str"},{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"show-popup","payload":"This is a message from the payload","payloadType":"str","x":310,"y":480,"wires":[["4349922788703674"]],"info":"Note that this is different to the `notification` uib command,\r\nthat shows an OS toast notification, this shows an on-page\r\npop-over.\r\n\r\nThe difference between this and the \"notify\" method is that\r\nthis does not autohide and it has a warning icon.\r\n\r\nOtherwise, it is the same and both use the `showDialog`\r\nmethod under the skin."},{"id":"1259b61fd05d7e1f","type":"group","z":"778eb3505a1c57cf","name":"Base node. Change ALL URL's to match.","style":{"label":true,"fill":"#ffffbf","fill-opacity":"0.28","color":"#3f3f3f"},"nodes":["5e8cef7a75c274e9","f6768800e8264e5b","3c3912ba1979a325","8e45322c03ef2242","8cb6282875850f48","05c8d0d1e7edc284","0711e37d3a8911b0","397cdb56724e22b1","70b3cb25d897b7f3","e3208e97a697d27f","609e62d7e38c2c5c","c55028d432f00c22","bc4ada305fa307f9","c012c86ba02b3767","569fa7fdb720f802","f9cbec6afedd2d2e","d11899a22de0067f"],"x":114,"y":79,"w":882,"h":262},{"id":"5e8cef7a75c274e9","type":"uibuilder","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"","topic":"","url":"low-code-tests","okToGo":true,"fwdInMessages":false,"allowScripts":false,"allowStyles":false,"copyIndex":true,"templateFolder":"blank","extTemplate":"","showfolder":false,"reload":true,"sourceFolder":"src","deployedVersion":"6.8.0","showMsgUib":true,"title":"Zero-code examples","descr":"A collection of flows that demonstrate and test all of the uib-element node's output types.","editurl":"vscode://file/src/uibRoot/low-code-tests/?windowId=_blank","x":620,"y":180,"wires":[["f6768800e8264e5b"],["c012c86ba02b3767"]],"info":"This example uses the default blank template.\r\n\r\nIt does not use any front-end framework, just\r\npure HTML, CSS and JavaScript."},{"id":"f6768800e8264e5b","type":"debug","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"uibuilder standard output","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"","statusType":"counter","x":755,"y":160,"wires":[],"l":false,"info":"This shows the data coming out of the\r\nuibuilder node's Port #1 (top) which is\r\nthe standard output.\r\n\r\nHere you will see any standard msg sent from\r\nyour front-end code."},{"id":"3c3912ba1979a325","type":"debug","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"uibuilder control output","active":true,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"","statusType":"counter","x":905,"y":200,"wires":[],"l":false,"info":"This shows the data coming out of the\r\nuibuilder node's Port #2 (bottom) which is\r\nthe control output.\r\n\r\nHere you will see any control msg either sent\r\nby the node itself or from the front-end library.\r\n\r\nFor example the \"client disconnect\" and\r\n\"client connect\" messages. Or the \"visibility\"\r\nmessages from the client.\r\n\r\nLoop the \"client connect\", \"cache replay\" and\r\n\"cache clear\" messages back to a `uib-cache`\r\nnode before the input to uibuilder in order\r\nto control the output of the cache."},{"id":"8e45322c03ef2242","type":"inject","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Reload","props":[{"p":"_ui","v":"{\"method\":\"reload\"}","vt":"json"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"reload","x":210,"y":180,"wires":[["05c8d0d1e7edc284"]],"info":"Sends a pre-formatted msg to the front-end that\r\ncauses the page to reload itself."},{"id":"8cb6282875850f48","type":"comment","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Chk Description in each node","info":"","x":560,"y":140,"wires":[]},{"id":"05c8d0d1e7edc284","type":"junction","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","x":420,"y":180,"wires":[["5e8cef7a75c274e9"]]},{"id":"0711e37d3a8911b0","type":"link in","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"link in to low-code-tests","links":["4349922788703674","bc4ada305fa307f9"],"x":285,"y":140,"wires":[["5e8cef7a75c274e9","f9cbec6afedd2d2e"]]},{"id":"397cdb56724e22b1","type":"inject","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Toggle Visible Msgs","props":[{"p":"_uib","v":"{\"command\":\"showMsg\", \"pageName\": \"index.html\"}","vt":"json"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":250,"y":220,"wires":[["05c8d0d1e7edc284"]]},{"id":"70b3cb25d897b7f3","type":"inject","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Log Lvl 5","props":[{"p":"_uib","v":"{\"command\":\"set\",\"prop\":\"logLevel\",\"value\":5}","vt":"json"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":220,"y":260,"wires":[["d11899a22de0067f"]]},{"id":"e3208e97a697d27f","type":"inject","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Log Lvl 0","props":[{"p":"_uib","v":"{\"command\":\"set\",\"prop\":\"logLevel\",\"value\":0}","vt":"json"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","x":220,"y":300,"wires":[["d11899a22de0067f"]]},{"id":"609e62d7e38c2c5c","type":"switch","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Connect Only","property":"uibuilderCtrl","propertyType":"msg","rules":[{"t":"eq","v":"client connect","vt":"str"}],"checkall":"false","repair":false,"outputs":1,"x":805,"y":260,"wires":[["569fa7fdb720f802"]],"l":false},{"id":"c55028d432f00c22","type":"uib-element","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Set title/heading on load","topic":"","elementtype":"title","parent":"body","parentSource":"","parentSourceType":"str","elementid":"myh1","elementId":"","elementIdSourceType":"str","heading":"","headingSourceType":"str","headingLevel":"h2","position":"first","positionSourceType":"str","confData":{},"x":905,"y":260,"wires":[["bc4ada305fa307f9"]],"l":false},{"id":"bc4ada305fa307f9","type":"link out","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Set title/heading on page load","mode":"link","links":["0711e37d3a8911b0"],"x":955,"y":260,"wires":[]},{"id":"c012c86ba02b3767","type":"junction","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","x":720,"y":200,"wires":[["3c3912ba1979a325","609e62d7e38c2c5c"]]},{"id":"569fa7fdb720f802","type":"change","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","name":"Set title text","rules":[{"t":"set","p":"payload","pt":"msg","to":"uib-element tests","tot":"str"},{"t":"delete","p":"uibuilderCtrl","pt":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":855,"y":260,"wires":[["c55028d432f00c22"]],"l":false},{"id":"f9cbec6afedd2d2e","type":"debug","z":"778eb3505a1c57cf","d":true,"g":"1259b61fd05d7e1f","name":"uibuilder standard output","active":false,"tosidebar":true,"console":false,"tostatus":true,"complete":"true","targetType":"full","statusVal":"","statusType":"counter","x":395,"y":120,"wires":[],"l":false,"info":"This shows the data coming out of the\r\nuibuilder node's Port #1 (top) which is\r\nthe standard output.\r\n\r\nHere you will see any standard msg sent from\r\nyour front-end code."},{"id":"d11899a22de0067f","type":"junction","z":"778eb3505a1c57cf","g":"1259b61fd05d7e1f","x":420,"y":260,"wires":[["5e8cef7a75c274e9"]]}]

It also automatically blurs the background. You don't actually need to set autohide as that is the default. But you might want to set the autoHideDelay as the default is 10 seconds.

[papka1974]

@TotallyInformation

Which is why I added a clientId which will be stable for the browser until it is completely restarted.

Yeah... Then I know why messages not working. They are sent based on _socketId and I'm doing reload to get blank form after flow processed. Since I'm doing reload then trying to show message....

@TotallyInformation

This msg content creates a popover that goes away automatically after 10 seconds

This I know. But it's 10 seconds or mouse click. This is what I don't want, that message disappear before.... Will think about.

[TotallyInformation]

@papka1974

This I know. But it's 10 seconds or mouse click. This is what I don't want, that message disappear before.... Will think about.

So, try this:

    "_ui": [
        {
            "method": "notify",
            "variant": "info",
            "title": "A Manual Title",
            "content": "A message from low-code",
            "payload": "This is a message from the payload",
            "topic": "show-popup",
            "autoHideDelay": 1000,
            "autohide": true
        }
    ],

1000ms = 1 second. Adjust according to need.

[papka1974]

This approach is working. Now messages working. Not really understood what was wrong, but it's fine now. Also removed page reload call. So notification by _socketId is fine as well. Thank you.