Posting PDF-files multipart/form-data solution

General
1

As I did not find a good solution before, this is how you post a binary file to http-request. The solution is: make sure that payload always is a buffer and never a string. No need to do all kinds of binary or base64 conversions, just make sure always a buffer is sent and never a string.

I started with File Upload using HTTP Request node (flow) - Node-RED that works well for text-files. For a version that works with both text and binary data (and has some boundary-=optimizations), you need to replace the contents of Format the header and payload with:

const genRanHex = size => [...Array(size)].map(() => Math.floor(Math.random() * 16).toString(16)).join('');

let boundary = "------------------------"+genRanHex(16);

msg.headers = {
    "Content-Type": "multipart/form-data; boundary="+boundary
}

let data_pre = Buffer.from("--"+boundary+'\r\n'+
"Content-Disposition: form-data; name=\"file\"; filename=\""+msg.filename+"\"\r\n"+
"Content-Type: application/octet-stream\r\n"+
"\r\n", "utf-8");

let data_post = Buffer.from("\r\n"+
"--"+boundary+"--\r\n", "utf-8");

msg.payload = Buffer.concat([data_pre, msg.payload, data_post]);

return msg;

Tested with a posted file being redirected to a http-request (with this code) and a resulting file being sent back to the browser.

2

Hi Vincent -- thanks for taking the time to post your solution! I, too, have spent a good bit of time trying to piece together a multipart/form-data compliant payload that my Spring apis would accept. I'm not sure why there is no editor panel for the core http-request node, that would allow the user to include an array of files, buffers, and key/value pairs... or why I never offered to contribute that feature... hmmm.

Lately I've been referring to this answer from Nick for a bit simpler JS function example. It does require that your binary data be in a Buffer, but it avoids the user generated boundaries and interspersed CR/NL bits that are hard to find when they are not perfectly placed.

Welcome to the node-red community!

3

That is much cleaner! Thanks!

This is the same as the above, but then a change-node instead of a function-node.

[
    {
        "id": "6cfb8852c3a4e6b0",
        "type": "change",
        "z": "792580e9.3d65e8",
        "name": "",
        "rules": [
            {
                "t": "set",
                "p": "payload",
                "pt": "msg",
                "to": "{\t   \"file\":{\t       \"value\":payload,\t       \"options\":{\"filename\":filename}\t   }\t}",
                "tot": "jsonata"
            },
            {
                "t": "set",
                "p": "headers",
                "pt": "msg",
                "to": "{\"content-type\":\"multipart/form-data\"}",
                "tot": "json"
            }
        ],
        "action": "",
        "property": "",
        "from": "",
        "to": "",
        "reg": false,
        "x": 820,
        "y": 2060,
        "wires": [
            [
                "fb27d4abe3afbb2a",
                "fda054cb98806e70"
            ]
        ]
    }
]

Lots of NodeRed is quite messy and buggy, so it's difficult to find good answers.

4

The help sidebar for the HTTP Request node tells you how to do a file upload.

File Upload

To perform a file upload, msg.headers["content-type"] should be set to multipart/form-data and the msg.payload passed to the node must be an object with the following structure:

{
   "KEY": {
       "value": FILE_CONTENTS,
       "options": {
           "filename": "FILENAME"
       }
   }
}

The values of KEY , FILE_CONTENTS and FILENAME should be set to the appropriate values.

1 Like
5

For completeness, allow me to ask the follow on question in this thread...

What is the syntax to use for a multipart body that contains an array of form data, json content, and several files to be uploaded?

6

Ah, true! I did not scroll to the end. Just I assumed I had to fix things, again. Good your node is a good exception to messy NodeRed-land.

7

We try to ensure all the core nodes set a good example.

If you have feedback on other nodes, I encourage you to share it with the node author to help improve things.

8

Biggest problem is that many nodes are at WFM-level. Then you get even situations like the complete msg being thrown away, or nodes that work under very specific situations only. A standard reaction is to fork to fix things quickly, but node-red-controb-xxx-3 gets abandoned too.

I'm working on multiple ones needed for interaction with gitlab and mattermost APIs.

Is there a design-doc for plugins? Are there ways to mark projects as abandoned?

9

Put the name of the file in "KEY". If one, then this is often defaulted to "file". Example of more files:

{
   "file1":{
       "value":payload.file[0].contents,
       "options":{"filename":payload.file[0].filename}
   },
   "file2":{
       "value":payload.file[1].contents,
       "options":{"filename":payload.file[1].filename}
   }
}
10

Again we would encourage you to point them out more publicly (in a polite manner) rather than jump straight to forking so we can possibly "help" the author try to improve their nodes. The latest packaging details are here - Packaging : Node-RED - and the recent work on scoring is also looking at ways to take note of nodes that seem to be abandoned.

11

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.