JSONata $base64decode does not support UTF8 strings

General
,
1

Hi There,

Taking this string Teiüäösß utf8 fïléñámê.m4a, encode as base64[1]: echo "Teiüäösß utf8 fïléñámê.m4a" | base64 and get this string VGVpdcyIYcyIb8yIc8OfIHV0ZjggZmnMiGxlzIFuzINhzIFtZcyCLm00YQo=

If I take the b64 string and push it through the JSONata $base64decode(..) function, I get Teiüäösß utf8 fïléñámê.m4a.

Demonstrated by this flow:

[{"id":"9a543998ea22e0eb","type":"inject","z":"6378c786ada4ddc2","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"VGVpdcyIYcyIb8yIc8OfIHV0ZjggZmnMiGxlzIFuzINhzIFtZcyCLm00YQ==","payloadType":"str","x":500,"y":430,"wires":[["a56ae1312fde1caf"]]},{"id":"a56ae1312fde1caf","type":"change","z":"6378c786ada4ddc2","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"$base64decode($$.payload)","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":728,"y":477,"wires":[["f98636d0205a090e"]]},{"id":"f98636d0205a090e","type":"debug","z":"6378c786ada4ddc2","name":"debug 1","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1016,"y":489,"wires":[]}]

Tested with NR 4.0.9

Work around is to use the JS code NodeJS code[2] in a function node instead of using $base64decode in a change node.

[1]: can also be done using the following JS code (gives the same b64 string):

    const blobToBase64 = (blob) => {
        return new Promise((resolve, reject) => {
            const reader = new FileReader();

            reader.onloadend = () => {
                const base64String = reader.result.split(',')[1]; // Remove the prefix
                resolve(base64String);
            };

            reader.onerror = reject;
            reader.readAsDataURL(blob); // Read the Blob as a data URL
        });
    };

    blobToBase64(new Blob(["Teiüäösß utf8 fïléñámê.m4a"], {type: 'text/plain'})).then( base64 => console.log(base64))

[2]: NodeJS function code

msg.payload = Buffer.from(msg.payload, "base64").toString("utf-8").trim()
return msg
2

Or [3] you could use node-red-node-base64 (node) - Node-RED.

3

Appears to be a bug in jsonata - not something we fix in Node-RED itself.

image
image2162×658 45.6 KB

4

To avoid future confusion, one could remove the two functions from NR's JSONata support, i.e., that $base64encode and $base64decode cannot be accessed from within NR.

Technically speaking then there is something that NR could do but probably shouldn't do because of backward compatibility and also it does not seem to have bothered anyone until now anyway.

So this becomes more of an FYI for future "end" users.