Extra Characters after base64 Decoding

Hi

I am reading accelerometer data from chip and sending it to the server and my node red is subscribed to the topic that reads the accelerometer data.
In Node Red, I am using function node with the following code:

var NewBuffer = Buffer.from(msg.payload, 'base64');
var decoded_msg = NewBuffer.toString('ascii');

//To remove u+Z from the start of output string
decoded_msg = decoded_msg.substr(3,msg.payload.length);
msg.payload = decoded_msg;
return msg;

I can see the output string decoded into ASCII but sometimes when I gave sudden movements to accelerometer chip, my output data starts collecting miscellaneous characters in the end.
I could not find the reason why it is collecting the random data at the end of string.
I would really appreciate any help.

(upload://aKgfA3FPVlWFD3fW3DOJb9hVD4f.jpeg)

Are they there in the buffer coming in? If not which conversion is adding them?
Why are you specifying msg.payload.length in the substr call?

Hi Colin
Thanks for your reply.

Initially, I am receiving base64 encoded data. Then I have applied above function to decode the base64 to ASCII. This is the only function that I am using in my Node Red that has input from the server that it is subscribed to and output of a debug node.
I figured out during my manual testing of sending string with negative number to node red, it generates random data in the end of string. If the data is all positive numbers then there are no extra characters in the output data.
So does it mean that base64 is not decoding negative numbers correctly?
For the substr function, I was passing the arguments of start and end index of the string. I have now removed the msg.payload.length and it works without mentioning the end of the string length. Thanks for correcting me.

I meant you to check whether the extra characters are on the end of the base64 data. And if not whether they are on the end of the Buffer after conversion to a Buffer. You can do this by checking the length I suppose. You can send diagnostic data to the debug window using node.warn() so for example at the start you could put

node.warn(`payload length is ${msg.payload.length}`)

and include similar code after the buffer conversion.
With substr the second parameter (if supplied) is not the end index but the number of characters to return, but if you don't provide it then it retuns the rest of the string.
Your image in the first post has not appeared (for me anyway). You can include an image by pasting it here directly.

Yes, The extra characters are getting added after the buffer conversion to ASCII. Initial string does not have those characters. I checked the string length of both using the node.warn function.

Node%20Red

Hi Colin,

The issue has been resolved.
I was using msg.payload for decoding but instead I only required the message data not the whole message object. So I replaced Buffer.from(msg.payload, 'base64') in my code to Buffer.from(JSON.parse(msg.payload).data, 'base64').
It is working now without giving any random data.

Thank you for your help.

1 Like