Getting unexpected "replacement" UTF-8 character in the Serial steam

Hardware
1

Trying to use a serial port with node-red, but I'm running into an unexpected issue.
Data stream includes a custom characters which are normal ASCII with MSB set, for example "E" & 0x80
Issue is that they are among normal Ascii string, for example "Time now is 8PM", with "8" having the MSB set and the rest being normal. Devices getting the data know to blink the "8" due to MSB, but that is besides the point.

Issue: is that instead of expected "binary buffers" I get "ef bf bd" sequence for each of the unexpected characters. Interestingly purely binary data, such as checksum, is unaffected. Searching I've learned that the sequence represents replacement character (link) which is used when ever parser can't display the chaster itself. As en example from before, data stream would be "Time now is " & ef bf bd & "PM".

I've verified that the issue is with the data coming out of serial node, not debug display itself.

I'm using serial port (link) library / node

Did anyone else run into a similar situation?

2

I wish to add that the issue is not showing up while using "screen" to read serial data. Characters with MSB set are still not displayed correctly (as they should not be), but at least they are one character instead of three.

3

Are you sure that is what you are getting from the serial in? Can you put the output of the serial node into a debug node, expand it, and show us what it shows? Also show us how you have configured the serial node.

[Edit] Or are you actually talking about Serial Out? In which case show us the debug output of what is going into the serial node.

4

Found the issue within my code which was dealing with the buffer as a string during an operation. Rewriting the code in question to keep data as a buffer has solved the issue.

Thank you @Colin for your time.