Utf8 decoding from serial buffer

General
1

I'm getting data from a serial port., including danish characters æøå = c6 f8 c5
I need to convert this to c3 a6 c3 b8 c3 a5
If i send the data through the iconv node, set the encoding to utf8, I just get <?> characters

I think the iconv is trying to convert the other way

Someone with the physical hardware is reading the data with the serial node and sending it over mqtt to me.
If he selects string, I receive each æøå as <?>
If he selects buffer I get "Test æ ø å" as
[84,101,115,116,32,198,32,248,32,197]
I want to convert it to
[84,101,115,116,32,195,166,195,184,195,165]
If I put it thru iconv(utf8) I get
[84,101,115,116,32,239,191,189,32,239,191,189,32,239,191,189]
239,191,189 = <?>

To see what actual bytes I'm getting, I use

const buf = Buffer.from(msg.payload)
msg.payload=buf
return msg;

But even without converting to buffer, I still get the <?>

2

can you provide 2 samples...

  1. Some data from an actual payload using copy value (see instructions below)
  2. The equivalent value (what you expect it to be)

I ask because it is not clear if you have the serial node set to return buffer or string.

How to get payload value (in a format that can be used by someone else)

There’s a great page in the docs (Working with messages : Node-RED) that will explain how to use the debug panel to find the right path to any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

BX00Cy7yHi

Posting code or data

In order to make code readable and usable it is necessary to surround your code with three backticks (also known as a left quote or backquote ```)

``` 
   code goes here 
```

You can edit and correct your post by clicking the pencil :pencil2: icon.

See this post for more details - How to share code or flow json

4

If I in shell do iconv -f latin1 I get the desired result, but if I select latin1 in the iconv node, I get

[84,101,115,116,32,63,32,63,32,63] = Test ? ? ?

Getting desperate now...
Saving message to file. exec'ing cat infile|iconv -f latin1 |tee outfile
I now get Test ý ý ý

5

Sorry, I am not following. first you say it is from a serial port, next you say its via MQTT. It still isnt 100% clear if you are getting a string or buffer.

So, I'll make some assumptions.

Going back a step to your original question...

You can achieve this with function node.

Buffer.from("æøå", "utf-8")

Proof...
image

However, if you have the MQTT node set to return a string, there will be generic toString operation against the incoming MQTT packet.

You would be better switching the MQTT node to return a buffer - the value may already a buffer of the values you expect.

6

Right now while I'm testing, I'm reading from a file, which contains æøå as c6 f8 bd, I read it binary to simulate what I'll get from the serial port set to buffer (and sending over mqtt)

I tried

const buf = Buffer.from(msg.payload,'utf-8')
msg.payload=buf//.toString();
return msg;

I still get c6 f8 c5 in the buffer, and if I use toString, I get <?><?><?>

I got it!

const buf = Buffer.from(msg.payload)
msg.payload=buf.toString('latin1');
return msg;```
7

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.