# Utf8 decoding from serial buffer

**URL:** <https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884>\
**Category:** General\
**Created:** [15 March 2022 12:55 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884 "2022-03-15T12:55:48Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![leifnel](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/leifnel/32/3053_2.png) [@leifnel](https://discourse.nodered.org/u/leifnel)\
**Post date:** [15 March 2022 12:55 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884/1 "2022-03-15T12:55:48Z")

</div>

I'm getting data from a serial port., including danish characters æøå = c6 f8 c5  
I need to convert this to c3 a6 c3 b8 c3 a5  
If i send the data through the iconv node, set the encoding to utf8, I just get \<?\> characters

I think the iconv is trying to convert the other way

* * *

Someone with the physical hardware is reading the data with the serial node and sending it over mqtt to me.  
If he selects string, I receive each æøå as \<?\>  
If he selects buffer I get "Test æ ø å" as  
`[84,101,115,116,32,198,32,248,32,197]`  
I want to convert it to  
`[84,101,115,116,32,195,166,195,184,195,165]`  
If I put it thru iconv(utf8) I get  
`[84,101,115,116,32,239,191,189,32,239,191,189,32,239,191,189]`  
239,191,189 = \<?\>

To see what actual bytes I'm getting, I use

```auto
const buf = Buffer.from(msg.payload)
msg.payload=buf
return msg; 

```

But even without converting to buffer, I still get the \<?\>

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [15 March 2022 13:20 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884/2 "2022-03-15T13:20:45Z")

</div>

can you provide 2 samples...

1. Some data from an actual payload using `copy value` (see instructions below)
2. The equivalent value (what you expect it to be)

I ask because it is not clear if you have the serial node set to return buffer or string.

* * *

### How to get payload value (in a format that can be used by someone else)

There’s a great page in the docs ([Working with messages : Node-RED](https://nodered.org/docs/user-guide/messages)) that will explain how to use the debug panel to find the right path to any data item.

Pay particular attention to the part about the buttons that appear under your mouse pointer when you over hover a debug message property in the sidebar.

![BX00Cy7yHi](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/b/d/bd1f333a9e043b061b39917c328b0094d3e52d30.gif)

### Posting code or data

In order to make code readable and usable it is necessary to surround your code with three backticks (also known as a left quote or backquote `````)

````auto
``` 
   code goes here 
```

````

You can edit and correct your post by clicking the pencil ✏ icon.

See this post for more details - [How to share code or flow json](https://discourse.nodered.org/t/how-to-share-code-or-flow-json/506)

---

<div class="post-metadata">

**Author:** ![leifnel](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/leifnel/32/3053_2.png) [@leifnel](https://discourse.nodered.org/u/leifnel)\
**Post date:** [15 March 2022 14:06 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884/4 "2022-03-15T14:06:20Z")

</div>

If I in shell do iconv -f latin1 I get the desired result, but if I select latin1 in the iconv node, I get

`[84,101,115,116,32,63,32,63,32,63] = Test ? ? ?`

Getting desperate now...  
Saving message to file. exec'ing cat infile|iconv -f latin1 |tee outfile  
I now get `Test ý ý ý`

---

<div class="post-metadata">

**Author:** ![Steve-Mcl](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/steve-mcl/32/4826_2.png) [@Steve-Mcl](https://discourse.nodered.org/u/Steve-Mcl)\
**Post date:** [15 March 2022 15:53 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884/5 "2022-03-15T15:53:42Z")

</div>

Sorry, I am not following. first you say it is from a serial port, next you say its via MQTT. It still isnt 100% clear if you are getting a string or buffer.

So, I'll make some assumptions.

Going back a step to your original question...

> [@leifnel](#):
>
> I'm getting data from a serial port., including danish characters æøå = c6 f8 c5  
> I need to convert this to c3 a6 c3 b8 c3 a5

You can achieve this with function node.

```auto
Buffer.from("æøå", "utf-8")

```

Proof...  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/3/9/39e0949716435444e4bbdce59f51bb3dd6c67c88.png)

However, if you have the MQTT node set to return a string, there will be generic `toString` operation against the incoming MQTT packet.

You would be better switching the MQTT node to return a buffer - the value may already a buffer of the values you expect.

---

<div class="post-metadata">

**Author:** ![leifnel](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/leifnel/32/3053_2.png) [@leifnel](https://discourse.nodered.org/u/leifnel)\
**Post date:** [15 March 2022 16:24 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884/6 "2022-03-15T16:24:09Z")

</div>

Right now while I'm testing, I'm reading from a file, which contains æøå as c6 f8 bd, I read it binary to simulate what I'll get from the serial port set to buffer (and sending over mqtt)

I tried

```auto
const buf = Buffer.from(msg.payload,'utf-8')
msg.payload=buf//.toString();
return msg;

```

I still get c6 f8 c5 in the buffer, and if I use toString, I get `<?><?><?>`

## I got it!

````auto
const buf = Buffer.from(msg.payload)
msg.payload=buf.toString('latin1');
return msg;```
````

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/1X/d073cd938eafa2e558d7c2cd59003b3ef4963033.png) [@system](https://discourse.nodered.org/u/system)\
**Post date:** [14 May 2022 16:24 UTC](https://discourse.nodered.org/t/utf8-decoding-from-serial-buffer/59884/7 "2022-05-14T16:24:23Z")

</div>

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.