Special character encoding /Utf8 problem

General
1

Hi,

I need to use special characters (from portuguese, pt-br) like ç, ã and so forth, in a flow, specifically to write to a xml file. At the debug window on node-red, the characters looks all correct but when writing to disk, the special charcaters get messed.

The xml is being writtern as this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><video><title/><keywords>palavras,chave</keywords><description>descri��o</description></video>

the charset of the file is iso-8859-1 as returned by the "file" command

file -bi undefined.xml 
text/xml; charset=iso-8859-1

the flow is below (change your file path - msg.filename - at the "format-xml" node)

[{"id":"160cecbf.c27973","type":"inject","z":"4a0b25ab.85b84c","name":"","topic":"","payload":"joão","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":270,"y":1300,"wires":[["c942a09.f98966"]]},{"id":"c942a09.f98966","type":"function","z":"4a0b25ab.85b84c","name":"format xml","func":"var title = msg.payload ;\n\nmsg.payload = \n{ \n    \"video\": {\n            \"title\"     : title,\n            \"keywords\" :  \"palavras,chave\",\n            \"description\": \"descrição\"\n\n    }\n}\n              \nmsg.filename = \"YOUR_PATH\" + title + \".xml\" ;\n\nreturn msg;","outputs":1,"noerr":0,"x":450,"y":1300,"wires":[["181f599b.af4696"]]},{"id":"181f599b.af4696","type":"xml","z":"4a0b25ab.85b84c","name":"","property":"payload","attr":"","chr":"","x":630,"y":1300,"wires":[["4c14ec7f.c0e354"]]},{"id":"4c14ec7f.c0e354","type":"file","z":"4a0b25ab.85b84c","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","x":790,"y":1300,"wires":[["68469962.d91c58"]]},{"id":"68469962.d91c58","type":"debug","z":"4a0b25ab.85b84c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":970,"y":1300,"wires":[]}]

how can I have the special characters in the written file ?

Many thanks in advance

2

please read How to share code or flow json and the edit the code you posted

3

done @ukmoose, thanks for pointing that out

4

Taking your flow and using MacOS it outputs the special characters.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><video><title>joão</title><keywords>palavras,chave</keywords><description>descrição</description></video>

What are you using to read the file?

5

I tested your flow and got the special characters correctly spelled/printed/displayed in the file name as well as in the xml content (opened with notepad++). Also all looks good in Node-RED debug.

My setup is Node-REd running in Windows10.

r-01
r-01.PNG773×61 5.12 KB

r-02

1 Like
6

I wonder why your file has been generated with such forward slash at the end of the title tag.

<title/>

Or perhaps this issue was introduced when you copy and pasted the excerpt to this post thread ?

7

I'm guessing its because the flow that generated that didn't include a title hence undefined.xml

8

Hi @Andrei , the title tag is like that cause the title value was somehow broken due to the special character. My file was named undefined.xml ..
I forgot to mention node-red is writing to and installed in a debian 9 machine. The strange thing is that now I get the joão.xml written ok and as @ukmoose mentioned, the xmls looks messy depending on the way I read it. The one I pasted in this thread was a "cat" command from the linux terminal. If I open with gedit on a Ubuntu machine, It reads correctly.
The last issue now is that I'm appending that xml to a http form to send to a API. the api is reading it wrongly as well. Anything I could do or the problem is only on the API side ? Thanks you both for helping me out
cheers

9

I guess the better place to look now is on how you build the http request. I guess you are using an http request node in your flow, right ? Can you share this part of this config (removing any tokens / passwords) ?

10

That may be down to whether the application you are using to view it is using a font containing those characters.