Special character encoding /Utf8 problem


I need to use special characters (from portuguese, pt-br) like ç, ã and so forth, in a flow, specifically to write to a xml file. At the debug window on node-red, the characters looks all correct but when writing to disk, the special charcaters get messed.

The xml is being writtern as this:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><video><title/><keywords>palavras,chave</keywords><description>descri��o</description></video>

the charset of the file is iso-8859-1 as returned by the "file" command

file -bi undefined.xml 
text/xml; charset=iso-8859-1

the flow is below (change your file path - msg.filename - at the "format-xml" node)

[{"id":"160cecbf.c27973","type":"inject","z":"4a0b25ab.85b84c","name":"","topic":"","payload":"joão","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":270,"y":1300,"wires":[["c942a09.f98966"]]},{"id":"c942a09.f98966","type":"function","z":"4a0b25ab.85b84c","name":"format xml","func":"var title = msg.payload ;\n\nmsg.payload = \n{ \n    \"video\": {\n            \"title\"     : title,\n            \"keywords\" :  \"palavras,chave\",\n            \"description\": \"descrição\"\n\n    }\n}\n              \nmsg.filename = \"YOUR_PATH\" + title + \".xml\" ;\n\nreturn msg;","outputs":1,"noerr":0,"x":450,"y":1300,"wires":[["181f599b.af4696"]]},{"id":"181f599b.af4696","type":"xml","z":"4a0b25ab.85b84c","name":"","property":"payload","attr":"","chr":"","x":630,"y":1300,"wires":[["4c14ec7f.c0e354"]]},{"id":"4c14ec7f.c0e354","type":"file","z":"4a0b25ab.85b84c","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","x":790,"y":1300,"wires":[["68469962.d91c58"]]},{"id":"68469962.d91c58","type":"debug","z":"4a0b25ab.85b84c","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":970,"y":1300,"wires":[]}]

how can I have the special characters in the written file ?

Many thanks in advance

please read How to share code or flow json and the edit the code you posted

done @ukmoose, thanks for pointing that out

Taking your flow and using MacOS it outputs the special characters.

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><video><title>joão</title><keywords>palavras,chave</keywords><description>descrição</description></video>

What are you using to read the file?

I tested your flow and got the special characters correctly spelled/printed/displayed in the file name as well as in the xml content (opened with notepad++). Also all looks good in Node-RED debug.

My setup is Node-REd running in Windows10.


1 Like

I wonder why your file has been generated with such forward slash at the end of the title tag.


Or perhaps this issue was introduced when you copy and pasted the excerpt to this post thread ?

I'm guessing its because the flow that generated that didn't include a title hence undefined.xml

Hi @Andrei , the title tag is like that cause the title value was somehow broken due to the special character. My file was named undefined.xml ..
I forgot to mention node-red is writing to and installed in a debian 9 machine. The strange thing is that now I get the joão.xml written ok and as @ukmoose mentioned, the xmls looks messy depending on the way I read it. The one I pasted in this thread was a "cat" command from the linux terminal. If I open with gedit on a Ubuntu machine, It reads correctly.
The last issue now is that I'm appending that xml to a http form to send to a API. the api is reading it wrongly as well. Anything I could do or the problem is only on the API side ? Thanks you both for helping me out

I guess the better place to look now is on how you build the http request. I guess you are using an http request node in your flow, right ? Can you share this part of this config (removing any tokens / passwords) ?

That may be down to whether the application you are using to view it is using a font containing those characters.