Writing using file node UTF-8, encoding becomes iso-8859

#1

I used the file node to write an XML file to disk. The XML has UTF-8 special characters like ä, however when written to file it is written as iso-8859, as shown on the linux command line :
file original_alert.xml
original_alert.xml: XML 1.0 document, ISO-8859 text, with very long lines

I am running:
Linux debian 4.9.0-8-amd64 #1 SMP Debian 4.9.130-2 (2018-10-27) x86_64 GNU/Linux

locale
LANG=en_US.UTF-8
LANGUAGE=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

And here the code:

[{"id":"72d46eda.d53ee","type":"tab","label":"Flow 2","disabled":false,"info":""},{"id":"ea39c940.9e1328","type":"inject","z":"72d46eda.d53ee","name":"Create alert","topic":"","payload":"{\"alert\":{\"$\":{\"xmlns\":\"urn:oasis:names:tc:emergency:cap:1.2\"},\"identifier\":[\"httpPost.128.Alert\"],\"sender\":[\"public-warning-portal@one2many.eu\"],\"sent\":[\"2019-02-14T09:35:30+01:00\"],\"status\":[\"Actual\"],\"msgType\":[\"Alert\"],\"source\":[\"SYSTEM\"],\"scope\":[\"Public\"],\"info\":[{\"language\":[\"fi-FI\"],\"category\":[\"Safety\"],\"event\":[\"Alert\"],\"urgency\":[\"Immediate\"],\"severity\":[\"Extreme\"],\"certainty\":[\"Observed\"],\"effective\":[\"2019-02-14T09:35:30+01:00\"],\"expires\":[\"2019-02-14T09:40:30+01:00\"],\"senderName\":[\"Group for system-wide operations. Do not remove.\"],\"headline\":[\"Öljysäiliö on tulessa Turun satamassa\"],\"description\":[\"Tulipalo aiheuttaa myrkyllistä savua, joka kulkeutuu Turun yllä. On suositeltavaa, että suljette kaikki ikkunat ja ovet. Viestin on lähettänyt Varsinais-Suomen hätäkeskus.\"],\"area\":[{\"areaDesc\":[\"Somewhere in the country\"],\"polygon\":[\"52.26929376035382,6.095581491740518 52.22725450454942,6.175232370646768 52.2449158438951,6.304321726115518 52.283578033962094,6.216431101115518\"]}]}]}}","payloadType":"json","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":110,"y":280,"wires":[["4cee15f0.c2d79c"]]},{"id":"4cee15f0.c2d79c","type":"xml","z":"72d46eda.d53ee","name":"","property":"payload","attr":"","chr":"","x":270,"y":280,"wires":[["c227b5d9.ea38e","7ced047d.8f5624"]]},{"id":"c227b5d9.ea38e","type":"function","z":"72d46eda.d53ee","name":"write","func":"var fs = global.get(\"fs\");\nfs.writeFileSync(\"test_alert.xml\",msg.payload)\nreturn msg;","outputs":1,"noerr":0,"x":450,"y":320,"wires":[[]]},{"id":"7ced047d.8f5624","type":"file","z":"72d46eda.d53ee","name":"","filename":"/home/bitnami/original_alert.xml","appendNewline":true,"createDir":false,"overwriteFile":"true","x":530,"y":240,"wires":[[]]}]

I found that I could overcome this by writing function in using node.js with fs and this seems to write UTF-8 correctly. But nevertheless I thought to share this issue with you.

And versions:
18 Mar 14:47:04 - [info] Node-RED version: v0.19.5
18 Mar 14:47:04 - [info] Node.js version: v11.7.0

Now I hope my first post is correct! :slight_smile:

0 Likes

#2

Not quite :slight_smile: , to make your flow importable you have to follow these instructions:

0 Likes

#3

This was broken in 0.19.5 and fixed in 0.19.6. Current is 0.20.2.

1 Like

#4

Thanks @elmicha. I've upgraded node-red to 0.20.2 and indeed the problem was fixed!

0 Likes