# The file node, change the syntax

**URL:** <https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178>\
**Category:** General\
**Created:** [25 November 2018 11:04 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178 "2018-11-25T11:04:46Z")\
**Posts on this page:** 20\
**Page:** 1

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [25 November 2018 11:04 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/1 "2018-11-25T11:04:46Z")

</div>

Hello,  
I have a problem with the file node.  
When I extract data with the file write node, everything is fine; no syntax problem. On the other hand, when I want to recover the data with the file reads node, I have syntax errors in the payload. the 'é' turns into '?'. If anyone could enlighten me.  
Thank you

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [25 November 2018 11:27 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/2 "2018-11-25T11:27:04Z")

</div>

Can you post a small flow that shows the problem so we can try it?

---

<div class="post-metadata">

**Author:** ![ukmoose](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/ukmoose/32/13_2.png) [@ukmoose](https://discourse.nodered.org/u/ukmoose)\
**Post date:** [25 November 2018 12:18 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/3 "2018-11-25T12:18:44Z")

</div>

What system are you running on? As on a mac I cannot reproduce

```auto
[{"id":"7131ffea.7413a8","type":"inject","z":"efbf8bf.35920f8","name":"","topic":"","payload":"joãoéßæ","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":140,"wires":[["8c4e484d.c8b418"]]},{"id":"8c4e484d.c8b418","type":"file","z":"efbf8bf.35920f8","name":"","filename”:”/<full path to file>/test.txt","appendNewline":false,"createDir":true,"overwriteFile":"true","x":450,"y":140,"wires":[]},{"id":"f6715482.41c258","type":"debug","z":"efbf8bf.35920f8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":550,"y":240,"wires":[]},{"id":"9eaa97fa.eb3b18","type":"file in","z":"efbf8bf.35920f8","name":"","filename":"/<full path to file>/test.txt","format":"utf8","chunk":false,"sendError":false,"x":350,"y":240,"wires":[["f6715482.41c258"]]},{"id":"697fa3d0.4bc9bc","type":"inject","z":"efbf8bf.35920f8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":240,"wires":[["9eaa97fa.eb3b18"]]}]

```

Produces the following in the debug panel:

```auto
_msgid: "a463efe0.17f5b"
topic: ""
payload: "joãoéßæ"
filename:

```

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andrei/32/10446_2.png) [@Andrei](https://discourse.nodered.org/u/Andrei)\
**Post date:** [25 November 2018 12:54 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/4 "2018-11-25T12:54:09Z")

</div>

If you write a file with Node-RED `file in` node and then read this file with Node-RED `file` it should work flawlessly. The character "é" will be saved as the UTF-8 sequence of two bytes "c3a9" as expected from the [UTF-8 character set.](https://www.fileformat.info/info/charset/UTF-8/list.htm)

![r-01](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/7/71da5f8240526dc1e7ce57f986658ddf2dd68285.png)

Now, if you saved the file with another text editor it may happen that the character will be saved with another encoding, for instance, ISO-8859-1 (ISO Latin 1). In such case the character "é" will be saved as byte "e9" , which is not a valid byte in UTF-8, resulting that in Node-RED node it will be displayed as the **replacement character** �

![r-02](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/a/ac20a543af6c85b1e474898fb4b0d74fc85806ed.png)

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [25 November 2018 16:56 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/5 "2018-11-25T16:56:32Z")

</div>

Node red runs on a raspberry with linux. I'm using the file node to save the data:  
(ciel dégagé 7.7 degré 1.5 pourcent )

then I use the file in node to retrieve and read the data:  
( 25/11/2018 à 17:49:22[node: 57c59eab.874de8](http://192.168.1.14:1880/#)

msg.payload : string[36]

"ciel d�gag� 7.7 degr� 1.5 pourcent " )

file name: / home / pi / logs / folder name .

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andrei/32/10446_2.png) [@Andrei](https://discourse.nodered.org/u/Andrei)\
**Post date:** [25 November 2018 18:14 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/6 "2018-11-25T18:14:05Z")

</div>

This testing may help to clarify ( peut-être).  
What do you get on the debug panel if you change your flow with this two changes ?

1- Firstly change the config node of your file node (read) as below:

![r-01](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/c/c0aeb125ec4e7cccc51fe28c27d98f56db7f1dbb.png)

2- add this function node after the file read node

```auto
[{"id":"ee5b88b8.33e098","type":"debug","z":"1a1dea55.020d86","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":670,"y":180,"wires":[]},{"id":"8797bab8.7d5eb8","type":"function","z":"1a1dea55.020d86","name":"latin1","func":"pay = msg.payload.toString(\"latin1\");\nmsg.payload = pay;\nreturn msg;","outputs":1,"noerr":0,"x":470,"y":180,"wires":[["ee5b88b8.33e098"]]}]

```

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [25 November 2018 19:26 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/7 "2018-11-25T19:26:47Z")

</div>

That's what I get:  
[110,117,97,103,101,117,120,32,32,54,46,55,32,100,101,103,114,233,32,48,46,53,32,112,111,117,114,99,101,110,116,32]  
thank you for your interest in my problem

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andrei/32/10446_2.png) [@Andrei](https://discourse.nodered.org/u/Andrei)\
**Post date:** [25 November 2018 20:15 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/8 "2018-11-25T20:15:55Z")

</div>

Good. It means that we do have a workaround for the issue. Missing to know why the file was saved in a way that is not usual (I can imagine a couple of reasons, but have no way to test).

This is what you will see in the debug panel:

![r-01](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/6/649c38ae15ded19b1f6152eef9f114f6fb7d2492.png)

But you need to add the function node after the file node, as below:

 ![r-02](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/d/d061126cd1c8f20ddbda4f591ae8490fe65cecdb.png)

Explanation:

This is the code in the function node:

```auto
pay = msg.payload.toString("latin1");
msg.payload = pay;
return msg;

```

It will decode the data read by the file node using the **8859-1 (Latin-1)** character set.

This is my testing flow. You need to **change the file name and path** in the file node to test again on your side.

```auto
[{"id":"87eb973e.4accd8","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"81036dd5.c91ce","type":"file in","z":"87eb973e.4accd8","name":"Read from file","filename":"C:/Users/OCM/.node-red/static/nrfiles/coffee.txt","format":"","chunk":false,"sendError":false,"x":340,"y":140,"wires":[["3c2937f5.6c7648"]]},{"id":"f6c975f9.5bf1c8","type":"inject","z":"87eb973e.4accd8","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":140,"wires":[["81036dd5.c91ce"]]},{"id":"39cd1e82.01f6f2","type":"debug","z":"87eb973e.4accd8","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":650,"y":140,"wires":[]},{"id":"3c2937f5.6c7648","type":"function","z":"87eb973e.4accd8","name":"latin1","func":"\npay = msg.payload.toString(\"latin1\");\nmsg.payload = pay;\nreturn msg;","outputs":1,"noerr":0,"x":510,"y":140,"wires":[["39cd1e82.01f6f2"]]}]

```

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [25 November 2018 21:31 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/9 "2018-11-25T21:31:09Z")

</div>

I tested, but it did not work:

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/f/f81b3d9553bbe099cec1d23f37ce521ec0a54e06.png)

the file is overwritten each time there is a new value of the openweathermap node

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andrei/32/10446_2.png) [@Andrei](https://discourse.nodered.org/u/Andrei)\
**Post date:** [25 November 2018 22:02 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/10 "2018-11-25T22:02:39Z")

</div>

Puzzling indeed. In such case we need to debug a little further.

Could you please confirm if the function indicated with a red arrow is the one I suggested ?

Can you possibly add a debug node where it is indicated with number 3 , run the flow and show a new output from the debug panel ? I wanted to compare the outputs of debug nodes 1, 2 and 3.

 ![r-03](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/a/a32d983f3bfc1f5cc6c1fdc70f8a3e9eeb6813e3.png)

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [26 November 2018 17:46 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/11 "2018-11-26T17:46:43Z")

</div>

Good evening,  
it is the node of function that you proposed to me;

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/d/d2535501258f2b7dd0454e03dff13d0b42f3d498.png)

and here is the result of the debug;

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/a/ae75c89768862f088aeb4138fd291335f5b98fe3.png)  
debug 2 and 3 are the same.

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andrei/32/10446_2.png) [@Andrei](https://discourse.nodered.org/u/Andrei)\
**Post date:** [27 November 2018 17:52 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/12 "2018-11-27T17:52:45Z")

</div>

Interesting. My best gues now is that the data is being saved wrongly so the conversion function does not work, but not having a raspberry I can not test by myself.

I wanted to know what this the default language and encoding used on your raspberry.

Google points these possible commands in the terminal. Could you please test and see what you can find ? (merci)

locale

![r-02](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/4/4e1c55d321421c8c688d0453ae9582e3e7aaad7c.png)

or maybe using the command `echo $LANG` in the terminal ?

or perhaps you can check in this system menu:

 ![r-01](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/6/67b340d54b067ff38438b272ff95011a72692b21.png)

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [27 November 2018 20:31 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/13 "2018-11-27T20:31:12Z")

</div>

hello,  
this is what gives

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/1/174ac292eb037257178cd12f982b5c690289359f.png)

if it can enlighten you

---

<div class="post-metadata">

**Author:** ![Andrei](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/andrei/32/10446_2.png) [@Andrei](https://discourse.nodered.org/u/Andrei)\
**Post date:** [27 November 2018 22:21 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/14 "2018-11-27T22:21:05Z")

</div>

Looks like your raspberry is configured with the default UTF-8, so nothing suspicious here.

I was not able to identify the root cause neither the solution for the reported problem.

Summary in case someone else has some clue:

1- Text from Openweathermap is `nuageux 6.7 degré 0.5 pourcent `

2- This text is saved with `node-file`

3- Binary data is saved and read as below. Note that the character `é` is the 17th byte which is `decimal 233 , hexa e9`. The assumption is that it was saved as ISO-8859-1 (ISO Latin 1) given that 0xe9 is invalid in UTF-8

> [110,117,97,103,101,117,120,32,32,54,46,55,32,100,101,103,114, **233** ,32,48,46,53,32,112,111,117,114,99,101,110,116,32]

4- The issue is that after reading the file with the `node file in` it appears in the debug panel as `nuageux 6.7 degr� 0.5 pourcent `

5- We tried to use a function node to convert from latin-1 character set to UTF-8. It works in my system (windows10) but it did not work for @fabien (raspberry pi) .

 ![final-01](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/6/6b55516c09cff7ef77fcd5042f41bfab050d247b.png)

 ![r-01](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/9/95ff44ce647b58d87ebe22dbf88528035709e38b.png)  
Testing flow:

```auto
[{"id":"ee5b88b8.33e098","type":"debug","z":"1a1dea55.020d86","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":550,"y":180,"wires":[]},{"id":"8797bab8.7d5eb8","type":"function","z":"1a1dea55.020d86","name":"latin1 - buffer","func":"msg.payload = Buffer.from([110,117,97,103,101,117,120,32,32,54,46,55,32,100,101,103,114,233,32,48,46,53,32,112,111,117,114,99,101,110,116,32]);\n\nmsg.latin1 = msg.payload.toString(\"latin1\");\nmsg.utf8 = msg.payload.toString(\"utf-8\");\n\n\nreturn msg;","outputs":1,"noerr":0,"x":350,"y":180,"wires":[["ee5b88b8.33e098","60224a4a.26e6c4","8f4ceaa9.6bd968"]]},{"id":"98ddfefc.df073","type":"inject","z":"1a1dea55.020d86","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":100,"y":180,"wires":[["8797bab8.7d5eb8"]]},{"id":"60224a4a.26e6c4","type":"ui_text","z":"1a1dea55.020d86","group":"90301f70.997cd","order":0,"width":0,"height":0,"name":"","label":"Latin1:","format":"{{msg.latin1}}","layout":"row-center","x":550,"y":240,"wires":[]},{"id":"8f4ceaa9.6bd968","type":"ui_text","z":"1a1dea55.020d86","group":"90301f70.997cd","order":0,"width":0,"height":0,"name":"","label":"UTF-8:","format":"{{msg.utf8}}","layout":"row-center","x":550,"y":280,"wires":[]},{"id":"90301f70.997cd","type":"ui_group","z":"","name":"Teste","tab":"1e2aea29.e92e76","disp":false,"width":"10","collapse":false},{"id":"1e2aea29.e92e76","type":"ui_tab","z":"","name":"LAB","icon":"dashboard"}]

```

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [27 November 2018 23:24 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/15 "2018-11-27T23:24:16Z")

</div>

Which version of node.js is @fabien running ?  
Versions prior to 6,14.0 did not have the latin1 support your function node uses.

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [28 November 2018 10:56 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/16 "2018-11-28T10:56:37Z")

</div>

yet my version of node.js = 8.12

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/2/21b5ecdc811ae8357e3857cceb376fba0fb5f6ae.png)

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [28 November 2018 12:46 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/17 "2018-11-28T12:46:22Z")

</div>

I have pushed a beta version of dashboard to master (not npm) that has a fix to try to handle binary buffers in a better manner - ie not force them to utf8 when displaying text,  
So it should now just handle the binary buffer msg.payload direct.

You can get it by

```
cd ~/.node-red && npm i node-red/node-red-dashboard

```

Let me know if it helps / makes things worse - etc

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [28 November 2018 15:06 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/18 "2018-11-28T15:06:41Z")

</div>

I try it, it's always the same

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/e/e5c8e17690df05aaea0f30727c72cf2c31f49f9f.png)  
 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/e/e02d087d1f81b71cf038f1787fadb8c2cbdbeb3a.png)

can be starting node-red with PM2 will it work?

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [28 November 2018 15:27 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/19 "2018-11-28T15:27:04Z")

</div>

Did you restart node-red and force a full refresh of your browser?

---

<div class="post-metadata">

**Author:** ![fabien](https://avatars.discourse-cdn.com/v4/letter/f/85e7bf/32.png) [@fabien](https://discourse.nodered.org/u/fabien)\
**Post date:** [28 November 2018 16:42 UTC](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178/20 "2018-11-28T16:42:34Z")

</div>

I redid the operation, then I restart node-red

 ![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/d/d84f99a884233d790a40103b0966290b538890f9.png)

as for the browser, I try chrome, fierfox; yes I  
refresh page. the same

[Next page](https://discourse.nodered.org/t/the-file-node-change-the-syntax/5178.md?page=2)