Reading File - Odd character return output?

aidanvenn · 2 May 2023 21:15

Hi,

Pic below hopefully helps.

When I read a simple text file it reads each line but appears to add a character return for all expect the last line(which has no return). This causes issues processing the output in next steps as the value isn`t as expected. For example:

"XXXXXXXXXX"

is actually

"XXXXXXXXXX
"

Text file created in notepad++ on windows 11.

I`ve noticed that when I select text in notepad++ it highlights a space at the end of each line except the last. Pic:
node red issue pic2

Maybe encoding of file? Tried utf-8 but not luck.

Any ideas?

Thank you

smcgann99 · 3 May 2023 01:22

Humm I couldn't replicate this using notepad as the source, regardless of encoding?

In notepad++ if you select view then show symbol then show all it might give you a clue.

Alternatively there is a handy node to remove white space and many other options.

ralphwetzel · 3 May 2023 05:53

As you're on Windows, I'm guessing you're going with the standard Notepad++ & Windows setting: NewLine there is CR LF (2 characters,\r\n) - as stated in the footer line of NP++:

The FileIn-Node splits the incoming stream yet Linux/Unix-like @ \n:

github.com

node-red/node-red/blob/67dd7e30faf1a6fca07f5b745de75f4ad3036ce4/packages/node_modules/@node-red/nodes/core/storage/10-file.js#L352


      
          var rs = fs.createReadStream(fullFilename)
              .on('readable', function () {
                  var chunk;
                  var m;
                  var hwm = rs._readableState.highWaterMark;
                  while (null !== (chunk = rs.read())) {
                      if (node.chunk === true) {
                          getout = true;
                          if (node.format === "lines") {
                              spare += decode(chunk, node.encoding);
                              var bits = spare.split("\n");
                              for (var i=0; i < bits.length - 1; i++) {
                                  m = {};
                                  if (node.allProps == true) {
                                      m = RED.util.cloneMessage(msg);
                                  }
                                  else {
                                      m.topic = msg.topic;
                                      m.filename = msg.filename;
                                  }
                                  m.payload = bits[i];

What you see could be the leftover \r portion of the Windows-like NewLine sequence.

Possible solution: Change the format of the file created by NP++ to UNIX (LF) ... by e.g. right-clicking on the field in the footer & appropriate selection ... or propose a PR to extend the capabilities of the FileIn-Node, allowing to select the LineSplit character sequence.

aidanvenn · 3 May 2023 09:31

@smcgann99 Did the trick - Thank you

@ralphwetzel - Thank you

system · 17 May 2023 09:32

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Read file node - read from file does not work General	28	1287	23 May 2022
Reading in file missing parts General	9	143	27 January 2024
Message payload - String? General	11	6836	10 July 2020
Read File node returns one line too many? General	15	259	9 December 2022
Reading and Writing into a text file from the function node General	34	22891	23 January 2019

Reading File - Odd character return output?

Related Topics