File that looks fine but may be encoded wrong

[zenofmud]

OS: macOS 12.2.1
Node-RED version: v2.2.2
Node.js version: v14.16.0
Darwin 21.3.0 x64 LE
Dashboard version 3.1.3 started at /ui

So I have a program creating a CSV file. I need to change the order of the items and rename some of the column headers in order to import it into another program.

When I read the file in it looks fine in the debug panel but trying to convert a string "$14.35" I remove the $ and then try a parseFloat and get a NaN.

After spending (way) to much time on this, I know my function code is correct so I used the file-in node to read the file (a single utf8 string)

"status","Receipt Date","Vendor","Category","Payment Type","Sales Tax","Amount","Tax Category","Purpose","Created Date","Modified Date","Reimburse"
"","2022-04-18","Home Depot","Garden","Visa Card","$1.65","$56.06","No Form:Non-deductible","Personal","2022-04-24 09:53:28","2022-04-24 10:32:40",""
"","2022-04-20","Stop&Shop","Grocery Store","Visa Card","$0.00","$128.64","No Form:Non-deductible","Personal","2022-04-24 10:00:07","2022-04-24 10:26:13",""

and sent the output to a file-out node (encoding default) and when I edit the new file this is what I see:

��"status","Receipt Date","Vendor","Category","Payment Type","Sales Tax","Amount","Tax Category","Purpose","Created Date","Modified Date","Reimburse"
"","2022-04-18","Home Depot","Garden","Visa Card","$1.65","$56.06","No Form:Non-deductible","Personal","2022-04-24 09:53:28","2022-04-24 10:32:40",""
"","2022-04-20","Stop&Shop","Grocery Store","Visa Card","$0.00","$128.64","No Form:Non-deductible","Personal","2022-04-24 10:00:07","2022-04-24 10:26:13",""

In addition as I'm entering the info in this post, this is what I see:

Any ideas?

[jbudd]

Is it possible your file is in utf32 (= Unicode ?) format?

I'm playing with a Raspberry Pi not a Mac. if I put your data into a file and convert it with

iconv -f utf8 -t utf32 foo > foobar

Then read foobar into Node-red and save it to a new file it looks like this

and in vi it looks like this

Edit

pi@ZeroTwoTwo:~ $ file -i foo foobar
foo:  application/csv; charset=us-ascii
foobar: application/csv; charset=utf-32le

[zenofmud]

@jbudd

pi@ZeroTwoTwo:~ $ file -i foo foobar

Oh, that's was the key!! I did it on my mac with out the -i and got this:

Paul@PaulsM1 Brother % file BR_export2.csv output1.csv
BR_export2.csv: Unicode text, UTF-16, little-endian text
output1.csv:    data

so in the file-in node I set
Output: 'a single utf8 string'
Encoding: 'utf-16le'
and it is working!!

Thank you very much!