POSTed CSV files lose its newlines - how to use CSV-node in this case?

General
1

I receive a csv-file from the HTTP IN POST node. It is a Buffer so I use a msg.payload.toString() function.
Then I pass it into the CSV-node.

image

This is the original csv that is POSTed to Node-Red:

apparaat;datetime;Vx;Vy;Vz;fx;fy;fz;temp1;temp2;temp3;lat;lon
A3203001;2022-12-20T21:03:04.000Z;0.07;11.0;0.89;8.0;0.00;0.0;0.04;0.48;0.00;51.123;5.123
A3203001;2022-12-20T21:03:34.000Z;0.71;93.5;1.53;8.0;0.00;0.0;0.26;0.84;0.00;51.123;5.123

And this is the SVV debugnode:
image

I have "Skip first 0 lines" in the CSV-node. Because when I skip 1 line then SVV-debug is empty. So this looks like msg.payload from Payload to String results in one line and does not take into account the CR and or LF?

image
image508Ă—592 26 KB

I am sure I lack some basic understanding. But where do I go wrong?

Thanks in advance!

2

Feed the output of the Payload to String node to a debug node and show us what you see,

3

"apparaat;datetime;Vx;Vy;Vz;fx;fy;fz;temp1;temp2;temp3;lat;lonA3203001;2022-12-20T21:03:04.000Z;0.07;11.0;0.89;8.0;0.00;0.0;0.04;0.48;0.00;51.123;5.123A3203001;2022-12-20T21:03:34.000Z;0.71;93.5;1.53;8.0;0.00;0.0;0.26;0.84;0.00;51.123;5.123"

4

What does the buffer look like in a debug node? Have you checked that the newline chars are there?

5

Hm, seems there are no newline chars in the Buffer.

[97,112,112,97,114,97,97,116,59,100,97,116,101,116,105,109,101,59,86,120,59,86,121,59,86,122,59,102,120,59,102,121,59,102,122,59,116,101,109,112,49,59,116,101,109,112,50,59,116,101,109,112,51,59,108,97,116,59,108,111,110,65,51,50,48,51,48,48,49,59,50,48,50,50,45,49,50,45,50,48,84,50,49,58,48,51,58,48,52,46,48,48,48,90,59,48,46,48,55,59,49,49,46,48,59,48,46,56,57,59,56,46,48,59,48,46,48,48,59,48,46,48,59,48,46,48,52,59,48,46,52,56,59,48,46,48,48,59,53,49,46,49,50,51,59,53,46,49,50,51,65,51,50,48,51,48,48,49,59,50,48,50,50,45,49,50,45,50,48,84,50,49,58,48,51,58,51,52,46,48,48,48,90,59,48,46,55,49,59,57,51,46,53,59,49,46,53,51,59,56,46,48,59,48,46,48,48,59,48,46,48,59,48,46,50,54,59,48,46,56,52,59,48,46,48,48,59,53,49,46,49,50,51,59,53,46,49,50,51]

This is the syntax I use for POSTing:

curl --header "Content-Type: text/csv" --request POST -d "@svv.csv" "https://endpoint....

Maybe I should have another Content-Type?

6

This translates to:

apparaat;datetime;Vx;Vy;Vz;fx;fy;fz;temp1;temp2;temp3;lat;lonA3203001;2022-12-20T21:03:04.000Z;0.07;11.0;0.89;8.0;0.00;0.0;0.04;0.48;0.00;51.123;5.123A3203001;2022-12-20T21:03:34.000Z;0.71;93.5;1.53;8.0;0.00;0.0;0.26;0.84;0.00;51.123;5.123

What is done with the newlines that are in the original file?

7

I think you had better edit the subject line of the thread, now we know it is nothing to do with the csv node or the parsing. As to why they are not there, I don't know.

8

Problem solved!

The curl-command I used to POST had a

-d file

should have been

--data-binary file

Now complete format is coming in and CSV-node handles all records correctly!
Thanks!

2 Likes
9

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.