Object export problem in CSV

Hi, me again, coming with a small problem concerning CSV export :slight_smile:

Most of the Array I export to CSV using the default CSV node is output correctly, but I have a trouble with objects in the array :

here is the object content . I tried different methods but no one work :

var csv = '{"desc":"","label":"123_CH07_RxA04"}'
csv = JSON.parse(csv)

and the result in CSV export :

descriptor
"{""desc"":"""",""label"":""123_CH07_RxA04""}"

How can I get rid in a proper way of the extra """ , which are not supposed to be there ?

Thanks

Jerome

It is best to use variable names that have correct meaning otherwise you will always get confused or someone trying to read your code will (both probably).

So your first assignment (which should use let not var by the way) is a string. So it should be something like this:

let myJson = '{"desc":"","label":"123_CH07_RxA04"}'
try {
   myJson = JSON.parse(myJson)
} catch (e) {
  // Do sometihng if needed if the parse fails - JSON.parse will ALWAYS occasionally fail, bet on it
}

Which makes more sense and myJson now actually contains a JS object as expected.

However, not sure why you want to turn that string into an object and then write it to CSV, that will likely always be an issue. You WANT your JSON to be a string before trying to write it to a CSV. So if anything you should be doing all this the other way around. Start with the JS object, use JSON.stringify (which also needs a try/catch around it) and then write that to CSV.

Hi, thanks for your reply, and sorry for the var/let mistake, I'm still learning :slight_smile:

I adapted the quote a little bit following your advices, but still no luck.

Actually, indeed it's fine by using only the string , it's non-sense to parse then stringify, that was just for testing a few different possibilities.

so, when I output the string I get the correct content, however, to get it in a CSV file, I need to send a Array with objects, like this :

let descriptorContent = '{"desc":"","label":"123_CH07_RxA04"}'
//csv = JSON.parse(csv)

        //csv = JSON.stringify(csv)
        var csvArray=[{"descriptor": csv}]
        //csv = (csv).replace('"')
   
msg.payload =  csvArray;

return msg;

Then I get the proper column name (descriptor), but the CSV content is not good :

descriptor
"{""desc"":"""",""label"":""123_CH07_RxA04""}"

Thanks in advance,

Jerome

So what I did was to create a workbook and save as CSV:

image

Opening the CSV as raw in VSCode which shows this:

Col1,Col2
"{""desc"":"""",""label"":""123_CH07_RxA04""}","{""desc"":"""",""label"":""123_CH07_RxA04""}"
"{""desc"":"""",""label"":""123_CH07_RxA04""}","{""desc"":"""",""label"":""123_CH07_RxA04""}"

So as you can see, the result you are getting is absolutely correct.

Yes, but I'd like to get the exact same content is what we can see in the original string...

I could get rid of the " " " from the string, but I need in the end also "" when there is no value for any of the key ( as we can see in desc in this specific case...)

Don't know how I can handle this, if I use just a space in desc then I'd get a " " in the final csv....

Sorry, I'm lost now. A CSV is a defined schema and complex strings HAVE to be escaped otherwise you don't have a working CSV. The double double-quotes are REQUIRED in the CSV. Without them you cannot use the CSV anywhere, it won't work.

To prove the case, try with the quotes in place and then manually edit one of the entries. In both cases, open the csv in Excel or similar and see what happens.

Hi, so I'm lost too.

Here is a snippest of the csv(-like ?) export I get from my system in a txt file, and from which I want to manipulate datas.

control	custom	descriptor	extraAlertFilters	Factory label	isIgmpSource
off	{}	{"desc":"","label":""}	[]	receiver_s06_a03	FALSE
off	{}	{"desc":"blabla","label":""}	[]	receiver_s05_a01	FALSE

When I load this one in Excel, I get it right :

And the csv import in NR is fine as well .

So on the input side all looks good to me, but I can't get it exported correctly.....

That is not a CSV. It looks like a TSV (Tab separated). That would actually be easier to work with for your data probably. But it isn't a CSV. You can work with that in Node-RED I think - still using the incorrectly named "CSV" node, just set it to use tabs instead of comma's as the separator. TSV does not need the double quoting as long as you don't try to include tabs in your fields.

this is part of the source file, you think it's TSV ?

#;active;bidirPartnerId;codecFormat;configPriority;control;custom;descriptor;extraAlertFilters;Factory label;isIgmpSource;mainDstIp;mainDstMac;mainDstPort;mainDstVlan;mainSrcGateway;mainSrcIp;mainSrcMac;mainSrcNetmask;public;sdpSupport;sipsMode;spareDstIp;spareDstMac;spareDstPort;spareDstVlan;spareSrcGateway;spareSrcIp;spareSrcMac;spareSrcNetmask;tags;useAsEndpoint
device73.484735f6-a1ea-596e-8282-0cd1108e6be2.R01a52f67-554c-5791-8503-fcbee07f93d9;true;null;Audio;off;off;{};{"desc":"","label":""};;receiver_s06_a03;false;null;null;null;null;null;null;null;null;false;true;NONE;null;null;null;null;null;null;null;null;;false
device73.484735f6-a1ea-596e-8282-0cd1108e6be2.R02d8318e-62ea-5d74-8693-9d3e7024f560;true;null;Audio;off;off;{};{"desc":"blabla","label":""};;receiver_s05_a01;false;null;null;null;null;null;null;null;null;false;true;NONE;null;null;null;null;null;null;null;null;;false
device73.484735f6-a1ea-596e-8282-0cd1108e6be2.R030c78b8-e95c-5012-bfc8-bd7ef3188d16;true;null;Ancillary;off;off;{};{"desc":"","label":""};;receiver_s02_m00;false;null;null;null;null;null;null;null;null;false;true;NONE;null;null;null;null;null;null;null;null;;false
device73.484735f6-

Well, in the end I made a small piece of code by myself in a function node directly connected to a file write node, and now gets the proper result in the target file :

const separator = ";";

let csvlines = ""

var columns = (Object.keys(msg.payload[0]).join(separator));

for (let index = 0; index < msg.payload.length; index++) {

    msg.payload[index].descriptor =  JSON.stringify(msg.payload[index].descriptor);
        
    var line = (Object.values(msg.payload[index]).join(separator) + "\n" );
    csvlines = csvlines + line;

    }

//node.warn("csvContent" + csvlines)
msg.columns = columns;
msg.payload = msg.columns + "\n" + csvlines;

return msg;

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.