Node-RED v1.3.3 CSV Node Changes Broke Several Flows

After upgrading to v1.3.3 several flows did not function properly. I isolated this to behavior changes in the CSV node. I am using CSV Node to convert space-delimited CSV strings to arrays of objects. The CSV Node Help documentation states the following:

  • Note: the column template must be comma separated - even if a different separator is chosen for the data.

In my case, my columns were specified similar to "Col1,Col2" (Example 1) and this worked fine. With the latest Node-RED update, this is now interpreted as a single column named "Col1,Col2" rather than 2 columns named "Col1" and "Col2".

Adding a space after the comma "Col1, Col2" (Example 2) does not help as this results in 2 columns named "Col1," (note the extraneous comma) and "Col2".

Using a space delimiter instead of a comma "Col1 Col2" (Example 3), contrary to the Help note above, works and yields 2 columns named "Col1" and "Col2".

And if the string and CSV node delimiter are both changed to comma (Example 4) this works as well.

So... It looks like the behavior has changed such that the specified delimiter in the CSV and the Columns specifier must now match. This is probably for the best. But the Help documentation needs updated to reflect same.

As for my flows, I have made the necessary corrections to get everything working again with v1.3.3 and am not requesting any assistance. I just wanted to note the documentation defect and also call out the change in case anybody else is struggling with broken flows after upgrading.

[{"id":"cdbe5f8a.176fd","type":"tab","label":"Flow 1","disabled":false,"info":""},{"id":"a261bda1.88667","type":"inject","z":"cdbe5f8a.176fd","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Row1Col1 Row1Col2\\nRow2Col1 Row2Col2\"","payloadType":"jsonata","x":190,"y":280,"wires":[["9c7cea70.541a48"]]},{"id":"1b0562a9.0b248d","type":"debug","z":"cdbe5f8a.176fd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":530,"y":280,"wires":[]},{"id":"9c7cea70.541a48","type":"csv","z":"cdbe5f8a.176fd","name":"CSV to Array","sep":" ","hdrin":"","hdrout":"none","multi":"mult","ret":"\\n","temp":"Col1,Col2","skip":"0","strings":false,"include_empty_strings":"","include_null_values":"","x":350,"y":280,"wires":[["1b0562a9.0b248d"]]},{"id":"cd89f537.96b0e8","type":"inject","z":"cdbe5f8a.176fd","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Row1Col1 Row1Col2\\nRow2Col1 Row2Col2\"","payloadType":"jsonata","x":190,"y":360,"wires":[["509f0186.3b62c"]]},{"id":"5fd0bd70.8954c4","type":"debug","z":"cdbe5f8a.176fd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":530,"y":360,"wires":[]},{"id":"509f0186.3b62c","type":"csv","z":"cdbe5f8a.176fd","name":"CSV to Array","sep":" ","hdrin":"","hdrout":"none","multi":"mult","ret":"\\n","temp":"Col1, Col2","skip":"0","strings":false,"include_empty_strings":"","include_null_values":"","x":350,"y":360,"wires":[["5fd0bd70.8954c4"]]},{"id":"c942af0e.4ca3a","type":"inject","z":"cdbe5f8a.176fd","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Row1Col1 Row1Col2\\nRow2Col1 Row2Col2\"","payloadType":"jsonata","x":190,"y":440,"wires":[["bc1bb13f.7e728"]]},{"id":"eac448d2.ec2488","type":"debug","z":"cdbe5f8a.176fd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":530,"y":440,"wires":[]},{"id":"bc1bb13f.7e728","type":"csv","z":"cdbe5f8a.176fd","name":"CSV to Array","sep":" ","hdrin":"","hdrout":"none","multi":"mult","ret":"\\n","temp":"Col1 Col2","skip":"0","strings":false,"include_empty_strings":"","include_null_values":"","x":350,"y":440,"wires":[["eac448d2.ec2488"]]},{"id":"57bb3b35.91baa4","type":"inject","z":"cdbe5f8a.176fd","name":"","props":[{"p":"payload"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"\"Row1Col1,Row1Col2\\nRow2Col1,Row2Col2\"","payloadType":"jsonata","x":190,"y":520,"wires":[["c99a9bfb.fdaad8"]]},{"id":"63c21856.bdd3b8","type":"debug","z":"cdbe5f8a.176fd","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":530,"y":520,"wires":[]},{"id":"c99a9bfb.fdaad8","type":"csv","z":"cdbe5f8a.176fd","name":"CSV to Array","sep":",","hdrin":"","hdrout":"none","multi":"mult","ret":"\\n","temp":"Col1,Col2","skip":"0","strings":false,"include_empty_strings":"","include_null_values":"","x":350,"y":520,"wires":[["63c21856.bdd3b8"]]},{"id":"6ce81155.7e2ef","type":"comment","z":"cdbe5f8a.176fd","name":"1. CSV delimited by space, Columns specified using comma - NO LONGER WORKS","info":"","x":350,"y":240,"wires":[]},{"id":"1afe693.be23997","type":"comment","z":"cdbe5f8a.176fd","name":"2. CSV delimited by space, Columns specified using comma space - DOESN'T WORK","info":"","x":360,"y":320,"wires":[]},{"id":"89804d51.4f135","type":"comment","z":"cdbe5f8a.176fd","name":"3. CSV delimited by space, Columns specified using space - WORKS","info":"","x":300,"y":400,"wires":[]},{"id":"4b98f9f8.190f58","type":"comment","z":"cdbe5f8a.176fd","name":"4. CSV delimited by comma, Columns specified using comma - WORKS","info":"","x":310,"y":480,"wires":[]}]

Peter

This may be related to issue 2950 and several other recent CSV Node defects/fixes. Please include Help documentation correction in the fix.

Functionality should not change in a fix release - if the behaviour has changed from what the node documents then that needs to be reverted.

There have been a number of fixes in the CSV node, this may be wrapped up in that. Need @dceejay to comment.

I, personally, did not like the fact that though my CSV was comma delimited I had to use spaces to list the columns for the CSV node. I think the “best” solution is when specifying columns in the CSV node allow use of either space (backward compatibility) or the identified delimiter (which can be replaced with a space during a pre-parse step).

If you revert back to only spaces again, then my flows will once again break when I update to that version (and for others).

The template was always comma separated, if it's broken, then we need to fix it.

@plfinch Hi - using your test case - I would expect tests 1, 2, and 4 to work as they are all correctly comma separated - and 3 to fail. I agree they don't at that version - but that is a bug I introduced, and will be reverted.

@knolleary Actually I think the commit to fix CSV node not working when field delimiter is | in version 1.3.3 It worked in version 1.3.1 · Issue #2950 · node-red/node-red · GitHub has already fixed this - certainly the extra tests now all pass. As does this test case.

2 Likes

Acknowledged. Thank you!

Node-RED v1.3.4 does restore the comma delimiter as the only delimiter for specifying the Columns value in a CSV Node. Examples 1, 2, 4, which broke in v1.3.3, now work as expected. And Example 3, which worked in v1.3.3, no longer works.

So all good now. Well, all good after I went back and once again “fixed” every CSV Node in my flows to make them work again with v1.3.4.

Thanks again…

Peter

1 Like

The moral is, if you find a breaking change following an update, then report it here before implenting a workaroud, it is virtually guaranteed to be fixed very quickly, and in the meantime you could revert to the previous version of node-red very easily.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.