Problems with csv file reading and its representation in a chart

General
1

Hi,

I am trying to build a simple data detector.but i face some problems in file read and its representation. I plan to use a dropdown to give users the choice they want. And their choice will be shown in form of single line chart or multi-line chart. The data will be real-time, of course.
My Node-red version is v4.0.2 and Node.js version is v20.16.0. Dashboard version is 3.6.5.

My structure is
[dropdown=>function(select)
timestamp=>file read(.csv*15)=>join=>function(process data for chart)=>chart]

image
image1854×908 124 KB

but debug return :
TypeError: Cannot read properties of undefined (reading 'csv') ReferenceError: done is not defined

My function code:

let armNumbers = flow.get('selectedArm');  
if (!armNumbers) return null;

if (!Array.isArray(armNumbers)) {
    armNumbers = [armNumbers];


}

let selectedData = [ ];
if (Array.isArray(msg.payload)) {
    msg.payload.forEach(fileData => {
        if (fileData && Array.isArray(fileData)) {
            fileData.forEach(line => {
                let parts = line.split(',');  
                let armNumber = parseInt(parts[0]);  
                if (armNumbers.includes(armNumber)) {
                    selectedData.push({
                        arm: armNumber,
                        timestamp: parts[1],  
                        angle: parseFloat(parts[2]) 
                    });
                }
            });
        } else {
            node.error("Expected an array of file data but got something else.", msg);
        }
    });
} else {
    node.error("Payload is not an array.", msg);
}

msg.payload = selectedData;
return msg;
2

Why are there 15 CSV files? Where is this data being generated? Do you have control over how this data is generated and written?

Which node generates this error? I don't believe that is generated by your function.

Ps, pretty much all of your function node can be made redundant by using the CSV node (it can parse the values etc)

Lastly, add debug nodes after the join node and function node - what do you get?

Oh, one more thing, sending an array of objects to the chat node will not work as you expect. Read the chart nodes built in help. There is a very specific format of data, series, labels that the payload needs to contain.

3

I’m working on a project to monitor the angles of robotic arms, but I’m just starting, so all the simulated data is generated using random numbers. They can also be properly recorded in local files, so I don’t think that’s causing any issues.
Moreover, I’ve already taken note of the data structure for the charts you mentioned earlier, so thank you for the kind reminder. Honestly, I’m quite frustrated with this error; it’s so different from the ones I’ve encountered before that it feels like there’s no clear starting point for troubleshooting.
You mentioned debugging after joining nodes, which I haven’t tried, but I did test them separately (so join wasn’t necessary), yet the same error occurred. Therefore, I suspect that the issue might be related to timestamp and readfile interference under global variable conditions.

4

In fact, I've already bypassed this issue. Initially, I was too focused on storing and retrieving data through files, but files are more for archival purposes. I realized I could directly convert the data into charts before storing it, which also avoids time delays during data transmission. However, I still want to understand whether it's possible to use global variables for control in Node-RED (I’ve encountered errors before when using global variables), or if there are recommended modules for implementing such functionality. Thanks again for your response.

1 Like
5

Your use of flow context to store the user's preferred robot arm seems a reasonable solution.
It permits regular screen refresh without the user having to make another selection from the dropdown.
As long as the processing of input data is reasonably fast, there is little chance of the user making another selection before it's finished.

6

What errors have you come across?

There are a couple of issues that you might come across but these aren't errors:

  • File-based context variables only write to storage periodically. This might be an issue if NR crashes. The timing can be changed though.
  • The way that JavaScript/Node.js handles asynchronous activities means that you can come across timing edge-cases where either multiple flows or multiple actors could trigger overlapping activities.

I can't really think of much else.

7

"by reference" user errors (stores an object or array from msg.payload -> flow.data, modifies msg.payload, finds flow.data has changed)

8

Yes.They did happen when I was dealing with asynchronous messages earlier, overlapping as you say with multiple actors coming together. But those issues have been solved. Thank you for your kindness.

1 Like