# Read data from two different csv files as input for a function node

**URL:** <https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933>\
**Category:** General\
**Created:** [14 November 2019 09:03 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933 "2019-11-14T09:03:26Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [14 November 2019 09:03 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/1 "2019-11-14T09:03:26Z")

</div>

I have a function node which has a few variables. They have to be altered depending on the useage. Therefore i dont anyone to alter the Code .Instead I want to read these values from csv file so that changes can easily be made in that file. The Problem is I am already reading the Input data from another csv file. So how can I proceed with this? Please help me out.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [14 November 2019 10:52 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/2 "2019-11-14T10:52:06Z")

</div>

Well, this is partly a timing problem. How often will the config be updated? Will config changes happen while the flow is active?

One approach might be to have a separate flow that reads the config csv into a flow or global variable.

Another approach would be to read the csv in-line with the flow but put it onto `msg.config` or something similar rather than `msg.payload`. Then your function node always gets the latest version whenever it receives a msg. However, that is quite inefficient and slow.

---

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [14 November 2019 11:24 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/3 "2019-11-14T11:24:37Z")

</div>

The config will only be updated once at the begining before starting the flow. How can i assign the 4 values to to say msg.config? And how can i access it? Can I use an array and access it?

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [14 November 2019 13:36 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/4 "2019-11-14T13:36:58Z")

</div>

> [@varun](#):
>
> How can i assign the 4 values to to say msg.config?

Change node if they are in a flow/global variable. To get them from a file, a csv node followed by a change node to move the input from msg.payload to msg.config. You might find it easier though to use a JSON file since it means that you can just copy from msg.payload to msg.config and don't have to mess with the complexities that a csv-in will leave you with.

> [@varun](#):
>
> And how can i access it?

Once it is on the msg, you can consume it directly in your function node with `msg.config.xxxx`

> [@varun](#):
>
> Can I use an array and access it?

`msg.config` will be either an array or an object. As mentioned, if you use a json file, you _could_ do either but personally I would use an object since you can then access the 4 config properties by name `msg.config.property1`, etc. Rather than having to worry about what order they are in.

---

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [15 November 2019 08:17 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/5 "2019-11-15T08:17:15Z")

</div>

I have to create a json file in which i have the varibales. I have written the below Code.

var text = '{ "configuration":[  
{"FIELD\_WIDTH":"100"},  
{"FIELD\_HEIGTH":"100"},  
{"FIELD\_HORIZONTAL":"21"}  
{"FIELD\_VERTICAL":"6"}  
]}';

var obj = JSON.parse(text);

---

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [15 November 2019 08:20 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/6 "2019-11-15T08:20:57Z")

</div>

![json](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/2X/5/572a7f1b52dac1a51ac60a6116b87f6208d01b6b.png)

This is my desired flow. I Need to get the variable used in the function node from the json file.

---

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [15 November 2019 08:25 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/7 "2019-11-15T08:25:19Z")

</div>

Is the json file i created correct? And what msg.payload be converted to in the Change node and how should i use the variable in the function node? Will this be correct FIELD\_WIDTH = obj.FIELD\_WIDTH;?  
Could you please help me with this as I am completely new to node red and could not much online that would help me? Thanks in advance.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [15 November 2019 12:01 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/8 "2019-11-15T12:01:03Z")

</div>

No, that isn't correct because you've split the flow so that you cannot know which order data will reach the function node. You need a single flow:

1. inject
2. Read the config json file
3. move the msg.payload to msg.config using a change node
4. read the csv file  
...

> [@varun](#):
>
> Is the json file i created correct?

You can just create it manually if you have access to the server filing system, easier than trying to create it.

But you've wrapped the actual fields in an array and you would only want to do that if you had multiple configurations.

```json
{ 
    "FIELD_WIDTH": "100",
    "FIELD_HEIGTH": "100",
    "FIELD_HORIZONTAL": "21",
    "FIELD_VERTICAL": "6"
}

```

Also, you don't really need quotes around the numbers unless you really want to treat them as strings. Numbers are valid data types for JSON.

You also had an error in your original, you were missing a comma. Of course, you also don't need to create it as text and then parse it, JSON is a first-class object type in JavaScript, you can just use it as-is.

I think that there may be a json file node that may help you avoid parsing and such, do a search on the flows site.

---

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [20 November 2019 08:31 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/9 "2019-11-20T08:31:01Z")

</div>

Hello, I have tried to implement what you suggested but still I am getting an error called Unexpected token c in JSON at position 2. Can you please have a look at it.

```auto
[{"id":"678e3899.0cdf4","type":"inject","z":"99a7958.4ae3d68","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":100,"wires":[["11facda2.ce0aa2"]]},{"id":"11facda2.ce0aa2","type":"fs-ops-dir","z":"99a7958.4ae3d68","name":"File location","path":"C:\\Users\\Public\\Varun","pathType":"str","filter":"","filterType":"str","dir":"files","dirType":"msg","x":310,"y":100,"wires":[["8435d0d6.264858"]]},{"id":"3bb10800.6e3de","type":"file in","z":"99a7958.4ae3d68","name":"","filename":"C:\\Users\\Public\\Varun\\PreHeating_2019-09-13_19-23-41.csv","format":"utf8","chunk":false,"sendError":false,"x":350,"y":340,"wires":[["d4fd98ff.6fd27"]]},{"id":"a4dc0c6d.b5db38","type":"function","z":"99a7958.4ae3d68","name":"Selecting the file","func":"\nmsg.filename = \"\\\\\\\\C:\\\\Users\\\\Public\\\\Varun\\\\PreHeating_2019-09-13_19-23-41.csv\";\nreturn msg;\n","outputs":1,"noerr":0,"x":630,"y":220,"wires":[["3bb10800.6e3de"]]},{"id":"3685a12.f68965e","type":"function","z":"99a7958.4ae3d68","name":"Calculate average ","func":"const inputArray = new Array (msg.payload.length);\n for (var n = 0 ; n < inputArray.length ; n++) {\n inputArray[n] = Object.values(msg.payload[n]);\n }\n\n // width and height of heating fields in pixels\n // dependant on distance between camera and object\n\n FIELD_WIDTH = msg.config.FIELD_WIDTH;\n const FIELD_HEIGTH = 100;\n\n // Width resolution of Images\n\n const RES_WIDTH = 100;\n\n // # of heating fields to calculate\n\n const FIELDS_HORIZONTAL = 6;\n\n // because of zig-zag recording of camera 1 to generate same layouts.\n\n const FIELDS_VERTICAL = 21;\n\n // first field starts half the field amount in length from middle of the image.\n\n // const HEATING_FIELD_START_HORIZONTAL = RES_WIDTH/2 - FIELDS_HORIZONTAL/2 * FIELD_WIDTH;\n const HEATING_FIELD_START_HORIZONTAL = 0;\n\n // Detect start of Fields// emperic values maybe?\n\n var HEATING_FIELD_START_DETECTION = detectStartHeating(inputArray);\n var HEATING_FIELD_END_DETECTION = detectEndHeating(inputArray);\n // var HEATING_FIELD_START_VERTICAL = (HEATING_FIELD_END_DETECTION - HEATING_FIELD_START_DETECTION )/ 2 + HEATING_FIELD_START_DETECTION - FIELDS_VERTICAL/2 * FIELD_HEIGTH;\n var HEATING_FIELD_START_VERTICAL = 0; \n\n // Creating an empty output Array with the set no fields\n\n var outputArray = new Array (FIELDS_VERTICAL);\n for (var k = 0 ; k< outputArray.length ; k++) {\n outputArray[k] = new Array (FIELDS_HORIZONTAL);\n for ( var l = 0; l< outputArray[k].length; l++) {\n outputArray[k][l] = 0;\n }\n }\n\n var tmp2 = 0;\n\n // loop through the fields\n\n for (var v = 0 ; v < FIELDS_VERTICAL ; v++) {\n for ( var h = 0; h < FIELDS_HORIZONTAL ; h++) {\n \n //calculate median of each field\n var tmp = 0 ;\n for ( var y = HEATING_FIELD_START_VERTICAL + v * FIELD_HEIGTH ; y < HEATING_FIELD_START_VERTICAL + v * FIELD_HEIGTH + FIELD_HEIGTH; y++) {\n for (var x = HEATING_FIELD_START_HORIZONTAL + h * FIELD_WIDTH; x < HEATING_FIELD_START_HORIZONTAL + h * FIELD_WIDTH + FIELD_WIDTH; x++) {\n tmp += parseFloat(inputArray[y][x]);\n }\n }\n \n // round to 2 decimals\n tmp = tmp / (FIELD_WIDTH * FIELD_HEIGTH)\n \n // no of decimal digits\n \n tmp = Math.round(tmp);\n outputArray[v][h] = tmp;\n\t\t}\n }\n \n // At this point, the array is a 21x6 matrix\n // The new output will have 6 rows\n var newOutput = new Array (FIELDS_HORIZONTAL);\n for (var k = 0 ; k<FIELDS_HORIZONTAL ; k++) {\n newOutput[k] = new Array (FIELDS_HORIZONTAL * FIELDS_VERTICAL);\n for ( var l = 0; l< newOutput[k].length; l++) {\n newOutput[k][l] = 0;\n }\n }\n\n let firstRowIndex = 0;\n for (let rowIndex = 0; rowIndex < outputArray.length; rowIndex++) {\n const row = outputArray[rowIndex];\n for (let colIndex = 0; colIndex < row.length; colIndex++) {\n const col = row[colIndex];\n // Determine the name\n const pixelName = `${getLettherForRowNumber(rowIndex)}${colIndex}`;\n newOutput[0][firstRowIndex] = pixelName;\n newOutput[1][firstRowIndex] = FIELD_WIDTH; // Width\n newOutput[2][firstRowIndex] = FIELD_HEIGTH; // Height\n newOutput[3][firstRowIndex] = colIndex * FIELD_WIDTH; // X\n newOutput[4][firstRowIndex] = rowIndex * FIELD_HEIGTH; // Y\n newOutput[5][firstRowIndex] = col; // Average\n firstRowIndex++;\n }\n }\n\t\n\tmsg.payload = newOutput;\n\t\n return msg;\n\t\nfunction detectStartHeating(inputArray) {\n\t \n\t// Haet difference from cool row to first heated row, how much hotter does the first heated row to be ?\n \n const THRESHOLD = 5;\n\n var hotRow = 0 ;\n var currentRowAverage = calcRowAverage(inputArray, hotRow);\n hotRow++;\n \n var nextRowAverage = calcRowAverage(inputArray,hotRow);\n while( currentRowAverage + THRESHOLD > nextRowAverage && currentRowAverage > -1 && nextRowAverage > -1) {\n\n currentRowAverage = nextRowAverage;\n hotRow++;\n nextRowAverage = calcRowAverage(inputArray,hotRow);\n }\n if (nextRowAverage == -1 || currentRowAverage == -1) {\n return 0;\n } else {\n return hotRow;\n }\n }\n\nfunction detectEndHeating(inputArray) {\n // heat difference from cool row to first heated row , how much hotter does the first heated row have to on average?\n \n const THRESHOLD = 40;\n var hotRow = inputArray.length - 1;\n var currentRowAverage = calcRowAverage(inputArray, hotRow);\n hotRow--;\n\n var prevRowAverage = calcRowAverage(inputArray,hotRow);\n\n while ( currentRowAverage + THRESHOLD > prevRowAverage && currentRowAverage > -1 && prevRowAverage > -1 && hotRow>0) {\n currentRowAverage = prevRowAverage;\n hotRow--;\n prevRowAverage = calcRowAverage(inputArray,hotRow);\n \n }\n if( prevRowAverage == -2 || currentRowAverage == -1) {\n return 0 ;\n }\n\n else\n {\n return hotRow;\n }\n }\n \nfunction calcRowAverage(inputArray,row) {\n var res = 0;\n if (inputArray.length < row) {\n return -1;\n }\n\n\n for (var g = 0; g< inputArray[row].length; g++) {\n res += inputArray[row][g];\n }\n return res/inputArray[row].length;\n}\t \n\nfunction getLettherForRowNumber(rowNumber) {\n // ASCII code for 'a' is 97, 122 for 'z'\n if (rowNumber < 26) {\n return String.fromCharCode(97 + rowNumber);\n } else {\n const asciiFirstLetter = Math.round(rowNumber / 25) - 1;\n const mod = rowNumber % 25 - 1;\n const firstLetter = String.fromCharCode(97 + asciiFirstLetter);\n const secondLetter = String.fromCharCode(97 + mod);\n return `${firstLetter}${secondLetter}`;\n }\n}","outputs":1,"noerr":0,"x":290,"y":480,"wires":[["31a3b225.c03346"]]},{"id":"31a3b225.c03346","type":"debug","z":"99a7958.4ae3d68","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":590,"y":480,"wires":[]},{"id":"d4fd98ff.6fd27","type":"csv","z":"99a7958.4ae3d68","name":"ReadCSV","sep":";","hdrin":"","hdrout":"","multi":"mult","ret":"\\n","temp":"","skip":"0","x":760,"y":340,"wires":[["3685a12.f68965e"]]},{"id":"8435d0d6.264858","type":"function","z":"99a7958.4ae3d68","name":"Selecting the file","func":"\nmsg.filename = \"\\\\\\\\C:\\\\Users\\\\Public\\\\Varun\\\\config.json\";\nreturn msg;\n","outputs":1,"noerr":0,"x":510,"y":100,"wires":[["c9bb7218.572258"]]},{"id":"c9bb7218.572258","type":"file in","z":"99a7958.4ae3d68","name":"","filename":"C:\\Users\\Public\\Varun\\config.json","format":"utf8","chunk":false,"sendError":false,"x":780,"y":100,"wires":[["4c2e8d38.1cd0a4"]]},{"id":"a0970e0b.a00a2","type":"change","z":"99a7958.4ae3d68","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"msg.config\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":360,"y":240,"wires":[["a4dc0c6d.b5db38"]]},{"id":"4c2e8d38.1cd0a4","type":"json","z":"99a7958.4ae3d68","name":"","property":"payload","action":"","pretty":false,"x":150,"y":240,"wires":[["a0970e0b.a00a2"]]}]

```

---

<div class="post-metadata">

**Author:** ![varun](https://avatars.discourse-cdn.com/v4/letter/v/e9bcb4/32.png) [@varun](https://discourse.nodered.org/u/varun)\
**Post date:** [20 November 2019 08:36 UTC](https://discourse.nodered.org/t/read-data-from-two-different-csv-files-as-input-for-a-function-node/17933/10 "2019-11-20T08:36:23Z")

</div>

{ config":{  
"FIELD\_WIDTH":100,"FIELD\_HEIGTH":100,"FIELD\_HORIZONTAL":2,  
"FIELD\_VERTICAL":6}  
}

@TotallyInformation This is the edited Json file.