Read data from two different csv files as input for a function node

I have a function node which has a few variables. They have to be altered depending on the useage. Therefore i dont anyone to alter the Code .Instead I want to read these values from csv file so that changes can easily be made in that file. The Problem is I am already reading the Input data from another csv file. So how can I proceed with this? Please help me out.

Well, this is partly a timing problem. How often will the config be updated? Will config changes happen while the flow is active?

One approach might be to have a separate flow that reads the config csv into a flow or global variable.

Another approach would be to read the csv in-line with the flow but put it onto msg.config or something similar rather than msg.payload. Then your function node always gets the latest version whenever it receives a msg. However, that is quite inefficient and slow.

The config will only be updated once at the begining before starting the flow. How can i assign the 4 values to to say msg.config? And how can i access it? Can I use an array and access it?

Change node if they are in a flow/global variable. To get them from a file, a csv node followed by a change node to move the input from msg.payload to msg.config. You might find it easier though to use a JSON file since it means that you can just copy from msg.payload to msg.config and don't have to mess with the complexities that a csv-in will leave you with.

Once it is on the msg, you can consume it directly in your function node with msg.config.xxxx

msg.config will be either an array or an object. As mentioned, if you use a json file, you could do either but personally I would use an object since you can then access the 4 config properties by name msg.config.property1, etc. Rather than having to worry about what order they are in.

I have to create a json file in which i have the varibales. I have written the below Code.

var text = '{ "configuration":[
{"FIELD_WIDTH":"100"},
{"FIELD_HEIGTH":"100"},
{"FIELD_HORIZONTAL":"21"}
{"FIELD_VERTICAL":"6"}
]}';

var obj = JSON.parse(text);

This is my desired flow. I Need to get the variable used in the function node from the json file.

Is the json file i created correct? And what msg.payload be converted to in the Change node and how should i use the variable in the function node? Will this be correct FIELD_WIDTH = obj.FIELD_WIDTH;?
Could you please help me with this as I am completely new to node red and could not much online that would help me? Thanks in advance.

No, that isn't correct because you've split the flow so that you cannot know which order data will reach the function node. You need a single flow:

  1. inject
  2. Read the config json file
  3. move the msg.payload to msg.config using a change node
  4. read the csv file
    ...

You can just create it manually if you have access to the server filing system, easier than trying to create it.

But you've wrapped the actual fields in an array and you would only want to do that if you had multiple configurations.

{ 
    "FIELD_WIDTH": "100",
    "FIELD_HEIGTH": "100",
    "FIELD_HORIZONTAL": "21",
    "FIELD_VERTICAL": "6"
}

Also, you don't really need quotes around the numbers unless you really want to treat them as strings. Numbers are valid data types for JSON.

You also had an error in your original, you were missing a comma. Of course, you also don't need to create it as text and then parse it, JSON is a first-class object type in JavaScript, you can just use it as-is.

I think that there may be a json file node that may help you avoid parsing and such, do a search on the flows site.

Hello, I have tried to implement what you suggested but still I am getting an error called Unexpected token c in JSON at position 2. Can you please have a look at it.

[{"id":"678e3899.0cdf4","type":"inject","z":"99a7958.4ae3d68","name":"","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":100,"wires":[["11facda2.ce0aa2"]]},{"id":"11facda2.ce0aa2","type":"fs-ops-dir","z":"99a7958.4ae3d68","name":"File location","path":"C:\\Users\\Public\\Varun","pathType":"str","filter":"","filterType":"str","dir":"files","dirType":"msg","x":310,"y":100,"wires":[["8435d0d6.264858"]]},{"id":"3bb10800.6e3de","type":"file in","z":"99a7958.4ae3d68","name":"","filename":"C:\\Users\\Public\\Varun\\PreHeating_2019-09-13_19-23-41.csv","format":"utf8","chunk":false,"sendError":false,"x":350,"y":340,"wires":[["d4fd98ff.6fd27"]]},{"id":"a4dc0c6d.b5db38","type":"function","z":"99a7958.4ae3d68","name":"Selecting the file","func":"\nmsg.filename = \"\\\\\\\\C:\\\\Users\\\\Public\\\\Varun\\\\PreHeating_2019-09-13_19-23-41.csv\";\nreturn msg;\n","outputs":1,"noerr":0,"x":630,"y":220,"wires":[["3bb10800.6e3de"]]},{"id":"3685a12.f68965e","type":"function","z":"99a7958.4ae3d68","name":"Calculate average ","func":"const inputArray = new Array (msg.payload.length);\n  for (var n = 0 ; n < inputArray.length ; n++) {\n    inputArray[n] = Object.values(msg.payload[n]);\n  }\n\n  // width and height of heating fields in pixels\n  // dependant on distance between camera and object\n\n  FIELD_WIDTH = msg.config.FIELD_WIDTH;\n  const FIELD_HEIGTH = 100;\n\n  // Width resolution of Images\n\n  const RES_WIDTH = 100;\n\n  // # of heating fields to calculate\n\n  const FIELDS_HORIZONTAL = 6;\n\n  // because of zig-zag recording of camera 1 to generate same layouts.\n\n  const FIELDS_VERTICAL = 21;\n\n  // first field starts half the field amount in length from middle of the image.\n\n // const HEATING_FIELD_START_HORIZONTAL = RES_WIDTH/2 - FIELDS_HORIZONTAL/2 * FIELD_WIDTH;\n  const HEATING_FIELD_START_HORIZONTAL = 0;\n\n  // Detect start of Fields// emperic values maybe?\n\n  var HEATING_FIELD_START_DETECTION = detectStartHeating(inputArray);\n  var HEATING_FIELD_END_DETECTION = detectEndHeating(inputArray);\n   // var HEATING_FIELD_START_VERTICAL = (HEATING_FIELD_END_DETECTION - HEATING_FIELD_START_DETECTION )/ 2 + HEATING_FIELD_START_DETECTION - FIELDS_VERTICAL/2 * FIELD_HEIGTH;\n  var HEATING_FIELD_START_VERTICAL = 0;   \n\n  // Creating an empty output Array with the set no fields\n\n  var outputArray = new Array (FIELDS_VERTICAL);\n  for (var k = 0 ; k< outputArray.length ; k++) {\n    outputArray[k] = new Array (FIELDS_HORIZONTAL);\n    for ( var l = 0; l< outputArray[k].length; l++) {\n      outputArray[k][l] = 0;\n    }\n  }\n\n  var tmp2 = 0;\n\n  // loop through the fields\n\n  for (var v = 0 ; v < FIELDS_VERTICAL ; v++) {\n    for ( var h = 0; h < FIELDS_HORIZONTAL ; h++) {\n   \n      //calculate median of each field\n      var tmp = 0 ;\n      for ( var y = HEATING_FIELD_START_VERTICAL + v * FIELD_HEIGTH ; y < HEATING_FIELD_START_VERTICAL + v * FIELD_HEIGTH + FIELD_HEIGTH; y++) {\n        for (var  x = HEATING_FIELD_START_HORIZONTAL + h * FIELD_WIDTH; x < HEATING_FIELD_START_HORIZONTAL + h * FIELD_WIDTH + FIELD_WIDTH; x++) {\n          tmp += parseFloat(inputArray[y][x]);\n        }\n      }\n          \n      // round to 2 decimals\n      tmp = tmp / (FIELD_WIDTH * FIELD_HEIGTH)\n        \n      // no of decimal digits\n      \n      tmp = Math.round(tmp);\n      outputArray[v][h] = tmp;\n\t\t}\n  }\n  \n  // At this point, the array is a 21x6 matrix\n  // The new output will have 6 rows\n  var newOutput = new Array (FIELDS_HORIZONTAL);\n  for (var k = 0 ; k<FIELDS_HORIZONTAL ; k++) {\n    newOutput[k] = new Array (FIELDS_HORIZONTAL * FIELDS_VERTICAL);\n    for ( var l = 0; l< newOutput[k].length; l++) {\n      newOutput[k][l] = 0;\n    }\n  }\n\n  let firstRowIndex = 0;\n  for (let rowIndex = 0; rowIndex < outputArray.length; rowIndex++) {\n    const row = outputArray[rowIndex];\n    for (let colIndex = 0; colIndex < row.length; colIndex++) {\n      const col = row[colIndex];\n      // Determine the name\n      const pixelName = `${getLettherForRowNumber(rowIndex)}${colIndex}`;\n      newOutput[0][firstRowIndex] = pixelName;\n      newOutput[1][firstRowIndex] = FIELD_WIDTH; // Width\n      newOutput[2][firstRowIndex] = FIELD_HEIGTH; // Height\n      newOutput[3][firstRowIndex] = colIndex * FIELD_WIDTH; // X\n      newOutput[4][firstRowIndex] = rowIndex * FIELD_HEIGTH; // Y\n      newOutput[5][firstRowIndex] = col; // Average\n      firstRowIndex++;\n    }\n  }\n\t\n\tmsg.payload = newOutput;\n\t\n  return msg;\n\t\nfunction detectStartHeating(inputArray) {\n\t \n\t// Haet difference from cool row to first heated row, how much hotter does the first heated row to be ?\n    \n  const THRESHOLD = 5;\n\n  var hotRow =  0 ;\n  var currentRowAverage = calcRowAverage(inputArray, hotRow);\n  hotRow++;\n      \n  var nextRowAverage = calcRowAverage(inputArray,hotRow);\n  while( currentRowAverage + THRESHOLD > nextRowAverage && currentRowAverage > -1 && nextRowAverage > -1) {\n\n    currentRowAverage = nextRowAverage;\n    hotRow++;\n    nextRowAverage =  calcRowAverage(inputArray,hotRow);\n  }\n    if (nextRowAverage == -1 || currentRowAverage == -1) {\n      return 0;\n    } else {\n      return hotRow;\n    }\n  }\n\nfunction  detectEndHeating(inputArray) {\n // heat difference from cool row to first heated row , how much hotter does the first heated row have to on average?\n \n   const THRESHOLD =  40;\n   var hotRow = inputArray.length - 1;\n   var currentRowAverage = calcRowAverage(inputArray, hotRow);\n   hotRow--;\n\n    var prevRowAverage =  calcRowAverage(inputArray,hotRow);\n\n    while ( currentRowAverage + THRESHOLD > prevRowAverage && currentRowAverage > -1 && prevRowAverage > -1 && hotRow>0) {\n         currentRowAverage = prevRowAverage;\n         hotRow--;\n         prevRowAverage = calcRowAverage(inputArray,hotRow);\n \n         }\n         if( prevRowAverage == -2 || currentRowAverage == -1) {\n             return 0 ;\n         }\n\n          else\n         {\n           return hotRow;\n       }\n   }\n   \nfunction calcRowAverage(inputArray,row) {\n     var res = 0;\n     if (inputArray.length < row) {\n         return -1;\n     }\n\n\n     for (var g = 0; g< inputArray[row].length; g++) {\n         res += inputArray[row][g];\n       }\n      return res/inputArray[row].length;\n}\t  \n\nfunction getLettherForRowNumber(rowNumber) {\n  // ASCII code for 'a' is 97, 122 for 'z'\n  if (rowNumber < 26) {\n    return String.fromCharCode(97 + rowNumber);\n  } else {\n    const asciiFirstLetter = Math.round(rowNumber / 25) - 1;\n    const mod = rowNumber % 25 - 1;\n    const firstLetter = String.fromCharCode(97 + asciiFirstLetter);\n    const secondLetter = String.fromCharCode(97 + mod);\n    return `${firstLetter}${secondLetter}`;\n  }\n}","outputs":1,"noerr":0,"x":290,"y":480,"wires":[["31a3b225.c03346"]]},{"id":"31a3b225.c03346","type":"debug","z":"99a7958.4ae3d68","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"payload","x":590,"y":480,"wires":[]},{"id":"d4fd98ff.6fd27","type":"csv","z":"99a7958.4ae3d68","name":"ReadCSV","sep":";","hdrin":"","hdrout":"","multi":"mult","ret":"\\n","temp":"","skip":"0","x":760,"y":340,"wires":[["3685a12.f68965e"]]},{"id":"8435d0d6.264858","type":"function","z":"99a7958.4ae3d68","name":"Selecting the file","func":"\nmsg.filename = \"\\\\\\\\C:\\\\Users\\\\Public\\\\Varun\\\\config.json\";\nreturn msg;\n","outputs":1,"noerr":0,"x":510,"y":100,"wires":[["c9bb7218.572258"]]},{"id":"c9bb7218.572258","type":"file in","z":"99a7958.4ae3d68","name":"","filename":"C:\\Users\\Public\\Varun\\config.json","format":"utf8","chunk":false,"sendError":false,"x":780,"y":100,"wires":[["4c2e8d38.1cd0a4"]]},{"id":"a0970e0b.a00a2","type":"change","z":"99a7958.4ae3d68","name":"","rules":[{"t":"set","p":"payload","pt":"msg","to":"msg.config\t","tot":"jsonata"}],"action":"","property":"","from":"","to":"","reg":false,"x":360,"y":240,"wires":[["a4dc0c6d.b5db38"]]},{"id":"4c2e8d38.1cd0a4","type":"json","z":"99a7958.4ae3d68","name":"","property":"payload","action":"","pretty":false,"x":150,"y":240,"wires":[["a0970e0b.a00a2"]]}]

{ config":{
"FIELD_WIDTH":100,"FIELD_HEIGTH":100,"FIELD_HORIZONTAL":2,
"FIELD_VERTICAL":6}
}

@TotallyInformation This is the edited Json file.