Looking for suggestions on how to condense an array of arrays (nRF24L01 scan results)

[zenofmud]

I'm looking for some ideas (before I go reinventing an existing solution) on how to process some data.

Background: I'm playing with the nRF24L01+ transceiver and scanning the channels available. There is a program that will repetitively scan the 128 channels (2.4GHz) and for each channel, listen 100 times and then output the count (up to 10) for anything it receives. Si I get a list like this (shortened for space)

0000000000000000
0123456789ABCDEF
0200000011110100
0001100023000000
0000000015000100
0000000032010000
0001000000000000

The first two lines are the channel number in hex. In this sample, you can see that channel 8 and 9 are the most used. What I want to do is create total the number from each channel so I can create a graph to see which channels are in use.

I'm thinking I need to create a 128 element array then split each line and add each value to the associated array element. But for some reason I'm tinking there might be an easier way to do this.

Any ideas/comments are welcome.

[UnborN]

Hi Paul,

interesting .. how are the data being received in node red .. channel by channel (line by line) ? or in a big chunk of information after the scan ? Can you post a sample of the payload as it is received in node red

ps. i have no idea about receivers ..

[zenofmud]

I’m doing old school. I’ve got the Arduino IDE running and the device connected. the sketch prints out the data in the serial monitor. I copy and paste it to a text file that I’ll read in node-red

The sketch is the scanner example from the RF24 library

[UnborN]

@zenofmud

I'm thinking I need to create a 128 element array then split each line and add each value to the associated array element. But for some reason I'm tinking there might be an easier way to do this.

That was my approach also ... i dont know if this helps ..

[{"id":"2ab49846.91b01","type":"file in","z":"61cb8088.6af39","name":"","filename":"c:/temp/data.txt","format":"lines","chunk":false,"sendError":false,"encoding":"utf8","x":620,"y":280,"wires":[["2db15e2e.f0ee02","3fd92a74.b1094e"]]},{"id":"366db2c2.f6c9a6","type":"debug","z":"61cb8088.6af39","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":1040,"y":280,"wires":[]},{"id":"3fd92a74.b1094e","type":"function","z":"61cb8088.6af39","name":"","func":"let arr = [...msg.payload]   // spread string to array\narr.pop();    // trim last char - new line character ?\n\nlet channels = flow.get(\"channels\")\n\nif (msg.parts.index > 1) { // skip first two lines of file - its hex\n\narr.forEach((el, i) => {\n\n    channels[i] = channels[i] + parseInt(el);\n})\n \n\nflow.set(\"channels\", channels)\nreturn\n\n}\n\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":870,"y":280,"wires":[["366db2c2.f6c9a6"]]},{"id":"a81a0f61.b9d178","type":"function","z":"61cb8088.6af39","name":"init 0s","func":"let arr = []\n\nfor (let i = 0; i < 128; i++ ) {\n    arr[i] = 0; // fill with zeros\n}\n\nflow.set(\"channels\", arr)\nreturn msg","outputs":1,"noerr":0,"initialize":"","finalize":"","x":450,"y":280,"wires":[["2ab49846.91b01"]]},{"id":"2db15e2e.f0ee02","type":"debug","z":"61cb8088.6af39","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":850,"y":380,"wires":[]},{"id":"14e4e318.3180a5","type":"comment","z":"61cb8088.6af39","name":"set flow.channels array","info":"","x":900,"y":200,"wires":[]},{"id":"1133fbd7.d8c834","type":"inject","z":"61cb8088.6af39","name":"sum values","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":270,"y":280,"wires":[["a81a0f61.b9d178"]]},{"id":"92a769a8.2ee19","type":"function","z":"61cb8088.6af39","name":"","func":"let channels = flow.get(\"channels\")\n\nmsg.payload = channels\n\nreturn msg;","outputs":1,"noerr":0,"initialize":"","finalize":"","x":610,"y":540,"wires":[["fb205f6f.68f08"]]},{"id":"db86dadc.52b7c8","type":"inject","z":"61cb8088.6af39","name":"plot","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":360,"y":560,"wires":[["92a769a8.2ee19"]]},{"id":"fb205f6f.68f08","type":"debug","z":"61cb8088.6af39","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":820,"y":540,"wires":[]}]

ps. needs some error checking ... there is an error for array element 16 .. most probably the new line character when reading the file. could use arr.pop(); // trim new line character ?

[zenofmud]

Andy, thanks for your suggestions. I used some of your code and had to do a couple extra things. The scanner saves the data as a single hex value so the array could contain 0 thru 9 or A, B, C, D, E, or F so I had to convert them to 10-15 n the array or they woulb become NAN's in the array when converting to numbers.

Here is my first scan (1210 rows of results) graph:

and the full flow:

[{"id":"9e026685.6586b","type":"tab","label":"RF24 scanner data plotting","disabled":false,"info":""},{"id":"1308b4aa.b0e293","type":"catch","z":"9e026685.6586b","name":"","scope":null,"uncaught":false,"x":620,"y":160,"wires":[["47c86ca3.1ebf74"]]},{"id":"47c86ca3.1ebf74","type":"debug","z":"9e026685.6586b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":800,"y":160,"wires":[]},{"id":"26253aa0.1c94d6","type":"file in","z":"9e026685.6586b","name":"read data file","filename":"/Users/Paul/temp/scan_results.txt","format":"lines","chunk":false,"sendError":false,"encoding":"utf8","x":150,"y":280,"wires":[["c951dce2.55c0d"]]},{"id":"760f1171.1baa58","type":"debug","z":"9e026685.6586b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","statusVal":"","statusType":"auto","x":350,"y":420,"wires":[]},{"id":"c1c711f2.e0af5","type":"function","z":"9e026685.6586b","name":"","func":"// count the total number of scans\nlet total_scans = flow.get(\"total_scans\")||0\nflow.set(\"total_scans\",total_scans+1)\n\n// convert the payload to an numeric array\nlet data_n = [...msg.payload]   // spread string to array\n\n// need this code because the scanner saves data as one hex character\nvar l = data_n.length\n\nfor (var i=0; i<l; i++) {\n    switch (data_n[i]) {\n        case \"A\":\n            data_n[i]=\"10\"\n            break;\n        case \"B\":\n            data_n[i]=\"11\"\n            break;\n        case \"C\":\n            data_n[i]=\"12\"\n            break;\n        case \"D\":\n            data_n[i]=\"13\"\n            break;\n        case \"E\":\n            data_n[i]=\"14\"\n            break;\n        case \"F\":\n            data_n[i]=\"15\"\n            break;\n        default:\n            break;\n    }\n}\n\n //convert to to numeric array\ndata_n = data_n.map(Number)    \n\n// get/initilize the channels array\nlet channels = flow.get(\"channels\")||[]\n\n// add the new data to each channel's count   \nvar sum = []\nsum = data_n.map(function (num, idx) {\n  return num + channels[idx];\n}); \n\n//Store the new channel counts\nflow.set(\"channels\",sum)\nmsg.payload = sum\nreturn msg;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":140,"y":420,"wires":[["760f1171.1baa58"]]},{"id":"56a9dfb1.5a153","type":"debug","z":"9e026685.6586b","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","statusVal":"","statusType":"auto","x":530,"y":380,"wires":[]},{"id":"21433828.b489a","type":"inject","z":"9e026685.6586b","name":"sum values","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":150,"y":100,"wires":[["b1e486d5.9bc528"]]},{"id":"7627ebd6.d3d5fc","type":"function","z":"9e026685.6586b","name":"","func":"let channels = flow.get(\"channels\")\nlet total_scans = flow.get(\"total_scans\")\n\nvar sum = []\nsum = channels.map(function (num, idx) {\n    msg.payload = channels[idx] ///total_scans\n//    msg.payload = channels[idx]/10 ///total_scans\n//    msg.payload = channels[idx]/total_scans\n    msg.topic = 'foo'\n    msg.topic = \"ch:\"+idx\n    node.send(msg)\n    return num;\n}); \n\nreturn null;\n","outputs":1,"noerr":0,"initialize":"","finalize":"","x":760,"y":340,"wires":[["f35f2887.b1dd28"]]},{"id":"7e6cc330.65740c","type":"inject","z":"9e026685.6586b","name":"manually plot data","props":[{"p":"payload"},{"p":"topic","vt":"str"}],"repeat":"","crontab":"","once":false,"onceDelay":0.1,"topic":"","payload":"","payloadType":"date","x":790,"y":280,"wires":[["7627ebd6.d3d5fc"]]},{"id":"308ef0ce.52b3d8","type":"function","z":"9e026685.6586b","name":"init channels to 0's","func":"let arr = []\n\nfor (let i = 0; i < 128; i++ ) {\n    arr[i] = 0; // fill with zeros\n}\n\nflow.set(\"channels\", arr)\nreturn msg","outputs":1,"noerr":0,"initialize":"","finalize":"","x":170,"y":220,"wires":[["26253aa0.1c94d6"]]},{"id":"b1e486d5.9bc528","type":"change","z":"9e026685.6586b","name":"set flow.channels array and scan counter","rules":[{"t":"delete","p":"channels","pt":"flow"},{"t":"delete","p":"total_scans","pt":"flow"}],"action":"","property":"","from":"","to":"","reg":false,"x":240,"y":160,"wires":[["308ef0ce.52b3d8"]]},{"id":"c951dce2.55c0d","type":"switch","z":"9e026685.6586b","name":"ignore first two - on last line, plot the data","property":"parts.index","propertyType":"msg","rules":[{"t":"lt","v":"2","vt":"num"},{"t":"eq","v":"parts.count-1","vt":"jsonata"},{"t":"else"}],"checkall":"false","repair":false,"outputs":3,"x":240,"y":340,"wires":[[],["7627ebd6.d3d5fc"],["56a9dfb1.5a153","c1c711f2.e0af5"]]},{"id":"f35f2887.b1dd28","type":"ui_chart","z":"9e026685.6586b","name":"","group":"8a5bb62c.cf1c68","order":2,"width":"21","height":"7","label":"RF24 Scanner results","chartType":"bar","legend":"false","xformat":"auto","interpolate":"linear","nodata":"","dot":true,"ymin":"0","ymax":"2600","removeOlder":"1","removeOlderPoints":"16","removeOlderUnit":"60","cutout":0,"useOneColor":true,"useUTC":false,"colors":["#1f77b4","#aec7e8","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"useOldStyle":false,"outputs":1,"x":800,"y":400,"wires":[[]]},{"id":"383ce6a5.2d21ca","type":"comment","z":"9e026685.6586b","name":"RF24 scanner output to graph (ReadMe)","info":"This flow will take a file - created by copying the Arduino IDE Serial Monitor output from teh RF24 'Scanner' example program and pasted into a test file - and add up the data for each of the 128 scanned channels and display the results on a graph.","x":350,"y":40,"wires":[]},{"id":"bc0ca12.937786","type":"comment","z":"9e026685.6586b","name":"change path to location of the file you created","info":"","x":470,"y":280,"wires":[]},{"id":"8a5bb62c.cf1c68","type":"ui_group","z":"","name":"Default","tab":"9146457a.eda6d8","order":1,"disp":false,"width":"21","collapse":false},{"id":"9146457a.eda6d8","type":"ui_tab","z":"","name":"Home","icon":"dashboard","order":2,"disabled":false,"hidden":false}]```

[UnborN]

Very good result Paul, and very interesting project .. but those 2 dimensional arrays are a headache

@zenofmud

the array could contain 0 thru 9 or A, B, C, D, E, or F

I was wondering about that .. i thought only the first two lines had hex values.

Im glad you found some of the code useful .. all the best automating it further so you dont have to copy/paste the values in the text file