Can CSV being used as a profile in a graph

Dashboard
1

Hi All,

I am making a PID reflow oven with an ESP8266. I get current temperature and the ESP will do action based on a set temperature or a pre loaded curve. This is all monitored by Node Red.

I was reading the forum and bumped into this topic. Which is almost the same as I am asking but slightly different so maybe you can help me out.

From the process my reglow oven is going to set temperature in time based on PWM and control the heating elements by a set temperature. (Solder flow table). What i would like to do is load a csv file with the perfect curve and load every update time the current profile setting in the perfect curve and display it in the dashboard.

So to explain it more easy:
I get temp update every .2 seconds. These are displayed in a graph (set temp and current temp). From the CSV i have a table where for every -.2 second i do have a reference or profile temperature. Is it possible to merge this with the other messages so i can display that value in the graph?

So when a temp update comes in I generate a message with the next value in the CSV.

Thanks Frank

2

The first thing to do is to write a function that will interrogate the CSV file and give you back the current profile value. Then you can put it on the chart. Are you able to do the first bit?

3

I got a csv but it load all in once. So fields are time, temp. Every line is 1 seconds. So at button press i would like to load one entry every second generating a message with temp. This as imput for the graph.

Thanks
Frank

4

Do you want it to use the timestamps in the file to know what value to output?

5

So, if I've read this right - you are trying to update a CSV file every 200ms (0.2s) and maintain a chart with the history?

So in reality, you will need to keep the data in memory or read from the file whenever a new client connection occurs - assuming you are sending updated values to the chart in realtime as well.

But honestly, this is a lot of hard work that others have already done far better. This is exactly what InfluxDB is designed for and precisely what Grafana is designed to pick up and use in dashboards.

Updating files so rapidly means that the file won't be available for reading when you need it to be. A database will handle that problem easily. Using InfluxDB, you also get many more benefits such as being able to keep the table to a manageable size or aggregate to different time periods such as getting an average/max/min by hour.

6

@TotallyInformation I think you have this the wrong way round. The file contains a profile that he wishes to show on the chart as the process data is coming in.

7

@Frank2 does this represent the current setpoint that the ESP32 is controlling to? If so can you not get the ESP32 to report the setpoint as well as the process and plot them together?
What s/w are you running in the device?

8

though looking at the example data you showed... they are not one second apart ?

9

Hi,

The example is of another topic. I want to read the csv for display the curve of the datasheet of the solder. In the esp i use a json object wich only stores 4 points in time. If it reach a level it goes to the next. Of course i can send it from esp but i would love to have it in node red. Less memory usage no difficult uploads. Use OTA for esp so alread thight on storage. It’s very easy to get a csv in any timestamp from a curve. This data i would like to display together with the actual curve. I don’t know how to insert one line by one line and generate an outcome wich i can merge and display it in the dashboard.

So I am looking for something like this.
On button click the reflow starts.
After Period one I get actual temperature from sensor.
Read row 1 of csv and display it in the dashboard together with actual.
Repeat for next actual and next row.

If there or no lines left in csv display 0
If button is pressed agan to reflow again repeat process

Frank

10

It’s another example. I can generate any csv matching interval of reading.

11

Cirrect Collin

12

I get this. But if an easy read say every second can be merged easily I dont have to learn influxdb or something else. Also endusers don’t need to install an maintain all.

13

That might be manageable. Possibly depends how many clients are attached as to whether you would get significant clashes. Suck it and see I guess.

14

I don't think you answered the question, is the value from the csv file the current setpoint that the ESP is controlling to?

Also what s/w are you running in the ESP?

15

Hi Colin,

Sorry. I am running from the arduino IDE the IOTappStory software for over the air update. I made a sketch communicating with node-red. The software loads this profile from flash to set temperature. Max 6675 for thermocouple sensor. Software PID control of the relais.
{
"NumCurves": 2,
"Curves":
[ {
"Name": "Lead (Sn63 Pb37)",
"Points": [
[150, 60, 0],
[165, 120, 0],
[235, 25, 20],
[35, 60, 0]
]
},
{
"Name": "Lead-free (SAC305)",
"Points": [
[150, 60, 0],
[180, 120, 0],
[255, 25, 15],
[35, 70, 0]
]
}
]
}

Supported ESP code:
String handleJSON_Curve()
{
["Points"][j][0]);
String response;
StaticJsonBuffer<800> jsonBuffer;
JsonObject& root = jsonBuffer.createObject();
root["NumCurves"] = listProfiles.numProfiles;
JsonArray& curves = root.createNestedArray("Curves");

for (int i = 0; i < listProfiles.numProfiles; i++)
{
JsonObject& thisCurve = curves.createNestedObject();
thisCurve["Name"] = listProfiles.targetProfile[i].profileName;
JsonArray& points = thisCurve.createNestedArray("Points");
for (int j = 0; j < listProfiles.targetProfile[i].numPoints; j++)
{
JsonArray& thisPoint = points.createNestedArray();
thisPoint.add(listProfiles.targetProfile[i].reflowCurve[j].targetTemp);
thisPoint.add(listProfiles.targetProfile[i].reflowCurve[j].targetTime);
thisPoint.add(listProfiles.targetProfile[i].reflowCurve[j].targetHold);
}
}

It gives a message every second with the current temperature with timestamp to node-red, so i am able to display where we are. It's giving more info for dashboarding so I am can selecting a profile, PID control (0-100%) or set temperature.

Every second the message looks like this:
{"IP":"192.168.xxx.xxx","SigStrength":-40,"SWVersion":"1.0.3","NetBIOSName":"ReflowOven","useNTP":1,"NTPServer":"nl.pool.ntp.org","ntpTimeZone":1,"mem":28440,"uptime":1200,"currenttime":1552227328,"PIDParams":{"KP":7,"KI":0,"KD":2.5e8}}

The csv looks like:

1,00; 26,22
2,00; 27,54
3,00; 28,87
4,00; 30,37
5,00; 31,80
6,00; 33,10
7,00; 34,75
8,00; 36,30
9,00; 37,70
10,00; 38,99
11,00; 40,66
12,00; 42,15
13,00; 43,51
14,00; 45,16
15,00; 46,73
16,00; 48,06
17,00; 49,64
18,00; 51,36
19,00; 53,38
20,00; 54,07
21,00; 55,26

I think i can also load the csv in a array and with an counter (needs a global var i think) add a message when the message comes in from the ESP.

16

Can you not get it to send up the current setpoint along with the temperature? In fact I don't see the current temperature in the message.

17

I do have one client attached. Makes no sense in this case more clients are subscribing to the same topic. Every reflow oven needs to have it's own topic.

18

Sorry debug on the wrong node:

{"CurrMode":1,"HeatLevel":0,"SetTempC":0,"CurveNr":0,"CurrTempC":33.25,"PIDOutput":82.08334,"HeaterStatus":0}

19

This is the complete node-red code:

It uses also the dashboard add in and mqtt server local IP.

[{"id":"1852f7f5.acee98","type":"tab","label":"Reflow Oven Control","disabled":false,"info":""},{"id":"23fcdf82.f57d1","type":"mqtt out","z":"1852f7f5.acee98","name":"Heat Control","topic":"rflCmd","qos":"1","retain":"true","broker":"36f961f1.c4f7ee","x":796,"y":131,"wires":},{"id":"71727ce7.5d4154","type":"function","z":"1852f7f5.acee98","name":"Heat Control","func":"if (msg.topic == "PWMLevel")\n context.set("PWMLevel", msg.payload);\nif (msg.topic == "WorkMode")\n context.set("WorkMode", msg.payload);\nif (msg.topic == "TargetTemp")\n context.set("TargetTemp", msg.payload);\nif (msg.topic == "CurveSel")\n context.set("CurveSel", msg.payload);\n\nvar workMode = context.get("WorkMode");\nif (workMode === undefined) \n workMode = "0";\nvar pwmLevel = context.get("PWMLevel");\nif (pwmLevel === undefined) \n pwmLevel = 0;\nvar targetTemp = context.get("TargetTemp");\nif (targetTemp === undefined) \n targetTemp = 0;\nvar activeProfile = context.get("CurveSel");\nif (activeProfile === undefined) \n activeProfile = -1;\n\nvar payloadStr = "{\"WorkMode\":\"" + workMode + "\"";\nswitch (workMode)\n{\n case 0: \n break;\n case 1: //PWM\n payloadStr += ", \"HeatLevel\": " + pwmLevel;\n break;\n case 2: //PID\n payloadStr += ", \"TargetTemp\": " + targetTemp;\n break;\n case 3: //PID\n payloadStr += ", \"SelectProfile\": " + activeProfile;\n if (msg.topic == "StartReflow")\n payloadStr += ", \"RunProfile\": 1";\n else\n payloadStr += ", \"RunProfile\": 0";\n break;\n}\npayloadStr += "}"\n\n\nmsg = {payload:payloadStr}\nreturn msg;\n","outputs":1,"noerr":0,"x":505.00002670288086,"y":199.00001907348633,"wires":[["23fcdf82.f57d1","f3c06f42.aa0a8"]]},{"id":"6c0813de.087a1c","type":"ui_dropdown","z":"1852f7f5.acee98","name":"","label":"Work Mode","place":"Select option","group":"c319e987.2d27c8","order":1,"width":"12","height":"1","passthru":false,"options":[{"label":"OFF","value":0,"type":"num"},{"label":"PWM","value":1,"type":"num"},{"label":"PID","value":2,"type":"num"},{"label":"Reflow Curve","value":3,"type":"num"}],"payload":"","topic":"WorkMode","x":173.07987213134766,"y":154.93063354492188,"wires":[["71727ce7.5d4154"]]},{"id":"3cef0d47.118d32","type":"ui_slider","z":"1852f7f5.acee98","name":"","label":"Target Temperature","group":"c319e987.2d27c8","order":4,"width":"12","height":"1","passthru":true,"topic":"TargetTemp","min":0,"max":"250","step":1,"x":188.07638549804688,"y":249.8577117919922,"wires":[["71727ce7.5d4154"]]},{"id":"99261193.e928d","type":"ui_slider","z":"1852f7f5.acee98","name":"","label":"PWM Level","group":"c319e987.2d27c8","order":3,"width":"12","height":"1","passthru":true,"topic":"PWMLevel","min":0,"max":"100","step":1,"x":170.0833282470703,"y":198.8611297607422,"wires":[["71727ce7.5d4154"]]},{"id":"fa65d0bf.5ed78","type":"ui_dropdown","z":"1852f7f5.acee98","name":"","label":"Select Curve","place":"Select Curve","group":"c319e987.2d27c8","order":2,"width":"12","height":"1","passthru":true,"options":[{"label":"Lead (Sn63 Pb37)","value":0,"type":"num"},{"label":"Lead-free (SAC305)","value":1,"type":"num"}],"payload":"","topic":"CurveSel","x":166,"y":302,"wires":[["71727ce7.5d4154"]]},{"id":"a8115f96.fd4ce","type":"mqtt in","z":"1852f7f5.acee98","name":"","topic":"dataReflow","qos":"1","broker":"36f961f1.c4f7ee","x":149.00000762939453,"y":457.99998664855957,"wires":[["3fd84da8.6ad3c2","98d18022.15d8a"]]},{"id":"441c322d.8d0bec","type":"ui_chart","z":"1852f7f5.acee98","name":"","group":"c319e987.2d27c8","order":6,"width":"24","height":"8","label":"Current Temperature","chartType":"line","legend":"true","xformat":"HH:mm:ss","interpolate":"linear","nodata":"","dot":false,"ymin":"0","ymax":"300","removeOlder":"15","removeOlderPoints":"","removeOlderUnit":"60","cutout":0,"useOneColor":false,"colors":["#1f77b4","#e81a2a","#ff7f0e","#2ca02c","#98df8a","#d62728","#ff9896","#9467bd","#c5b0d5"],"useOldStyle":false,"outputs":1,"x":770.632080078125,"y":419.96697998046875,"wires":[]},{"id":"3fd84da8.6ad3c2","type":"json","z":"1852f7f5.acee98","name":"","property":"payload","action":"","pretty":false,"x":341.632080078125,"y":457.96697998046875,"wires":[["48eb1067.15b8f","d5242d74.7cb4b"]]},{"id":"48eb1067.15b8f","type":"function","z":"1852f7f5.acee98","name":"Output","func":"if (msg.payload.CurrTempC >= 0)\n { msg1 = {payload:msg.payload.CurrTempC}\n msg1.topic = 'Current Temperature'\n }\nelse\n msg1 = null\nif (msg.payload.SetTempC >= 0)\n {msg2 = {payload:msg.payload.SetTempC}\n msg2.topic = 'Set Temperature'}\nelse\n msg2 = null\nif (msg.payload.HeatLevel >= 0 )\n msg3 = {payload:msg.payload.HeatLevel}\nelse\n msg3 = null\nif (msg.payload.HeaterStatus >= 0)\n msg4 = {payload:msg.payload.HeaterStatus}\nelse\n msg4 = null\nif (msg.payload.VoiceMsg > 0)\n switch (msg.payload.VoiceMsg)\n {\n case 1: msg5 = {payload:"Please open the oven and let it cool off"}; break;\n case 2: msg5 = {payload:"Profile Completed. Please remove and verify parts"}; break;\n default: msg5 = null;\n }\n \nelse\n msg5 = null\n \nreturn [msg1, msg2, msg3, msg4, msg5];","outputs":5,"noerr":0,"x":523.6320877075195,"y":458.9669704437256,"wires":[["441c322d.8d0bec"],["718aa450.5e10ec","441c322d.8d0bec"],["1e8652d4.4be2dd"],["2137b98a.f3b786"],["323941a8.c430ee"]]},{"id":"1e8652d4.4be2dd","type":"ui_gauge","z":"1852f7f5.acee98","name":"","group":"c319e987.2d27c8","order":7,"width":"8","height":"4","gtype":"gage","title":"PWM %","label":"%","format":"{{value}}","min":0,"max":"100","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":737.7050552368164,"y":496.7968330383301,"wires":},{"id":"718aa450.5e10ec","type":"ui_gauge","z":"1852f7f5.acee98","name":"","group":"c319e987.2d27c8","order":9,"width":"8","height":"4","gtype":"gage","title":"Target Temp","label":"Deg C","format":"{{value}}","min":0,"max":"300","colors":["#00b500","#e6e600","#ca3838"],"seg1":"","seg2":"","x":744.7050514221191,"y":461.8072500228882,"wires":},{"id":"34e62921.5cfec6","type":"ui_button","z":"1852f7f5.acee98","name":"Start Reflow","group":"c319e987.2d27c8","order":5,"width":"3","height":"1","passthru":false,"label":"Start Reflow","tooltip":"","color":"","bgcolor":"","icon":"","payload":"1","payloadType":"num","topic":"StartReflow","x":165,"y":346,"wires":[["71727ce7.5d4154"]]},{"id":"323941a8.c430ee","type":"ui_audio","z":"1852f7f5.acee98","name":"","group":"c319e987.2d27c8","voice":"nl-BE","always":true,"x":738,"y":580,"wires":},{"id":"d604fda1.e1278","type":"mqtt in","z":"1852f7f5.acee98","name":"","topic":"pingReflow","qos":"1","broker":"36f961f1.c4f7ee","x":151.00000762939453,"y":399,"wires":[["f3c06f42.aa0a8"]]},{"id":"2137b98a.f3b786","type":"ui_template","z":"1852f7f5.acee98","group":"c319e987.2d27c8","name":"Oven ON","order":8,"width":"8","height":"4","format":"<md-button class="vibrate filled touched bigfont rounded" style="background-color:#FFFFFF" ng-click="send({payload: msg.payload })"> \n\n\n<svg width="260px" height="90px" version="1.1" viewBox="0 0 800 200">\n <g id="Button_Long">\n \n <rect fill="#FFFFFF" width="800" height="200"/>\n <g ng-style="{fill: (msg.payload || 0) ? 'lime' : 'red'}">\n <rect width="800" height="200" rx="80" ry="80"/>\n \n \n <g ng-style="{fill: (msg.payload || 0) ? 'lime' : 'red'}">\n <rect x="11" y="10" width="778" height="180" rx="90" ry="90"/>\n \n <g ng-style="{fill: (msg.payload || 0) ? 'black' : 'white'}">\n \n <text x="400" y="125" style="text-anchor:middle" font-weight="bold" font-size="80" font-family="Arial">{{(msg.payload||0)? "ON" : "OFF"}} \n \n \n\n\n\n\n","storeOutMessages":false,"fwdInMessages":false,"templateScope":"local","x":738,"y":539,"wires":[]},{"id":"98d18022.15d8a","type":"debug","z":"1852f7f5.acee98","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":349,"y":612,"wires":},{"id":"d5242d74.7cb4b","type":"debug","z":"1852f7f5.acee98","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1033,"y":245,"wires":},{"id":"f3c06f42.aa0a8","type":"debug","z":"1852f7f5.acee98","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":1032,"y":200,"wires":},{"id":"36f961f1.c4f7ee","type":"mqtt-broker","z":"","name":"Air-van-Frank","broker":"Air-van-Frank","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"c319e987.2d27c8","type":"ui_group","z":"","name":"Reflow Oven Control","tab":"21e9fb8e.a29114","order":1,"disp":true,"width":"24","collapse":false},{"id":"21e9fb8e.a29114","type":"ui_tab","z":"","name":"Reflow Oven","icon":"dashboard","order":5}]

20

Isn't that the value you want to plot along with CurrTempC?