Can CSV being used as a profile in a graph


#21

I already do that. That is the temperature to reach and set by the ESP. The values in the csv are the profile of the supplier. So I can see if it's following this profile.

Frank


#22

I don't understand why they are not the same thing.


#23

I would like to see someting like this as example in the dashboard:

image


#24

OK, I don't understand why you are not using the profile curve as the setpoint to follow, but that is up to you.
The way I would solve this is to start with a Join node set to generate key/value pairs and into this feed the csv data on one path and a trigger, once a second or once each time you get a new pv, or whatever, on the other path. Then feed that into a function node. In there you will have access to the csv curve and, if the message topic is that for the trigger input, then you will know that a value from the curve is required. Then the right point on the curve can be found and the current value passed on to put on the chart.


#25

Hi Colin,

Thanks. I was not allowed to response due to the fact I am a new user and my replies are limited. I don't do it in the ESP because of memory reserve and easy of use. It's a pain to get some data files stored on the EPS. Due to OTA i am limited to 500K. So i thought why don't fix this in Node-Red. Build someting like you suggested. Was not capable to make a join. I store the time in a flow variable and from the csv I take the right message. This is not optimal because every time a message with the temperature comes in all lines of the csv are checked. I got an idea to store the array in a flow variable and read the array but my programming experience are not that well so I gave up after 6 hours of trying.

Hope someone comes with a good example to store the csv as a array. Based on the index which are matching the time get the value out of it an push it into the dashboard.

Frank


#26

I was thinking you could generate the current profile setpoint in node-red and pass it down to the ESP as the current setpoint, rather than pass the whole profile down. You may have good reasons for not wanting to do that though.
So for charting the profile in real time you could start with something like this

[{"id":"29202d62.2414ea","type":"inject","z":"cfa39195.99bef","name":"Get file","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":90,"y":546,"wires":[["1d7d4e90.1985f1"]]},{"id":"1d7d4e90.1985f1","type":"file in","z":"cfa39195.99bef","name":"","filename":"/path/to/profile.csv","format":"utf8","chunk":false,"sendError":false,"x":259.5,"y":545,"wires":[["5e84b499.467634"]]},{"id":"5e84b499.467634","type":"csv","z":"cfa39195.99bef","name":"","sep":",","hdrin":"","hdrout":"","multi":"mult","ret":"\\n","temp":"","skip":"0","x":272.5,"y":589,"wires":[["31e94c8a.bc689c"]]},{"id":"454e9bde.2b681c","type":"debug","z":"cfa39195.99bef","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":705,"y":574,"wires":[]},{"id":"31e94c8a.bc689c","type":"change","z":"cfa39195.99bef","name":"topic: csv","rules":[{"t":"set","p":"topic","pt":"msg","to":"csv","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":410.5,"y":589,"wires":[["837fa64c.1e87f"]]},{"id":"837fa64c.1e87f","type":"join","z":"cfa39195.99bef","name":"","mode":"custom","build":"object","property":"payload","propertyType":"msg","key":"topic","joiner":"\\n","joinerType":"str","accumulate":true,"timeout":"","count":"2","reduceRight":false,"reduceExp":"","reduceInit":"","reduceInitType":"","reduceFixup":"","x":552.5,"y":636,"wires":[["454e9bde.2b681c","f5828d81.87dcc"]]},{"id":"ced9bf3c.b771d8","type":"inject","z":"cfa39195.99bef","name":"Tick","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":89,"y":648,"wires":[["7e40e90d.0305c8"]]},{"id":"f5828d81.87dcc","type":"function","z":"cfa39195.99bef","name":"Calculate value","func":"if (msg.topic === \"csv\") {\n    // this is a message containing the csv data\n    context.set(\"profile\", msg.payload.csv)\n    msg = null\n} else {\n    // otherwise it is a tick message\n    // get the csv array\n    var csv = context.get(\"profile\")\n    // timestamp is in msg.payload.tick\n    // do the stuff\n    msg.payload = \"some calculated value\"\n}\nreturn msg;","outputs":1,"noerr":0,"x":728.5,"y":636,"wires":[[]]},{"id":"7e40e90d.0305c8","type":"change","z":"cfa39195.99bef","name":"topic: tick","rules":[{"t":"set","p":"topic","pt":"msg","to":"tick","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":406,"y":647,"wires":[["837fa64c.1e87f"]]}]

You may have to tweak the csv node to match your data. I have assumed comma separated. The join won't output anything till it has received both the file and a tick input, then it passes the data on to the function node each time there is a tick.


#27

Wauw Colin what nice of you to do. I tried this one yesterday and was not able to implement it. I did an other implementation. But my node-red refuses to respond after reading 255 messages every time i update the temperature. So with a fresh start today I did the following:

So I read on starting the profile the csv into context memory of the flow and set start timer to 0:

var ProfileData=flow.get('ProfileData') || 0;
var ProfileTime=flow.get('ProfileTime') || 0;

flow.set('ProfileData',msg.payload);
flow.set('ProfileTime',0);
return null;

When there is an update from the current temperature in the reflow I read based on the time between the messages from the object array out of memory and get the profile temperature.

var ProfileData=flow.get('ProfileData') || 0;
var ProfileTime=flow.get('ProfileTime') || 0;

ProfileTime = Math.min(ProfileData.length-1,ProfileTime + Math.round(msg.payload/1000));
var ProfileTemp = ProfileData[ProfileTime].ProfileTemp;

msg = {payload:ProfileTemp};
msg.topic = 'Profile Temperature';

flow.set('ProfileTime',ProfileTime);

return msg;

Then i feed this into the dashboard and it final works.

Thanks for thinking with me and the file you made. Great you helped me in how to solve this issue. Without you I had no idea how to fix this. On the fly learned about join node, interval node and context variables. Even accessing o JSON object was new for me :wink:

I didn't know about the change node before the join. Great idea.

Thank you very much.
Frank


#28

Using the Join node makes for a much neater solution than using flow context I think. There may be times when flow or global context may be useful, but I have never used them (except for fixed global data set in settings.js).


#29

Hi Colin,

Also very nice solution except node red has to join 255 messages with every update of the temperature. In 10 minutes about 30.000 messages from the CSV. But from a total flow idea I get your point it's more message a like thinking.

Really appreciate your help and effort.
Thanks.

Frank


#30

I don't understand what you mean, where are these 255 messages that have to be joined each time? For a start there is only one message from the file node and one from the csv node, and it is only sent once, not each time.


#31

To clarify my point, you only need to inject into the file node at the start of the profile. The file node reads the whole file and passes it as one message to the csv node, which builds an array of points and passes it to the Join node which remembers it in the node context. A reference to the array is then passed to the function node each time a tick comes in. It does not even copy the array so there are no significant overheads at each tick, the array is available in the function node for a lookup of the current time into the profile.


#32

Hi Colin thanks. It’s my incompetence to get it right at once :face_with_raised_eyebrow:

Many thanks
Frank