Storing Data in an Array

[TizaouiT]

Hello everyone,

I am trying to store data coming from a sensor in an array. The sensor output is actualised every 5 sec. Every time a new value comes only 1 value will be stored in my array. I am trying to save this set of data so I can use it to fit some curves to the data coming from my system.

Any ideas how can I sove this problem ?

Thanks :).

[Colin]

The contents of temp_data will be lost each time a new message comes in, the value will not be held over between messages. You will need to use the node's context to remember it. Have a look at the docs Writing Functions for details. In the meantime try replacing
temp_data = []
with
temp_data = context.get("temp_data") || []
and at the end (before the return) put
context.set("temp_data", temp_data)

Note, untested so there may be typos.

[BartButenaers]

Hi Colin,

Your example indeed works.

var myArray = context.get("myArray") || [];
myArray.push(msg.payload);
context.set("myArray", myArray);
msg.payload = myArray;
return msg;

But when I read your answer the first time, I thought: no no Colin, you don't need the contex.set because the context.get gets a reference to the array instance on the context. So when you change the instance, it will automatically be saved on the context memory also (since it is only a reference). However it seems indeed you need to call the setter... So I assume you get a new instance of the array every time you get it (??), even when you have an in-memory context store. Makes sense of course, otherwise the code would behave differently when you would start using e.g. an file-based context store...

Thanks for teaching me the basics of this
Bart

[Colin]

@BartButenaers

So I assume you get a new instance of the array every time you get it (??)

I don't think that is it. Consider what happens if the context.set() is removed. The first time through it tries to get the array, but it doesn't exist so it initialises it to an empty array. Then it pushes the payload and then returns. The next time in it tries to get it from context again, but because the set has been removed it still doesn't exist in context so again it is set to an empty array. Without the set nothing is ever written to the context so it won't work. It is obvious once you see it but it took me a little time to realise what was going on.

[BartButenaers]

@Colin

it still doesn't exist in context so again it is set to an empty array

Damn Colin,
you are right. I hadn't thought about that.
Did a little test where I only initialize the array on context memory once, and from then on I only get the array but I don't set it anymore:

var myArray = context.get("myArray");

if(!myArray) {
    console.log("Set the array on context memory");
    context.set("myArray", []);
    myArray = context.get("myArray");
}

myArray.push(msg.payload);
msg.payload = myArray;
return msg;

So the get returns a reference to the array on the context memory, which means I don't have to set it after I have changed it.

I understand that it works like this in an in-memory context storage, but not with other kind of storages.
For example when I have an File-base storage, I don't see how it could work unless there is a caching layer that keeps all objects into memory. Or I am just getting nuts

@TizaouiT: I hope you don't mind that we dive a bit deeper... Hopefully your question is already answered in our posts above? If you do it like in @Colin's example (i.e. always set the array to context after changing), it will work fine...