Confusion regarding retrieving array from context

I've run into a bit of confusion...

I'm working with some rather large CSV files of time series data with a bit of overlap. This is patient data so I can't share exact examples, but what I'm trying to do is combine many months of vital signs alarms but also separate each bed and alarm type into its own context variable (bed-1-SPO2, bed-1-tachycardia, bed-2-SPO2, etc.)

I've been pretty successful so far with this by splitting the input file into a stream of messages assigning topics based on the content of specific columns, and applying the following function:

if (flow.get(msg.topic) === undefined){
flow.set((msg.topic), msg.payload);
}
else {
var oldval = flow.get(msg.topic)
var newval = msg.payload
var finalval = oldval.concat(newval)
var deduped = Array.from(new Set(finalval));
msg.payload = deduped
flow.set((msg.topic), msg.payload)}
return msg;

which I'm using to check if any data of this type has already been saved and if there is any add the new data to it and remove any identical rows to eliminate overlap. From what I can tell from the context explorer, this portion seems to be working like a charm and my resulting stored variable looks to be an array of strings.

Now comes my problem: When I attempt to call back this data using a function node containing node.send({"payload":flow.get(msg.payload)}); what I expect is an object containing that same array of strings, but what I'm actually geting is an object containing an array of single element arrays with the expected string living in element 0. I feel like I have to be overlooking something simple, but my limited experience has me for a loss as to what it could be.

Are you using that node.send in a different function? In my playing it looks the same using a new function.
What's interesting is the first time a new topic appears (say 'topicA'), the payload (call it 'cat') is written out as a string. cat. When a second msg with 'topicA' - payload 'frog' then end result is
["c", "a", "t", "f", "r", "o", "g"]
If another msg with 'topicA' - payload 'cat' comes in you end up with
["c", "a", "t", "f", "r", "o", "g", "cat"]
and if another msg with 'topicA' - payload 'frog' comes in, you get
["c", "a", "t", "f", "r", "o", "g", "cat", "frog"]
Is that what you are expecting?

test flow:

[{"id":"e2c81220.152bf","type":"function","z":"6223ebc3.0ec744","name":"use node.send","func":"node.send({\"payload\":flow.get(msg.payload)});\nreturn;","outputs":1,"noerr":0,"x":440,"y":400,"wires":[["33b16c0e.fcd7a4"]]},{"id":"33b16c0e.fcd7a4","type":"debug","z":"6223ebc3.0ec744","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","targetType":"full","x":600,"y":400,"wires":[]}]
1 Like

it's in a new function with a query (inject node for testing) sending the desired bed-alarm value for retrieval.

I just did a restart of the node-red application to clear context storage and my smaller sample just ran fine with no issue. Re-testing using my larger sample (about 450,000 rows) which will take several minutes to execute and look over. I may have failed to follow IT rule #1 (turn it off and back on again.)

And we have confirmation: I must've not cleared context before the test that gave me errant results after making changes. All is well now.

2 Likes

This behavior has happened to me in the past -- usually when I expected an array of strings, but received a plain string instead. This was because i was iterating over the length() of the incoming data (expecting to process rows on an array), but I ended up processing each char of the string instead... d'oh!

If that's the root cause, then Array.isArray(msg.payload) is your friend!

1 Like