Is there a known issue where flow.set is inconsistently failing?

General
1

Is there any known issue where flow.set is inconsistently failing? I have run into this three times now, where an existing working flow that does flow.set (write to file system, i.e. persistent) seems to fail to write out changes to the object before the flow.set invoked.

I can't consistently cause the behavior explicitly over and over, but once it happens, it keeps happening for the given flow. Stopping and restarted NR does not fix the issue, deleting the context object and creating it does not seem to fix the odd behavior.

As far as I can tell there is no errors or exceptions reported anywhere that might provide clue how or when this happens. I am going to export the flow in question, and re-image the SD card and/or use a different SD card. But I would think any writes to SD card that fail would be visible in the NR logging right?

What is odd is the issue happen only to a specific attribute of an object, that is written flow flow set. For example, where 'persistent' the name I set in settings.js for localfilesystem context variables, versus memory only.

obj = flow.get('object', 'persistent')

Where obj is...

obj = {
a:1,
b:2
}

Change one attribute/property...

obj.b=4

flow.set('object', obj, 'persistent')

obj2=flow.get('object', 'persistent')

Where obj2 should be...

obj2 = {
a:1,
b:4
}

But when I look at the object...

obj2 = {
a:1,
b:2
}

Why is b not equal to 4? Given this is in a functional node... not an explicit deep copy (that I know of). I can change any other property in the object and it is correct after each change, but the b property seems to locked or stuck at value 2, and never changes to value 4, once this odd behavior starts... it can be minutes, hours, even days, until the behavior happens... but once it happens, it is consistent. It is almost like b = 2 becomes read only?

This to me implies some type of write failure on the SD card, but a write error specific to a specific attribute or property of an given object? Is that even reasonable? Or is something else going on?

Is there a way to explicit test for context write issue in NR? Some enhanced logging I can turn on, to catch this?

2

Well that is odd, it works differently for me. obj2 shows the revised value even though I have not called flow.set, (flow.get returns a pointer to the object not a copy of it.)

image
image1108Ă—513 29.6 KB

What's more, this revised value survives a reboot.
image

If you leave 'object' in place and add another context variable, does that one ever fail to work?
What if you hold 'object' in memory instead of localfilesystem?

3

That is one of things I am testing for... the flow that exhibits the behavior creates a number of context variables, I have seen this happen with simple variables and variables that hold complex objects. If it was a memory only variable, I would say it might be a deep copy issue, but functional notes don't have a deep copy option? Like a change node does?

In one case, I had a simple boolean flow variable, set to true or false, it suddenly seems to be locked at false. So the type of variable/data type does not seem to be deterministic... but I have only seen the issue with persistent, i.e. localfilesystem context.

At the moment, if I can get the same flow to work 'wrong' on a different SD card. Then exist on a different Pi device, etc. To try to isolate the problem in some way to say a specific SD card, or a specific Pi device, etc. I am not yet, in a position to say or suggest it is a NR issue... but something odd is going on, at least for me.

If it happens on different SD card and different Pi device, then that suggests it might be NR code somehow, but a lot of testing before I consider that in any real sense.

I also plan to recreate the flow from scratch, it is a pretty complex flow, but worth the effort to try to get this issue understood/solved.

By way, I am using Pi 3, with Pi OS latest (bookworm) 64 bit and the latest NR install, defaults per the Pi installation script... Node v18? if memory serves.

4

I suspect this is caused by the fact that objects are accessed by reference in javascript. The result is that if you do

let obj = {
  a:1,
  b:2
}
flow.set('object', obj, 'persistent')
let obj2=flow.get('object', 'persistent')

Then obj and obj2 are pointing to the same object in memory, so if you do

obj.b=4

Then inspect obj2 you will find that obj2.b is also 4.

1 Like
5

Hmm.

:thinking:

6

We have not got enough information to know exactly what is going on. I suspect, though, that if/when it is finally understood then the reference issue will be at the heart of it.

It does explain that symptom.

The SD card would only come into it on restarting node-red. When node-red is started it reads the data into RAM and all operations on it are in RAM. Changes are written to the SD card but if that were failing it would not be until the restart that the error would be seen.

@Nodi.Rubrum any time you are fetching anything from context and intend to do something to it and not write it back to context (such temporarily modifying it or passing it on in a message for example) then clone it when you fetch it so that you get a newly created copy of the context data not a pointer to the original.
let obj = RED.util.cloneMessage(context.get("obj", "persistent"))
or, if it is the default context store
let obj = RED.util.cloneMessage(context.get("obj"))

2 Likes
7

@Colin, Ah! So that is how you can do clean/deep copy in a function node! Will give that shot. Just for qualification... if I do... Where 'T' say is an array of objects...

var t=flow.get('T', 'Persistent')
t[1].me=x
flow.set('T', t, 'Persistent')
msg.payload.me=t[1].me

All values by reference, meaning if somewhere down stream of the given function node, msg.payload.me is changed, that change will be refelected in t[1].me? And... if I do flow.get('T', 'Persistent')[1].me... that will also reflect the changed value back in the context. Unless at some point downstream I used a change node with deepcopy enabled (for sake of this example). A reference is a reference is a reference. LOL... guess my old C skills are not completely dated.

8

Is x an object or a simple type? This only applies to objects, arrays etc.

I am not sure whether deep copy is anything to do with cloning, I will have to look that up.

9

I maybe thinking of deepcopy ala python where a complete and separate object is created, that has no reference to the original object. I would call that a clone. :slight_smile: Is that not what happens in change node when you select the 'deepcopy' option?

10

I think you may be correct, but I can't see it documented anywhere. Hopefully someone with more knowledge can confirm this.

11

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.