❓ DISCUSSION: Should nodes save/restore their own context?

General
41

Well I thought my idea of having a context node was a good option, but it appears not :confounded:

42

What do you mean by Context node? What would it actually do? You can use the Change node to get/set values in context.

43

Put all the configuration options in one place, like persistent vs transient, variable context names, global vs flow tab vs individual flow ...

And by using a context node in a flow you can see at a glance there is context associated with that particular flow. Then you can open the node to check all the context properties.

Originally the discussion appeared to be about making life a bit easier, especially for newcomers. I think the above would help.

44

This discussion may have gone on past the point where I can add anything useful, but since a node I contributed was mentioned at the start, I'd like to make a few comments.

  • The node that @Paul-Reed mentioned saves only its local node context -- in fact, just one variable. It makes no attempt to restore the state of the rest of the flow, and I agree with most of the comments that efforts to do that can be problematic.

  • On the principle of doing one thing well, the gate node does nothing except be open or closed to incoming messages. Optionally persisting its state seemed like a reasonable thing to offer to users who understand the feature, which is off by default.

  • In hindsight, the decision to use only the default context store, which was done for the sake of simplicity, has its down side. If the user wants persistence, he is forced to use the filesystem (or something else persistent) as the default. As @TotallyInformation points out, however, this is not as great a danger to SD cards as some may think, and a dropdown menu to select the context store might be one complication too many.

  • I did not set out to defend the design or implementation of the gate node. This discussion has raised important points, but I would have liked to hear ideas as to which nodes, core or contributed, might benefit by having an option for persistent storage of node context. One that comes to mind immediately is the schedex node. (The lack of persistence is enough of a potential issue for users that it is documented specifically in the README.) Nodes that keep a history of previous inputs, for example, the smooth node, might have an option to restore that history on a restart, and so on. A close look at such cases might show whether this is truly a good or bad idea.

2 Likes
45

Can it happen that we introduce memory issues in case of chart type nodes will rewrite it's context storage with every incoming payload?

46

These are, indeed interesting potential good use-cases for node instance level persistence. Though you could argue that, for example, the smooth node might as easily produce incorrect information on a restart because time has moved on and the starting environment might be nothing like the previous ending environment.

I don't think anyone is saying that there is no case to be made for persistent storage at the node instance level, there clearly are cases where it is useful.

However, what I think we are all saying is that it is a complex issue with many different use-cases and that it would be wise to be cautious about jumping in with a single solution without thinking things through carefully.

1 Like
47

I agree completely. Each node or use-case is probably unique, and developers will have to decide whether it's worth the effort to implement a feature that presumably is optional, may not see much use, and could produce unforeseen issues. Still, I think the discussion is worth having.

1 Like
48

I also agree with that.

I said above that I was using the chart node 'as an example' to explain the discussion context, as I wanted to avoid using a contrib node 'example' so that I didn't offend any of the community contributors.

1 Like
49

I am delighted that the debate has been so detailed that I can reshape my opinion.
I am now standing somewhere in between but this between means that state-trail node gets the option.

And yes, the hardest part is this

I'm really in trouble to describe this functionality when things the user will see are the checkbox and a few words next to it.

1 Like
50

I'm not sure it can be just a checkbox if you need to be able to choose the context it uses... If the user hasn't configured any context (apart from memory) then there is no point (imho) of having the option at all. If they have then you need to be able to select which to use as some may be more useful than others.

1 Like
51

That goes deep now :slight_smile:
If there is multiple available and user switches between them and the previous one was persistent and then after while user switches it back, the very old data still exists and provided. Ouch.

52

Yup - all part of the problem.

53

Could it not use the default storage, and then add a few lines in the readme, and a link to the storage documentation to add appropriate file storage as default.
I think that's what Mike has done.

If the user left it as a memory store, I assume that it would survive a full deploy & node-RED restart, but not a system reboot?

54

Might not be as hard as you think actually. That's because the data is well formed and easily accessible in settings.js.

All you really need to know is:

  1. Is there a contextStorage property in the settings.

    If not, then disable the feature

  2. Is there a contextStorage property that contains a module property who's value isn't memory?

    On the assumption that anything not memory would be persistent - hopefully a reasonable assumption.

    If there is, then populate your drop down with it - repeat for any other contextStorage properties similarly configured.

For reference, an example of the contextStorage settings:

    contextStorage: {
        default: {
            module: 'memory'
        },
        file: {
            module: 'localfilesystem'
        }
    },

Once you have a standard set of code, perhaps it could be considered to be added to the RED object as a standard library function?

1 Like
55

I have to disagree here. The way I have implemented saving and restoring the state of the gate node does not depend on the persistence of the context store. The option is labeled "Restore from saved state," and it tells the node to try to restore its state from context, regardless of where the context is stored. (Obviously the node has no way of knowing whether that storage is persistent.) The main value of this option is that the if the flow is restarted from the Deploy menu the gate will not return to its Default State, which is what would happen if the option were not selected. Being able to choose between the two behaviors turns out to be a help in development/debugging, even when context is being kept in memory.

2 Likes
56

Well, it is a little more complicated. Question is what to offer and how to offer.
Easy way - (checkbox) offers option to store node state into context the user declared as default. Node survives if default is "memory" or not configured at all, just the functionality can not be guaranteed. Can only say "the node tries to..."
Advanced way - Nodes configuration tries to make smart decision by examine what is configured in settings for contextStorage and offers functionality which it can then provide for as it says. If user has configured contextStorage for memory only, there is no reason to offer any boot/recovery option based on persistent storage as it will not work.

But if user has configured file based option as default or both contextStorage modules what is provided by default by Node-RED, (memory and file based) then it will be reasonable may be offer again simple way (checkbox) to even use or not use but internally use file based store.

And there may be custom memory module defined (and it can exist together with default modules). And this is black box. So the functionality may fail.

All that should be done with easy to understand manner in nodes configuration page where is not much place for long explanations.

57

Exactly right. I'll be watching to see what you decide -- and whether I can steal something from it. Meanwhile, no surprise, I've taken the easy way out...

1 Like
58

If you do decide this option, then maybe we could all contribute to help with the wording.
Just let us know :+1:

59

I have it already as it really was couple of minutes to implement but I'm not still decided if I want it that way. So waiting for maybe more opinions from here...

But for wording I'll probably need it all the time so .. :smiley: :ok_hand:

1 Like
60

@drmibell - if you just store it in global context (memory) it may survive a deploy - but won't survive a stop start.
It would however also survive (without being saved) if one of the other deploy options was used - ie only changed nodes or flows as long as your node isn't one of those changed...