❓ DISCUSSION: Should nodes save/restore their own context?

General
1

There was a post last night which touched upon nodes saving their own context data, which would then be retrieved if node-RED was restarted, a full deploy, or a system reboot.
I'll use the ui chart node as an example, where currently if we want persistence, we need to add a change node to the flow to save the context, and an inject node to inject the context data back after a reboot. It's not a big deal (if you know how to do it), but couldn't this be rolled into the node as a config option? It would make it much easier for new users, and also simplify flows.
The same applies to many other nodes, which unless the user manages persistance, their state/data could be lost.

Some time ago, I raised this with @drmibell in relation to his excellent node-red-contrib-simple-gate, and he kindly integrated the feature into the node config, so all users need to do, to ensure persistence is to tick the box. It works extremely well, and has never let me down!
I believe that Mike added the feature here, and this is the node's config;

gate
I realise that developers are free to develop nodes how they wish, and if they don't want a particular feature, then that's fair enough - it's their node! My point in airing this is to ask what other members think? I got the impression that @dceejay was opposed to this suggestion, but I'm failing to see why....

1 Like
2

As it sound appealing in the beginning there are at least two reasons against a general “rule” for persistent data.

  • The world doesn’t stop spinning during the time Node-RED is down. So whatever state you restore could be outdated.

  • The state of the flows where Node-RED is shut down is not the same as when it starts. So simply restoring the Data doesn’t restore the state of the flow. (See the complexity of hibernation in modern OS)

An Example: “all” my states of actors and sensors are stored on the mqtt server as retained messages (including configuration data like datatype, units ...). That gives not only Node RED the opportunity to replay all state changes happen in the time of being offline also the device can catch up after a reboot simply by subscribing to the same topics again. So any persistence here within Node-RED would be contra productive.

It is good to think about persistence early on and make wise decisions. Is it ok or even better to save data in memory? Or is the data context better limited to a node, a flow or globally? A good practice during testing (and from time to time) is rebooting to see if everything comes up as expected.

There are certainly nodes where it is a good option to save data on non volatile media but as a general “design advice” I have my doubts.

One of many reasons why I took the effort to migrate from openHAB to Node-RED was a (in my experience) unreliable persistence implementation.

1 Like
3

Hi Chris, this wasn't really about the pros & cons of whether to implement persistent data, it was more about how persistent data is implemented IF the user wishes to use it.
In the example I gave above, if a user wished to use persistent data, they would only need to click the tickbox, and it would be done.
If they didn't want to persist data - then they would not tick the tickbox, and the node would function as it does now.

1 Like
4

Sorry if I got you wrong. As I mentioned there are several cases where it makes sense (with or without user intervention) to save data persistent.

I took a look in node-red-contrib-simple-gate and yes if I use the same node I would had perhaps the same wish to save the state depending on where the state came from in the beginning. Because if the state of this node is restored by itself, the node modified the state before does not, resulting in a dashboard switch showing the state open when it is actually closed for example.

But I stay with my opinion that it is better to restart the flows and let them restore the state of the machine after reboot. In my flow I have a single inject node on startup to get the startup sequence right passing then to many setup nodes in a predictable order and finally to a out node where other flows can link into for their final setup tasks.

5

I always assume that node red is up, my Pi running the weather sensors has been running for 15 months :slight_smile:

2 Likes
6

Reliability of the whole system should be in hands of the flow creator. What if you have hundreds of nodes with outdated states and those states may then affect overall system differently? Can you allways stand nearby to stop whole thing and figure out where the stored state should be cleared out?
Way too many things can go wrong...

3 Likes
7

:+1: ... running until the cf-card wears out.... :crazy_face: I had this too often, thank you, and no I didn`t used no name cards, solved that since I use a server HDD (after a ssd went down the drain too)

You can`t have nice weather all the time and what can go wrong will go wrong ... as Murphy always proof us.

2 Likes
8

...and it would be...
Users would have the option to use persisted data if they wished by ticking, or not ticking the tick-box.
Let's remember that writing and reading context data from disk is the documented and prescribed way to handle node-RED persistence.
Even if they wanted to use a change & inject node to restore data, or even some other implementation, then the user would not tick the tickbox and use their preferred option.

1 Like
9

That is the option where they take it and then may or may not trust what the node does for them. But how can the creator of node figure out where and how the node will be used? Can it be created to cover all situations you can or can not imagine?

2 Likes
10

As longer as I think about this I get the feeling that it is not a good solution in general. The idea of Node-Red is that you have a flow starting with an input and then runs until it reaches the end (output, Termination, whatever). Changing the state of the flow before it even starts at the beginning depending on the state of the last shutdown is "unpredictable" for all nodes before. Especially for a node that changes the behavior of the flow itself (not only the own task) like the gate node you mentioned.

Even if it does not matter in most cases it "feels" wrong in the back of my head.

11

The card is a usb stick. And everything gets stored in an InfluxDB database anyway (retention policy 1 week). If I lose the outside temperature data from yesterday at 10:17pm I don't lose any tears either :slight_smile:

12

If Node-RED was created only as fun tool to read out some temperatures ...

1 Like
13

Sometimes that is all a user needs my friend!

1 Like
14

Getting of topic here but between the cf-cards I used usb sticks with the same result (damaged databases, safe mode or no boot at all). If I had done the research before I would have spent many hours and nights with my family instead. And by the way I said goodbye to the raspberry too, because personally It was not a good Idea for me to rely on a system/OS I`m not using very often and I do not understand in depth.

So I treat my mosquitto db and my flows like raw eggs (with clones far away my home somewhere in the cloud) always assuming that I have to restart someday at a time and state I never expected.

15

So if the user "just ticks a box" which persistence would it use ? The default context in settings is ram, so not very persistent. We wouldn't want to create yet another storage api so it would want to use whatever is there, which is then out of the control of the node author., as to whatever the user may decide to configure, which may also lead to performance impacts (and indeed SD card wear).

Where does that gate node store it ? Dies it work in cloud where there may be no reliable disk ?

2 Likes
16

If the health and life of your plants relies on the temperature reading in your greenhouse then my plants have a dedicated opinion if the reading is important or not. Especially tonight when the temperature will certainly drop under 0°C. :wink:

17

Isn't it whatever the user has set as 'default' in node-RED settings?

    contextStorage: {
      default: "file",
    memoryOnly: { module: 'memory' },
          file: { module: 'localfilesystem',
          config: {
                  flushInterval: '300'
                  },
          },
    },
1 Like
18

And the default (unedited) is memory. So ticking the box would give the user unrealistic expectation.

19

Sorry to be sarcastic it is late: default:file and say goodbye to your flash - sooner or later.
I think everyone should think twice if file is the right decision. That’s why “file” is not configured out of the box. (Perhaps I’m wrong)
But everyone is and should be free use any system as they prefere.

20

It's one sentence in the readme!!!
The node help in node-red-contrib-simple-gate seems pretty explanatory (to me)...