# ❓ DISCUSSION: Should nodes save/restore their own context?

**URL:** <https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590>\
**Category:** General\
**Created:** [19 January 2020 19:16 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590 "2020-01-19T19:16:08Z")\
**Posts on this page:** 20\
**Page:** 4

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [21 January 2020 21:18 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/61 "2020-01-21T21:18:05Z")

</div>

I think I said the same thing, perhaps not as clearly.

> [@dceejay](#):
>
> as long as your node isn't one of those changed...

Right, but with the option selected, it survives in those cases too.

---

<div class="post-metadata">

**Author:** ![hotNipi](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hotnipi/32/383_2.png) [@hotNipi](https://discourse.nodered.org/u/hotNipi)\
**Post date:** [22 January 2020 09:00 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/62 "2020-01-22T09:00:19Z")

</div>

It seems like there is no legal way to get information about the **module** configured for **context store(s)** in `contextStorage` if asked from nodes configuration page. (`RED.settings.context`)

Making smart decisions how to offer the persistence is not possible.

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [22 January 2020 14:36 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/63 "2020-01-22T14:36:30Z")

</div>

> [@TotallyInformation](#):
>
> Might not be as hard as you think actually. That's because the data is well formed and easily accessible in settings.js.

Have you been able to get access from the nodes configuration page Julian.

---

<div class="post-metadata">

**Author:** ![shrickus](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/shrickus/32/517_2.png) [@shrickus](https://discourse.nodered.org/u/shrickus)\
**Post date:** [22 January 2020 16:56 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/64 "2020-01-22T16:56:40Z")

</div>

I do agree that this has been a lively and potentially useful discussion -- thanks for starting it, Paul...

One distinction that I have NOT seen made is _what the purpose of the stored data would be_ -- the initial request was a simple checkbox to save the previously received data points, so the chart could be redrawn easily (i.e. for display purposes). I would not be worried if there were a few dropped points, since it would not influence any downstream systems or controllers...

On the other hand, things like SCADA data that are used to control physical systems need to be handled _very carefully_ to ensure out-of-sync signals do not trigger "bad things" -- but I don't think that's what is being requested here.

Probably 1.5 years ago (on the Google groups, i think), we discussed developing "Caching" nodes that could be used inline to cache msg data (like a ring buffer) and play it back (maybe on some signal, or redeploy). IIRC, it was more for regression testing, but it may also be useful in front of the ui display nodes for just the purpose that the OP envisions. So no changes needed to the chart node itself, and the author can add that behavior by dropping a node in front of the ui node, maybe? I've not looked, but there may well be some contrib-\* nodes like this already...

---

<div class="post-metadata">

**Author:** ![Colin](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/colin/32/17040_2.png) [@Colin](https://discourse.nodered.org/u/Colin)\
**Post date:** [22 January 2020 17:16 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/65 "2020-01-22T17:16:10Z")

</div>

A simple replay from caching node would not work for the chart as it also uses the time at which the message is received.

---

<div class="post-metadata">

**Author:** ![dceejay](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/dceejay/32/38_2.png) [@dceejay](https://discourse.nodered.org/u/dceejay)\
**Post date:** [22 January 2020 18:05 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/66 "2020-01-22T18:05:44Z")

</div>

The chart node in particular does send out its complete array already. There is an example how to use it in the flows library

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [22 January 2020 19:55 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/67 "2020-01-22T19:55:04Z")

</div>

I have an example caching function for uibuilder that I use. I also have a repo waiting to be written for something more robust as a custom node.

In uibuilder though, I control the cache via control messages that let you do a cache replay or cache clear.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [22 January 2020 20:00 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/68 "2020-01-22T20:00:31Z")

</div>

> [@Paul-Reed](#):
>
> Have you been able to get access from the nodes configuration page

Yes 😀

Sort of.

You can use the `settings` property of the `registerType` function to pass information back to the configuration panel:

In your js file:

```auto
    RED.nodes.registerType(uib.moduleName, nodeGo, {
        credentials: {
            jwtSecret: {type:'text'},
        },
        settings: {
            uibuilderNodeEnv: { value: process.env.NODE_ENV, exportable: true },
        },
    })

```

Example use in your html file:

```auto
                    if (window.location.protocol !== 'https') {
                        if (RED.settings.uibuilderNodeEnv !== 'development') {
                            console.error('HTTPS NOT IN USE BUT SECURITY REQUESTED AND Node environment is NOT "development"')
                        } else {
                            console.warn('HTTPS NOT IN USE BUT SECURITY REQUESTED - Node environment is "development" so this is allowed but not recommended')
                        }
                        // TODO: Add user warnings
                    }

```

Not sure if standard Node-RED settings are accessible directly via `RED.settings` in the html file.

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [22 January 2020 21:11 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/69 "2020-01-22T21:11:08Z")

</div>

> [@shrickus](#):
>
> we discussed developing "Caching" nodes that could be used inline to cache msg data (like a ring buffer) and play it back ...there may well be some contrib-\* nodes like this already...

Yes, searching on "buffer" and "queue" produces a few candidates. As @dceejay and @TotallyInformation point out, charting nodes are (or should be) designed with this functionality in mind. More general kinds of message replay are possible with the contributed nodes, and (self-promotion) the [q-gate](https://flows.nodered.org/node/node-red-contrib-queue-gate) node is easily configured to retain an arbitrary number of the most recent messages, replay them singly or all at once in the order received, and use persistent context storage to restore its state after a shutdown or restart (/self-promotion). As you said, using this kind of replay requires great care. Designing systems to fail safe is an art practiced by too few and mastered by hardly any.

---

<div class="post-metadata">

**Author:** ![hotNipi](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hotnipi/32/383_2.png) [@hotNipi](https://discourse.nodered.org/u/hotNipi)\
**Post date:** [22 January 2020 21:24 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/70 "2020-01-22T21:24:44Z")

</div>

So we are back at the main question. Is the state recovery safe enough to provide from nodes or should it left for users to figure out how to manage if needed.

Currently the custom node can not do it reliably. This can be changed. But I can't see any reason to do it if there isn't agreement about the way it goes.

Can we conclude some midpoint agreement on main topic? Then the solution proposals may be coming in well more targeted.

---

<div class="post-metadata">

**Author:** ![shrickus](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/shrickus/32/517_2.png) [@shrickus](https://discourse.nodered.org/u/shrickus)\
**Post date:** [22 January 2020 21:40 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/71 "2020-01-22T21:40:05Z")

</div>

> [@Colin](#):
>
> A simple replay from caching node would not work for the chart as it also uses the time at which the message is received.

Indeed, which is why I was pushing to get to a `msg.timestamp` field added to every msg as it was generated... but it never materialized. Seems that node-red is designed primarily for real-time message passing, and any temporal attributes need to be handled within custom flow logic, not the nodes themselves...

That said, I just checked the ui\_chart.js code from my old install, and it does indeed check each incoming msg for a `timestamp` property, which it uses if found but defaults to now()

```auto
if (node.chartType === "line") {
    var time;
    if (msg.timestamp !== undefined) { time = new Date(msg.timestamp).getTime(); }
    else { time = new Date().getTime(); }

```

Granted, that was _probably_ a local customization I made to verify that it could work, before suggesting it to Dave.

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [22 January 2020 22:23 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/72 "2020-01-22T22:23:33Z")

</div>

> [@hotNipi](#):
>
> Can we conclude some midpoint agreement on main topic? Then may be the solution proposals may be coming in well more targeted.

Yes we do seem to be drifting off-topic  
**IMO** there are probably 3 options (in descending order of preference);

1. Create a mechanism within the code to read what storage options are available, and let the user choose from a populated dropdown. I realize that you have been doing a lot of work on this @hotNipi (thank you), but to achieve this maybe we need better specific support from other developers, especially the Leaders.
2. Follow @drmibell example, and save the context to the default storage, with a brief explanation in the readme.
3. In respect of `node-red-contrib-state-trail` you could always add an array of the data as a node output, so that context could be saved/restored in pretty much the same way as the chart node does.

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [22 January 2020 22:33 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/73 "2020-01-22T22:33:01Z")

</div>

> [@hotNipi](#):
>
> Can we conclude some midpoint agreement on main topic?

Could we clarify the question we would like to answer? I think the original proposal was that we encourage developers of contributed nodes to add (when appropriate) a feature to restore the state of the node after re-deployment, restart, or reboot. The discussion has highlighted a possible technical limitation in managing multiple, non-default context stores. That is clear. The possibility that even attempting to do this could be "unsafe" or "unreliable" seems greatly exaggerated to me. Unless I'm mistaken, the simple-minded feature in the `gate` and `q-gate` nodes works exactly as advertised and should not cause problems for anyone who understands node context. Whether it is useful is up to the user.

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [22 January 2020 22:56 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/74 "2020-01-22T22:56:44Z")

</div>

> [@Paul-Reed](#):
>
> 1. Create a mechanism within the code to read what storage options are available

Use the `RED.settings.context` property that is already available in the editor:

![image](https://us1.discourse-cdn.com/flex026/uploads/nodered/original/3X/f/9/f9fcda521e3ff7393b7d7a7ecc89568d7d740802.png)

This is how the TypedInputs already know about what stores are available. It doesn't _not_ provide any information about the nature of the stores - just the list of store names and which one is considered the default store.

* * *

I would suggest the next step is to identify some specific nodes (more than the sample of one or two we have) that would benefit from this and to creating some examples that show that benefit and how it would actually work.

My concern is more on the unexplored edge cases this introduces. For example, if a user wants to explicitly reset the stored state of the node, how can they do that in a consistent way?

From that we can see if there needs to be a consistent way to present the option(s) in the UI - especially if it needs to provide a choice of context stores. We can then see if there is something more concrete the Editor could provide to help nodes be consistent.

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [22 January 2020 23:01 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/75 "2020-01-22T23:01:53Z")

</div>

> [@knolleary](#):
>
> My concern is more on the unexplored edge cases this introduces. For example, if a user wants to explicitly reset the stored state of the node, how can they do that in a consistent way?

Thank you Nick, I hope that helps @hotNipi  
Could that be achieved by passing msg.reset into the node, which seems to be a common theme amongst other nodes.

---

<div class="post-metadata">

**Author:** ![knolleary](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/knolleary/32/3_2.png) [@knolleary](https://discourse.nodered.org/u/knolleary)\
**Post date:** [22 January 2020 23:29 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/76 "2020-01-22T23:29:43Z")

</div>

I was thinking more of the scenario where they are developing the flow, make a change and want to clear any stored state so when the flow starts, it's starting with a clean slate. So more of an explicit developer-triggered action, rather than some piece of application logic.

---

<div class="post-metadata">

**Author:** ![drmibell](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/drmibell/32/8424_2.png) [@drmibell](https://discourse.nodered.org/u/drmibell)\
**Post date:** [23 January 2020 20:40 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/77 "2020-01-23T20:40:33Z")

</div>

> [@knolleary](#):
>
> clear any stored state so when the flow starts, it's starting with a clean slate

How about: select the node, go to the context panel in the sidebar, select and refresh the node scope, and delete any or all of the state variables saved by the node that you want to reset.

---

<div class="post-metadata">

**Author:** ![hotNipi](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/hotnipi/32/383_2.png) [@hotNipi](https://discourse.nodered.org/u/hotNipi)\
**Post date:** [23 January 2020 20:56 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/78 "2020-01-23T20:56:52Z")

</div>

May be there is chance to introduce another deploy option with cleanup procedure called if node has dedicated callback for that event?

---

<div class="post-metadata">

**Author:** ![Paul-Reed](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/paul-reed/32/66906_2.png) [@Paul-Reed](https://discourse.nodered.org/u/Paul-Reed)\
**Post date:** [25 January 2020 19:57 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/79 "2020-01-25T19:57:02Z")

</div>

`node-red-contrib-ui-state-trail` has now been updated to restore it's chart data after a full deploy, node-RED restart or system reboot. I've just been trying it and it works great, thanks @hotNipi.  
The changes implemented have also moved development potential further forward, as the node now has access to the node-RED's `setting.js` file, and therefore it's `contextStorage` settings, so rules have be introduced to ensure that Data Storage is only implemented whenever a persistent data store is available.

Knowing which storage modules are available, opens up the future potential to maybe being able to select different storage option from within the node config (as @TotallyInformation commented above), and also ensure that orphaned data is cleaned up if a different storage option is selected.  
Also as commented above, edge cases also need to be managed, but hopefully the clever guys here will eventually build this into a viable model for node data persistence.

---

<div class="post-metadata">

**Author:** ![TotallyInformation](https://sea2.discourse-cdn.com/flex026/user_avatar/discourse.nodered.org/totallyinformation/32/31_2.png) [@TotallyInformation](https://discourse.nodered.org/u/TotallyInformation)\
**Post date:** [25 January 2020 22:35 UTC](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590/80 "2020-01-25T22:35:19Z")

</div>

This whole discussion reopened my thoughts on a caching node to complement uibuilder allowing people to avoid having to write their own function node.

This will now use Node-RED's context store features to store the cached entries. I've taken loads of ideas from this thread to start to flesh out the design so thanks a lot to everyone.

As usual, I somewhat sidetracked myself today by going down a related rabbit hole for creating better node documentation - settling on using [VuePress](https://vuepress.vuejs.org) as it integrates well with both a docs folder and GitHub pages.

If you want to follow the progress of the design, the docs for the [node-red-contrib-infocache](https://github.com/TotallyInformation/node-red-contrib-infocache) node are [here](https://totallyinformation.github.io/node-red-contrib-infocache/design.html).

[Previous page](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590.md?page=3)

[Next page](https://discourse.nodered.org/t/discussion-should-nodes-save-restore-their-own-context/20590.md?page=5)