0.19 Node Red roadmap

General
1

Nick.... This persistable context, will that mean that globals will be optionally backed up to disk/flash between NR sessions (power down/up) ? I'm sure you've already documented it somewhere but if so I've missed it. I did look without success That would be good.

I'm writing in here because your forum will not let me do any more replies in "global numeric array trouble using global get and set". Anyway I think we've nailed it.

2

In 0.19 we will introduce the ability to store context data. Whether that is flow context or global context.

We will provide a File-based persistence with the 0.19 release and the API will be published so that other storage mechanisms can be written.

It will enable two specific use cases:

  1. persist context data across node-red restarts
  2. allow multiple runtimes to share a context store - useful for when you need to horizontally scale Node-RED, but less interesting for Pi users.

We haven't released 0.19 yet so there isn't the end-user documentation for it yet.

3

There is some estimated date for this? Just for curiosity.

Regards

4

I've stored Both your emails Nick - I can't do any more replies in the other topic for a while. Am I right to assume that storage will be optional... overall or on a per global item basis?

5

The feature is optional. By default it continues exactly as it does today - in memory context.

You will be able to enable one or more context stores. If you enable multiple context stores you will be able to pick which store a particular context property is stored. So you would, for example, be able to have both a memory store and a file store - so only certain properties to persisted fully.

0.19 will be ready to ship next week, but the actual release will wait until I am back from holiday in the second week of August.

6

Magic – so optional and check again week 2, August… sounds great to me.

Pete

7

Quick Q. Feel free to ignore if you are busy.

Will persistence happen on write or on NR shutdown?

BTW, have a great break - hope you are going somewhere cool :sunny: :wink:

8

The File store offers two modes of operation; caching (the default) and non-caching.

With caching mode, it loads the full contents of context into memory on startup. This means a get can be completed synchronously from memory. A set will also return synchronously once the value is updated in memory - but an asynchronous action is kicked off to write the update to disk in the background. If using caching mode, no changes will be required to existing code that accesses context.

With non-caching mode, all get and set calls involve reading/writing to disk. As such the code making the request must provide a callback function. This is a change to the existing context api. For example:

flow.get("foo", function(err, foo) {
   // check err in case something went wrong...
   // otherwise do something with foo
});

To avoid lots of nested callbacks, the api also allows you to get multiple values in one go:

flow.get(["foo","bar","name"], function(err, foo, bar, name) {
   // check err in case something went wrong...
   // otherwise do something with foo, bar and name
});

The same is possible with set -

flow.set(["foo","bar","name"],["red","blue","green"], function(err) {
   // check err in case something went wrong
});

Finally, as you will be able to have multiple context stores configured, you can specify which store a particular value should be put in.

var fooFromMemoryStore = flow.get("foo","memory");
var fooFromFileStore = flow.get("foo","file");

The "memory" and "file" arguments are the names of context stores as configured in your settings.js file.

I'm sure there's more to document - this is a bit of a brain dump ahead of writing the proper documentation later this week.

2 Likes
9

@scargill I've stumbled across the proposal / design document by accident last week. Long wanted feature. Looks promising.

10

Please note that is a living document and the implementation has overtaken what is written in the wiki in some areas. So whilst it gives a good sense of what is coming, it is not up to date with what will be released.

1 Like
11

Very nice. Thanks for the dump.

12

Nice.

I have two questions. First, will the new persistence mechanisms work the same way for the node context as for the flow and global contexts? Second, how will node context persist across the different kinds of editor re-deployments (full, modified flows, and modified nodes)? I've never quite sorted this one out for the current version.

Mike

13

Yes it will. It does raise an interesting question of whether nodes should maintain more of their internal state in context, rather than node-local variables. But that's a thread I'm not going to pick at until after 0.19 is done.

The contents of context will persist across all types of deploy.

14

Great. Just to be clear, this seems to be a change -- my experience has been that a function node has its context object deleted in a full deploy. Is that correct?

15

Slight mistake there. That won’t change. Memory won’t suddenly start persisting on a full deploy.

16

No. If you enable persistent context, even if you tell it to use the in-memory store, then context will not be cleared on deploys.

If you do not enable persistent context, then it behaves as it does today.

17

Neat ! That is extra goodness (as long as you tell it to persist)

18

I suspect that node developers will naturally drift towards that anyway as it seems logical. I'd also expect to start seeing some additional switches on some nodes allowing optional state keeping.

Some cool things will ensue!

19

As long as they realise that not everyone will run with persistence switched on so can’t guarantee it will be there

20

Hmm, good point. Perhaps a future enhancement would be an alternative entry point into the persistence API for node developers then. Anyway, that's for post 0.19 as Nick says.

Good stuff as always though.