Development plan for 0.20

Core Development
1

With 0.19 out the way, work is already well underway on 0.20. I wanted to share a bit of the plans for 0.20, how we're going about it and provide guidance to anyone wanting to contribute.

Runtime/Editor split

The headline feature of 0.20 will be a complete restructuring of how Node-RED is packaged. We're splitting the one node-red npm module into seven separate modules. This introduces a more formal API for the runtime component and opens up a number of interesting ways that Node-RED can be embedded and extended. To the end-user, this change should be completely invisible. The node-red module will still exist and be what is installed via npm - but with this change, it will pull in the new component modules under the covers.

This work is incredibly disruptive to the code base - pretty much every file moves around in someway. We need to minimise the overhead of dealing with PRs against the existing code layout, versus how they will look moving forward.

There are two main branches in git - and its important to understand their role whenever a PR is raised:

  • master - this is the default branch and is for the current release - 0.19. If a PR is fixing a bug, it should target master so the fix can be included in the next maintenance release.
  • dev - this is the development branch - what will eventually be shipped as 0.20. If a PR provides new functionality, it should target this branch.

We also have a branch called repackage where the repacking work is being done. This work is largely complete and that branch will merge to dev in the next few days. At which point, many of the existing PRs will no longer be able to merge cleanly. We'll work on a case-by-case basis with the authors of those PRs to resolve any problems.

It's also true we have a large backlog of open pull-requests that needs tidying up. There's lots on there we realistically won't ever merge - they have either been long-since outdated, or they pull in the wrong direction.

The Design Note for this feature contains more details.

Node Message API

One of those PRs introduces a new messaging API for nodes to use - this defines how a node handles the messages it receives and, crucially, requires a node to tell the runtime it has finished handling a message.

This will allow new features such as timeout handling, better logging and correlation of flow activity, and introduces a new node (name TBD) that can be used to trigger a flow whenever a node finishes with a message. This feature will mean a flow can be triggered when a node that doesn't have an output, finishes its work.

Of course, introducing the new API is just the first step. There are 1600 modules in the flow library and they won't update overnight to take advantage of this API. The existing API will continue to work (of course), but we will be updating the docs for the new api once it is available. The complication, of course, is once a node updates to the new API, it will not work on older versions of Node-RED - so this does need to be done with some care.

There's more work to be done here - it is possible we choose to bump this to 0.21 depending on the progress we're able to make.

The Design Note for this feature contains more details.

What else...

We don't plan to have any other headline items in the release. That's mostly due to the fact there's no-one left to work on anything else.

There will be the usual assortment of incremental enhancements to the editor and nodes - but given the amount of work on the above items, it may be we don't throw too much else into the release.

When...

We want to minimise churn of major releases, but we also need to continue to make good progress on the roadmap to 1.0. There's a balance to be struck between shipping regularly and giving users a stable base to work on. The 6-month gap we had between .17 and .18, and again to .19 was never the plan.

One of the reasons for writing this post is to put a stake in the ground for the 0.20 release.

The goal is to have 0.20 ready to ship by the end of September - although there are a couple factors outside of our control on that (specifically, getting access to the @node-red npm organisation that sits registered but unused - we're working with npm support on that).

Contributing

If you're interesting in contributing, get in touch. Either here, or on slack. We have a healthy backlog of ideas on the whiteboard and there's always things we haven't thought of.

The one golden rule is that we ask you to talk to us before opening a pull-request.

All of the items in the roadmap have basic designs behind them - not all of which we've written down. Some have design notes on the wiki, some don't. No-one should start implementing anything on there without talking to us. This is simply because those design notes don't always stay up to date with our current thinking.

There are plenty of other ways to contribute beyond lines of code.

Whatever you are interested in, get in touch.

Nick

5 Likes
2

As always, fantastic work underway.

I wonder, would it be sensible at this point to create a simple guide for installing previous versions in case anyone needs to stay on 0.19.x but needs to do an install after 0.20 comes out? Or in case someone needs to do a debugging comparison? Not sure if that will happen, possibly depends on further update schedules. But the joys of npm mean that it is easy enough.

Just a thought.

One other thought - does the repackaging have an impact on resource use?

3

Hi @knolleary,

Would like to make my contributions ready to deal with the new API, as soon as possible (i.e. when API is finished and stable). But not sure what is the best way for us to make sure the contributions still run fine on older Node-RED versions?

So I did a quick test, starting from this simple node using the old (current) API:

module.exports = function (RED) {
    "use strict";
    function HelloWorldNode(config) {
        RED.nodes.createNode(this, config);
        var node = this;
        
        this.on('input', function (msg) {
            msg.payload = "Hello World";
            node.send(msg);   
        });
    }
    RED.nodes.registerType("hello-world", HelloWorldNode);
};

Did some small changes to update to something the new API 'could' look like:

module.exports = function (RED) {
    "use strict";
    function HelloWorldNode(config) {
        RED.nodes.createNode(this, config);
        var node = this;
        
        this.on('input', function (msg, send, done) {
            msg.payload = "Hello World";
            node.send(msg);   
            
            if (this.done) {
                this.done(null,msg);
            }
        });
    }
    RED.nodes.registerType("hello-world", HelloWorldNode);
};

This still works fine on my Node-RED version 0.19.1:

  • The 'send' function parameter is undefined
  • The 'done' function parameter is undefined
  • The 'this.done' function is undefined

Is this a correct refactored design, or did I overlook something?
Thanks !
Bart

4

Bart, whilst I applaud your enthusiasm, please don't do anything yet. We're still working on this item and I simply don't have the bandwidth to support people trying to consume the api at the same time as we're developing it.

Once it is stable and available to use, it will be documented and published.

5

Haha, thanks for the applause!
Well I have tried to transform myself in a rather cautious person, after spending two years on this forum :thinking:
That is why I had already added following quote to my question:

Let's rephrase: Suppose (200% hypothetical !!) that the design note on the wiki would be stable and final. Would my new of the hello world node be a good solution, or not. Or why not.

Consider it as a basic Javascript lesson, to prepare this hobbyist for the near future ...

6

With all due respect, I don't have the energy to delve into hypothetical critiques of code using an API that is still being worked on. I appreciate your example - when we're at a point to look at how nodes could be written to be backward compatible, I'm sure it'll be a helpful example.

7

I have now opened the Pull-Request to merge the repacking work into the dev branch on git - https://github.com/node-red/node-red/pull/1890

I'll be merging it in the next day or so - and once it goes in, any existing PRs against dev are likely to need some amount of manual merge conflict resolution. But given this is the main development item for 0.20, it has to take priority. The sooner the developer community are working against the repackaged code base, the sooner we can ship 0.20 with confidence.

1 Like