[Article] Node-RED Development: The Numbers

General
1

Hi There!

Following up on a discussion had over here, I wrote an article on analysing Node-RED Development by numbers.

It's a collection of my interpretations of those numbers, there is a link in the article to the flow that generated everything (i.e. a bunch of dashboard graphs).

TL;DR: There is a core group of maintainers who maintain and update Node-REDs core node packages. Many developers have contributed and have potentially moved on, however because of Node-REDs Flow Based Programming nature, this has had no technological consequences. The use-cases for which Node-REDs is utilised remain narrow and focussed, thus continuing to ensure Node-RED remains a niche product.

Cheers!

3 Likes
2

Thanks for sharing @gregorius

On the topic of who 'controls' the project, it is important to acknowledge the project is part of the OpenJS Foundation; something your article doesn't mention. This requires an open governance model that prevents any one company taking 'ownership'.

Anyone is able to be an active participant in the core of the project and through merit of contribution and collaboration have just as much say in what happens in the project as anyone else does.

https://nodered.org/about/governance/

Nick

3 Likes
3

Thanks @knolleary saying that but I can't really see in which direction Node-RED will go.

I have heard that Node-RED shouldn't become a dashboard, fair enough even if I don't exactly hold myself to that but what do you see coming into Node-RED core:

Do these things have a future in the Node-RED core? Or do you and the contributors have a different vision for Node-RED?

As a potential and interested contributor, I'm missing the vision for Node-RED - sorry I have missed something and it's written down somewhere but I haven't found it yet :slight_smile:

Cheers!

4

I have heard that Node-RED shouldn't become a dashboard,

That has been one of the guiding lines we've had, but there's always room to explore what it possible and what makes sense.

but what do you see coming into Node-RED core

One of the things we have to balance is what makes sense to be a core functionality of Node-RED, and what could/should be an optional extension.

The use cases for Node-RED are many and varied. IoT scenarios get a lot of air time in the forum, but there are lots of users of Node-RED that have very little to do with IoT. That's something we mustn't lose sight of.

In terms of the list of items you've shared - I'm happy to share my thoughts on individual items. But I also want to separate the concept from your specific implementation. It's great to see how active you've been developing all of these things, but it doesn't mean the precise approach is going to be appropriate for dropping straight into the core.

  1. Auto Layout. I've said before I think it would be good to have some sort of auto-layout option in the core of the editor - it's a feature that is useful regardless of what flows are being created. You've seen the draft PR I raised ages ago to record some very simple experimentation I did on the subject. There are many ways it could be done and the nodes you and Bart have created are a good way to figure out what works. Ultimately it needs someone to drive the conversation to move it forward.

  2. Node dev within the editor. As you note we already support packaging a subflow and publishing to npm. We have a command-line tool that can help with that - GitHub - node-red/node-red-nodegen: Command line tool to convert Node-RED function node, Swagger document, or WoT Thing Description to a custom Node. - but yes, it is still quite a manual process. Being able to do the full development lifecycle of an npm module is not something I personally think belongs in the core of Node-RED. There are many uses of Node-RED for which that would not be a required use case. So my personally opinion is this would be a good thing to supply as a Plugin that can be installed into the editor.

  3. flowhub.org. We already have flows.nodered.org, so I'd prefer we focused the core project's efforts in improvements to that site and how it could integrate in the editor. Having a means to import/export flows from the editor directly to the existing Flow Library would be a great thing to have. Off the top of my head, the existing Import/Export dialog is already pluggable so would seem to be the place to do that. In terms of it being a built-in feature, vs a plugin - I'm not sure. Again I am aware of a number of Node-RED integrators who would explicitly not want to expose that functionality - but unlike item 2 above, this does feel like something we could have in the core as long as it could be disabled via the settings file. We'd also have to look at what updates would be needed on flows.nodered.org to support this workflow.

  4. visual diffs/flow comparison - having something like this integrated in the existing flow diff view in projects mode would be useful - as well as having something in the regular non-projects mode.

  5. converting to uml - can't say this is something I've ever seen requested, so not sure it's something on my wish list. My instinct is to say it feels right to be a contrib plugin for now so users who need it can get it.

In terms of a vision, we don't necessarily have something concrete written down that will give you what you're looking for. Ultimately, Node-RED is shaped by the community and those who chose to contribute to the discussion and contribute to the code - there's no shortage of great ideas from the community, but there is a shortage of who is able and willing to invest time to implement them in collaboration with the project.

We want to keep the balance right between what Node-RED does 'out of the box' vs how it can be extended by installing additional plugins. In the latter case, that may well require some amount of core Node-RED development work in order to expose the right APIs or extension points. I cannot give you a specific definition of where the balance is - it's something I'm always open to exploring.

Nick

5 Likes
5

flowhub.org.

I note that this site has a non-standard license. I think this might restrict its use for many organisations. I'm also not sure about the legal validity of that license. While these things don't often cause issues, it only takes one person or organisation to create waves and things can easily unravel.

6

Imagining Node-RED as the kernel of an operating system (e.g Linux), then those are most important tasks that Node-RED can do: provide APIs for applications (i.e. packages) to dock onto. Linux had the advantage of ~30 years of Unix development and many APIs were already well defined and stable, so it was "easier" to build something that people could use, i.e. Unix developers could easily port their stuff to Linux because of the standard APIs.

Node-RED does not have that, Node-RED has to invent the standards.

Using this analogy has helped me to better place Node-RED into a category. I just hope Node-RED remains a modular kernel and does not become a monolithic kernel - I think that is something you are quite aware of thankfully.

Thank you the clarity and honesty. I agree with your take on those features.

FlowHub.org has developed something more like a flows package manager and I might write something that explains better what I my vision is for it. Flows.nodered.org does a great job of being a node package manager, but not for flows. Hence flowhub.org tries to make flows first-class members of the Node-RED family. I didn't really plan this but it just happened as I began using it.

Which going back to the analogy with the Linux kernel, providing pluggability becomes the "APIs" that Node-RED should define. So perhaps less is more: less core features and more pluggability in every part of the editor.

For example something like the image drag&drop in NR 3.1.0 I personally find problematic since it bloats the flows.json with base64 image data. I don't know whether I would have included it or instead provided a setting for image uploading and then maintain a reference to the image in the flows.json.

If there were a way to make that plugable, I would replace that with something that uploads the image to some service and adds a link to the flows.json - much as this discourse does. The drag&drop part is a great idea, its the storage that I find problematic - to clarify my critical point of view.

Original you definitely created Node-RED for IoT application, at least that's what I've seen. Has Node-RED outgrown that in your opinion? Did you model it on the concepts of FBP or did that happen? What were your influences/thinking in creating Node-RED in the way you did, i.e. visually? I ask to find out whether you came to FBP via solving the itch you had or did you happen upon FBP after the itch was scratched?

7

What you mean the don't do evil license? Is not that what the christians have been saying for 2k years, yet it's a non-standard license?

I consider licenses to be something that is BS in a world where git clone will give you a complete history of all changes - in two seconds. Do you truly think that anyone holds themselves accountable to licenses? How many companies have been caught out cheating? How many companies "bend the rules"? Who is enforcing licenses? Licenses only work if they are enforced and who knows of the internal workings of many companies that make illegal use of Open Source licenses?

What I want to achieve with the don't do evil license is a moral awareness for the people around us, its more about making people aware than making people stop copying my code.

This a philosophical point of view not a legal one.

Btw the idea isn't from me it was IBM in fact:

Quote:

Douglas: That's an interesting point. Also about once a year, I get a letter from a lawyer, every year a different lawyer, at a company – I don't want to embarrass the company by saying their name, so I'll just say their initials – IBM…

[laughter]

…saying that they want to use something I wrote. Because I put this on everything I write, now. They want to use something that I wrote in something that they wrote, and they were pretty sure they weren't going to use it for evil, but they couldn't say for sure about their customers. So could I give them a special license for that?

Of course. So I wrote back – this happened literally two weeks ago – "I give permission for IBM, its customers, partners, and minions, to use JSLint for evil."

Douglas Crockford and JSLint.

1 Like
8

https://flows.nodered.org/search?type=flow

That is one of the core uses for flows.nodered.org? Yes, it provides the npm packages for the Palette, but its also a common place to share and document flows (and collections of flows).

1 Like
9

Woah! Sorry for the trigger, obviously a touchy subject.

Not sure what religion has to do with licensing? And I'm not in any way criticizing the desire, I rather like it actually. However, that is separate from legal license issues.

However, I can assure you that from a professional perspective, there are lots of (often overpaid) people who totally pay attention to this stuff. All I'm pointing out is that even (or especially) open source projects need to pay attention to licensing no matter how much we might find it distasteful. And most certainly it has to be factored in if you want to be taken seriously in commercial and government settings.

10

But there isn't a flow palette manager - don't get me wrong, flows @ flows.nodered.org is great centralisation and sharing of knowledge. But it lacks the integration into Node-RED the editor. I am not criticising, I'm just making aware of parallels: I can install nodes in Node-RED, why can't I install flows into Node-RED? (Ok, I can install example flows from node packages using the importer ...)

I could say the same for collections - how can I quickly install a collection into Node-RED without copying each flow and importing each flow?

I don't want to go-it-alone with flowhub.org, I want to demonstrate features that might be useful for others and with that, have a proof-of-concept that works. Admittedly I removed the "download flow button" from flowhub because I wanted to use a different workflow, i.e., an integrated workflow within Node-RED. Nodes have to be installed to install flows, yes but I don't have to leave Node-RED to install them.

If I install packages in Linux, I don't download stuff from a website (although that was typical before the days of package managers), I want to have an integrated environment: apt-get install banana.

I would - personally speaking - like to have a more collaborative development within the Node-RED community. FlowHub allows that - not very easily admittedly, but someone can pull a flow, then push changes to that flow, which I in-turn can install and modify and upload again. All without leaving Node-RED. (GitHub integration is inbuilt so there is nothing lost.)

Hence there are three nodes for FlowHub:

Screen Shot 2023-11-03 at 12.30.26

Pull to get a flow, diff to check what changed locally compared to the version currently at flowhub and thirdly a push put up a new version of the flow.

Documentation for a flow is the tab info details, so I can describe a flow within Node-RED and have that documentation appear at flowhub.

A future direction I can imagine is remote execution: flowhub could become an execution engine (much as a lambda function of AWS fame) whereby I upload a flow and then can execute that flow - remotely - using a remote execution node within my Node-RED instance. Just an idea that I had. (Spoiler: In fact I've already tried it and works now - see this flow which uses a raspberry pi and node-red as the execution engine)

That, for me, is making flows first-class passengers on airplane Node-RED!

11

Nothing! It's not a trigger, I was philosophising and drawing parallels. But remember: parallels meet at infinity.

I can definitely understand that but Amazon doesn't seem to care much about licenses - I recall something about Elasticsearch I believe. And there was a case with Large-Network-Gear-Company where they had installed Some-Open-Source-Networking package on devices without consent. The developers found this out because the networking package had a specific why of responding to packets - so the developers could prove that the package was on the device.

It's strange that I as a programmer, now have to become a lawyer to program, yet a lawyer does not have to become a programmer to use their PC. /s

12

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.