Noisecraft - anyone heard of it?

Node RED needs a shape mode - black&white, shape for Node-RED - +1 from me!

I played around with RSS feeds and got this flow - it creates nodes from RSS feeds. These nodes can then be searched via the info sidebar - i.e. just as if they were nodes! It actually creates subflows of different colours so that feeds are visually differentiatible. I found it highlight addictive since news continues to arrive with zero duplication and nodes are continually popping up.

Fantastic! Thisā€™ll be a good rabbit hole to go down :slight_smile: looks like some of the projects may not be maintained any more?

To be fair to UML, shapes are more accessible than colours in visual representations, but yes, the 800 page spec can be a bitā€¦ much.

Probably since there is less shape-blindness than colour blindness :wink: It's an interesting thought to pursue: would shapes be simpler to understand? I find myself remembering the form of my flows not the colours. I look at the icons of nodes before the colours. Colours are nice when rearranging flows but generally its form, i.e. icons that I have in mind.

Although really bright colours stick out, the colour scheme in Node-RED is low-key, i.e., no fluorescent, bright colours which probably make them less dominate. On the other hand, bright colours should never be over done ...

For those interested, the end product is over here - there is an animated gif for your viewing pleasure! :slight_smile:

I was actually going to build a 3d bar graph but did the ticker instead ...

Just going back to NoFlo, I was perusing their README can came across these ideas:

Flowhub -- browser-based visual programming IDE for NoFlo and other flow-based systems
noflo-nodejs -- command-line interface for running NoFlo programs on Node.js
noflo-browser-app -- template for building NoFlo programs for the web
noflo-assembly -- industrial approach for designing NoFlo programs
fbp-spec -- data-driven tests for NoFlo and other FBP environments
flowtrace -- tool for retroactive debugging of NoFlo programs. Supports visual replay with Flowhub

Has Node-RED got similar tools? Obviously Flowhub is Node-RED as we know it but is there a command line tool for executing flows.json?

(Btw NoFlo does seem to have stalled with their last "human" updates being December 2020)

This has been on the design log for a long time. I think that Nick tries to push forward on separating the admin from the runtime when he can but I suspect that it is a complex job.

Not sure this is terribly feasible. I suspect it could be done - with a lot of work - but who would have the time and what benefits would it bring?

Not sure that Node-RED needs that does it? I'd have to look at what it does to understand what they mean I think.

There is a flow test helper for Node-RED isn't there?

Node-RED does not have visually traceable flows since it doesn't work that way.

And a lot of their referenced other projects seem to be offline.

testing out Node-RED without installing anything, i.e., for gaining broader acceptance of Node-RED.

I guess this is what FlowFuse does or wants to do, i.e. industry application of Node-RED - so this isn't really relevant for Node-RED.

How does Node-RED not work that way? If I have a test packet and I want to visual how it travels through my flows that could be a very useful debugging tool. At least that is my take on flowtrace, similar to traceroute on a unix command line but visually shown in the Node-RED editor.

So obviously, this is just my view, but the installation of Node-RED is far easier than that of NoFlo. There are helper scripts and my alternate installer is useful for having multiple instances of Node-RED even when different versions. Also, you can install on any platform that supports node.js and even on some ESP32 microcontrollers!

As I say, I think it would be possible to have a browser-only version - and if my failing memory serves me correctly, I think someone already did that in the past? I just don't think that it is quite as necessary as the more complex NoFlo.

Sorry, have the call from the wife so need to go. If nobody has answered this by the time I get back, I'll respond then :grinning:

Should say this isn't a comparison to NoFlo, this is "Does this idea make sense for Node-RED?" question.

I understand that there are folks here who are relatively, for whatever reasons, insular in their outlook for Node-RED, which is ok. I personal would like to spread the word and for that I consider anyone and anybody to be a potential candidate. In that group (not the insular group), there are people who don't want to install before they try - try before you buy.

These people just want to click on a link and have a feeling of Node-RED. Now, if these people are to be ignored, then fine, forget it. However for those that have a broader outlook, the question remains: how to attract a bigger audience to Node-RED?

Is it by applying Node-RED to new problem spaces other than IoT? Or is it by making access to Node-RED even simpler? By having a browser version or a version that one can try out - much as you do with your service (if I recall correctly).

I don't know what the magic recipe is but tossing around ideas is a start.

I think this should also be in the interest for FlowFuse even though their target group is industry. Even industry is made up of people and if these people have experience of Node-RED outside from their companies, then they might also be willing to purchase a subscription with FlowFuse for company purposes because they have made their experience with Node-RED. But of course, this is very abstract and esoteric, i.e., isn't a profitable activity.

Just as an aside, I did also spent the week doing a analysis of the community packages, these are the top 100 download numbers for last week:

It's a long tail which does suggest that Node-RED application is very limited to specific problem spaces with isolated islands of creativity. It's those islands of creativity that interest me.

2 Likes

Interesting analysis! But isn't a time interval of 1 week not a bit too short to make conclusion?

If I e.g. look at the top downloads of my own nodes:

Then only my msg router is in your top 100, but not my svg node (unless I have read over your top 100 too quickly). So you would not conclude that users want to visualize stuff with Node-RED (like floorplans, manufacturing processes, ...).

So personally I would have a look at longer time intervals ...

Not really what I was trying to convey though it came across like that. What I meant was that it makes less sense for Node-RED because it isn't needed as much.

The Node-RED admin interface is an EDITOR and not a DASHBOARD. Also, Node-RED is relatively heavily async so it often does not make sense for the Editor to have visible trails. Also, for anything happening at speed - such as the 100's-1000's of MQTT messages being processed per minute, any kind of general tracing would be of little use. And also, Node-RED flows can be spread not just on 1 "page" but across many tabs. Unless you could isolate a specific part of the flow and limit to single messages, any tracing would be fairly pointless I think?

Not to say that such a tool would not have zero value - of course it would have value to some. But Node-RED was built around the assumption that the Editor is just that. Node-RED is certainly open so if someone wanted to build a visual tracer, I'm sure many people would use it. But it doesn't (in my view) need to be part of the core.

No dissent from me there at all. I've long spread the word on Node-RED - for over 10 years now in fact. :grinning:

Not going to disagree and I really hope that more people will offer up a web-based version of Node-RED that has a low entry point. I know that several such initiatives have already happened.

It is already in use vastly beyond IoT. I'm even seeing enterprise-level uses from big IT outsourcers. And various big tech companies such as Hitachi and Siemens already use it.

Without wishing to seem to put down FlowFuse at all, as a business, it can only meet certain types of use-case because of its size and current capabilities. Please don't think that is a put-down at all, it isn't. But as an Enterprise Architect dealing with organisations spending 100's of millions of Ā£'s each year and working in highly-sensitive areas, it can be hard to introduce an open source tool such as Node-RED. For example, we would need significant investment in security testing and accreditation to be able to introduce it as a core tool. This would naturally be hard for a small company like FlowFuse to break into. A classic chicken-and-egg problem.

But I'm drifting off-topic now.

Indeed. And some of that does come down to how the core of Node-RED is architected. Some to the use of JavaScript.

JavaScript for example, is relatively poor at handling data science and analytics tasks. Something that I find endlessly frustrating.

Indeed. But which are you focused on? The existing islands? Or the gaps?

Which would bring more people to Node-RED?

Tricky isn't it. When I look at my own stats, I see a pretty consistent 2000 or so downloads per month for uibuilder and its been that way for the 6 1/2 year lifespan. I find it hard to believe that out of the total 193,155 downloads, even a small fraction of those are actually in-use. So are downloads even a useful measure of things in use?

I could certainly change that interval but will it change the trend, probably make the tail even longer.

I found this also interesting:

those are the updates per day for the last 2000 days i.e. overall updates made per day over all packages. Mostly less than 10 updates per day to Node-RED packages. Which either means that little needs to be fixed or there is a core of developers that maintain Node-RED packages or there are many individuals who post one or two versions of their package and then don't maintain them.

Although if you look closely (today is to the left), the number of updates is slightly increasing, showing more interest in maintaining packages. A good thing.

1 Like

Possibly at least in part due to the recent hacks. :frowning:

I think it is a mix. My moment node for example virtually never needs updating because the package it relies on is very stable (actually, it is on life support and it really needs changing to a different library - if only I could find the time) and yet it was in the upper 1/2 of your earlier list.

uibuilder though has a higher than average number of dependencies and so even if I wasn't actively developing it, it would need more rapid updates.

If I knew that, I won't be here having this conversation! All I can say, spread the net wide and see what happens to get caught. Those user surveys that get done every year would be a good starting place but is that new users or old users? How would you find topics for new people? By asking the existing users? I don't know.

Literally one of the first questions I got was: can I replace my data pipeline tool with Node-RED. Answer: No, keep your Airflow. But in way that's good, it means that Node-RED isn't perfect since of perfect things people are suspect. So having floors is a good thing in this case.

It seems that many companies seem to think so, downloads are everywhere - from the app store to firefox being more popular than chrome (or whatever). It's the numbers! Gotta be doing the numbers to have the insights :wink:

But its important to see trends and not specifics when looking at the numbers, the numbers only provide hints for the underlying story, numbers aren't words.

Yep, the stories one creates from numbers .... still its interesting looking at the numbers, here's another:

These are the updates made to packages grouped by year. A steady growth which either means more packages have been added or maintainers are updating more often. My suspicion is that more packages have appeared rather the older ones being updated more. But definitely a good sign if those packages are also maintained. Although, as you rightly point out, many don't even need maintenance since they just work.

Which does make me thing of unix tools: these are solid and rarely get updated (talking of sed, awk, ls, &c) - why might that be the case? In comparison my mobile apps get updated every week nearly. The reason is the Unix principle that each program does one thing and one thing only. I think we're seeing that principle at work here. Nodes are specific little components that only do one thing and that really well.

Along with that, Node-RED itself hasn't modified its core so that all nodes would stop working. That's great! Look at Android or iOS - each update forces thousand of developers to update their apps because some API changes to the underling operating system.

Ok, enough story telling ... back to the numbers!

2 Likes

But I WANT my JavaScript!

ē”œé¦Øļ¼Œč·ŗ脚ļ¼Œē”Ÿę°”GIF

Because they do one thing and do it well. :slight_smile:

1 Like

Ok so here is the same but with the last months download figures:

not much difference ...

Just one more stat before I write all this up: how long are maintainers onboard? I.e. actively maintaining their packages.

On average about 5 months. Although the mean is probably a more sensible value to take in this case, 16 months, so about a year and a bit. But the 90th percentile does indicate that there is a solid core of developers who stay around longer.

Are you assuming that a node that does not have any updates for a period is unmaintained? I don't think that is a good way of determining that. Better might be to check for unaddressed issues.

If there are no issues, or issues that are submitted are at least replied to by the author then the node is still maintained.