Reimplementing Node-RED in Rust: EdgeLinkd project update

Share Your Projects
41

Well, there is an element of truth in this statement but really not so much. Node-RED is a runtime server environment, in fact, that is often what I use it for most. It is the convenient environment I use to run code - whether data transformations or user presentation - embodied by flows.

This is the thing about Node-RED, there is no One-Right-Way to use it. I often use it to host a data extract (using nodes) and transformation (often using a function node or two) and UIBUILDER for display and user input. But others use it differently.

And this is the key thing about Node-RED. It is a general purpose compute environment. Using the common language of the Internet. Anything that restricts this is a retrograde step in my view, anything that enhances it is interesting.

42

And AI isn't? Especially when combined with a bunch of agents utilising MCP servers.

I think a lot of what you say is muscle memory and convenience: the way you solve your solutions is related to your experiences with NR. Anyone starting out will not have those experiences and will take a different route, probably using LLMs to create their first programs.

There is literally nothing special about NR other than it implements a visual programming interface and uses a data-flow-programming or flow-based-programming paradigm for creating programs. AI by default doesn't do that but can if told to do that, if so desired.

The other thing NR could excel at is giving non-programmers a visual entry to programming. Much as Blocky or Logo or Processing do, NR could provide a tool for teaching folks how to program in a data-first approach (as opposed to functional first or object first approach).

One further interesting use case for NR could be programming-language agnostic visual code sharing tool. And that's where EdgeLinkd comes in. Sharing visual code that can be equally well be run in an Rust environment or a NodeJS environment (or Erlang), without modification, could be an interesting approach to leverage the abilities of different computing environments. Something that Java promised about 30 years ago: "Compile once, run everywhere". In the case of NR that would be "Draw once, run better".

Or NR just remains that IoT tool used for routing data from A to B via MQTT. This has always been its initial and main use case. It has morphed into a home automation tool for turning lights on and off. Or as some describe it: it's that duct-tape solution for connecting various other systems that actually do something. The "do something" part I added but that implication is clear if you describe NR as being a duct-tape solution.[1]

So NR might well be a glorified ENIAC machine along with most AIs today. NR might become the COBOL of the future with many companies still having NR installation that "duct tape" "do something" systems and that "no one should touch" for fear of breaking something but that no one neither understands.

[1]: "do something" are software systems that management sees as customer-facing, profit generating systems. But since companies grow over time and folks leave, there are many "do something" systems which become independent of one another. So "duct tape" software is required to connect the "do something" systems. Unfortunately management does understand that "duct tape" is just as important as "do something".

43

I think that to limit the description of Node-RED to 'morphed into a home automation tool for turning lights on and off' rather limits it's abilities. True, there are many on here who use it for that, and do so successfully, I hold my hand up, I am one. But NR is capable of so much more.

In past life, I was an Engineer who fixed things in a Manufacturing Industry. Many of the Systems I worked on involved pieces of equipment that were very good at doing their own specific task. These were items brought together using a controller that acted as 'glue' (Duct Tape), why re-invent the wheel?. You might have such things as Vacuum Throttle Valves, Vacuum Pumps, RF Power Supplies with tuning coils, Mass Flow Controllers, Developer dosing pumps, various Gas/Fluid Control Valves etc, etc. These systems were all brought together by the 'glue' to create a complete system that Operators were able to use, through a Human Machine Interface, able to load a recipe for the system to complete a specific task in a defined way.

The Interfaces on all these disparate systems ranged from a simple On/Off to complete Interactive Control involving real time feedback. However, the driving technology could vary from a simple wire, a Serial or Parallel I/face, Custom signals, MODBUS, SCADA, SECSII Protocol and all variations in between.

I came across NR after I retired, but to have had this while we were building systems, it would have made our lives so much easier. Never mind the PLCs, or the Custom Processors and Interfaces, all of which have their own methodology, this would have been a great asset. But then I suppose when you could have been paying up to $1M+ for a piece of equipment, I suppose it is a good way of keeping IP 'clouded' and justifying the price!! Especially when the Companies Engineers are called out to fix the systems on site (too big and delicate to move).

But then I see this from the point of view of a 'non-software writer'.

44

No, of course not.

So-called AI is mostly a Large Language Model hosted in an input/output engine. Any compute capability is secondary to its main function. MCP Services are "simply" lookup engines.

Indeed, which is why I was careful to say that was my way of working. And equally careful to say that one of the main benefits of Node-RED is that it can be used in many different ways. It is that flexibility that makes it stand out and makes it useful whether you have no programming experience or tons.

We may need to disagree there. Node-RED isn't just a programming "interface", it is a programming "host". A flexible way of hosting many, sometimes wildly differing workflows within its unifying environment. It is a programming server/service, it hosts/runs as well as programmes.

But that isn't its primary purpose and I suspect that you would either rapidly run out of money or be kicked off the platform if you tried to use it as a program host and server. Even if running your own AI, it would be horrendously expensive to make it do what Node-RED does easily and cheaply.

Totally agree, and I believe many people do use it this way.

Perhaps. It is certainly an idea worth exploring. However, as I've said multiple times in the past - visual programming is useful until it isn't. Like any visual paradigm (e.g. flow and other graphing and charting methods), there is a point at which the visual "clutter" overwhelms any usefulness. And this is, in my opinion at least, the reason that the ability to embed written code into the visual model is so important and useful. It is a key aspect of Node-RED (and most other general purpose flow-based tools).

This could not be further from the truth. I've seen it used in so many different contexts including large-scale ETL, SCADA and other commercial contexts.

Sorry, time for my Tai Chi lesson - will come back to this later. :smiley:

45

Hm don't know where you get that from, but first link from ddg:

What can MCP enable?

    - Agents can access your Google Calendar and Notion, acting as a more personalized AI assistant.
    - Claude Code can generate an entire web app using a Figma design.
    - Enterprise chatbots can connect to multiple databases across an organization, empowering users to analyze data using chat.
    - AI models can create 3D designs on Blender and print them out using a 3D printer.

MCP allows definitely more than "lookup" table. And when AIs start solving obscure and ancient maths problems, you do start wondering whether Node-RED will ever solve such maths problems or support lean proofs. Then there was that "incident" with OpenAI agents hacking into hugging faces to cheat their way to a solution.

Don't know where to start but AIs are literally solving maths problems and aiding in discovering new super conductors. Pretty good for a randomised, brute force, probabilistic prediction calculator. Even creating a fly brain simulator in three hours is possible.

And yes, those examples probably cost much more than using NR but if folks are doing them, then that becomes the new "cheap". If folks are ok with paying X hundreds/thousand/millions yen/dollar/euro/pound for their tokens to get the bucket of satisfaction, then that becomes the new cheap - because, thanks to capitalism, people don't do uneconomical things.

And perhaps there are folks out there solving really hard problems like whale communication using Node-RED, but they aren't here talking about it. Which is a shame.

So all I can assume either there is a giant silent majority doing really cool stuff with NR but not talking about it nor sharing their knowledge, or there is no such silent majority. All I ever read here is that "I've seen folks using it for ...." but those folks aren't here talking about their solutions.

Perhaps folks start out using Node-RED but then hit a wall and move to something else. That might also be a reason for "having seen folks" (i.e. past tense). Certainly I have heard of several folks moving on from NR, that certainly also does happen.

That's probably one point where folks do move on.

Much as the "clutter" many companies are now experiencing in their text-based codebases and hence have started using AIs to make sense of their codebases - to create documentation for their codebases.

Clutter seems to be a commonality shared by many human endeavours.

46

Quite possibly. But these are exceptions and not the rule.

There are amazing things happening with domain specific AI's - that's been true for some years and I've been involved with some in the health domain. Nobody is disputing that AI can do amazing things. But this is far off-topic. If you want to use generic AI (e.g. LLM's) to run your home or do your shopping - sure, go ahead. But please don't complain when the bill comes in.

Jury's out on that. Personally, I expect the cost to fall and it already is in some areas, but not in most of the leading generic AI's. I can run Node-RED on a v1 Raspberry Pi of pennies for a whole year. And, with some care, I can get an AI to write some code I can run on that Pi - under Node-RED (or not) - also for pennies. But to get AI to run my compute - well, good luck with that, let me know how you get on. I look forward to being proven wrong.

This is exactly the case. One example I was involved with is the consultancy giant Accenture who I know (because I reviewed and approved the architecture) use Node-RED to gather and coordinate enterprise-scale data to feed into their reporting system for part of the NHS. But we know of other examples beyond the more well documented examples of Siemens and Hitachi. Not everyone wants to shout about using Node-RED. Some of these may not be "hard" problems because Node-RED probably lends itself to quiet, long-term, background problems. But that is exactly my point. Of course AI has its place. But so does the "boring" old Node-RED compute platform.

Which kind of makes my point. Node-RED needs function nodes (or something similar) and the flexibility to call out to externals (exec, custom nodes, ...) to allow visual clutter to be collapsed when needed. There is no Right or Wrong here, just a need for flexibility.

Indeed. And one of the interesting use-cases for AI is to get it to reduce clutter, not just document it. Still, that doesn't take away from the original discussion.

To conclude - an important aspect of Node-RED is its FLEXIBILITY. Visual programming is useful to a certain point. Node-RED lets us bury complexity when that is useful. All the while, giving us a convenient and easy to use PLATFORM for running a variety of workflows.

My original point was that tools similar to Node-RED (e.g. a Rust version) will be far less useful without that inherent flexibility.

47

So your original comment:

was related to to:

Imagine you're the person presenting some tool they have worked on for months and that they are "pretty' proud of and then imagine someone like you coming along and saying "pretty useless".

How would you fell with all your years of experience working as an "enterprise architecture" when someone just says "pretty useless" to the work you put in? Especially since the "pretty useless" is not helpfully provided with a clear explanation for improvements or tips for improvements or with considerations of possible limitations of the underlying technology.

It's nice that we got to the point that you explain what you mean with "pretty useless" because I didn't understand what you meant and that you meant flexibility wasn't clear to me.

Indeed a tool that doesn't have a function node or JSONata is not as flexible as NR but then again AI is far more flexible than NR. Using natural language to program something will always be more flexible but also less accurate than drawing lines and rectangles.

So I can also understand the authors comment:

So Node-RED could become "pretty useless" in the age AI because it doesn't provide the flexibility that AI does.

If NR were augmented with alternative implementations in different programming languages, perhaps that could provide more "flexibility" for NR (or rather the visual approach it takes). But that is a "shared" flexibility between different implementations of the concepts of NR and would also a provide a larger platform for running a variety of workflows.