Hi, I'm new to Node-RED, it looks very impressive. I was thinking about building a workflow processing system on a server and eventually found Node-RED. I see it can be very effective for many workflows. I had a look into the docs and searched this forum but I am still wonering if I can use Node-RED as a tool to define a workflow and then pull the workflow description into another service e.g. I define an algorithm using the workflow UI of Node-RED and then another service/node pulls that entire workflow as a formal description and executes it locally (e.g. it may run in a single process so there is little/no overhead)? Another way of thinking about this might be - is it simple to use Node-RED as a frontend and use a different servce to execute a flow (e.g. it might just take the entire flow and send back the output to Node-RED) ?
Hi @MarkNZed
Welcome to this forum!
The flows Node-RED creates are held & stored in a JSON format, that usually can be read by a lot of tools. The challenge is to create an alternative runtime to execute this definition & bring it to life. We've realized this idea for node-red-mcu ... and it's not simple! 
EDIT: @BartButenaers recently did a nice sketch of the ecosystem we created for node-red-mcu that might give you an additional insight.
Thanks @ralphwetzel That is a complicated diagram 
Where are the flows stored in the default Node-RED setup ? Is there some modular functionality in the Node-RED stack for dealing with those flows ? I guess what I'm asking for is an example of a small kernel that runs a flow. If I understood the node-red-mcu does not use the same JSON description of the flow.
For a standard installation not using projects: <user-dir>/.node-red/flows.json
The flow is just a mean to organize a bunch of nodes. The logic sits in the nodes. The Node-RED runtime does little more than spawning the nodes & providing a system for message forwarding between the nodes.
It starts with the same description ... that is then run through a lot of optimization steps - and in cases adapted to a different runtime implementation of a node.
I understood there is a processing step that means the JSON is not run on the MCU (it is trnasformed into an executable on the MCU). Which seems a lot more complicated.
What I want seems much simpler - the flow (not the code in nodes in the flow). The projects concept seem really neat, so may I would be better to assume that? In the case of a project is there something similar like the flows.json for a specific project?
Is there a schema for the JSON defining a flow?
Thanks!
[edit: maybe something that takes https://github.com/node-red/node-red/blob/master/packages/node_modules/%40node-red/runtime/lib/flows/Flow.js and replaces the remote calls to nodes with local calls (functions if we stay in node.js). Then I guess translating that code into anoher language would give a kernel in a different language).
You seem to step into a sidepath here. It doesn't matter if you pre-process the json file - and in fact, this was / is one of the easier tasks.
There's a lot of documentation available that describes the core principles of Node-RED. My proposal is always to make oneself familiar with that before digging into the details...
Yes, there's then a flows.json per project.
I don't remember. Basically it's an array of nodes, organized as a linked list by node.id.
Maybe. You could as well look into nodered.js to get an impression of what's necessary to create an alternative runtime.
is it simple to use Node-RED as a frontend and use a different servce to execute a flow
Doubt it.
Instead you can pass on what data you have to node-red, have it process and let it return output.
It literally takes 10 seconds to define an API endpoint in node-red.
But it might be nice to define the workflow in that other service using the Node-RED GUI.
In general that may work but I would still be reading the docs about the first system I came across if I did not dive into the details and discard it for some reason.
I could not find much relevant documentation. This Trello has the "Internal Runtime API docs" but it is empty. Here API Reference : Node-RED there is no mention of that API. Regarding the developer details it states " If this is an area you are interested in, come talk to us in the forum or Slack team" which I (erroneously?) did.
What I was trying to get across: The documentation gives you an impression of how default Node-RED works. Once you (or other) understand this, you may find a point to position the lever for making Node-RED do other things as well - like the functionalities you're looking after.
From personal experience: Node-RED is a brilliant piece of software; if you're interested in doing something freaky like modifying it's core, most of it's secrets yet can only be discovered by reading the source...
Additional remark - if you're just looking after an easy way to get the JSON data: There's an Export function available in the menu; you may use this to copy the json to the clipboard...
I fail to understand what the advantage would be.
Could you elaborate on the use-case ?
Workflows might be quite good for describing algorithms.
So we should ask questions after having mastered the tool? I am not blaming anyone for not having answers but I get the impression you are defending not having answers. There could have been people doing this already and I don't see the harm of asking on the forum.
I was trying to support by sharing with you the experience I made.
But hey, I can live with that impression... .
The schema of the json file for a flow is all that complex. It is fairly easily understood just by reading it. It is just a set of configuration information that describes, nodes, groups, tabs (confusingly called flows) and wires.
But Node-RED is already sophisticated workflow processing system - as others have commented - is there much to be gained by using a different editor? The Editor already has a number of features that lets you change the look and feel and control what people can see and do.
The Node-RED runtime is slowly being separated from the Editor but isn't quite there yet. But it is the runtime that orchestrates the code loaded and executed as standard Node.JS modules in a way that lets the flows pass information between the nodes.
You also talk about handing off workflows to another process and Node-RED can already do that in a number of ways simply by managing flows and passing data.
The question was not about using a different editor but using a different "runtime" or combining different runtimes. The editor does look good. Regarding the handing off of workflows it was about transferring the control of a workflow to another process (i..e not simply IO)
Nodes have their own speciality, some require compilation when they are installed or binaries, others require credentials that are encrypted and stored in separate files. The nodes themselves are living in their own realm, other than that there is a visual wire between them they are not aware each of other. Next there all kinds of complexities with environment variables, flow/node/global variables, links, subflows, link call nodes. what does it solve when you can serve the data to node red directly
What you are describing is how the "backend" of the system is typically used. But the "frontend" does not need to assume that nodes have a low communication cost compared to their computation cost. If the nodes have high communication cost relative to their computation then the current backend (or "runtime") may not be appropriate for a particular flow.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
