Following up on a long discussion and a previous discussion and also the pull request, I've come up with a flow that demonstrates various approaches to auto layout. To use the flow, you must install the introspection package which includes the ClientCode node.
I would have included the flow here directly but it's too large for discourse (see screenshot)
The flow is hosted at flowhub and if you install the flowhub nodes, you can pull the flow directly into Node-RED, flowid being: 291e3f0b7f652839
These layout engines (elkjs and dagre) can be used with any flow you happen to wish to layout, you can play with the layout algorithms and even use other engines. The whole thing works within Node-RED client, i.e. browser.
Unfortunately I don't get it running. I think I have replaced all the escaped html characters by their original symbols, but there are quite a lot (in conditions, lambda expressions, ...). So most probably I fixed one of these incorrectly...
A couple of questions about your demo with the 2 layout engines (elkjs and dagre). Not sure if I ask them too soon, because you had not much time yet to experiment with this...
Are these the 2 best layout engines for Node-RED (for some reason), or is there perhaps another reason why you choose those two (e.g. because they are the most easy to embed)?
Are you going to explore other engines, or are these good enough?
Is there already one of the two that you like most for some reason?
Do you think one of those engines is good enough to be used in real life?
So these engines offer lots and lots of options. I assume most of these can have default values that will do for all/most users, and that only some of them are useful to be customizable by users. Do you already have a subset in your mind, that might be useful?
Are you already aware of some severe limitations, that makes it hard to build a general usable solution?
Do you have plans to integrate this (e.g. using a plugin) into the editor in the near future, or are you just having an immense amount of fun with your lovely experiments?
Thanks for sharing your knowledge and endless creativity with us!!
Ay, that there be many questions! As proviso I have to say that is this is very much an experiment, it's a side track of other things I'm doing, so no, I'm not going to push this to become part of Node-RED - that's for others to decide. I'm happy to help where I can, if others want to run with these ideas, all the better!
The only reason was that a) dagre there was example code from @lizzardguki and b) elkjs was the simplest to implement from the list from @TotallyInformation (it wasn't react related and seems to be stable and in usage). My main intention is for others to say "hey, I've got experience with engine XYZ and here's a flow that shows me how to integrate it and test it in Node-RED - let me try!". I built the flow to be extendable: extracting the including of a javascript lib, extracting the moving of the nodes (provided the right array of nodes is passed over) and I extracted the selection of the nodes which should be laid out.
All that was together in one client code node in the original flow
Probably not since dagre does a pretty good job - for my tastes but I'm not doing this for laying out regular flows, rather flows that I generate on the fly from other sources. Also elkjs has some 198 options for 12 different layout algorithms! There is plenty of room to experiment - IMHO
I'm simply not a layout engine expert (depth and rank where new to me!) so objectively, learning the meaning of 198 options for something that is very much personal test (everyone has their own opinion on what is a good layout) is a little off the radar for me. Hence if someone has the knowledge, then that would be great.
What I'm trying to do is make experimenting with auto layout simpler since there is no checking out of branches nor hacking around in the browser console, it's just hacking around in a flow in Node-RED and pressing deploy.
dagre since it simply did a good layout from the start - admittedly from taken from the example code. Although if elkjs mrtree layout algorithm would go right, then it might will also be a good candidate.
How long is a piece of string? What is real life and what is good enough? I do think that elkjs must be able to do the job, after all it's from Eclipse layout engine ... if they're using it ... but it's a matter of experimenting with the various options. There are a bunch of examples and configuration possibilities ... it's a rabbit hole!
Not gotten that far but you're right, a subset or options for configuration would be great. On the other hand, why not have flows that have different lay-outing possibilities? Have a competition using the flow I've built and everyone can configure their own examples. One could define a set of ten different flows and the person who comes up with the best configuration options for a layout engine (any layout engine), gets a free subscription to the Node-RED monthly magazine! Of course, before that, one would have to have a Node-RED monthly magazine
I've been playing around with very simple flows and nothing bad has yet happened Whether this is a general usable solution depends on whether another JS library should be bundled into Node-RED or whether an inbuilt solution would be better - after all these libraries aren't tiny - dagre 100k v. elkjs 1.5MB - space v. time!
I'm pursuing other things but will help if others are interested in moving this along. I have my solution and am happy to work with an extra node - i.e. I don't need to have it integrated into Node-RED. IMHO it's definitely a feature that should be in Node-RED but whether that happens ... there are others here that have more to say about that.
@gregorius has created a flow to experiment with the different layout algorithms. Hopefully some folks here can find some time to play with it on different kinds of flows (straight lines, loops, ...) and share here their experiences. If one of the algorithms can solve most common flow layouts, it might be a good idea to integrate that one in our flow editor (via a plugin or whatever...).
Hi Dave,
One of the algorithms that @gregorius demonstrates already in his example flow, is the algorithm from that pull request. That way people can compare all the algorithms currently available, and discuss here which they like most and why...
In fact the animated gif above is the the algorithm from #2267 - since the entire code is frontend, I put it in a ClientCode node to be executed when triggered (btw the gif is from an earlier version of the flow).
I created this because I wanted to do some auto layout'ing for something else I was working on and the PR was not on my radar - else I might not have tried out dagre and elkjs, who knows.
As I said, this is an FYI not a request to have auto layout in Node-RED itself. At the moment, I prefer the mediative work of aligning nodes by hand and won't want the computer to have that pleasure!
Thanks!
I installed it, and I really love this node
I have one feature request. Currently the nodes are rather close to each other:
For some users that will be fine, but I like to have them a bit more apart from each other.
Is it possible (in your limited free time) to implement something like this:
Your node's config screen shows a json typedinput field.
When another algorithm is selected, the most common used properties (and their corresponding default values) are automatically being displayed in the json.
You can edit property values (e.g. distance between the nodes), and those changes can be deployed.
From then on these updated values are being used by the the algorithm in this node.
When another algorithm is selected from the dropdown, then the typedinput is again filled with the default values of that algorithm.
And perhaps a hyperlink on your readme page to all available settings.
Feature requests are best done as pull requests with code modifications and additions - the code is all there and just as your time, mine is also limited. I only created the node because I'm testing nodedev setup - in this case adding template nodes to split code up.
Another curious node :') nice, personally i like them a little bit closer, but besides this point - would it be possible to add a checkbox that also performs a "hide node labels" at the time of re-layout ? I am terrible at node development (although I get the "idea", the workflow is quite cumbersome and confusing to me).
No that is not trivial, but why would this be important? You can export the flow as an SVG and then edit it that to remove the labels.
Yes unfortunately that is the case. The workflow is still very technical however every node developer has to master those steps eventually, be it outside or within Node-RED. There is no way around it.
Using Node-RED at least provides the node developer with a familiar environment to do that development.
Thanks i dont mean removing the labels but hiding the node labels, it reduces/minimises the nodes to a little square not sure if there is an api call for it (i use the keyboard shortcuts to do this currently)
I'm curious why you are publishing all of these utilities as nodes - given they are purely editor enhancements. You could be publishing them as a plugin that adds everything to the editor without expecting a user to drag a node into their workspace to make them functional.
You could then register an action so a keyboard short-cut can be assigned, have them show up in the Action menu and so on.
How do I install a plugin within Node-RED? How does a user install a plugin? How do I define a menu entry for a plugin? I had a look at Documentation : Node-RED but there was nothing on plugins that I found.
Is there a manage palette equivalent for plugins? Is there a plugin download or plugin collection @ flows.nodered.org? Plugins don't seem to have a first-class representation as nodes or flows do.
I think the best thing would be that auto layout be integrated into Node-RED but that has been tried and failed, so instead people go their own way. What does it then matter whether it be a plugin, node or flow - it gets the job done. The nice thing, this being a node, is that I can also trigger a layout event in a flow.
Also, I created this for me much as flowfuse creates features for themselves, but I shared it in the form that I need it. I'm not here to create feature per-request from non-paying users, much as flowfuse is not in the business of giving away money.
@gregorius,
I am by far a specialist in this area, so others are welcome to append or correct my information!!
First of all I am already a fan of your layouting node. I am just wondering - like Nick said - if a node is the best way to store this kind of functionality. NOT saying at all that I expect you to re-implement your stuff. Because nobody has to tell us open-source developers what to do in our free time . But I like to discuss and compare in this community the different ways to implement something. Because that way I can learn a LOT from feedback of other users. And at the end of these discussions, I have no free time left to implement their feedback
I "think" these are the available ways to implement something like this:
Using a node. When I want to layout the selected nodes on tab X, I first need to add (or find) your layouting node. So although I love nodes, it somehow feels not the best option in this case.
Using a plugin. I must admit that I am not familiar with plugins in Node-RED. This is the only thing I know about them:
The plugin framework has been introduced in Node-RED version 1.3
It has been implemented in pull-requests 39 and 2779.
You can find an example in node-red-contrib-plugin-header. Although I am not sure how it works: because this one is registered as plugin in it's package.json file, but I don't know where the RED.plugins.registerPlugin is being called. Or perhaps that is not required. Don't know...
So I "assume" you could for example use such a plugin to register an action (via RED.actions.add as described in the docs), so your layout algorithm could be triggered via an action in the flow editor. But perhaps other things are possible with plugins, which I am not aware of...
However I "think" plugins have some disadvantages for this use case:
When I want to change the properties of the layout algorithm (e.g. distance between the nodes), where can I adjust that when there is no config screen available? You could show a popup dialog, but that will become annoying when you don't want to change the properties.
When I want to save my updated properties of the layout algorithm (e.g. every time I trigger the algorithm, I always want to have the same distance between my nodes). Since there is no node involved in a plugin, I assume there is no way to deploy my updated properties? But I might be mistaken.
Using a sidebar. This seems (IMHO) the best solution:
There is a visual sidebar, where you can show and update the layout algorithm (in a dropdown) and also the corresponding layout algorithm properties.
There is a config node in the background attached to this sidebar. The layout algorithm and the updated properties can be deployed inside that config node.
The sidebar is always at your disposal, whenever you want to start auto layouting.
A sidebar is a mean to create a UI for a plugin (if the plugin needs a UI).
A plugin may store its (config) data wherever it likes, e.g. in a subdirectory of userDir.
Plugins may be registered at the flow library like any other node, and installed via the palette manager. Further / full support via the palette manager is in the making - short final.
For an example, you may reference to node-red-mcu-plugin.
@ralphwetzel,
Thanks for the addition! Indeed I should have mentioned that. It is indeed correct that your node-red-mcu-sidebar is a plugin (as we can see in your package.json file). However my /node-red-contrib-xterm sidebar was developed in 2019, long before the plugin mechanism was introduced in Node-RED (version Version 1.3 was released in 2021). So the original sidebars were no plugins, which made me forget to add it in my overview above...
From that point of view, it indeed makes sense to implement an auto complete feature as a plugin that visualizes the algorithm settings in a sidebar. And that these settings are persisted e.g. in a node via deploy (or somewhere else). Back on track
Ok that's great to know, then there is a certain amount of visibility that a plugin can gain.
A feature I'd prefer to implement as a plugin would be a visual unit testing - far more interesting and would make Node-RED that much more approachable for non-developers.
It seems a shame that no one has made an effort in that direction since it would allow people who do testing to also have a use case for using Node-RED.
