I don't like very much how node-red represents node. I think they are over simplified in some aspects and that the connections tend to tangle.
I am wondering if there is a mechanism where you can provide a different way of rendering the nodes on the editor. If it does not exist, are there plans to do such thing?
If this is not on the plans, is there at least a formal description of the nodes so others can provide an alternative representation of the editor that parses node-red flows and shows a different UI? Do you know of someone who tried already?
I'm sure others will chime in to ask questions about your issues with the flow editor... I'll say that 'tangles' are (mostly) resolved through using the link in / link out nodes - especially with labels enabled. functionality is best described (visually) using the 'group' feature which draws a border around a set of nodes, colours them, and labels them.
Spending some time learning the available functions goes a long way towards comprehensibility of the displayed flow. To a certain extent you will need to relearn how to visualize and architect your code if you want to feel comfortable with node-red.
There is/was an effort to display the flow vertically - each side has proponents.
Oversimplified probably means you need to enable the label and add comments to the node / group.
- the github project Node-RED IS the plan... There are designs, wiki, trello boards full of things to do node-red · GitHub.
Any different mechanism to render the nodes will have to be provided by some other project.
The formal description of the nodes is the json flow that that the editor creates. It is this that describes what nodes are called and how they are linked together. By parsing that file another program could render another UI if they wish.
'll say that 'tangles' are (mostly) resolved through using the link in / link out nodes - especially with labels enabled. functionality is best described (visually) using the 'group' feature which draws a border around a set of nodes, colours them, and labels them.
Groups are nice, and links has their utility. I also tried creating abstractions using subflows, but I still se lacks on the node rendering.
The main one is that nodes are oversimplified. Having them being the minimum expression of themselves is fine when you want minimalism, but it should be configurable to show more things. For example, the names of each output, or what the expected input is will be very handy and will reduce the amount of things you need to remember. I think that for being a visual tool it requires you to keep a lot of stuff on your head.
Talking about what to show, I also think that the UI can be a lot more dynamic. On the example above, all those details could be hidden when you zoom-out, to see the bigger picture and appear when you zoom-in enough.
The state of each flow is restricted to a single line. Again, I don't think this makes any sense in several situations and it should be configurable.
On top of all that, I think the UI of each node should be more configurable, giving node authors flexibility to build nodes able to represent different UI elements. How nice will be to have a buffer node that shows how much of the buffer is filled in? It can be a counter, but it will be even cooler to have a bar that fills.
And finally the connecting lines. Not being able to decide how they are puts me into trouble many times. Why they have to be soft? I prefer straight lines. Deciding line colours will also be cool so you can have more visual clues of what is connected to what.
Those are just a bunch of improvements from the top of my head.
Json is not a very strict format. You can put whatever you may want there and it will still be valid JSON.
Aren't there any rules? Like this props are mandatory, this is the format of the lines of code, this set of names are reserved, etc, etc?
Of course anything able to read JSON and understand it can display an UI. The thing is, there are still some things I suggested that will not be possible because they can never be part of a flow (like internal state)
"For example, the names of each output" - the output name / label of each node CAN be be specified and appears when you hover over the out node itself.
Expected input is (generally) very diverse - every app expects data to be presented in different ways. That's why there is a full markup editor for storing specifications etc.
The node-red UI is not a dashboard - though there are some elements that CAN be displayed dynamically on the editor like you requested.
"How nice will be to have a buffer node that shows how much of the buffer is filled in? It can be a counter, but it will be even cooler to have a bar that fills." This is possible with some nodes - you need to look at using a Debug node with options enabled:
Line colours have been discussed. Complications with very little / no benefit. What would a line colour show you? Generally, lines show connections... The data flows along the lines. Visualize the data using Debug nodes or Function nodes to get the insight you need.
Good design / architecture results in a flow that is easier to understand than classic languages such as JavaScript / VB.
Keep in mind this is open-source crewed by a small team of people. Contributions / pull-requests are always welcomed along with discussions.
Hello @paulkeates ,
I know all the features you outlined, and I think they are just not enough.
Let me explain.
I know, but I don't want to have to hover to see them, I want them to be part of the node body and be shown all the time. I can't even see both at the same time with the method that exists now. Take a look at my screenshots to better understand this
Again, being able to name/restrict the input will be very nice. What if I want to only allow number payloads? Or at least specify the main property needs to be called fileName without having to look at the documentation tab or the code? Will improve overall visibility and reduce mental overhead (was it fileName? filename? just file?)
If the status line were more functional, I would agree. But all you can show is a single line of text with a small coloured dot. I need more flexibility. I know it is not a dashboard, but it doesn't hurt having available more flexible ui representations.
That is up to each individual. I may decide to colour all my error handling lines on red, and the OK ones in green. I don't think having the option will hurt anyone.
This is precisely one of the weakest points of node-red. The debug output is very limited, disconnected from the flow and only captures when you have the editor open.
I may decide to open a PR, but I don't want to put hours of effort into it just to get it rejected because doesn't fit the project goals. That's why I'm discussing it first.
To answer some of your initial questions, there is certainly a desire to make the node appearance more pluggable - to allow for more varied and custom Node-RED appearances. But it isn't something that's in the short-term plan as it is a significant piece of work to untangle the existing view code.
In terms of the specific enhancements you're interested in, we're always wary adding items that lead to a significant change to the flow appearance. That doesn't mean we're not open to the idea, but nor are we going to instantly turn around and welcome such changes without some consideration.
We'll never have a design that satisfies everyone all of the time, but we do try to get the balance right between function and form.
As for the wire colours - currently the wires don't exist as discrete objects in the flow file format. As such, there's nothing in there to have properties such as colour (or choice of straight vs curved etc). That makes it a much bigger consideration as it would require a change to the flow file format, rather than just a different rendering choice in the editor.
Indeed, it would be a large, breaking change to suddenly start keeping track of wire properties within the flows.json file -- but perhaps that could be an add-on optional file (e.g. wires.json)? That is, if the file exists, use it for display, but if not then use the current default style...
I spent some time 2 years ago modifying the editor to include a pluggable wire routing function, so the shape of each wire would be still be calculated solely from the endpoints of the nodes -- but depending upon which wire shape was selected in the settings menu, they would display as straight lines, or angled like printed circuit runs, or even more rounded cubic splines.
Essentially, each wire would call the wire routing function, to generate an svg element (polyline, bezier curve, cubic spline, arc, etc.) best suited to connect the two nodes, according to that routing style. At the time, I believe the focus was to have a certain "look and feel" across all node-red flows. However, if there was ever a desire to provide alternate wire routing logic (without changing the flows.json definition), I would be happy to work on a PR to contribute back to the core editor. And if done properly, pluggable wire routing logic could serve as the basis for vertical flow layouts (although node and port placement would also have to be pluggable, I suspect).
Anyway, interesting discussion -- but clearly there are far more pressing projects to be worked on, than how to provide alternate flow layouts...
Node-RED is what it is and simplicity is one of its core design features. After all, its main purpose started as a way to enable non-programmers to be able to quickly create and execute logic.
Some of the things you are suggesting are similar to other flow-based tools and I have to say that I would mostly be against them since I've always found them hard to work with. Whereas Node-RED was very easy to install, pick-up and work with.
Dynamic displays are certainly nice - but they are "expensive" both in processing terms and in development/maintenance terms.
There are certainly some additional features related to lines that some of us would like to see. However, are those things more important than other developments? Really probably not since what we have works for most people most of the time and is robust and simple. We aren't trying to reproduce Visio.
I know others mention this - but the Editor is not a dashboard. Though actually you could probably bend the status display to do something similar if you really wanted to.
It might be nice to have a discussion about what might sensibly be added to the Editor display but again, this must be weighed against the available design & development resources and the existing roadmap.
Your preference but not everyone's. Again, consider this against the available design and development resources and the existing roadmap - what is more important?
Personal choice vs complexity and design/development effort and available resources.
We all want things.
This is untrue. It hurts design and development effort, if can hurt stability and complexity.
Again, you are suggesting that added complexity doesn't hurt - but it absolutely does.
Not entirely correct, you can direct debug output to log. There are also contributed logging output nodes.
The bottom line being that many of us aren't against some of the things you are saying but they need to be considered in a wider context. And we need to be sure they add enough value compared to the extra code, resources, instabilities and bugs they might introduce.
Commend, Council, Commend. That is the proper way to begin a discussion where you are proposing big changes or recommending corrective action. In practice that would be: Recognize the work already done and highlight positive aspects of it, make limited recommendations/suggestions, Express gratitude and appreciation for the fine efforts and contributions of others.
A more sensitive delivery makes your audience more willing to listen and consider what you are proposing. We live in a very critical society and if we can relieve some of that in our technical discussions, well then the exchanges become much more pleasant and beneficial.
Regarding wires... I will often place labels as comments on my flows to serve as a simple mechanism for documenting in place.
I have contributed to several open source projects. Sometimes with pull requests, sometimes with suggestions / improvements or bug reports and I usually don't get such negative feedback.
I saw some good arguments against introducing complexity, about limited resources and maintainability concerns. Topics that every open source project has to deal with. Technical and resource reasons are some things I can digest and deal with, but something I can not do anything about are people getting offended because I am suggesting what I think are improvements.
I opened this thread with some honest questions about what is supported and what route I can go with if it is not. I didn't demand anything, I didn't diminished anybody's work, but what I got in returns was a strong river of pushback and what looks like some offended users.
Some people even suggested me to go and do my own thing, even after I suggested that I am willing to put effort into this and open a merge request. Well, that was always one of the options I was thinking, but I decided to ask what others did already so I don't reinvent the wheel.
If human resources are limited I don't think this is going to attract many contributors.
As I said, I also got some good arguments against my suggestions and even some people trying to solve my problems with existing features. I really appreciate all those answers and I am thankful about them.
If you put my sentences one after another like that I look like a 5 year old child, but I don't think it's fair.
I appreciate if we can re-route the conversation around technical concerns, what is possible and what not, internal architecture of node-red and what the project is willing to accept to better support third party implementations of the editor.
For example, is the API full featured? Can node-red be operated headless without any UI and still be feature complete? Is the entire API documented or are there some internal unexposed/undocumented methods used by the editor?
If I am suggesting this is because I have been using node-red for a while and I have some ideas about it. If I suggest those ideas is because I think it is a great tool, and I want it to improve so I can keep using it because I appreciate what it brings. I usually start most of my suggestions with "Thanks for this impressive project", this time I omit it not because I don't think it is an awesome project, I did it just for the shake of brevity.
But yes, I appreciate that node-red exists, I am thankful to their maintainers, I think it is a great idea, I like it and I want continue using it.
I'm not wanting to get at you nor diminish what you could contribute. And it is the end of a long tiring week so probably not the best time to get an easy-going response. However, I spend a LOT of time working with written strategies, business cases and the like and I've had to get quite good at analysing language. I pulled those out the way I did because that's how the writing appears when read cold.
You did start with some quite aggressive language to a forum full of passionate techies. Maybe re-read your posts with that in mind. You came in with very strong and direct statements about not liking things that a lot of us like. So that is very likely to set off a passionate response don't you think?
In my responses I tried to pull back from the emotional response that your wording triggered and to look at what positive things could be taken from it as I realised were the conversation was going to end up.
As I said in my response, we all want better and more for Node-RED and many of us have lived and worked with it for years now. You will have to cut us some slack if we react emotionally when someone seems to want to come in and sweep away a lot of what has gone before without first asking whether other people have the same opinion.
Yes, taking out the emotion a little would get some better discourse going. If you have followed the forum for any length of time, you will know that this is actually one of the best open source forums on the Internet with lots of people spending lots of personal time helping others get to grips not only with Node-RED itself but also all of the related subjects.
I don't believe it is quite there yet but if you look at the roadmap you will see that is certainly an aim. You can certainly run flows headless but whether everything is feature complete I couldn't say, perhaps Nick or Dave would have a clearer view.
I think we all recognise that the API documentation could still gain some detail and we often do end up reading code to get some of the finer details.
If asked to guess, I think that most of us would say that we would prefer consolidated improvements to the existing Editor rather than effort being spent on something else. Again, it isn't that we are against improvements or think that the editor is perfect. It is that we've seen the struggle for development resources and what that does to development times. Lots of us would love to contribute more if we could but time and experience places limits on many of us.
It would be good if Nick were able to respond with some thoughts about which of these ideas works to further the development of Node-RED on its current trajectory or perhaps indicate whether there were some background development that you could participate in that might open up the ability for the editor to be more flexible. That would seem like a great place to start and would benefit everyone? In doing that, it might help you and others understand current issues or limitations that could be further worked on - one step at a time maybe? This part is just some random thoughts from me on a Friday evening though so take them with a pinch of salt if needed.
I think Nick's previous answer is pretty much where we are... there is an aspiration to make the view code more flexible and able to accept parameters to allow greater customisation - but it is a way down the current development path. See the whiteboard for other items that are also up for consideration. - Trello.
Likewise just adding colours to wires, is as pointed out, totally non-trivial as it will break the existing flow format and thus create a serious migration problem for all users... which when balanced against it makes it look nicer... really doesn't stack up well... it may do one day... but I don't think we are there yet.
As for headless - not sure what you mean by feature complete... certainly the intention is that flows in production should not ever need the editor open. That is one reason we keep saying the editor is not a dashboard - that should be handled elsewhere. Certainly the editor is needed for debug - and indeed that could definitely be improved (eg see upcoming flow debugger and flow linter features - see Nick's video from this weeks OpenJS conference - https://youtu.be/Xhx3OoaWIaw ) - and I'm sure there's more. But once deployed the flow should just run with no need for the editor - and indeed you can turn it off in settings. As you can see from the project structure node-red/packages/node_modules/@node-red at master · node-red/node-red · GitHub it is partitioned into packages that can potentially be separated - but again we know that isn't 100% clean or complete - but would need someone to step up and do it to help find out all the issues.
As for API completeness - well - there are no doubt some internal ones that aren't documented - but that is usually as they are still open to change - and if we documented them as-is then they become set in stone - and some of the features on the roadmap we have then become much harder to accomplish. Indeed part of the view work already mentioned is to try to work out what that particular API should be.
I can understand that people gets emotional about one of their favorite tools their used for years. But the thing is, I never wanted to change anything for anybody, it was all focused on allowing more customization, and customization is all opt-in.
I re-read my first post and I still don't find it aggressive. But that may because in my mind all that I wanted was to ask for guidance, help and I didn't wanted to be aggressive.
That said, I'm happy we are leaving that behind and move ahead on actionable items.
When I ask if the API is feature complete I am asking if I can build a separate editor that has all the same features as the official one that just communicates with node-red with the API. If the current editor just uses the API, then I would say the API is feature complete. If the current editor needs to run on the same host as the node-red engine because it "writes some files to disk" or "calls some binaries", then I will be interested to know for what so I can understand which features can not be implemented on an independent editor.
I keep reading that the editor is not a dashboard, and I agree. My suggestions are for better showing the internal state of the nodes, so you have a better insight of what is happening on your flows without having to read the debug log. In some cases, it is even impossible to know the internal state of a node unless the node dumps it to the debug console.
I took a look at the code on github and I find it a bit difficult to navigate for a newcomer. is the @node-red/nodes the library that renders the nodes? How does the editor package interact with it? I want to start hacking it a bit, but I will appreciate some help to understand/outline how things interact.
- the github project Node-RED IS the plan... There are designs, wiki, trello boards full of things to do 
