Wish for new Nodes

General
1

Hey guys.
Firstly, I'd like to thank everyone here to help on this project. I tried to build the very same multiple times and failed, now I found this project and I LOVE IT!

Unfortunately, I am not into JS-Programming, but I have some wishes and since I did not find a good place to put that topic on, it is now in the “General” area. Here we go:

  1. I'd like to communicate with Virustotal, with and without API-Key

  2. I'd like to communicate with MISP, local and cloud.

  3. I'd like to see the internals of the data packages in the move from A to B on the node itself, not just on the debug-window

  4. A UI-overhaul would be awesome. It would be way cooler if the UI would look and behave like, e.g., Unitys Shadergraph. And the grid should be zoomable and moveable infinitely,

    image
    image922×649 39.2 KB

    LInk: Pure Circuit - Grafisches Programmieren (Node Editor) - PureBoard

  5. A better way to link my own programs: I can call an exec-node, but then I have to care about so many things like: where do I store the file, how does Linux or Windows behave (would be nice to drag-n drop the pe/elf file inside and make it store somewhere internally, give it a name and save it as new node, exchangeable. Also, the input should allow infinite long messages. I think currently, only stdin is supported which is limited to a few KB.

  6. Better integration for Layouts. It should not be necessary to import them manually, there should be a simple Dropdown to select it. In best cases, that is an individual, client-based thing.

I know I ask a lot, but this will make the project grow even larger.
If someone feels competent to care about one of these issues or is just a good JS-Coder, I can offer help, since I already solved all the problems above in other languages. So the code just needs to be adapted.

Thanks alot for reading& even more fore participating

2

Where to start replying to this :slight_smile:

!=

:slight_smile:

3

You can already do that - see the http-request node. Actually, I'm making an assumption about MISP as I've no idea what it is or how it works - but if it has an API interface of some kind, you can almost certainly talk to it.

Why? A debug node gives you everything. With the minor caveat that if you have a massive msg object, it might not show everything and you may have to filter it down (max size can be set in settings but making it too big slows things down).

You can also output to the node-red log so that's another option. And you can create custom logging for specific purposes. I've even created a custom log that outputs to MQTT and a uibuilder web page that listens for the MQTT-based log data and displays it on its own page. I use it to get trace level logs just for uibuilder without all of the other trace outputs for other parts of node-red.

Why? The current layout is simple and effective. Also, the example you give has multiple node inputs and this would be inappropriate for the way that a mainly asynchronous, flow-based, message-based tool works. Further, the example you give is too simplistic to be realistic. A more complex flow without names showing would be a nightmare. Similarly, once you start needing names for clarity, the odd shapes become unmanagable - I know because I do a fair bit of diagram drawing in my professional life including "standard" diagrams with odd shapes.

Really? Infinite?! That is certainly a recipe for a browser crash. :wink:
However, it would be nice if the flow workspace were a bit larger, I'd agree with that.
Zoom isn't really an issue though I don't think?

Well, you can kind of do that right now if they are either: (a) JavaScript modules, or (b) C++ modules callable from node.js. Both of those approaches work fine and are inherent in how node.js works. They are also cross-platform.

An interesting and nice idea, to be sure. But I suspect extremely hard to do. But maybe someone with a bigger brain than mine can work out a way and provide the programming to make it happen. :slight_smile:

There's that i word again.

Can you explain this one? What "layout" do you mean?

4

Hey there and thank you for the long response :slight_smile:

Starting at the top:

  1. Virustotal and MISP
    I probably have to dig deeper into that, thanks for the info.

  2. Seeing internal Messages
    It would make it easier to see which data is available. Let's say there is msg.payloag and msg.topic, then it'd be nice if I could just drag a connection from msg.topic to my new node, without having to type anything. So I do not have to care about coding something and always looking up what I did code there. Visual representation is juist simpler, imo. I woudn't ask to discard curent features, just extend it. In the end, I'd love to see Flows that build up 'highly layered' chuncks of data like a complete TCP-Protocol without a single line of code. (Building all the layers and concating them.)

  3. UI-Overhaul:
    Multiple input are already possible, so that one is not the no-go.
    Having no text on the node makes clearly no sense, I agree with you there. And that the UI should be simple, yes. But would it not be as simple as now if it just has some better colors and more responsive UI-Elements. I am not talking about changing the shapes of the nodes, although my example picture was choosen poorly, I'd like to see vector graphics and pretty connection lines, bright colors on dark ground and an optional grid-snapping (currently, this one is forced).
    Maybe a better example is this one:

    image
    image1440×1080 115 KB

  4. About the zoom and move:
    Zooming and moving can easely and very efficiently go infinite. Do not computate everything on the graph but only th visible part. However, you are right, zoom is not really an issue. But not being able to move further to the left top is a bit strange.

  5. About Layout:
    If you want to apply a dark mode at the moment, you have to shut down the node-red server, update the settings.js file and restart it. (Note: Imo even the dark mode does not look really 'fresh'). It would be better if users could download them via the package manager and could select it somewhere on the page to update the current view. There should not be a server-restart.

Thanks for the constructive citicism btw, it's nice to see a place in the internet where ppl are not killing each other for diffrent opinions :wink:

Cheers

5

Let's say the node does not know...and it doesn't.

1 Like
6

Just to clarify @bakman2's comment. A node can not know what will be contained in messages that it will receive in the future. Also it would use up valuable memory and processor time if it tried to remember all the properties of all the messages that it has ever received. There could be any number of such properties.

7

The nodes already are vector graphics, so perhaps there's something else you're looking for?

Grid Snapping is on by default but can be turned off (as can display of the grid) in the settings dialog.

Particularly because you can already zoom in and out using the controls in the bottom right of the workspace.

2 Likes
8

The focus was on the indefinetly. It'd also feel better if the scrollwheel would do the zooming. But everyone here seems to be pretty happy with the current solution.

9

But I dies know it's own outputs. Why not displaying them each as individual connector?

10

How much have you used Node-RED? Some of your proposals seem to come from some confusion about how it's typically used. I would recommend you play around with it for a bit more and perhaps also look at some of the flows published in the library.
Note that Node-RED is not classic dataflow, which has multiple inputs per operator and has a notion of token (message) matching to fire the operator. Node-RED doesn't have that.
The parts of a message, such as payload and topic, belong together, they're generally not separable data items but rather one is a qualifier of the other.

11

That is probably very true. I did not use it alot yet (It also does not feel very intuitive for my purpose). Do you know and "classic dataflow" ? Like Data-pilot just open source and free?

12

Not in advance it doesn't.

13

In terms of more data-flow-ish open source stuff there has been Pure Data for many years, used heavily in the music synthesis arena.

To me the difference is the typical level of abstraction being used. I like Node-RED because I can use it at a fairly high level of abstraction where nodes are fairly chunky. The graph gives me a high-level visual overview of the flow of control in my "program". E.g., here MQTT messages come in, they're classified in those nodes, then these node operate on them, and over there are the nodes that save the data or send replies. I don't do small nodes that calculate 1+1=2 as one might do in a more classic dataflow. So I don't think of Node-RED really as programming but more as a way to hook little program snippets together (of course that's "programming" as well, but at a higher level).

Others will disagree and have a different way of using Node-RED, and that's OK. I think in the end you need to look at what you think you want, try some of the tools available, and be flexible about revising your opinion on what you're looking for as you gain a bit of experience.

1 Like
14

OK, I get it, you want the message properties to be exposed as output ports so you can drag a wire to another node.

Unfortunately, I think you have missed the difference between Node-RED and something like the Unity game engine. Node-RED is MESSAGE based. The whole idea is that a JavaScript object is exchanged between nodes so you only ever need 1 wire between the nodes (well that's a simplification of course, but close enough). With things like Unity, you connect node settings to other node's, in Node-RED, you connect whole objects between nodes.

So no, there would be no point in trying to go through the complexity of exposing each individual msg property.

I think we have to agree to differ here. The visual representation of a node-red flow is typically an order of magnitude simpler than a flow represented in something like Unity. If for no other reason than a single wire carries complex information instead of a single setting.

To be clear, multiple input PORTS are not available in Node-RED and likely never will be for reasons that have been well explained previously in various threads over the years. Much of that has to do with the asynchronous nature of node.js and the fact that node-red is message-based.

Grid snapping is already available and configurable.

Bright colours on a dark background are a personal preference - Node-RED should never adopt such a scheme as default since it would most certainly not be accessible out of the box which is a legal requirement for much of the world. However, you can certainly achieve that already by simply using a different editor Theme. There are a bunch of them already and you can create your own as well.

The one thing from the new image that might possibly be interesting would be the introduction of a 2nd line which has a different font size and weight. I could see that being useful for some things. At the moment, you can have multiple lines but they all have the same font.

I think our definitions might be slightly different! :wink: But on the other hand, I did say that personally, I would like to see the flow area have some more freedom. Not being a D3 expert, I've no idea how easy or otherwise it might be to implement.

OK, well that is a decent point and you are right - I'd like to be able to switch Editor themes on the fly so I could get rid of my half-baked theme that is a bad compromise between something that is comfortable in the day and comfortable at night.

I have discovered by experience that getting a dark mode theme right is extremely difficult so I'm certainly not going to criticize the people who have taken the trouble to create themes - even though I don't like any of them :grin: (which I don't but that's my problem).

There is certainly room for you to create a new theme, there are likely to be many people who will like it.

Well, many people have commented about the friendliness of this forum over the years and those of us who've been around a long time with Node-RED certainly work to keep it that way. I know that I have to reign myself and my comments in sometimes :slight_smile:

My own take from this discussion is that I think you have a couple of good points here that could potentially be turned into requests. However, we've not heard from any of the full-time devs, particularly Nick as to what their thoughts might be.

But there is also a challenge for yourself to put together a dark theme that meets your needs. :sunglasses:

1 Like
15

I think there are a couple points we can follow up on here.

Switching Themes in the Editor

That was the original plan, but it got a bit awkward to achieve due to the way user settings are loaded and themes are injected into the page - basically we load the theme before we know who the user is, so we don't know if the user has picked a different theme.

It will be doable, just got bumped for time.

Infinite Scroll

Certainly doable - just needs to be done in a way that minimises issues with backward compatibility. Specifically, I'm thinking that it would mean we end up with some nodes having negative x/y positions (because we have to have a central position that matches what users see when they open a flow today) - and they would get forced to (0,0) when imported into an older version of NR. I'm sure with some more thought a proper solution can be found.

2 Likes
16

Very well-spoken, thanks for the nice answer!
In fact, I already created a bunch of themes, even a 'create a random theme'-program but at the end, they all look bad imo. I think it has something to do with the response, not with the colors.
Some ideas of what could be the problem, even though I have no idea if these are correct:

  1. No shadows around nodes
  2. no color gradient like here on this image:
    image
  3. Poor outliners - how about something fancy like this?
    image
    image1000×600 69.7 KB
  4. One-colored connections and ports. Maybe adapt to the node they come from or the information they carry (oh yeah, you guys already said that's not possible :frowning: ...)
  5. Colored background for the title
  6. Allowing any icon to be used, not just those from the own set

Cheers

17

That can be done of course using CSS added to the theme. Though it is added complexity.

Again possible - though not sure if this would need a change to node-red code since nodes are currently defined with a single background colour. Might be possible using amended CSS theme though.

Both of those might add some polish to the Editor UI but also add some visual complexity that perhaps not everyone would want.

However, such nice additions are not core to the operation of Node-RED. Since there are still many developments that people want that enhance the operation of Node-RED, I suspect that the general view would be that contributions to enhance the standard themes might be welcome rather than it being a focus for limited core development resources.

An ongoing discussion about metadata on wires has happened on and off for some years :slight_smile:

That might be an occasionally useful enhancement.

That would indeed be nice. Or at least the ability to use an image, especially an SVG since most of the icon collections have SVG image options.

18

re icons - In theory you could create a dummy node that contains a whole bunch of icons - that would then get added to the existing list - which other nodes could then reference.

We do want icons to be sourced locally (I.E. no requirement for internet access) - and ideally they need to be available to all users - otherwise if someone shares a flow with a custom icon then it would broken for the recipient.

19

Not sure if I should already know this. But is there a guide somewhere on how to do that? Certainly would be nice to add some custom icons.

20

If you add an icons folder to your published node filled with images, they will automatically get hosted at a location (based on your config) such as mywebsite:1880/admin/icons/your-published-node-package-name/theicon.svg The icon will then show up in the icon picker. As a quick example, I added the coffee.svg to my mp4frag node and can then use it in my ffmpeg node.

Screen Shot 2023-01-19 at 4.48.47 PM
Screen Shot 2023-01-19 at 4.48.47 PM1030×1210 60.3 KB

For me, the image is directly linked at https://xxxxxxxxxxxx/admin/icons/@kevingodell/node-red-mp4frag/bmc-logo.svg.

Unless I am misunderstanding @dceejay, then I think you could make a node that has a variety of icons and other nodes can use them.

2 Likes