Ideas for simplifying custom node creation

Developing Nodes
1

Hi all, this topic stems from An idea for third-party UI in ui-builder - #5 by jackcarey. It is intended to garner some ideas about how the development of good quality custom nodes could be eased and simplified for everyone.

Some opening "issues" that might be good to try and deal with:

  1. Writing the edit panel for a node requires:

    1. A (relatively) complex mix of two different parts of HTML (or HTML and Markdown) and JavaScript. This makes it very hard to manage as the node gets more complex. It also makes it hard to lint correctly - especially since the HTML for the Editor is in the same folder as the JavaScript for the runtime.
    2. A lot of boilerplate HTML (often 4 or more lines per input field.
    3. Long element ID references that can be confusing to read and write.
    4. More complex nodes can require a lot of boilerplate Javascript code and a reasonable understanding of the complex processing that Node-RED does.
    5. Typed input fields require a lot of boilerplate JavaScript code

  2. Writing the runtime for a node requires:

    1. A good understanding of the complex interactions between different parts of the runtime code. What does what and when and what objects exist at different parts of the process.

I've some thoughts about how we might be able to improve some of these issues but I would like to see if others have thought of things I've missed and start to get different ideas into 1 place.

Please note that I don't believe all of the possible improvements would necessarily need to be part of core, though that might be a desired eventual outcome. Some of what I'm thinking about could certainly be offered up as stand-alone enhancements for developers to use.

Anyway, time for tea (or dinner if you are that way inclined!) so I'll come back to this later. Happy to include further issues into the above list so that we have them in 1 place.

2 Likes
2

For 1.1 - I've already moved the JavaScript for the editor in my uibuilder nodes (or I've started to anyway), to a loadable resource file, this not only makes the html file smaller and easier to edit, it also moves the code to a folder that is guaranteed to only contain front-end (browser) code. This makes it easier to lint. Obviously, this does not require any changes to core Node-RED since the resources folder was introduced a while back now.

3

For 1.2, I had the idea that it would be possible to create some web components that would encapsulate each input and look something like this in the HTML file:

<nr-typed-input 
    id="myvar" types="msg,jsonata,global,string,number"
    label="My Var" 
    ....
>
    My Var
</nr-typed-input>

Maybe not the best example but you can see how the component encapsulates much of the complexity. The label is simply the slot content and the variable name myvar is simply the name used in the node - that becomes node-input-myvar in the shadow HTML so that Node-RED processes it as normal.

Most input types could almost certainly be dealt with using a single nr-input component.

But people could potentially then also contribute more complex inputs such as a combo-box input that I've long wanted (harder to do properly than it might first seem, especially if you want it to be reasonably accessible).

The components could issue custom events if needed so that your JavaScript could interact correctly. Again, helping reduce code complexity and improving consistency by offloading common processes into the component.

Custom web components can be built fairly easily by hand or can be created by a number of frameworks including LIT and Svelte. Hand-built ones would probably need a build step to produce outputs that can simply be included in script links but of course, they could also be combined into a single loadable library - both approaches are not mutually exclusive.

Also usefully, the gradual (or rapid) introduction of such components would not be dependent on changes to Node-RED core at all. Such enhancements could happen in parallel with core development and be incorporated over time if desired.

This overall concept is an idea I'm especially interested in getting feedback on. It would obviously be some amount of work, especially for the first go so I would want some encouraging feedback in order to warrant the investment of time.

Oh, one down-side of using web components is browser support generally only extends back to around 2018 for Edge and Firefox, 2017 for Safari. Some mobile browsers even later. Though there are polyfills even for IE.

"web components" | Can I use... Support tables for HTML5, CSS3, etc

2 Likes
4

Creating the settings page to be used in the editor was always the hardest part for me. I would prefer that nodered generate all of the forms for me from some simple list that I could provide. Of course, there sometime may be complicated settings that would still have to be handled manually.

A good second choice would be a nice input wrapped as a custom element. It seems that it could be created as an editor plugin. You simply link the resource and drop the custom element into the html and then add your change listener.

1 Like
5

This ^. You are reading my mind?. Definitely something I want to explore. There are sooooo many side benefits too (that I dont have time to list out right now) but this design would have been HUGELY simplified if we had a way of describing the UI instead (as well as) freeform.

2 Likes
6

:cough: I don't like to say this in public - but you do realise that such a thing may already exist? It is already in use in a fairly popular set of nodes. :wink:

7

A web component is a great solution and I like that it doesn't even require the runtime to be changed. The current typed widgets require jQuery, so these components will either not work outside of NodeRED (an acceptable solution), or need to redundantly include jQuery - just something to consider

1 Like
8

config UI just generated from a list... uurgh... so a vaguely random set of boxes users have to work through ? and how would it cope with hiding / revealing optional elements based on another field... unless you had a markup language for creating the list - hmmm - it's markup all the way down...

1 Like
9

Come on Dave, you knew "list" was a friendly, non-fleshed version of "Data Driven UI" :smiley:

There is a balance to be struck as always but I have used packages where the UI is "described" and build from definitions. And yes, it supports hide/reveal, cascades, custom classes, and much more (but then it starts getting complex)

The win, in my head is getting to a standard where the runtime can predict behaviour and provide things like Env Var overrides, autocompletion intelligence etc.

Lets call it a distant utopia :wink:

1 Like
10

friendly - only if you know the magic codes... but yes. Why not skip to "prompt engineering" to write it ? Maybe not so distant :slight_smile:

11

Great if you want your flows randomly changing because the AI started imagining things! :rofl:

12

Yup, I'm going to test it using my new node-red-testbed nodes that I set up specifically to enable this kind of experimentation. :grinning:

2 Likes
13

Hey - you've described my development style - bring it on !

2 Likes
14

Maybe some of us would prefer not to be part of the great AI takeover though! Would be a bit embarrassing if Node-RED caused it. :frowning:

1 Like
15

I wouldn't be surprised if nodes like function-gpt (Chat GPT in Node-RED Function Nodes • FlowFuse) garner a lot more usage in the coming years, possibly even becoming a default for some people. I say this as the combination of an LLM with visual programming allows one to not only see the structure of the application but to describe it in smaller, digestible chunks - it's powerful stuff! Not to say it isn't scary though, as removing the need to write code also removes the user's ability to check it for errors and that it works exactly against their requirements

16

Maybe I get late to this topic and I don't want to give any expectations. I'm working in a solution user-friendly for developers:

  1. A web GUI frontend which basically it is a form to introduce the whole info of the component.
  2. Then it generates all the boilerplate (html file + js file) and add things to the package.json (nodered section)
  3. The dev only need to worries to fill the JS file logic of each componente
  4. Then it just a normal flow with a npm package to publish, etc.

My experience creating custom components is:

  • Most of the job could be automatized
  • If you have any typo nothing work and you don't have any feedback (which is the worst part)
  • I really dislike the way we have know to create custom components

I hope in the near future I will create an entry in this forum releasing my solution for the current way to create custom components.

3 Likes
17

This sounds great! You’ll have to keep us updated as you develop it further

18

It's stopped while I'm developing some components. As I finish them, I release them. For now I have:

  • Septentrio GNSS (GPS) parser (parses its proprietary SBF binary protocol)
  • NMEA 01833 parser for GNSS (GPS) that can be expanded using yaml files.

I will post the components here when I have them working with real devices 24/7.

When I finish publishing the components that we need in my company, the next thing is to make that tool.

1 Like
19

Screenshot from the nodedev package:

Screen Shot 2024-01-16 at 13.17.11
Screen Shot 2024-01-16 at 13.17.11504Ă—1174 36.4 KB

That's the NodeFactory node that generates template files for a new node. That latest addition was the minify node that makes the node appear without a label in the workspace.

Using the NR UI and yes, have to agree all those buttons and dials makes it confusing to create a new node but then again, remember which html, js or json files I need for a new node is even more confusing... :wink:

As I've been going along and discovered the button feature or the sidebar nodes or the config nodes, I've added those features to the templates so I don't have to remember this stuff.

I talked a little more about NodeDev over here - it's certainly not simple but then again, copying a bunch of files around and finding all the spots where I need to change something for a new node isn't simpler.

Edit: another part of that package is the package importer sidebar node:

Screen Shot 2024-01-16 at 13.27.00

What that does is import the code of any package I like. It imports the code as nodes in my NR flow, with that, I can easily find out how others solve problems that I'm having.

It's a bit confusing since it opens the import dialog:

Screen Shot 2024-01-16 at 13.29.20
Screen Shot 2024-01-16 at 13.29.20771Ă—664 84.2 KB

but after doing the import, I have the entire code:

Screen Shot 2024-01-16 at 13.29.38
Screen Shot 2024-01-16 at 13.29.38728Ă—1090 154 KB

3 Likes
20

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.