Improved hardware support

No I'm suggesting you might have more luck starting a new topic for a new topic in the development category.

Most folk are advised this to do this when veering off topic - it's just forum etiquette & what trends to work best around here.

OK what category do you suggest. I am now ready to build a static implementation of a MQ connected Hardware Module namely MCP23017 ?

Hope is a fickle mistress indeed.

As long as you remember that, mostly, the experienced folk are also the busiest in life, family and work and we've already noted that what you are wanting may not be everyone's focus.

These days, Node-RED is so popular and the forum so busy that few have the time to attempt to follow everything, certainly I don't sadly.

2 Likes

If you want to create a node-red node you don't need to build node-red itself.
Start here Creating Nodes : Node-RED
Perhaps I have misunderstood exactly what you want to do though.

2 Likes

I now have a working fork of master within my IDE is that not a good starting point ?

My approach is designed to be a minimum burden on the community.

The plan is to see if I can single step into a working node and then create my own from an existing MCP23017 and try to detach it from I2c calls as a first step.

I don't believe it is necessary to clone node-red in order to step debug a contrib node. Not sure if it ultimately matters.

This is how I step debug my contrib nodes in VSCode - it might help? (Unsure because I am unfamiliar with the ide mentioned earlier)

A clone of what exactly? I haven't been following every single reply in this thread - there are now 92 posts. Scanning back through, I can't see you mention which git repo you are cloning - my apologies if I've missed it somewhere.

You don't need a clone of the node-red source code to start developing nodes. You can install and test nodes you are developing with a regular install of Node-RED.

Are you following the Creating Nodes guide? It won't go into the sort of depth you'll need eventually, but it's a good place to start as you're new to this environment and particular set of technologies.

I forked master and cloned that. Now I can mess around as much as I like and revert the mess ups.

It may not be necessary but it is done. I feel comfortable with JetBrains it has git built in. I can now keep in sync with master. Hey there could well be another version by the time I have got a hardware node working.

Also If I have working source I can see where execution goes with working code. Also navigating is incredibly easy. Plus knowing what issues exist, maybe have a go at fixing one. And just browsing the code !

Time is too short to be debating IDEs. It works. Shoul I shove off to another category or what.

I have asked that question where it shoul be !

The #creating-nodes category.

Regarding SPI I’ve been using this excellent library and it works great within node on a pi:

It’s very easy to integrate in Node-RED by either writing a node or using it within node-red-contrib-unsafe-function.

3 Likes

Thank you that looks good. The problem is maintenance. If it is not part of node-red distribution it will die by not keeping up with node version changes.

I propose a small collection of hardware nodes that expose a chips functions but communicates via MQ (as others have described). It may have to be a little prescriptive in determining the serial format to and from the device but as long as it exposes all the chips functions and responses then that does not matter at all and is never going to change or be made redundand by node (what I would call a static definition).

If a small collection is too big a burden then a single example should at least use check boxes, drop down list selection etc. rather than a singe temperature measurement for example.

One of the most important reasons why I think it should be kept totally separated from NR development & distribution, such "strong bindings" should be totally avoided. Instead such devices should be either INTEGRATED via a standard protocol like MQTT or they should be a device connectable to a usb port. That will make each part rather independent of each others. Imagine you have to follow and sync, having the same release dates as NR, a nightmare. Maintenance can instead be focusing on features (and ev bug fixes and user documentation!) and releases can be made freely without dependencies.

A very nice product series based on the cheap ESP device concept, using also MQTT over WiFi, is Shelly. So smart, simple to configure and operate, lots of features I personally don't need and they simply work. Reliable.Very clever design as well. And very competitive pricing, cheaper than ZigBee and much, much, much cheaper than Z-Wave devices

PS I don't use their cloud, I have everything just internally connected on my network using my own MQTT brokers

https://shelly.cloud/

1 Like

The specification of a chip is static and will never change without a part number change which does not happen anywhere near as often an node version changes.

Lets say a chip has two command registers and some device address bits. The difficult bit (for me at least) is creating a UI to those settings. The UI should allow address selection and data input fields for those command registers that get serialised and sent to MQ.

The chips responses are also fixed and will not change and so if the serial format of the reply is prescribed by the info page within NR then it can decode the response.

The maintenance problem has gone away (as far as the NR dev team is concerned). The developer/end user then chooses the hardware for the controller of the device (arduino or whatever) and unpacks the address and commands from its serial in, talks to the hardware (they should be good at that because that is within their comfort zone) and serialises a response in line with the info provided within NR.

So the strong binding is only against the chips functionality, and that will not change over time.

By all means say if it is a daft idea but please explain why ?

The core Node-RED nodes (the nodes you get when you npm install node-red) are domain neutral. They are not IoT specific. They are not hardware specific. They provide basic programming building blocks and are useful for all users of Node-RED.

We will not add hardware specific nodes to the core Node-RED module.

The Node-RED project (by which I mean, the Node-RED organisation on github) maintains a collection of extra nodes - GitHub - node-red/node-red-nodes: Extra nodes for Node-RED. This includes the Serial port node, the pigpiod nodes, the Pi Sense Hat nodes. They are all maintained as separate modules and can be updated entirely independently of the release cycles of the core project.

The Raspberry Pi install script we provide gives you the core Node-RED and adds a few of these non-core nodes because it makes sense to install them on the Pi. You can see the list of extra nodes here - linux-installers/deb/update-nodejs-and-nodered at master · node-red/linux-installers · GitHub

It is exceedingly rare for a pure JS node to need any upkeep due to newer versions of Node. The issues tend to be where nodes require C code that has to be built for newer versions of Node. An ideal model is for the Node-RED node to make use of existing well-maintained low-level libraries - as those libraries will typically have a much larger audience than just Node-RED. This is why some of our existing nodes lean on existing python libraries - its simply a pragmatic way of getting things working.

If the node has a maintainer who is interested in maintaining it, then it won't die. If they lose interest in it, the community can step in.

The key point is, at this point in time, its fairly irrelevant as to whether the code ends up in the Node-RED organisation on github, or someone's personal account. The question is whether there are people willing and able to do the work and take on the longer-term effort.

I speak from bad experiences of nodes being contributed to the Node-RED project on github and then getting abandoned - leaving Dave and I to add it to the long list of things we have to maintain.

8 Likes

Sorry if this is a long one, I have to unpick each point I am definitely not having a go, and sorry it is just terminology on my part. By core I meant distributed like the serial port that you sight. The out of the box experience is so important. Like an interview the first 5 minutes is all you need. First impressions are what it is all about if you want to attract more willing hands.

There is nothing very interesting or instructive about the serial UI interface. As for pigpiod unless it has been fixed recently I does not give access to SPI, I2C, or scripting (and never has) which renders it pretty useless if you want to read a temperature or whatever. Also the api has not changed for donkeys so if is not there now then it will never be with the current state of affairs. So get away from the hardware altogether and just supply UI serialised interfaces and forget it.

I and others have to go back to 8 lts to get a ui to a chip that works because the underlying hardware driver is easily broken with new node verstions. A ui for a remote connected MQ chip is completely static (no maintenance at all). All the team have to contend with is maintaining an interface with MQ which you do already.

Only if they work with the latest node lts / maintenance release that the community have time to support.

Others have sited that attempting to write hardware drivers in JS is futile and I think they are correct otherwise they would be everywhere and JS security issues would be even more problematic than they are already.

Python (chosen by Pi Foundation for educational purposes) is far too heavyweight for a Pi Zero. And, if you have to resort to multiple exec nodes calling python libraries every second in order to drive your sensors it will drive a Pi4 into the ground. Been there and wasted precious time and resources.

Yes I know. But a ui that serialises a chips i/o command structure is nothing more than passing strings back and forth via MQ. That would not break with node version changes.

Hardware people are good at C and I2C and SPI and Interrtupts and debounce and so on. Leave the fun stuff for them give them a UI to a chip and bingo they will definitely let the world know they have an arduino or whatever driver for your example chip front ends. No maintenance at all.

The low level C library apis have not changed for years. And as you say they have a very brittle interface with node. Just get rid of them all.

I got the impression that the NR team aim to reduce the current and future maintenance burden.

Great for education rubbish at anything fast and efficient.

The makers will look after their end with pride and conviction especially if given an example of a static UI definition that they can work with. The softies will look after their end with pride and conviction also. But for the life of me it seems like they will never meet and agree the terms of engagement. Simultaneous engineering was supposed to take care of this impasse.

So don't do it. Give the makers a UI or two, piece of cake for a softy.

I am only 50 50 but well over sixty so I have seen it and experienced it all before.

Best of luck, I have enjoyed most of the interaction but can smell the dislike towards an alternative point of view from an outsider.

The UI design, learning a new IDE (not JetBrains), a whole bucket load of uncoordinated tools that look like they have been made up on the hoof is all too much for my 50%.

Best regards to all.

Sorry, but for the record - for other people who might come across this thread in the future, I can't let this go unchallenged. We are not disliking an alternative point of view because someone is an "outsider". We are disliking it because our experience tells us that it is non-optimal.

A group of hardware-related modules dedicated to Pi GPIO would, as several people have said, be a great idea and would really help a certain class of users, probably educational users most of all since they may not have the resources to manage the additional complexity of multiple devices (e.g. Arduino/ESP + Pi).

But this is not core to what Node-RED now is. To make a success of such a set of modules would need community just like any other open source initiative and would doubtless gain help from some folk on the forum.

6 Likes

Ok - lets try this another way... it seems like your preference seems to be with SPI and i2C (or at least they have come up several times on both sides of the conversation) - so let's just focus on them (for now). You also mention you want a UI or two... What would they look like ? can you sketch out something - paper and pencil and photo would be fine - so at least we can try to get to grips with what seems to be missing.

The existing i2c node lets you scan the bus and send raw commands in and read things out - so is pretty low level in that regard - but often we find people want something slightly higher level than that - as they want to target a specific device and use known terms and commands - so they wrap the low level into a higher level package. This is often where the issues start in that they then abandon it and don't update the underlying code as and when required... - but hey - that's the "other" issue... Let's start with the basics and try to understand the design you are proposing properly.

You dont want JS because it's bad for low level hardware I/O - agreed.

You don't want python because it's slow.

You don't want C because it's brittle.

At this point I don't know what you want.

This isn't about you being new to this forum. Since the start of this thread I've tried to understand at a technical level what you are looking for.

I have agreed consistently that it would be great thing for good quality hardware nodes to exist. And that the community can make that happen if there are the people to do it.

But I continue to struggle to understand at a technical level what you are suggesting.

Welcome to the world of open source. The NR project is an example of how to do it right, and this community consistently embraces new people and ideas. I just don't believe that you are the first to have discovered the fatal flaw in the project and have found the only cure for it. (Yes, your posts clearly give that impression.) I think there is merit in some of what you suggest, but you are clearly failing to generate much interest in your ideas.

Hi,
at moment I work on nodes for connect control/automation world with node-red. It includes very popular hardware. But there are lot of nodes to write and test. I hope, that the nodes are ready for use in short time. On beginning, the nodes runs on raspberry pi. Later on other hardware. My targets are most cheap hardware avaible on market. The mcp23017 or gpio for example. Which digital and analog modules are needed urgent from the node-red user ?

3 Likes