Improved hardware support

Maybe this topic should be split if there's still people willing to discuss the matter? There's a nice continuous subthread of messages.

Edit: hardware category maybe?

I did say if HA is the largest group. I simply think that if there are no core hardware nodes then a lot of softies will be turned off. The Pi Zero is the Pi Foundations door into makerland with goodness knows how many devices out there. Unless one is 50 50 hardware/software orientated then all perspectives are incomplete and unbalanced.

Just an example. I won a contract to design and develop the software for an explosion protection fire alarm panel. This had to respond to sensor input and shut slam valves within a 500 micro second window. The company was primarily hardware orientated and the senior hardware engineer had plans for dual DMA controllers and all sorts of hardware related features to make my job easier. I am 50 50 so I challenged the proposed hardware design and saved them a lot of money. It was a natural thing for the hardware team to use tools and devices they were familiar with in order to make the software development possible in their eyes. Had I not been 50 50 I would have opened the data sheet on DMA controllers and designed the software to integrate them and be non the wiser.

As @ristomatti suggested, I have now moved this discussion to its own thread.

Out of curiosity, how would you propose to tackle the issue of Node.js not being particularly good at hardware/real-time communication? The existing library support is not IMO not very good on Node.js, likely due to actual reasons.

One way would be to write native C/C++ wrappers using N-API for the interfacing but like previously brought up, there's not a lot of those either regardless of Raspberry Pi's vast success and popularity among tinkerers. Like Nick mentioned, writing such libraries requires a specific set of skills so maybe it would have already happened given Raspberry's age? I haven't spent much time checking lately but the number of replies to this thread might give some idea: https://www.raspberrypi.org/forums/viewtopic.php?t=146712

Or are you suggesting IBM should pour more money / hire more developers to fill in the gap?

1 Like

So - when I get 5 mins I look at https://www.npmjs.com/package/pigpio as a possible enhancement to the gpio nodes - as indeed it wraps the beloved pigpio C library. But - so far it also has limitations - most notably

The pigpio C library and therefore the pigpio Node.js package requires root/sudo privileges to access hardware peripherals.

As we really don't want to have to run as root especially for devices that may be internet facing, I have parked that idea at least for now. If someone wanted to take it on and work out how to run as a normal user that would be a great unblocker.

Obviously this would only "fix" one small aspect - but that is the problem with these sorts of things... lots of small issues that all take time and dedication to fix.

There are lots of different temperature and humidity sensors that are supported - like the ds18b20 and the bmp180 and (heaven forbid) the DHT11/22

Re the MCP23017 there are 2 main nodes https://flows.nodered.org/search?term=MCP23017&type=node - a) which doesn't work ? - b) if neither - which would be the the one to target for upgrade ? The general i2c node has been recently updated so that can certainly work with node10,12 and 14.

So once again I ask - what is the (minimal) list of hardware support that would be required to make @ozpos happy ?

Is this view is from a software perspective by any chance ?

My personal needs do not enter into it and I trying to keep the tone non personal. People explode when they come across conflicting viewpoints on their pet subjects. Myself and others think js is a terrible language, I met and spoke with one of the authors of Occam who thought Kernighan and Richie had made a bit of a mess when designing C. JS is here to stay so I just have to live with it.

Focus: I have complemented the NR team. I do not think the focus has been wrong all along. Simply from a 50 50 perspective I think the complete lack of a working hardware node out of the box would be damaging to your user base.

I have not said it should be the one thing to focus on.

I solved my problems with hardware nodes not working with node. The results of which are of little use to others because I had to create C based executables run through an exec node. Not very pretty, poor performance but better than python.

I obviously touched a nerve somewhere. I have not complained and do not know what you are talking about (because you have no influence like in commercial projects) ?

If there was one example core hardware node it would show others (particularly users towards the hardware end) how to add additional nodes and how they integrate into the NR echo system.

@ozpos I have move this latest reply to the new thread as I suspect you started writing it before I moved this discussion to its own thread.

You have come on pretty strongly saying it should be the focus. Whether that was your intention or not, I think the reaction you've generate shows how it has been interpreted.

There are already lots of hardware nodes. Some maintained by the core project (Serial port, Pi GPIO of various types and so on), many are maintained (to varying degrees) by the community. There is always room for improvement, and this is where highlighting specific areas for improvement will be helpful to the individual maintainers of those nodes.

2 Likes

Because of performance I think it has to be a C library.

pigpio has a user-space C library pigpiod_if2.

Also tcp and pipe interface as well as scripting.

A 23017 would lead the way for others to contribute.

I think it needs to be linked with pigpiod_if2

Though the author seems fairly adamant that it can't be done https://github.com/fivdi/pigpio/issues/114 - Is this something you can bend your C skills towards ? Could you modify that code to link that other library ? As I said it would really help unblock this.

PS and no - I'm an ex-hardware guy also, who has drifted towards software. (and don't know C). That is one of the reasons I helped create Node-RED - I just hate writing things like serial drivers or O-Auth handshakes.

5 Likes

Dave, just passing by. This thread suggests...

You need to use sudo to start the pigpio daemon ( sudo pigpiod ) or run a program linked against the pigpio library.

You do not need sudo to run a pigpio Python script or to run a program linked against the pigpiod_if library (both of which use the services of the running pigpio daemon).

Not sure if it helps any (sorry if this is just noise)

yes the pigpiod need sudo - a client talking to it over a socket doesn't... BUT the https://www.npmjs.com/package/pigpio library is "just" a wrapper around the native C library (that is presumably used inside pigpiod) - it's not a client talking to pigpiod.

1 Like

If code is linked to pigpiod_if2 then it does not reed to be root to access the services provided by the daemon:- sudo pigpiod.

The community would already have a fix from me if I understood the js development methodology and debug tools. I only have RubyMine.

I looked at several hardware related node modules but could not figure out how to give them a front end for NR nor how to debug and set breakpoints.

I would give it a go if it was possible to start from a working development environment.

If the node package is linked to the pigpiod C api it requires root.
If the node package is linked to the pigpiod_if2 C api it does not require root.

Both require sudo pigpiod as the daemon.

Actually, a lot of people are moving to Rust for this. It aims to have the same performance but enforces the things that make C/C++ a dangerous language for system-level programming (e.g. not being memory safe which accounts for a consistent 70% of all errors).

I'd be interested to note whether sudo is required because of some kind of kernel interaction or whether it is due to poor configuration of the pseudo file-system links to the hardware. It is very typical on Linux to have hardware links assigned to root:root when they really should be something like root:wheel so that access can be granted via group membership.

Also worth noting that, if starting Node-RED via systemd, you get much more control and probably wouldn't need root.

And finally, please remember that many people use Node-RED on other platforms, not just Linux/Pi.

Though I've come to this discussion late, I'll add my support for the creation of a list of key Pi hardware libraries - I think that would be excellent. But as others have said, this is not core Node-RED. While Nick is currently doing his best to encourage more people to pick up and understand how to help with core changes & the input from Hitachi is extremely welcome, the available resource is extremely limited. The best "bang for the buck" is achieved by focusing on continuing to make Node-RED a solid, reliable, flexible low-code platform.

Node-RED has massively outgrown its original intent which is a great testament to Nick & Dave and other IBMers who contributed to the original and ongoing development. Just lurking in the forum for a while will soon build the understanding that Node-RED is now used in many commercial settings even though, as has been said, the small niche crowd of people using it as their primary HA system are the most vocal in the forum a lot of the time.

At the end of the day, Node-RED will only continue to grow and be successful if people with skills help it to develop and grow. Whether that is by contributing code, documentation, testing, and/or ideas. That means investing personal and possibly professional time and resources into it.

If you perceive that a set of Pi GPIO/Sensors nodes are important to the community, please help corral support and make it happen.

For me personally, Pi GPIO has never been useful, it is too easy to destroy a Pi via it's GPIO and too hard to control the pins due to the complexity of the OS. It was much easier for me to sit down and learn enough C/C++ to program Arduino's and then ESP8266/ESP32's. So much so that I am now moving most stuff off my Pi's back onto a PC. But again, that is just the perspective of using NR for HA.

3 Likes

I do not think there is a rust library for the pi gpio. I agree w.r.t. memory leaks but the pigpiod library is very mature and I have > 1yr up time without problems so far.

The author of the pigpiod library states that the daemon should be started with sudo and that all code linked to the pigpiod_if2 api does not require root.

Yes I appreciate that however the largest user base should be considered before other use cases especially when dev resources are so scarce.

I strongly believe that for a balanced outcome there should be some way of weighting opinion based on whether that opinion is from a hardware or software perspective or both.

Do you have a link re Hitachi ?

That is what I have spent the last 24h doing. I have had to be provocative in order to get contributors to recognise their own perspectives along side that of others. It felt a little like a bomb had gone off as a result.

With respect I do not think the OS or the library can be blamed for fried I/O.
Buffers or I/O expansion are the safest bet here and go naked gpio only when fully tested.

I found that NR for HA works fine if you are careful which js libraries are pulled in.
I agree that a PC lifts any code size and speed restrictions but cost also factors in and a portrait 7" touch screen will fit flush when mounted in the place of a light switch.
I have sited in an earlier post that one can include moment in order to format a date

I’m sorry to butt in to this topic so late, but do you have resources supporting that “raspberry pi” users are indeed the biggest userbase? Because so far I haven’t seen direct evidence of that, nor am I specifically supporting them as special beyond “arm linux” for nodes I work on. If so, it might be something I need to be more focused on as well, perhaps?

1 Like

Not necessarily. There are many possible approaches and at the end of the day, the product "owners" need to make the call. The largest number of users does not necessarily translate to the broadest reach and benefit over the longer term - especially given the limited resources.

I for one would not presume to know the best roadmap for Node-RED as a core.

And I think that we appreciate that viewpoint but it seems clear that not everyone agrees at this point. Such hardware nodes can be developed by anyone (with the skills or willing to learn them) whereas core development is much harder.

I think there was a blog post about it when it was announced.

Yes, that is common in forums. I think it safe to say that this is one of the best and friendliest tech forums around so the reaction tends to be "relatively" mild :smile: But there is a lot of very personal investment into the platform and people are naturally "enthusiastic" about their responses.

And with respect, I didn't. Though perhaps that's how it comes across. No, the issue re safety is with the hardware. The issue with the OS is that you are carrying maximum complexity for a tiny, niche feature set. Personally, I really wish that the Pi had been designed with an Arduino or similar on-board. But doubtless that would have had a hit on cost.

But that is not what has been discussed so far is it? Or did I miss something? And if you are going to use additional hardware, the cost of an Arduino nano or ESP8266 is so low, there seems little reason NOT to use one?

Indeed. That is true of all platforms. Whether node.js, Python, or whatever.

Of course. That is why I was careful to prefix that with it just being my experience - I happen to have an old work laptop available after an upgrade just pre-covid lockdown - it would have been scrapped so I have permission to reuse it.

Sorry, not sure what that relates to.

I do.


http://www.hitachi.co.jp/New/cnews/month/2017/08/0809a.html

Several more, but these give starting points.
2 Likes

Is it not evident from the survey results ?
There is no getting away from the enormous pi user base. NR is there already which is the hardest part. I think the peak interest and diversity of hardware nodes came a little too soon for the host hardware. I, like many others I suspect were waiting for the next boost in pi performance before revisiting hardware nodes in NR. Now we have the Pi4 and Pi Zero and 8GB RAM but it looked like hardware nodes were going to fall by the wayside.