Improved hardware support

Hardware
21

You are a maker therefore it is easy to use any hardware. A software engineer would not know where to start and after blowing a couple of Pis would give up (no disrespect for softies I have done it myself). Makers and softies have very different perspectives and capabilities when out of their comfort zone. I agree that when you try and put intensive I/O, GUI and control it takes the top end device to handle it. When you dig into why you find enormous js libraries when perhaps all you used them for is to format a date or something. I would use a Pi 4 with 7" touch screen as a main HA controller with battery backup etc and Zeros as remote I/O. Thermocouples for flexible cheap long distance temperature input and 20317 as port expanders.

With respect that is how most makers would perceive the situation. A maker can see hardware alternatives but may struggle to analyse why the software is consuming too many resources and a softy has the opposite problem how can they swap out the hardware without breaking things.

The Pi is first and foremost an educational tool so it will focus on ease of use (python) and to address the overkill of a Pi4 there is the Zero. So it must be made easy for its students hence my suggestion for some hardware specific nodes in NR using the existing C library so as not to pull performance down.

This is the price of convenience and ease of use. But it must leave a door open for softies to do hardware easily. I also believe that it would be good for non softies if nodes could indicate the level of resource burden they carry with them. I am sure that many js nodes have been build without any regard to performance and efficiency.

22

I'm not sure how much there is to be gained by this continual back and forth. It is well off-topic of the original post which was about how we plan to schedule our future releases.

The point about dropping Node 8 has been made and acknowledged.

You are raising perfectly valid concerns over how well supported some things are over other things. But bringing it back to the original post, this schedule is about the core project - the runtime and editor.

Nothing stops the community driving development of any particular nodes - whether that is through raising specific suggestions, feature requests, or where possible, contributing actual code.

3 Likes
23

Maybe this topic should be split if there's still people willing to discuss the matter? There's a nice continuous subthread of messages.

Edit: hardware category maybe?

24

I did say if HA is the largest group. I simply think that if there are no core hardware nodes then a lot of softies will be turned off. The Pi Zero is the Pi Foundations door into makerland with goodness knows how many devices out there. Unless one is 50 50 hardware/software orientated then all perspectives are incomplete and unbalanced.

Just an example. I won a contract to design and develop the software for an explosion protection fire alarm panel. This had to respond to sensor input and shut slam valves within a 500 micro second window. The company was primarily hardware orientated and the senior hardware engineer had plans for dual DMA controllers and all sorts of hardware related features to make my job easier. I am 50 50 so I challenged the proposed hardware design and saved them a lot of money. It was a natural thing for the hardware team to use tools and devices they were familiar with in order to make the software development possible in their eyes. Had I not been 50 50 I would have opened the data sheet on DMA controllers and designed the software to integrate them and be non the wiser.

25

As @ristomatti suggested, I have now moved this discussion to its own thread.

26

Out of curiosity, how would you propose to tackle the issue of Node.js not being particularly good at hardware/real-time communication? The existing library support is not IMO not very good on Node.js, likely due to actual reasons.

One way would be to write native C/C++ wrappers using N-API for the interfacing but like previously brought up, there's not a lot of those either regardless of Raspberry Pi's vast success and popularity among tinkerers. Like Nick mentioned, writing such libraries requires a specific set of skills so maybe it would have already happened given Raspberry's age? I haven't spent much time checking lately but the number of replies to this thread might give some idea: https://www.raspberrypi.org/forums/viewtopic.php?t=146712

Or are you suggesting IBM should pour more money / hire more developers to fill in the gap?

1 Like
27

So - when I get 5 mins I look at https://www.npmjs.com/package/pigpio as a possible enhancement to the gpio nodes - as indeed it wraps the beloved pigpio C library. But - so far it also has limitations - most notably

The pigpio C library and therefore the pigpio Node.js package requires root/sudo privileges to access hardware peripherals.

As we really don't want to have to run as root especially for devices that may be internet facing, I have parked that idea at least for now. If someone wanted to take it on and work out how to run as a normal user that would be a great unblocker.

Obviously this would only "fix" one small aspect - but that is the problem with these sorts of things... lots of small issues that all take time and dedication to fix.

There are lots of different temperature and humidity sensors that are supported - like the ds18b20 and the bmp180 and (heaven forbid) the DHT11/22

Re the MCP23017 there are 2 main nodes https://flows.nodered.org/search?term=MCP23017&type=node - a) which doesn't work ? - b) if neither - which would be the the one to target for upgrade ? The general i2c node has been recently updated so that can certainly work with node10,12 and 14.

So once again I ask - what is the (minimal) list of hardware support that would be required to make @ozpos happy ?

28

Is this view is from a software perspective by any chance ?

My personal needs do not enter into it and I trying to keep the tone non personal. People explode when they come across conflicting viewpoints on their pet subjects. Myself and others think js is a terrible language, I met and spoke with one of the authors of Occam who thought Kernighan and Richie had made a bit of a mess when designing C. JS is here to stay so I just have to live with it.

Focus: I have complemented the NR team. I do not think the focus has been wrong all along. Simply from a 50 50 perspective I think the complete lack of a working hardware node out of the box would be damaging to your user base.

I have not said it should be the one thing to focus on.

I solved my problems with hardware nodes not working with node. The results of which are of little use to others because I had to create C based executables run through an exec node. Not very pretty, poor performance but better than python.

I obviously touched a nerve somewhere. I have not complained and do not know what you are talking about (because you have no influence like in commercial projects) ?

If there was one example core hardware node it would show others (particularly users towards the hardware end) how to add additional nodes and how they integrate into the NR echo system.

29

@ozpos I have move this latest reply to the new thread as I suspect you started writing it before I moved this discussion to its own thread.

You have come on pretty strongly saying it should be the focus. Whether that was your intention or not, I think the reaction you've generate shows how it has been interpreted.

There are already lots of hardware nodes. Some maintained by the core project (Serial port, Pi GPIO of various types and so on), many are maintained (to varying degrees) by the community. There is always room for improvement, and this is where highlighting specific areas for improvement will be helpful to the individual maintainers of those nodes.

2 Likes
30

Because of performance I think it has to be a C library.

pigpio has a user-space C library pigpiod_if2.

Also tcp and pipe interface as well as scripting.

A 23017 would lead the way for others to contribute.

31

I think it needs to be linked with pigpiod_if2

32

Though the author seems fairly adamant that it can't be done https://github.com/fivdi/pigpio/issues/114 - Is this something you can bend your C skills towards ? Could you modify that code to link that other library ? As I said it would really help unblock this.

PS and no - I'm an ex-hardware guy also, who has drifted towards software. (and don't know C). That is one of the reasons I helped create Node-RED - I just hate writing things like serial drivers or O-Auth handshakes.

5 Likes
33

Dave, just passing by. This thread suggests...

You need to use sudo to start the pigpio daemon ( sudo pigpiod ) or run a program linked against the pigpio library.

You do not need sudo to run a pigpio Python script or to run a program linked against the pigpiod_if library (both of which use the services of the running pigpio daemon).

Not sure if it helps any (sorry if this is just noise)

34

yes the pigpiod need sudo - a client talking to it over a socket doesn't... BUT the https://www.npmjs.com/package/pigpio library is "just" a wrapper around the native C library (that is presumably used inside pigpiod) - it's not a client talking to pigpiod.

1 Like
35

If code is linked to pigpiod_if2 then it does not reed to be root to access the services provided by the daemon:- sudo pigpiod.

The community would already have a fix from me if I understood the js development methodology and debug tools. I only have RubyMine.

I looked at several hardware related node modules but could not figure out how to give them a front end for NR nor how to debug and set breakpoints.

I would give it a go if it was possible to start from a working development environment.

36

If the node package is linked to the pigpiod C api it requires root.
If the node package is linked to the pigpiod_if2 C api it does not require root.

Both require sudo pigpiod as the daemon.

37

Actually, a lot of people are moving to Rust for this. It aims to have the same performance but enforces the things that make C/C++ a dangerous language for system-level programming (e.g. not being memory safe which accounts for a consistent 70% of all errors).

I'd be interested to note whether sudo is required because of some kind of kernel interaction or whether it is due to poor configuration of the pseudo file-system links to the hardware. It is very typical on Linux to have hardware links assigned to root:root when they really should be something like root:wheel so that access can be granted via group membership.

Also worth noting that, if starting Node-RED via systemd, you get much more control and probably wouldn't need root.

And finally, please remember that many people use Node-RED on other platforms, not just Linux/Pi.

Though I've come to this discussion late, I'll add my support for the creation of a list of key Pi hardware libraries - I think that would be excellent. But as others have said, this is not core Node-RED. While Nick is currently doing his best to encourage more people to pick up and understand how to help with core changes & the input from Hitachi is extremely welcome, the available resource is extremely limited. The best "bang for the buck" is achieved by focusing on continuing to make Node-RED a solid, reliable, flexible low-code platform.

Node-RED has massively outgrown its original intent which is a great testament to Nick & Dave and other IBMers who contributed to the original and ongoing development. Just lurking in the forum for a while will soon build the understanding that Node-RED is now used in many commercial settings even though, as has been said, the small niche crowd of people using it as their primary HA system are the most vocal in the forum a lot of the time.

At the end of the day, Node-RED will only continue to grow and be successful if people with skills help it to develop and grow. Whether that is by contributing code, documentation, testing, and/or ideas. That means investing personal and possibly professional time and resources into it.

If you perceive that a set of Pi GPIO/Sensors nodes are important to the community, please help corral support and make it happen.

For me personally, Pi GPIO has never been useful, it is too easy to destroy a Pi via it's GPIO and too hard to control the pins due to the complexity of the OS. It was much easier for me to sit down and learn enough C/C++ to program Arduino's and then ESP8266/ESP32's. So much so that I am now moving most stuff off my Pi's back onto a PC. But again, that is just the perspective of using NR for HA.

3 Likes
38

I do not think there is a rust library for the pi gpio. I agree w.r.t. memory leaks but the pigpiod library is very mature and I have > 1yr up time without problems so far.

The author of the pigpiod library states that the daemon should be started with sudo and that all code linked to the pigpiod_if2 api does not require root.

Yes I appreciate that however the largest user base should be considered before other use cases especially when dev resources are so scarce.

I strongly believe that for a balanced outcome there should be some way of weighting opinion based on whether that opinion is from a hardware or software perspective or both.

Do you have a link re Hitachi ?

That is what I have spent the last 24h doing. I have had to be provocative in order to get contributors to recognise their own perspectives along side that of others. It felt a little like a bomb had gone off as a result.

With respect I do not think the OS or the library can be blamed for fried I/O.
Buffers or I/O expansion are the safest bet here and go naked gpio only when fully tested.

I found that NR for HA works fine if you are careful which js libraries are pulled in.
I agree that a PC lifts any code size and speed restrictions but cost also factors in and a portrait 7" touch screen will fit flush when mounted in the place of a light switch.
I have sited in an earlier post that one can include moment in order to format a date

39

Iā€™m sorry to butt in to this topic so late, but do you have resources supporting that ā€œraspberry piā€ users are indeed the biggest userbase? Because so far I havenā€™t seen direct evidence of that, nor am I specifically supporting them as special beyond ā€œarm linuxā€ for nodes I work on. If so, it might be something I need to be more focused on as well, perhaps?

1 Like
40

Not necessarily. There are many possible approaches and at the end of the day, the product "owners" need to make the call. The largest number of users does not necessarily translate to the broadest reach and benefit over the longer term - especially given the limited resources.

I for one would not presume to know the best roadmap for Node-RED as a core.

And I think that we appreciate that viewpoint but it seems clear that not everyone agrees at this point. Such hardware nodes can be developed by anyone (with the skills or willing to learn them) whereas core development is much harder.

I think there was a blog post about it when it was announced.

Yes, that is common in forums. I think it safe to say that this is one of the best and friendliest tech forums around so the reaction tends to be "relatively" mild :smile: But there is a lot of very personal investment into the platform and people are naturally "enthusiastic" about their responses.

And with respect, I didn't. Though perhaps that's how it comes across. No, the issue re safety is with the hardware. The issue with the OS is that you are carrying maximum complexity for a tiny, niche feature set. Personally, I really wish that the Pi had been designed with an Arduino or similar on-board. But doubtless that would have had a hit on cost.

But that is not what has been discussed so far is it? Or did I miss something? And if you are going to use additional hardware, the cost of an Arduino nano or ESP8266 is so low, there seems little reason NOT to use one?

Indeed. That is true of all platforms. Whether node.js, Python, or whatever.

Of course. That is why I was careful to prefix that with it just being my experience - I happen to have an old work laptop available after an upgrade just pre-covid lockdown - it would have been scrapped so I have permission to reuse it.

Sorry, not sure what that relates to.