You either did not reply or missed my question on the analogy to RPiEasy through MQTT. I've got another one. When you were talking about Arduino, would your idea with the serial interface / protocol to have something in common to Firmata for Arduino that is used for example by Node.js IoT/robotics library Johnny-Five? Wouldn't it be reinventing the wheel if so?
Good luck, and get well soon. I too am in the three score age bracket, and have had a minor heart problem, but we'll keep soldiering on 
After 40 or so years of coding, I find there are always new things to learn, and new projects to try out, so hopefully we can work out something which can help you in your quest, although it may mean another learning session ahead 
Apologies I am so slow at responding I lost track.
Possibly but unintentionally. I took a quick look at it. Are the function calls just writing to MQ ?
I would start with something a lot less granular and very basic and not needing any special interface library on the target.
That way you become target agnostic as well.
I gave an example somewhere (short term memory is a problem as you get on a bit verry frustrating).
To start with the only requirement for the target end node is to be able to read a string from serial in process the string talk to the chip and write a string out as a response back to NR. Those I/O streams lead back to the editor node for that chip via any means MQ cable or whatever (the protocol of the communication should be transparent).
Of course there may be blocking issues (I do not know I have not used MQ). By that I mean if the target reads serial in and there is no msg then the function call will not return untill a msg arrives or a timeout if that is available. And on the reply side if the traget writes to serial out and there is nothing taking your data then that write operation may not return unless a timeout is available. I would expect MQ has buffers in both directions so blocking does not occur. I assume the temporal order is maintained.
A simple 23017 command string from NR to target.
Data d from within a flow arrives at pinx on the chips node shown in the flow editor. the 23017node constructs the a string "channel_x=d" and writes it to an output which may be wired to MQ. At some point (a timeout simply shows via a status indicator that the device is not online) a message arrives on the 23017node input say "channel_6=0" (an input) which updates pin6 within the flow to the value 0.
This level of implementation does not need special software library in any target device.
I looked at the Jojnny-Five but could not get it to work with any I2C drivers (i think it was a while ago now).
Here is what my solenoid (sol) controller looks like.
When configured with 8 outputs and 1 input. The last output is the state of all ports in one message. The outputs go off to status widgets in the dashboard. I do not show the status of the sole input it drives a switch widget. I meant widget in this case!!
Is that any help. This node 'ncd MCP23017'. All the ncd nodes below. Best I could do is get them working with node 8 lts.
While I can definitely understand that approach (and I do the same when working on difficult nodes nowadays), I wouldn't advise that if you haven't made one before. While simplistic and not that practical, this page actually offers a good getting started for a "hello world" on a first node: Creating your first node : Node-RED
For simple nodes parsing in and out coming messages I take a couple days to write them from scratch. The one pictured in one of the above posts, node-red-contrib-html-entities looks very simple and is just a wrapper around an npm library, but took 4-5 days to properly put together, set up decent testing, and rework the interface until I had it the way I liked it. That may have been the first node I published to the flows library, and my programming speed for them has increased now, I definitely take more than a day for simple ones. "half a day for a bunch of them, 10 minutes for one" would be a pipe dream. I don't think I'll ever be able to write them that quickly.
Thank you.
1 Like
If the working code already exists and all that is required is to replace calls to I2c write with string out and calls to I2c read to string in ?
Sorry but I didn't understand what you are referring to (RPiEasy?) or meaning by the function calls? With MQ I assume you mean MQTT and yes RPiEasy (and ESPEasy) can be configured to send the data by MQTT.
With target agnostic you mean that the hardware/chip interface would need to be written separately for each host system (Arduino/Raspberry Pi) but it would communicate over serial (rx/tx) using the same protocol?
Given the amount of existing implementations for popular chip interfaces to Arduino, Tasmota, Espurna, ESPEasy, ESPHome, MicroPython etc. I just don't see the point in recreating them yet again using a new protocol just to interface with Node-RED.
It could be useful for some people but I can't see developers running in for this unless one ambitious and talented developer creates the framework and a big chunk of nodes to go with it first (like the devs behind the earlier mentioned frameworks).
To be honest I've only quickly tried it also as I didn't like the idea of an Arduino attached over serial as it was much more convenient to have it communicate over network/MQTT where I needed the sensors but didn't want my Raspberry Pi to be (200km away).
If that is your goal, why write a separate node for each of them, why not put together one generic one, and have the different targets handled on your abstraction level that would have to be added for every hardware target either way?
Quote from the creating nodes documentation:
Specifically the point "be well defined" and "be simple to use, regardless of underlying functionality" should matter here. The opposite of the "be well defined" point holds true as well: a collection of nodes that all do more or less the same with just one parameter changed might be less useful than one or a couple nodes that define those.
No only a single editor node is required for any number of targets and the multitude of libraries that they contain.
No need to. One node per chip for all platforms and libraries.
Sorry I do not understand what this has to do with frameworks. The nodes already exist within NR. They would just need to be copied and I2c calls replaced with strin in and string out. Nothing else to do.
Of course. The choice is yours. Whichever way it is connected the connection/protocol is completely transparent. Just a msg out and a msg in.
I was not talking about the Node-RED nodes but the actual hardware interface code that has to be somewhere. All the frameworks I've mentioned pretty much have their own implementations to all the popular sensors sold as modules for hobbyist and alike.
Also from what I've seen from various I2C implementations to popular sensors, they might be straightforward in theory but there's often some quirks and it's not just about sending command strings (if that is what you meant).
That is a somewhat simplistic view I'm afraid. At best is only true for devices without firmware and that tends to be fewer these days.
But the point remains, it isn't so much the hardware that is the issue here, it is what the hardware is connected to. That does change, and often. Even for simple sensors, the interactions may change if you are hitting the hardware directly and even when you aren't many sensors can be tweaked. Go see the libraries for things like the BME/P sensors or the SHT30, there is a lot you can do to get the best results out of those sensors and as knowledge of them improves, the software that interacts with them may also change. But the biggest changes are at the OS and higher levels as we keep pointing out.
Just because the hardware hasn't changed (and lets not forget about updated hardware revisions, not everything with the same number is exactly the same) it doesn't mean to say that you don't have to change how that hardware interfaces with the OS let alone anything higher up the stack.
Unless you later purchased a 2nd chip but that had actually been revised. But even if not, any change is a functional change in software and may have an impact on performance, compatibility, etc. It all needs testing and maintaining.
Believe you me, in my professional life I've seen many devices both in the comms world and in health where manufacturers haven't changed anything for years and other people pay the price, often badly.
Nothing stays static and thinking otherwise is very blinkered at best, dangerous at worst
OK, last time I'm going to say this - and I'm only saying it in case other people stumble across this thread. That is incorrect.
On a Pi, nothing is sending direct to hardware. Everything is filtered by the OS. Unless you are writing ring-0 drivers. Those are written in C/C++ or assembler or something that works at a low level, like Rust.
A Pi is a general purpose computer not an embedded microprocessor. Actually even then, you are not directly hitting the hardware very much any more. Most sensors are firmware controlled and microprocessors are now so powerful they commonly have an OS.
Once you have firmware and/or an OS, you have software which needs maintaining. Indeed, that is exactly why firmware is used, so that defects can sometimes be corrected in software without needing to release new hardware.
Doesn't come across very well when you've used so much aggressive language elsewhere I'm afraid.
Me too. Been in IT since the early 80's. And yes, absolutely still learning new things. That's what I love about IT.
1 Like
I am not sure I understand what you are suggesting. The code templates already exist they just talk to hardware drivers instead of simple msg in msg out. I am looking at this from a minimal impact minimal code change point of view. That sounds like a lot of work. From the flow editors node code point of of view it can be talking to any target that can be hooked up by any means available. Minimum effort.
Sorry but you are reading too much into my simple "channel_x=y" example.
Although it is not far from reality when you read the info page for that node ncd-mcp-23017 provides. Of course the incoming msg has to be parsed at both ends but that is not rocket science. And what can be more simple that a human readable delimited text string. Again my approach is based on minimal effort maximum gain. These chip definitions already exist. Just like a parts library in schematic capture. They do not change.
The hardware interface code is supplied by the manufacturer/seller. arduino python library as an example. As such it is of no concern to the NR end. Any target (ardino, pi etc.) that has a library for chip X and can connect serially/network to from NR can use the exact same flow editor node for chip X with out modification.
Yes and that is OK. The code examples provided by different manufacturers to drive chip X all do functionally the same thing. The function calls to read and write the device may look different and abstract the device differently but they must all present the same functionality. In the case of the 23017 turn out put x high/low. read input x.
On the target I have explained above that a few lines of code are required to parse the incoming message to determine the driver params to action the command. e.g. Split the incoming message to determine the channel number and state and call the driver to action it. If the device triggers an ISR because of an input state change then the ISR used the driver to read the new input state and build a simple response "channel_1=0, channel_2=1,..." in line with the response expected by the NR flow editor info page for the 23017 chip.
Information and data are diced and sliced everywhere on a flow. This is no different.
Sorry what more can I say. I do not have nor will have an aggressive bone in my body.
I am talking sensors and drivers here. The functional semantics will not change in these devices. Often the devices are deployed in hardware only situation with no software at all. Or a device is really and FPGA underneath. But what you can bank on is that device x (a port expander or humidity sensor) will remain semantically static as per datasheet.
Sorry but data sheets are data sheets. Even chips from different manufacturers of the same part number can be interchanged
There are no implementation details going up the stack. The command response is just a string of data values, commands and status info. Defined within the nodes info page. You decide how they look. "read temperature", "read analog channel 6" anything you want but statically defined and parsed to ensure no bad data gets in through the response. On the Si7021 humidity sensor the command may be "calculate dew point". Some python driver I was looking at exposed and handles heating element control etc within the device. But you do not care at the NR end. The detail is where it should be, on the target and under control of the driver for that specific chip.
Get this in perspective as long as there is a data sheet for a device then the operation of that device will adhere to that specification for the lifetime of that device. It would be a very poor library if it could not conform to the device specification.
Perspective again. If so prove it. A hardware driver is written against a data sheet and if it does not comply then it is rubbish.
You call a driver for the chip. If the chip has a write register then the driver will provide write access to it. If it has a status register the driver will allow you read access to it. If you compile your own code against pigpiod_if2 you can access the device in user_mode. where is the problem and what is the proble that is bugging you ?
No matter. A driver to a 23017 must provide the ability to turn output high or low, set pull-ups or pull-downd or what ever the data sheet says the chip can do. All that has to be done is send those commands in a form that is simple and easy to understand. The low level bti twiddlin that goes on is of no interest.
Sorry I hurt someones feelings it was not intended to do so.
I became suddenly alarmed by the responses that were coming back at me and I tried to defend my position as best I could for several posts before I realised why the responses were so off the wall (it really did feel like I had been turned of after 4 days of fielding questions and desperately trying to be understood). I meant flow editor node not widget.
I have corrected all my widget gaffs where I can but cannot edit the quotes in other peoples posts.
What am I to do ?
Have I answered all outstanding questions ?
Is this it? Is this what you want? I this the key?
What is stopping you trying? Why don't you just make a proof of concept, showing the benefits we all seem to miss? Sounds doable & simple enough. And please, age before beauty, you are not alone in retirement, me too, I spent many many hours also trying to learn as best I can. It's now we have time for fun, not only for grandchildren and repainting the porch
You know, when I see something goes wrong, like when there is a problem with the serial port after an upgrade, you see immediately a reaction, people start asking about it
So I don't know, is there a huge problem reported for those nodes you are mentioning? Or is it so that very few users actually need them? I would expect a massive reporting if an important node stopped working. Let's assume the MQTT nodes suddenly stopped working after a NR upgrade. What do you think would happen?
Also in professional life things like this happens. You believe in something that you think "is a must, a killer" and your customers will love it, they will not be able to resist once it is launched. You get your blessing & budget to start the project. And then...what, just a few customers are placing an order, the rest is not even interested. Demand and supply
1 Like
Exactly this! I've mentioned previously that my company makes a range of devices, and provides drivers and other support software, and an extensive API. We value feedback from our users and look carefully at suggestions for improvements or new features, but the trade-off between the time and effort to implement them and the likely user uptake and benefits are carefully weighed.
I had a "brilliant idea" to improve the API the other week, but after discussion it was decided against it. It did simplify things for a very small category of users, but the development time and future support would be fairly large. As my boss said, we don't want to over-complicate the API - keep it clean and simple.
I'm still not sure I fully understand what @ozpos is asking for, but it seems like it would be adding a large number of very device-specific nodes to the system, but which would have limited interest to most users, and the development and ongoing support would be considerable.
The best approach, which is pretty much what's there now, is for users who need these kinds of node to develop them, and hopefully maintain them in the future if the underlying libraries or architecture changes.
2 Likes
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
