What do you mean by fast ?
PWM would certainly not be possible, but you could configure the microcontroller to output pwm and control the parameters of that pwm via MQTT.
What kind of speeds do you need? What are your use-cases? You really should be more concrete 
If you don't want to play with ESP32 and wireless and stuff, you can also connect your i2c/spi devices to an arduino which in turn is connected to Node-RED via the serial port node (which works great, by the way).
Yes, calling Python is a massive overhead. If you really need something in Python and need better performance, you might look at creating a Python daemon and then using a UDP or TCP or UNIX Pipe connection between them.
There are a couple of things limiting throughput on a microcontroller/MQTT broker setup. Speed of the microcontroller (and the speed of the sensor which is more likely the limiting factor), the amount of data to be sent over TCP/IP or serial port and the speed at which the broker can react.
One example is SPI. I have not found a working SPI driver that automatically does a chip select. e.g. If you call spi write with a chip enable x the driver is supposed to drive the CEx low do the write and return x high. I have had to handle the CE manually to get any results. So if I need to read 4 spi thermocuples each second I must do this extra work. Ideally 1ms CE would be OK possibly 10ms.
In the light of :-
Is the wifi secure ?
All my iot devices are behind a pfsense firewall and have a rule that bans any internet in or out (I think this control your house from your phone malarky is just not worth the risk). Also I certainly do not trust the cloud based HA services with my data.
I would favour zigbee over wifi.
From a coding perspective is using arduino and MQ synonymous with the output from an exec node data stream ?
Can MQ work entirely within a network segment of my LAN ?
"Manual" CE for SPI is very common.
In my opinion, it does not make a lot of sense (I acknowledge that there are exceptions) to control a chip enable or SPI from a high level system like Node-RED.
Using an arduino or esp32 or teensy (highly recommend teensy) you can certainly read 4 thermocouples via SPI (assuming the thermocouples are not horribly slow) every second and send the result over the serial port to Node-RED. Node-RED does not and should not need to know that there is SPI or CE involved in that process, it should just get the data in a convenient format.
With "be specific" I mean what on a higher level do you want to achieve.
1 Like
Take the 23017 for example.
At the Arduino client I can visualise the C code required to de-serialise an incoming command string and drive the I2C bus. Presumably an Interrupt Service Routine could be triggered by a state change on the 23017 which does an I2C read, builds a serial data message and write it to MQ.
Within NR I would expect one command input from MQ and sixteen I/O lines plus a status output just like the ncd-red-mcp23017. It allows address selection, a few more check boxes, if the channel is i/p or o/p, power on state and a load of configurable display labels.
I have studied their node_module and cannot get a handle on it at all.
Yes of course. Mine does. It is just TCP/IP so you can encrypt it with TLS. I don't bother encrypting on my local network because I don't run anything that really needs that level of security as long as I isolate the LAN from the Internet (no inbound connections, only outbound). Though traffic to and from my smart heating system certainly IS encrypted as you should expect (I've not tested the level of security though so probably not that secure).
As with any other TCP/IP based system, you can also use multiple segments and route between them with firewalls too should you have a need.
If I were setting up an industrial/enterprise IoT system, I would certainly want IoT traffic on at least 1 independent VLAN with an MQTT connection on that VLAN then probably forwarding (you can bridge MQTT brokers) to a more secure broker on a different VLAN. Depends on the value both of the data, what is being controlled and the "value" of the organisation it was implemented for (SMB not so much, hospital, Rolls Royce missile factory or GCHQ would warrant a lot more!)
As always though, security needs to be proportional to risk. No point in having a super-secure network and then leaving the front door open.
And yes, Zigbee is going to be superior to WiFi in a number of ways - whether for security possibly open for debate, I'm not an expert there. However, the ESP chips revolutionised cheap IoT. They introduced WiFi at the cost of an Arduino with the same programming interface. Prior to that, WiFi units were really expensive and 433MHz (or your local equivalent of 355 or 868) were the only realistic cheap options for wireless connections. Zigbee I don't think has yet caught up with that low cost. Though Ikea have certainly helped.
Personally, I don't see any real point in having a smart home that costs the earth and will never return benefits commensurate with the cost. If it costs me £20+ to build or buy a sensor platform to measure basics like temperature and humidity - it won't happen. Similarly if I have to spend £40-50 on an LED smart bulb or on a switch, it won't happen. If I can do it for <£10 then I have something to work with. Currently I have about 1/2 dozen hand-built sensor platforms, a dozen or so remote switches, a dozen radiator TRV's and a small selection of odds and ends like PIR's, door sensors, etc.
Anyway, sorry, I'm wondering well off-target here so I'll stop.
But you probably don't need to since someone will have already done that and you can simply use their library. A search for "arduino lcd-mcp23017" seemed to show some potentially useful ideas - I'm no expert and I didn't follow any of them.
And that highlights another benefit of using a dedicated microcontroller for that stuff - someone likely already did it and it likely works on both Arduino's and ESP's.
Sorry, VERY late to the party.
ESP Easy appears to offer support for the MCP23017 IO expander.
I've not tried it, but here's a link to the docs.
https://www.letscontrolit.com/wiki/index.php?title=MCP23017
They also offer support for PCA9685 to do PWM.
https://www.letscontrolit.com/wiki/index.php?title=PCA9685
Here's a link to the Slot Racing Car project I did with some of my IoT students.
The project used a Wemos D1 Mini (ESP8266 12F) driven from a RPi-Zero-W via MQTT.
Thank you. If a hardware NR node_module has a set of parameters that reflect all the functional aspects of the chip and is capable of decoding all the different responses then if the chip is on an arduino node.js breakage is eliminated and NR could ship with a few hardware nodes without fear of long term maintenance burden and becomes a static entity.
Remote hardware nodes could be segregated from local hardware nodes to avoid any confusion w.r.t. their implementation.
That just leaves the static implementation within NR which as I have explained above is way beyond my capabilities.
Would jetbrains be an IDE of choice do you think ?
I have single stepped in to the browser using RubyMine and opal so I know that is possible.
I only know that from my PHP days. Many of us use VSCode these days - it is based on Electron which in turn is based on node.js and has excellent node.js and JavaScript support and many useful extensions. It is completely free.
For Node-RED nodes or Arduino? In either case, I like to use VSCode for smaller hacks/projects but for anything bigger I love to use IDEA Ultimate for Node.js (Webstorm would likely do also but for work I need the Java supported also) and CLion (with PlatformIO extension) for the Arduino projects.
All of the options support semantic jumps around code and debugging.
VSCode is the most used editor for Node.js/JS/web these days but as I'm used to Jetbrain's IDE's at work, I quickly start missing some of its features.
Thanks. I pay a subscription for RubyMine and love it.
I have used source debug into the browser with opal (ruby -> js) and I have a node.js plugin.
Are there any languages other than js within the node-RED echo system ?
What possible other language could you want other than beloved JavaScript! 
Well C/C++ (and Rust) is easily integrated via node.js. Quite a few people also combine Python with Node-RED though "integration" might be a bit of a stretch. Any shell script can be called. Various forms of SQL. HTML ... so quite a few possibilities.
Fundamentally though, Node-RED is built over node.js which is server-side JavaScript.
By languages I mean things like HTML, CSS, MarkDown, HTML5 BoilerPlate etc.
RubyMine as all JB IDEs have a load of plugins incl. Node.js and I have single stepped opal (ruby->js) into the browser.
Is that the same as server side JS ?
With questions like this you can see why I find NR node_module code almost incomprehensible!
Have a look at the basic node red node development tutorial here to get a feel for the structure:
https://nodered.org/docs/creating-nodes/
But its basically node.js javascript server side and html with javascript and jquery for the client side configuration and ui part of the nodes.
Johannes
As @JGKK says, to get the most out of NR, you will want some HTML and CSS for advanced things. HTML, CSS, JS & jQuery for custom nodes. You may wish some Angular v1 if you want more advanced Dashboards. You can also use uibuilder and add any front-end library you like to go with your data-driven bespoke UI.
Markdown will always feature for documentation.
I've seen a lot of veteran developers struggling to make sense of all the JavaScript madness (especially as the "scene" moves way too fast) but it's really not that bad if you can let go and stop fighting it. The language is quite forgiving and allows writing code in different paradigms so it's quite easy to fit your way of thinking. Reading code written by others might be a little more difficult though!
1 Like
But if we're now talking a out nodes to interface hardware through GPIO, I'd say it's enough HTML/CSS/jQuery knowledge to be able to copy paste stuff from existing nodes if you just do it with precision and thought (taking note of the conventions with naming to allow NR do its thing).
4 Likes
Hi, Thanks to all for the help just a quick progress report.
In the early hours of this morning I saw a glimpse of the light ahead.
I am going to attempt to create a static definition of a remote MCP20317 (hosted on whatever is connected to the other end of an MQ pipe (sorry for not using the correct fancy words like broker etc).
I have a paid-up licence for RubyMine (which supports Node.js) but am in the process of evaluating WebStorm. Many will already know that JB run a v.generous open source freebee licence but, if there is a paid core dev. then one has to apply (I am trying that, being a tight a*** old gimmer). I am still awaiting a response from JetBrains as to what I cannot do with RM v WS.
I have forked and cloned the git repo to two places, one in my RubymineProjects folder and the other in my new WebstornProjects folder. Both have successfully parsed the source and have attempted to install the nec. dependencies, both reported the same output (why wouldnt they ?).
A slack handful of depreciation warnings but what are these gyp errors (maybe a config setting) ?
