I have been trying to find a solution for my bluetooth project and I have spent a considerable amount of time on this. Before I muddy the post with details of the things I've tried and the problems I've encountered, I'd like to explain what I'm trying to do, and see if someone has an approach that I haven't tried.
I would like to design a system using a raspberry pi running Node-Red that connects to peripheral microcontrollers that are acting as wireless sensors and relays. There will be 3-5 peripheral devices when all is said and done. I know most people will wonder why I'm not using MQTT or some sort of wifi solution, and for this project I would like to find a solution without wifi. I am not opposed to Bluetooth Classic or BLE, but BLE is preferred. For the peripheral devices, I am planning on using either ESP32's or Arduino Nano 33 BLE/IoT's. I would use raspberry pi zeros, but I don't have any and they are extremely hard to come by right now. The data being transferred is not super time sensitive or large in size. Some of the devices only need one-way communication from the peripheral to the Pi, but a couple will need two-way communication.
To narrow some of the suggestions, I found that the following node packages can't be installed with the latest version of the Pi kernal. If someone has information on how they can be installed without having to downgrade to an older version of Raspberry Pi OS please let me know.
node-red-contrib-bluetooth
node-red-contrib-bluetooth-serial
node-red-contrib-bluetooth-serial-port
I have tried using the following packages but had different issues with each of them:
node-red-contrib-noble-bluetooth
node-red-contrib-ble-sense
node-red-generic-ble
I have looked into binding the bluetooth channel to a virtual serial port and then use the serial node to communicate in Node-Red, but I also found several threads saying the tools to do that have been deprecated. Also I'm not sure if that only works with Bluetooth Classic, or if there is a way to set it up with BLE. More information on this would be appreciated.
I would like to see if someone has some suggestions on a way to tackle this project. If someone wants a detailed account of what I have tried and what issues I've encountered I can post that as well.
This doesn't quite answer your questions, but maybe it'll give you something else to look at. Zigbee is a protocol that is neither Bluetooth nor Wi-Fi. It is a self-healing, secure, robust, mesh protocol that can scale to many devices across large areas.
Have a read-up & have a look around the forum. It's not too difficult to get up and running and you can buy off the shelf zigbee devices for quite a low price.
I appreciate the reply! I have heard of Xbee and Zigbee protocols and should've paid them more attention. I should have been more clear, but for this project I am locked into using one of the devices I mentioned. ESP32, Arduino 33 Nano IOT, or Arduino Nano 33 BLE. Do you know off the top of your head if there is a way to implement the Zigbee protocol on these mentioned devices? I will also look into it.
I realize I should have mentioned the purpose of not wanting to use wifi rather than just saying "I don't want to". The peripheral devices will be battery powered and I am trying to make them as efficient as possible. I have read that BLE uses much less power to communicate than wifi. Perhaps the gains in reality are not what I'm imagining, and a bluetooth solution isn't worth the hassle.
Another requirement is that there may not be an existing network present at each deployment location. So if wifi is the better way to go, then I will need to setup the Pi as an Access Point which I have no problem with.
I also considered using the ESP-NOW network and then have one of the ESP32's connected to the Pi through a USB cable but to accomplish this I will have to add an extra ESP32 to the project and since this could be mass produced at some point, I'm trying to limit potential unnecessary costs.
Another note I can add is that it appears most of the Node-Red packages rely on Noble or BlueZ or both. Some of the examples I have seen of bluetooth communication between a Pi and the peripheral devices don't use either. They use Blueman or PyBlue. Is there a route I could take to use one of these and still link the information to Node-Red?
The problem with Zigbee at least is that it really requires a specific chipset and, as far as I know, isn't compatible with the ESP32's. I suspect that ZWave is the same. You could switch to Zigbee compatible microprocessor platforms but I don't believe they have the kind of software support that the ESP's have so you would need skills on the specific platform. Here is an example platform from ST: ZigBee - STMicroelectronics, there are also ARM Cortex based options.
That is probably the best protocol for lowering the power requirement but keeping with low-cost, common hardware that supports the easy programming tools such as Arduino.
Doesn't need to be a USB cable, you could easily mount a small ESP (either an ESP8266 or an ESP32 would do if using ESP Now unless you want the encryption available from the ESP32) onto a Pi hat. ESP's are a few US$ each so not likely to really impact your costs that much and the whole thing would likely be cheaper than using Zigbee platforms.
Yes, that's because Bluetooth is a PIG to work with.
Blueman sits on top of BlueZ and is a higher-level controller for it. Both it and PyBlue are Python libraries not node.js JavaScript needed by Node-RED.
This search on npmjs will give you an idea about what suitable libraries might be available for use with Node-RED: bluetooth - npm search (npmjs.com)
Thank you for the very detailed response. After reading the responses and doing some more digging in this forum as well as others, it appears the consensus is that BLE (perhaps Bluetooth in general) is not a reliable method, especially when using a Linux system.
My reasons for using Bluetooth over the alternatives are diminishing. In practice, it doesn't appear that BLE has significant power consumption differences to a wifi protocol like MQTT, and it seems to perform worse (power-wise) than ESP-NOW. I believe for this project I will abandon using a BLE network.
I would like to not mark a solution yet as the curious part of me still wants to explore the Bluetooth route for my own knowledge and skillset. I'd be curious to hear more opinions and potential suggestions on how to get a Bluetooth/BLE system working. So feel free to continue the thread for myself and any others who may be curious in the future.
Calling @davidz . His company offers commercial solutions using Bluetooth sensors for industrial monitoring. Hopefully he can provide some general advice.
@jgreenmt BLE can give you the ultra-low power consumption with relative good range (300m/1000 feet line of sight with careful antenna design). There are other technologies such as NFC that does not need battery but with less than 1m range. Zigbee has lower data rate than BLE and is used mainly for home and office. For industrial wireless IOT applications, you may be surprised that BLE is extremely widely used. LoRa is another low-power wireless technology that is ideal for very low data rate and very long distance applications such as temperature or humidity measurement.
Some quick inputs:
You may not want to use ESP for serious BLE. ESP is fantastic with combined WIFI and other features, but it is not optimized for BLE. Use dedicated BLE chips from Nordic, Silicon Labs, TI or ST or some other companies. They can give you much lower power consumption.
BLE is challenging technology. It needs tremendous engineering effort to go down to ultra-low power. You need to use C language at the BLE chip for the best performance. Use Node-RED on top level such as a Raspberry Pi to improve productivity.
The end result could be extremely rewarding. To give you an example, our ultra-low power wireless vibration and temperature sensor consumes less than 200 uA in 50Hz sampling rate real-time mode. The power consumption includes 16-bit triaxial acceleration measurement, temperature measurement, and data transfer to the wireless gateway. It takes us many years engineering effort to reach this level.
BLE technology can give you ultra-low power, but you need to work on chip selection, circuit design, communication protocol, firmware development and software development. If one area has a bug, then you end up with tens or even hundreds of times more power consumption.
Hmm, I used a short phrase to describe Bluetooth in my previous post. To expand on that, and with many years of experience of dealing with IT "standards", it is clear that BT was written by "experts" who value their jobs as experts. It works well (though actually see the recently released BLE security failure reports) when you have the expertise and the industrial resources to put into it. Otherwise, as previously categorised - it is a PIG.
I appreciate your insight, and I have some follow up questions.
Would dedicated BLE devices such as the ones you mentioned play nicer with Raspberry Pi and Node-Red?
Are there better methods to use Node-Red with BLE devices than the available Node-Red packages? (Such as using other libraries to link BLE to serial port to avoid using any Node-Red BLE nodes)
If i needed a multipurpose microcontroller that also supports BLE, what would you recommend over an ESP32?
They will help you reach true low power. They are way more difficult to use. Prepare to get very frustrated using them
At the current stage, it is recommended to use C to handle the communication between BLE controllers. Use Node-RED to handle the interface between the BLE microcontroller and your controller such as Raspberry Pi or ESP as gateway. There is constant development work and there could be better and easier solutions in the future.
There are BLE controllers from Nordic, Silicon Labs, TI, ST that you can choose from. You may choose a chip such as nrf52840, EFR32BG22, BlueNRG-LP ....
Many of these microcontrollers also support Zigbee. BLE has much bigger market share than Zigbee though (actually more than WIFI and Zigbee combined according to Industry Insights in 2022). Do you know that Apple AirTag uses BLE microcontroller nrf52832?
I'd like to expound on my questions just a little.
I am currently using platformIO or Arduino IDE to program the ESP32 in C. I dont know of a way to program the ESP32 directly from Node-Red as I don't believe it can run on an ESP32. So as you are suggesting I am only using Node-Red to facilitate the communication between the ESP32 and the Raspberry Pi. But that is what I have been having issues with. I have not found a reliable way to achieve communication between the two. This is where I would like guidance/support.
If I remember correctly doesn't the Arduino Nano 33 BLE use one of the nrf chip variants? I have struggled to get that device to communicate as well.
Also I'd just like to clarify that BLE communication between BLE devices is working great (i.e. two ESP32's or ESP32 and smartphone). So I know the libraries and code I'm using works from strictly a BLE standpoint. It is when I incorporate the Pi and Node-Red that I run into problems.
Yes Arduino Nano 33 BLE uses nrf52840.
If you use Arduino IDE, then you need to figure out how to program the interface between your BLE microcontroller and Raspberry Pi (Node-RED). You can use either serial port, SPI, WIFI or Ethernet port (if you use ESP32) via MQTT protocol.
Either I'm not being clear enough or you are misunderstanding. Perhaps you didn't read the initial posts before you were tagged. I'm not having any trouble communicating between ESP32/Nano BLE and Raspberry Pi via any of the protocols you mentioned. I have done Serial Port, MQTT, and Ethernet with no issues.
The purpose of this post was to receieve help communicating via BLE or Bluetooth Classic between a BLE/Classic device and Raspberry Pi. I stated in post #1 the Node Red packages (and some other routes) I tried without success.
To go a little deeper, when using the node red packages I had various issues. I either couldn't connect at all, connected but couldn't see characteristics, had working operation for a short time but then connection issues that wouldn't resolve without reboots, then couldn't connect anymore. It got to the point where I wanted to reach out to see if someone had a reliable way to communicate via BLE/Classic to a Raspberry Pi running Node-Red.
I realize my description of the issues above is too vague to offer any meaningful advice, but the purpose in me stating it was to help explain what I'm trying to do. Hopefully this helps and I would appreciate anyone who can point me in the right direction.
I didn't mean for my last post to sound as sharp as it did. I felt like the last couple of responses have been going in circles and my intention was to get it back on track.
Don't worry. Now I understand that you still want to use BLE communication between Raspberry Pi and your wireless sensors.
Sorry I should have made it more clear: I would recommend to use a dedicated BLE controller (same BLE vendor as your sensors) to talk to your sensors, and the BLE controller talks to Raspberry Pi with either serial port, SPI or MQTT. The controllers from the same vendor use compatible BLE library and are thoroughly tested.
I agree with your point and for the project I'm working on, I decided the same thing.
However for the sake of learning I'm still interested in how to communicate via BLE between the BLE device and Raspberry Pi even if it is not preferred or efficient. Perhaps I'm just being stubborn and the answer is to just leave it alone. But I find it hard to believe that no one does it, or that there isn't a good way. I have seen several tutorials and posts about Bluetooth communication between a peripheral device and Raspberry Pi, but they either use depcrecated tools/software, or don't use Node-Red.
