Neural network - Internal Node or Watson

General
1

Hello everyone

I have integrated several home sensors in my nodered configuration with the use of several protocol nodes ( Modbus / Mqtt / HTTP), these sensors, (various PIRs / electricity meter / water meter / WIFI router record / electricity meters for specific appliances / garage door / Nuki lock / rachio smart sprinkler / air conditioning / House Alarm / temperature sensors + Light switches /IFTT cellphone location) collect a variety of data some of it stored in SQLite,

With all this information, I would like to create a neural network which could, for example. put the house on standby, vacation mode , or even more daring to adapt the "behavior of the house" to the habits and the number of occupants

I would appreciate some leads to start ...

2

There are two parts to machine learning. Creating a model via training and then using the model.

Of those, the most complex and intensive part is the training. It generally requires a lot of data to be processed in clever ways that I don't understand. This is not so easy as you can imagine and seems generally best left to big cloud solutions from Microsoft, Google or Amazon.

The running part however can be done locally in general.

Please bear in mind though that I am no expert on this. I'm sure there are local-only solutions, I'm just saying what I know.

The problem you have is how to get a specific handle on what adaptions you might want to make and what data those adaptions will take in. You've a lot of data across a number of different topics: garden/plants, weather, security, lighting, etc. Taken as a whole, I suspect you will never be able to come up with something. Broken down into components however - maybe.

Take the garden for example. You will likely want to combine current and near-future weather conditions with how much you have watered recently in order to predict how much and when to water. Alternatively, you could use a simple soil moisture sensor.

House empty/standby/vacation modes are easy enough. You will need to know when the house is empty or get into a discipline of arming the house when the last person leaves. Watch out for inadvertently advertising an empty house to burglars though!

And so it goes on.

The more "intelligent" you want things, the harder it gets because you need to understand what combination of data influences the choices.

3

Hi Jose,
Hopefully you get some good responses here, because I'm also interested as a newbie in this area of Node-RED.

https://discourse.nodered.org/t/standalone-raspberry-pi-security-camera-with-ai-person-detection

In this discussion they are talking about a 80$ Movidius Neural Compute USB Stick. Is that perhaps anything that could be if any use for you? Don't know if it is also used for training a neural network. Think @wb666greene or @krambriw could give you an intro about it...

1 Like
4

I think this could be a very relevant & interesting starting point for you.

Adrian at Pyimagesearch is extremely knowledgeable about this topic. Even if the focus is about detecting object in pictures it might be of great interest.

Some example scenarios I can imagine;

For 2 hours no tap water has been consumed & no motion detection registered from indoor PIR's, your cars are all parked in the garage, all doors locked, alarm system is fully armed -> house is empty, you are all away from house -> put house in empty mode

When house is in empty mode, someone comes walking up to the front door, the camera video image analytics finds and identifies your daughter (coming home from school), the alarm system gets disarmed, front door unlocks, lights in the entrance hall are turned on if dark outside, music starts playing...

Except for the face recognition analyze (this is also something Adrian has an article about) required to identify your daughter from the video, no need for neural networks. Scenarios like this can be designed & realized already today with Node-RED just using logic & available nodes (and running the deep neural network analyze as separate Python services).

My personal experience with video analytics is that it requires some reasonable computing power to run the analyze locally. In my attempts I have, in this order, used AWS Rekognition -> local analyze in RPi3's -> finally settled for an outdated Lenovo laptop running Debian and doing local analyze.

If you are looking for speed, forget the RPi3, it takes roughly 7 seconds to analyze an image that AWS or my Lenovo does in less than a second. The Movidius stick is also much faster than the RPi3 but I have no own experience of it. One thing is clear, the future will for sure bring faster "things", also the algorithms will be further optimized and refined.

1 Like
5

I've been running MobileNetSSD AI to detect people with a Movidius USB stick on a Raspberry Pi since about the 4th of July and my false alarm rate is essentially zero now. So for our home its been fantastic.

I've not trained any nets or tried any other networks capable of running on the Pi yet, but I can say software only AI on the Pi is borderline useful analyzing about one image every two seconds from the Pi Camera module. However I am finding reliability issues with the Pi Camera module where after some time from days to weeks it stops capturing new images and returns the same "old" image as a new capture, everything else continues to work -- ssh, node-red, etc. But the AI is effectively dead. A reboot seems to always fix it.

The Movidius stick is not 24/7 reliable either. Discussion boards suggests its marginal power from the Pi's USB port, or "thermal issues". But this failure is detectable with a "Timeout" error from the Movidius API. Restarting the process seems to cure it for another few days or weeks. Tough to track down.

I'm in the process of trying the new Movidius v2 SDK which does have a temperature monitoring call. But by detecting failures and restarting I've had near 24/7 operations with only a few minutes of blind time every couple of days from the recovery restarts. My major blind time is node-red running out of memory and forcing a restart every ~33 hours. Its been discussed here, seems the issue is in an underlying nodejs library used by one of the contrib nodes I use. I plan on doing a parallel rolling upgrade replacing about half the analog security DVR cameras with Onvif netcams one by one to minimize the disruptions. Now that its finally started to cool off, this process will start soon.

I'll be happy to discuss my results and experience. I don't do (or want) any AI for learning systems states, I use hardwired sensors in the door locks and Bluetooth Low Energy key fob beacons to give three states. Idle, Audio, and Notify.

When both beacons are out of range it means my wife an I are not home so we get an SMS and Email of the image whenever the AI detects a person in the camera view -- where the near zero false alarm rate really matters.

If the doors are locked but a beacon is in range its Audio mode where an Espeak-ng message says where the AI detection happened -- fantastic when the mail comes or a package is delivered.

If a beacon is in range and a door unlocked it means we are in and out doing things and don't need to be bothered by the AI. (not strictly true as if back door is unlocked and front door locked we'll still get audio alerts from the front of the house cameras or vice-versa).

Other than adding the key fobs its required no changes to our lives which is IMHO the halmark of good IOT.

6

Well, the hallmark of wife-acceptable IoT anyway :wink:

2 Likes
7

Hello all thank you for the ideas and experiences shared,
My main objective is not start with image recognition but start to get behavioral patterns and then start any actions.

For that I would like just start slow and small. (That´s why I have the Pi) ...

After browsing the several existing node red nodes and applying a scientific method and selected the one with more downloads.

node-red-contrib-machine-learning

Any one with experience about it? any good examples ?

Thank to every one who had dedicate some time to answer just a generic question

JLG

8

hello

Thank you! I will give it a try

JLG

9

You can also have a look at node-red-contrib-tensorflow. But same problem with this one: no documentation. Somebody already raised an issue to request for extra examples...

Have seen on the web already some quite interesting applications build with Tensorflow. Perhaps this article about "time series prediction on real-time data generated from a sensor" might be of any help to you?

1 Like
10

Hi guys,

I'm very interested in this topic, but I have ZERO knowledge about it (and not enough time). Have been googling a bit about 'time series prediction'. Would be great to have some kind of node-red-contrib-prediction that predicts future values based on the values in the past.

Found some tutorials (e.g. 1, 2, 3, 4) about time series prediction with neural networks in Tensorflow. However from this article, I understand (with my small shrinked brain) that machine learning is not the way to go: since machine learning needs large training datasets, and we most of the time have smaller datasets for sensors. Is that correct or have I completely misunderstood this?

In the latter case, would the timeseries-analysis nodeJs module be a good alternative to start developing a new node-red-contrib-prediction node? It contains a whole section about data prediction.,,

Bart

11

This has been the BIG challenge for the stock market the last...40 years, in technical analyze, thats where I first heard about using self-adapting neural networks aimed to predict stock prices (a bit off topic, I admit)

12

:joy:
Ok, that is true. But I can image that e.g. the temperature at my home (weekends, holidays, work hours, ...) is much more predictable, compared to the price of titcoins ...

13

Hello Bart

I am a bit sick but i have just browsed the youtube and i saw very interesting things, When I am back to myself again I will shares with

Thank you

JLG

Bart Butenaers nodered@discoursemail.com escreveu no dia sábado, 13/10/2018 à(s) 23:02:

1 Like
14

just some comments:

  • machine learning not always needs lots of data to work (although the more the better, usually)
  • however, I think that especially a home equipped with sensors generates a lot of data
  • there are other methods than neural networks....

What would be a concrete example what node-red should learn about your environment and act accordingly?

15

Hello Cinhcet

What I have today is a lot of sensors and information, some of it saved and other not.

I just want to explorer what is is available, because we could have simple decisions Tree made by AI or some more complex decisions.

The basic things is to adapt the house to the habits of the occupants, I dont wnat to have a key saying I am home. But I want that the house temp, the lights the water is being turned on or off based upon the people and the people habits.

if you have any of this things, normally based upon some time series data, it would be the best point to start
Kind regards

José Godinho

16

another way to bring in machine learning (or rather, the processing of pre-taught models) is to look at https://github.com/justadudewhohacks/opencv4nodejs
This is mainly a computer vision library, but also brings in opencv's DNN parts, i think capable of running tensorflow models... Bart and I have both played with the CV parts in NR, but some months ago....
It's quite 'heavy', but when trying it out, it installed easily on windows and RPi.

1 Like
17

Hello Mister Hailes,

nice having you back on this forum !

Have been doubting whether I had to propose openCv on this post, but I have remembered the following disadvantages from our series of experiments:

  • Not easy to install e.g. on a Raspberry
  • Lot of memory related issues issues while integrating it into Node-RED (garbage collection, instance cloning, ...)

Still having some traumas about the latter two issues, Tensorflow became more and more attractive to me. :wink: In particular the fact that Google released some months ago Tensorflow.js. While Tensorflow is written in C++ (similar to OpenCv), Tensorflow.js is pure Javascript. Their main focus is on machine learning in web applications (i.e. running it inside a browser), but they are also working on tfjs-node to run it on NodeJs. However the latter on is still under active development...

I don't know whether Tensorflow.js is fast enough, or whether you can let it calculate e.g. on the USB stick mentioned above in this post. And I don't know how much percentage of the C++ functionality is covered in the Javascript counterpart. But if it should be 'acceptable' performance and functionality, it would be a great library to integrate in Node-RED. Would be installed automatically as a dependency, and less memory related troubles ...

But I have had NO time at all to do any further investigation. So I would HIGHLY appreciate if somebody could give feedback about this !!!!

Bart

18

Hello

I am just going to several tutorial for keras and tensor Flow.

I decided to go with google colab has a platform for learning and training.

Besides that not much development

JLG

Bart Butenaers nodered@discoursemail.com escreveu no dia sábado, 13/10/2018 à(s) 23:02:

19

Unless openCV 4 dramatically speeds up the dnn module the Raspberry Pi3B+ is too slow. With python code and the PiCamera Module I get about 0.6 frames/sec using openCv 3.4.2

This is still useful to me as I have neighbors that ask us (being retired) to "keep an eye on their house" while they are away. I've set a couple of these up as stand-alone boxes that use Node-Red to email notifications to me and enable/disable with a dashboard UI, that I loan out to them when they are away. But I'm not sure its worth adding to node-red unless you are targeting i3 and better systems (which can get 4+ fps from the dnn module) -- not the kind of thing I think of for stand-alone IOT applications.

Integrating the USB Movidius NCS "AI co-processor" (~$70) would be IMHO a better bet as it can get 6 fps on a Pi3B+ with python code -- which I find adequate to support four ~$20 720p "onvif netcams". Downside is I don't think Windows is currently supported for the Movidius.

IMHO AI is not looking to "make a video" its looking to make alerts with a near zero false alarm rate.

2 Likes
20

Dear Bart,
Until I'm convinced having wrong.....
I do not believe javascript is even close to providing the performance if you need fast real-time responses. For machine learning where there is need for hurry, building a model type of thing, it might be ok

But for real-time applications where you need the speed, like video analytics, my experience is that applications written in C are still performing best

  1. Motion, fast, low on resources, written in C
  2. OpenCV, hungry on resources, written in C++
  3. Java applications but I have not seen any such
  4. Python eventually doped with Cython -> should make them almost as fast as C

I wonder when we can get our hands on the software used in self-driving cars for detection of pedestrians