Node-RED TensorFlow Object Detection with Telegram on macOS

The GitHub README file cautions that their TF Node-RED Object Detection demo only works with the Raspberry PI; but I got it working just fine on macOS today.

This is a fun little project, for sure; which I retooled for Telegram and changed the detection logic:

The details are here, including the flow:

Enjoy!

We have some seagulls which sometimes fly into my condo because the window are open most of the time! They go a bit crazy when they fly into the condo and cannot fly out so easily! Maybe we can detect them before they poop on my mac! LOL

5 Likes

What would be really impressive would be a Node-RED controlled drone that chased the seagull's out again! :rofl:

haha.....

I think I need some sexy seagulls to pop on on the window ledge to lure them out! Maybe some sweet seagull love songs too!

The darn things are easy excited so if I sent an army of NR controlled drones their way, they will surely poop all over the place, in wild panic! hahahaha

They are pretty smart too. Normally, if we are in the condo or asleep in the bedroom, they will not fly in.

Crazy Birds!

They are indeed crazy. Maybe a chip firing thing then, they will surely follow the chips out the window. Of course, they'd also be attracted to the chips in the first place - ho hum.

You could try hanging up a silhouette of a bird of prey outside the Window, that might work?

1 Like

haha..... tried that a few years ago..... they just fall in love with scarecrows, decoys, model birds.... we even have a windmill... that worked for a few days until they started to fall in love with the windmill too!

LOL. Life is tough!

1 Like

Installed Node-RED with TensorFlow.js on an older MacBook Air and now testing this "detect the seagulls" TensorFlow app using the built-in MBA web cam (the Logitech wide-angle HD camera on my MacPro trash can is too short to reach the window... LOL).

Come on birds!!! :slight_smile:

Hi @unixneo,
That tutorial uses the node-red-contrib-tfjs-object-detection package. Seems not to be available in npm for some reason. Do you know why (beta phase or ...)?
P.S. lovely view from your window ...
Bart

Hi Bart,

I installed node-red-contrib-tfjs-object-detection following directions from Git:

There is also node-red-contrib-tfjs-object-detection available in the Node-RED install palette, but I have had mixed results installing directly inside Node-RED and, personally, got the best results just following the Git directions (the manual step by step way), above. (clone, then do the npm install thang.... per the directions in the repo)

Still cyber-tensorflow-stalking seagulls ...... no "hits" yet. LOL

Post back if you have any issues, I have got this "cool TF stuff" running on two macs, one 12-Core MacPro (2013) and one older MacBook Air (both running Catalina).

PS: No idea why it is not directly available in npm as I did not go that route and just followed the Git repo install directions.... sorry. My best (bad, stab-in-the-dark guess) is that because the developer of node-red-contrib-tfjs-object-detection states it only works for the PI (based, in part I think because of the usbcamera and other requirements which the developer believes are currently "PI only", but as I mentioned, this is only a wild guess). As mentioned, I had to do a little "out-of-the-box" thinking and debugging to get it running on macOS, but it does work after "adjustments".

1 Like

Thanks! But my brain is a bit too small for this Tensorflow stuff ...
There are a number of tensorflow.js nodes for Node-RED available to offer object detection in images (via the coco-ssd model), but they all differ:

  • The node-red-contrib-tfjs-object-detection node (from IBM) is not on npm (yet?) but one of the advantages is that it installs both tensorflow and the coco-ssd model automatically. See their package.json file:

     "dependencies": {
         "@tensorflow/tfjs-node": "1.4.0",
         "@tensorflow-models/coco-ssd": "*"
     },
    
  • The node-red-contrib-tf-model on the other hand explicit advises not to install tensorflow.js as a dependency automatically:

    image

  • The node-red-contrib-tensorflowjs from @dceejay says this:

    image
    But on the other hand the Tensorflow documentation says they support Raspberry. Or does Tensorflow.js perhaps doesn't raspberry? But that again would be strange, because the tfjs-node documentation says this:

    image

Any thoughts to have this noob started in the right direction?

For me, so far, I have had the best success with the IBM demo node-red-contrib-tfjs-object-detection as mentioned in my little macOS summary writeup in the unix / linux forums.

I have not used the other repos and nodes you mentioned.

In addition, I have tried node-red-contrib-machine-learning because I want to refine and train object detection models; but node-red-contrib-machine-learning is a bear due to poor documentation.

I found node-red-contrib-tfjs-object-detection a piece of cake on macOS; and freely confess I have not tried it on any other platform, including the PI. It's working flawlessly at the moment, except the model needs to work better (better training).

Hence, from my perspective, I can only comment on node-red-contrib-tfjs-object-detection and I installed it from the IBM GitHub repo directions (manually with npm) and made a few tweakies for macOS and it's been running great (now on two macs).

I assume you are on the PI.

Have you tried just cleaning thing up and starting from scratch following only the node-red-contrib-tfjs-object-detection install directions on their repo?

No, but since you had the least problems with that, that probably will become my starting point.

Can you please explain that a little bit more?

Sure. Here are some high level concepts.

In this case, we are talking about processing images to detect objects. There are many other detection models; but this particular node-red-contrib-tfjs-object-detection gizmo is about object detection.

TensorFlow has created and provides some basic models in their TensorFlow offerings that detect a number of objects classes. When a computer algorithm "detects" an object, it will assign a probability or confidence estimate, for example:

"person" 0.92323232

However, the Google TF model in this case, based on my testing so far, only detects person and it can, fairly accurately, detect the number of objects in its field of view, in this example person objects. But the current model does not attempt to detect who that person is. That requires a different model and / or training the current model (with ML learning techniques), since we might only want to detect our family members to distinguish them from strangers, if we were interested, for example, in getting an alarm or message when a stranger enters into some space (instead of the coarse object person).

I advise you not to worry too much about that yet.

First, you need to get up and running and detecting the objects in the models out-of-the-box and when you are comfortable with that, then you can decide if you need different models and how deep you want to detect (how deep you want your models to learn). Of course it is more coarse
to detect a person versus detecting male person or female person etc. So, we have to crawl before we walk and walk before we run.

In my case, I am trying to detect bird and that is an object which files and poops on my balcony. At this point, we are not trying to detect if it is a seagull or a pigeon; just a bird which is a less refined object than seagull obviously.

In this app we are discussing, node-red-contrib-tfjs-object-detection, the focus is on coarse object detection based on the provide model. Or course, the more refined and less coarse an object model is the bigger and slower the detection process.

So, instead of digressing into discussions about false positives and false negatives and so forth and so on about detection theory, it might help you a bit if you review a paper I wrote this topic a number of years ago, regarding computer and network security. It's fairly high level and easy to read:

https://www.researchgate.net/publication/220420389_Intrusion_Detection_Systems_and_Multisensor_Data_Fusion

Back to node-red-contrib-tfjs-object-detection ....

I think this is a very well designed basic still image detection module for Node-RED and I like it so far.

Frankly, I am not a huge fan of surveillance capitalism and commercial applications of surveillance like the huge tech companies are doing (don't get me started on that rabbit-hole topic); but I am interested in home applications like stopping birds from pooping on my balcony or detecting strangers in the vicinity of my home.

In addition, I don't use any tech from companies which have surveillance capital business models; for example Blynk, Alexa / Amazon, FB APIs and the myriad of other tech companies which surveil all online activity and create human behavioral models (for sale to the highest ad bidder) as they offer us "free" services.

Node-RED is a great platform to build our own instrumentation and this is good, so people do not depend on third-party services which are "free" in theory but the cost is little privacy. The way to resist this surveillance capital business model is to build our own and manage our own data.

That is why I will never use Alexa for home automation. I don't use Blynk either, after reading their TOS (terms of service) where they have the right to share all your data, even your own messaging data, with third-parties.

This is also why I have become a huge Node-RED fan because we, all of us, can build our own instrumentation sans the surveillance capitalism business models of big tech.

Sorry to digress..... LOL

When you get old and long in the tooth, we start to digress often ... because of all those years of tech experience embedded in our brains... haha :slight_smile:

1 Like

Just to lob one in here - I also have a node that is basically the same but includes the model in the node - so it should work "offline" - https://flows.nodered.org/node/node-red-contrib-tfjs-coco-ssd

I appreciate this great introduction!!

Dave, is there any particular reason why you haven't added a Tensorflow dependency to your node?
And about person detection: you support two models (posenet and coco-ssd). Which one would be best for this purpose?

Eh ? The coco-sad node only does objects including people. No posenet. And absolutely does have a tensorflowjs dependency in package.json. Ah the npm point to the wrong master project... will fix.
should be this one - https://github.com/dceejay/tfjs-coco-ssd

Ah that is why I was confused. Ok I now see indeed your tfjs-node dependency. Thanks! I'm back on track...

(and so am I :slight_smile: - thanks for spotting it !)

1 Like

Update:

So far, no luck detecting birds in my setup; except COCO-SSD seems to "think" a bonsai tree moving in the breeze is "person" .....and the bright sunlight and / or reflections are "oven"..... LOL

Not really encouraged this setup is going to detect pooping birds , but let's keep testing.

In the meantime we are going to take a drive in the car today with Node-RED running this TF object detection model on a MacBook Air, and see what kinds of Telegram detection messages that brings us.

Yeaaaa! A First ! Haha!

After running for 4 days (LOL) and detecting the sun as "oven" and moving leaves on a bonsai tree as "airplane" and "keyboard", a small bird flew in and perched itself right on the balcony rail, and my MuppetBot detected "bird".... LOL

Little MuppetBot is certainly far from perfect {{ I need to install a more robust (large, slower) TF object detection model}}, but have been busy on another project).

Here ya go :slight_smile: - See the most recent entry at the bottom....

It's actually a bit exciting to detect a bird (for the first time) on the balcony with a Node-RED object detection flow.

1 Like

This is working well, but I need to take a few steps back when I have time (busy with other projects at the moment) and find a different object detection model; or I need to find out if there are more refined COCO models and how to install them without breaking my current setup.

The COCO model in this demo works very well detecting people, but is not very accurate with smaller objects like birds. Detecting people, however, is really fast and impressive.

Doing some quick research into this, the answer was not obvious how to proceed; but I did find an interesting paper by the developers of the COCO model:

Microsoft COCO: Common Objects in Context

Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, Piotr Dollár

(Submitted on 1 May 2014 (v1), last revised 21 Feb 2015 (this version, v3))

We present a new dataset with the goal of advancing the state-of-the-art in object recognition by placing the question of object recognition in the context of the broader question of scene understanding. This is achieved by gathering images of complex everyday scenes containing common objects in their natural context. Objects are labeled using per-instance segmentations to aid in precise object localization. Our dataset contains photos of 91 objects types that would be easily recognizable by a 4 year old. With a total of 2.5 million labeled instances in 328k images, the creation of our dataset drew upon extensive crowd worker involvement via novel user interfaces for category detection, instance spotting and instance segmentation. We present a detailed statistical analysis of the dataset in comparison to PASCAL, ImageNet, and SUN. Finally, we provide baseline performance analysis for bounding box and segmentation detection results using a Deformable Parts Model.

Also, directly available here in PDF.