I spend last year quite a huge amount of free time in the new object detection node from our friend @dceejay, based on Tfjs (i.e. the javascript port of Tensorflow). However I needed a fix in order to be able to finalize it. But looking at the commit history of Tfjs, Google seems not to be interested anymore in that nice library:
Which is kind of a killer for my video surveillance dream with pure Node-RED
Of course there is the python API for Tensorflow, which seems to be maintained still very well. So I assume Google has not ditched Tensorflow completely, but only Tfjs. However calling Python code from Node-RED is somehow perpendicular to our 100% Javascript ecosystem
Does anybody know if there is a (decent) replacement for Tfjs?
Not sure how to continue with this (when I should have enough free time)...
I know you will not want to hear this Bart - but Frigate is extremely easy to use (comes as a docker container) and can offload recognition etc onto a Coral (USB or PCIe) to take the workload off the processor - and it pumps out MQTT messages for all the stages of recognition - might be better off putting time into that ?
I have 8 cameras running and it uses less than 10% CPU on my Linux host and Inference times is below 10ms
Yes I have to admit that I also had a quick look at Frigate already...
The fact that Tfjs is slowly fading away makes it completely impossible for me to create a full Node-RED solution. Which is kind of a huge bummer, because I have spend really lots of precious free time in it.
There is one sentence on their introduction page that made me considering to start using it:
Use of a Google Coral Accelerator is optional, but strongly recommended.
Because that means that they have good support for Coral sticks.
I had object detection running via Tfjs on a Coral stick already at pretty high frame rates, but my scores were to low. So probably somewhere an issue in my preprocessing. But no free time to troubleshoot it further.
How do you connect between Node-RED and Frigate? If you should find some time to write down how you use it, that would really help ...
Yep - basically Frigate runs as a docker container and then you mount an external volume from the Linux host into the container for image and video storage.
As part of the Frigate setup there is an MQTT server that is sent messages at all stages of the processing /capturing process for each camera (it is nearly an overwhelming amount of information.
I do most of my work with Frigate in Home Assistant - but one of the things i do push off to Node Red is to look for suspicious behaviour.
An example
Our Driveway has Our EV in it and the back of it is directly adjacent to the Footpath that people walk along (all day and night) - i have a Dahua TIOC camera out there and a Shelly with a Large LED floodlight - i let the camera do its own thing - which is to flash Red and Blue lights when it detects someone walking past - and it sends this info as a MQTT stream - it basically has a heap of messages but essentially you get a message saying person positively identified in the frame and then when they walk out of camera range you get a message person left the area.
What i do is if it is after Sunset and i get a continuous stream of messages saying people detected, but no people leaving the frame (use a trigger node) - then after 10 seconds i turn on the Shelly controlled floodlight to highlight the whole area and leave that on until i get a message that they have left the area
If i still do not get the message i have an audio message from the camera that tellls them they are being recorded and then sounds an alarm on my phone using Home Assistant.
It works well and has saved kids messing around with my car a couple of times so far.
