YOLO is supposed to be "best" but it doesn't seem to run well on IOT class hardware and the readily available AI "co-processors" Google Coral TPU, Intel Movidius NCS.NCS2 (an improved model is in the pipeline), etc. aren't yet good enough for it as far as I can tell.
I'm using the Google Coral TPU and MobilenetSSD-v2_coco AI and getting very good results. I have 15 outdoor facing cameras 6 4K UHD and 9 1080p using an i7-4500U I'm getting about 38 fps AI inferences for multiday runs. Each camera rtsp stream is set for 3 fps which means processing every frame would be 45 fps. I'm currently in the process of finishing up the "permanent" installation for 24/7 use after a successful week of testing.
An early version of my code is here: https://github.com/wb666greene/AI_enhanced_video_security
But this is before I added the Coral TPU and MoblenetSSD-v2. I plan to update my GitHub soon, I just need to find time to write some description and instructions.
Some recent test results on Pi4B and Jetson Nano:
5DEC2019wbk some Pi4B tests with rtsp cameras, 3fps per stream:
4 UHD (4K) : ~2.8 fps (obvious Pi4 can't handle this)
4 HD (1080p): ~11.8 fps (basically processing every frame)
2 UHD 2 HD : ~6.7 fps (Pi4B struggles with 4K streams)
5 HD : ~14.7 fps (basically processing every frame)
6 HD : ~15.0 fps, -d 0 (no display) ~16.7 fps
6DEC2019wbk Some UHD tests on Jetson Nano
5 UHD (4K) : ~14.6 fps (effectively processing every frame!)
5 UHD 3 HD : ~10.3 fps, jumps to ~19.1 fps if -d 0 option used (no live image display)
4 UHD 4 HD : ~16.3 fps, ~22.5 fps with -d 0 option
5 UHD 10 HD (1080p): ~4.4 fps, ~7.6 fps with -d 0 option (totally overloaded, get ~39 fps with running on i7-4500U MiniPC)
Basically a Python program runs the AI and a node-red flow handles the control and sending of notifications.
Here is the flow I use for testing the IOT class hardware (Pi4B, Jetson Nano, etc.).
[{"id":"3060ae3b.5947f2","type":"mqtt in","z":"f0607957.14d578","name":"AIdetection","topic":"AIdetection/#","qos":"0","broker":"b5381f24.da8dd","x":115,"y":75,"wires":[["3b4c4658.42725a"]]},{"id":"3b4c4658.42725a","type":"change","z":"f0607957.14d578","name":"","rules":[{"t":"move","p":"payload","pt":"msg","to":"image","tot":"msg"},{"t":"move","p":"topic","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":285,"y":75,"wires":[["585ab982.05a2c8"]]},{"id":"717dd0d5.d5ca8","type":"file","z":"f0607957.14d578","name":"","filename":"","appendNewline":false,"createDir":true,"overwriteFile":"true","x":771,"y":74,"wires":[["982cbbae.069548","8a628e2e.8ba3e"]]},{"id":"8a628e2e.8ba3e","type":"debug","z":"f0607957.14d578","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":895,"y":35,"wires":[]},{"id":"311ef41a.e4d01c","type":"e-mail","z":"f0607957.14d578","server":"smtp.gmail.com","port":"465","secure":true,"name":"","dname":"Email Notification","x":1005,"y":645,"wires":[]},{"id":"5738cf5c.29578","type":"mqtt out","z":"f0607957.14d578","name":"Alarm/MODE","topic":"Alarm/MODE","qos":"2","retain":"true","broker":"a12f6f63.c32c7","x":650,"y":261,"wires":[]},{"id":"8b98ffdb.f2563","type":"debug","z":"f0607957.14d578","name":"MODE","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":630,"y":221,"wires":[]},{"id":"982cbbae.069548","type":"function","z":"f0607957.14d578","name":"Notification Filter","func":"var Idle = context.get('Idle')||0;\nvar Audio = context.get('Audio')||0;\nvar Notify = context.get('Notify')||0;\n\nif (msg.topic === \"Alarm/MODE\"){\n if(msg.payload.includes(\"Idle\")){\n context.set('Idle',1); \n context.set('Audio',0);\n context.set('Notify',0);\n }else if(msg.payload.includes(\"Audio\")){\n context.set('Idle',0);\n context.set('Audio',1);\n context.set('Notify',0);\n }else if(msg.payload.includes(\"Notify\")){\n context.set('Idle',0);\n context.set('Audio',0);\n context.set('Notify',1);\n }\n return [ msg, null, null, null ]; // change null to msg for debugging.\n}\n\nif (msg.topic === \"AIdetection\"){\n if (Idle == 1){\n return [ null, null, null ];\n }\n if (Audio == 1){\n return [ null, msg, null ];\n }\n if (Notify == 1){\n return [ null, msg, null ];\n }\n}\n\nif (msg.topic.includes(\"DetectionImageBuffer\")){\n if (Notify == 1){\n return [ null, null, msg ];\n }\n else{\n return [ null, null, null ]; \n }\n}\nreturn [ null, null, null ];","outputs":3,"noerr":0,"x":445,"y":340,"wires":[["c2061c74.31138"],["98eac8e0.b34058","be485b2c.ba6308"],["75f45acd.701a74","bc7c0fc5.ec88c"]]},{"id":"f153f958.178898","type":"debug","z":"f0607957.14d578","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":785,"y":685,"wires":[]},{"id":"affb5b8d.586098","type":"file in","z":"f0607957.14d578","name":"get file ","filename":"","format":"","chunk":false,"sendError":false,"x":460,"y":645,"wires":[["bc7c0fc5.ec88c"]]},{"id":"bc7c0fc5.ec88c","type":"change","z":"f0607957.14d578","name":"Setup Email","rules":[{"t":"set","p":"to","pt":"msg","to":"1234567899@mms.att.net","tot":"str"},{"t":"set","p":"cc","pt":"msg","to":"1234567899@message.ting.com","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"AI Detection Alert!","tot":"str"},{"t":"set","p":"bcc","pt":"msg","to":"xxxx@gmail.com","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":615,"y":645,"wires":[["311ef41a.e4d01c","f153f958.178898"]]},{"id":"75f45acd.701a74","type":"debug","z":"f0607957.14d578","name":"Notify","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":655,"y":510,"wires":[]},{"id":"bfe0b85.e076948","type":"inject","z":"f0607957.14d578","name":"Send Test Email","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":140,"y":645,"wires":[["87e601f4.50f97"]]},{"id":"87e601f4.50f97","type":"change","z":"f0607957.14d578","name":"attachment","rules":[{"t":"set","p":"msg.filename","pt":"msg","to":"/home/wally/AIdev/TestDetection.jpg","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":315,"y":645,"wires":[["affb5b8d.586098"]]},{"id":"175ecf37.75a801","type":"inject","z":"f0607957.14d578","name":"Set AI Mode Idle","topic":"Alarm/MODE","payload":"Idle","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":150,"y":215,"wires":[["982cbbae.069548"]]},{"id":"ddaa549a.47f9a8","type":"inject","z":"f0607957.14d578","name":"Set AI Mode Audio","topic":"Alarm/MODE","payload":"Audio","payloadType":"str","repeat":"","crontab":"","once":true,"onceDelay":"1.0","x":160,"y":255,"wires":[["982cbbae.069548"]]},{"id":"71619d77.b96584","type":"inject","z":"f0607957.14d578","name":"Set AI Mode Notify","topic":"Alarm/MODE","payload":"Notify","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":160,"y":295,"wires":[["982cbbae.069548"]]},{"id":"ead5388c.5278f8","type":"ui_dropdown","z":"f0607957.14d578","name":"Set AI Mode","label":"","tooltip":"Set AI Mode","place":"Select option","group":"fab579a8.55a868","order":3,"width":0,"height":0,"passthru":true,"options":[{"label":"Idle","value":"Idle","type":"str"},{"label":"Audio Alerts","value":"Audio","type":"str"},{"label":"Email Alerts","value":"Notify","type":"str"}],"payload":"","topic":"Alarm/MODE","x":120,"y":335,"wires":[["982cbbae.069548"]]},{"id":"235c0d72.ae6ec2","type":"inject","z":"f0607957.14d578","name":"Launch AI","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"","once":true,"onceDelay":"0.5","x":178,"y":1215,"wires":[[]]},{"id":"8d6f5e8a.3eb01","type":"exec","z":"f0607957.14d578","command":"/home/ai/ncs/startAI_mt.sh ","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"Autostart","x":480,"y":1215,"wires":[["bbd28bda.fc7608"],["bbd28bda.fc7608"],[]]},{"id":"166db571.a6f5ab","type":"mqtt in","z":"f0607957.14d578","name":"AI/Status","topic":"AI/Status","qos":"2","broker":"a12f6f63.c32c7","x":150,"y":1325,"wires":[["40e331b8.824c6","bbd28bda.fc7608"]]},{"id":"40e331b8.824c6","type":"switch","z":"f0607957.14d578","name":"","property":"payload","propertyType":"msg","rules":[{"t":"cont","v":"died","vt":"str"}],"checkall":"true","repair":false,"outputs":1,"x":290,"y":1325,"wires":[[]]},{"id":"bbd28bda.fc7608","type":"debug","z":"f0607957.14d578","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":635,"y":1265,"wires":[]},{"id":"6c0d3fc.db676c","type":"delay","z":"f0607957.14d578","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"1","rateUnits":"minute","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"x":800,"y":620,"wires":[[]]},{"id":"de6b06c7.1395c8","type":"template","z":"f0607957.14d578","name":"","field":"payload","fieldType":"msg","format":"handlebars","syntax":"mustache","template":"<img width=\"640px\" height=\"360px\" src=\"data:image/jpg;base64,{{{payload}}}\">","output":"str","x":955,"y":745,"wires":[["d66dda1d.b658a8"]]},{"id":"d66dda1d.b658a8","type":"ui_template","z":"f0607957.14d578","group":"ce7f592b.fc44c8","name":"Viewer","order":1,"width":"13","height":"7","format":"<div ng-bind-html=\"msg.payload\"></div>","storeOutMessages":false,"fwdInMessages":true,"templateScope":"local","x":1090,"y":745,"wires":[[]]},{"id":"f8c2fbe1.8f8318","type":"base64","z":"f0607957.14d578","name":"","action":"str","property":"payload","x":815,"y":745,"wires":[["de6b06c7.1395c8"]]},{"id":"48e30ccd.4f91a4","type":"ui_button","z":"f0607957.14d578","name":"filename","group":"ce7f592b.fc44c8","order":2,"width":"12","height":"1","passthru":true,"label":"{{msg.filename}}","tooltip":"","color":"#101010","bgcolor":"#83ed7b","icon":"","payload":"","payloadType":"str","topic":"","x":845,"y":795,"wires":[[]]},{"id":"be601c1c.c5064","type":"exec","z":"f0607957.14d578","command":" sudo /home/ai/ncs/PowerOffAI_mt.sh","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"shutdown","x":290,"y":1005,"wires":[["2a2bce20.a3f062"],["2a2bce20.a3f062"],["2a2bce20.a3f062"]]},{"id":"9bacceda.c39df","type":"ui_button","z":"f0607957.14d578","name":"PowerOff","group":"1f34e858.de7018","order":4,"width":"3","height":"1","passthru":false,"label":"Power Off","tooltip":"","color":"","bgcolor":"#793042","icon":"","payload":"","payloadType":"date","topic":"","x":150,"y":1005,"wires":[[]]},{"id":"2a2bce20.a3f062","type":"debug","z":"f0607957.14d578","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":460,"y":1005,"wires":[]},{"id":"7e2c499f.d7f6d8","type":"mqtt in","z":"f0607957.14d578","name":"espeak","topic":"Espeak","qos":"2","broker":"313c03c1.be782c","x":1045,"y":280,"wires":[["766406ad.3a2248"]]},{"id":"5e101c14.d152f4","type":"inject","z":"f0607957.14d578","name":"speak test","topic":"Espeak","payload":"'Multi word test'","payloadType":"str","repeat":"","crontab":"","once":false,"onceDelay":0.1,"x":1080,"y":231,"wires":[["766406ad.3a2248","d410603e.5fe27"]]},{"id":"766406ad.3a2248","type":"exec","z":"f0607957.14d578","command":"/usr/bin/espeak","addpay":true,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"Espeak","x":1275,"y":365,"wires":[[],[],[]]},{"id":"be485b2c.ba6308","type":"switch","z":"f0607957.14d578","name":"","property":"filename","propertyType":"msg","rules":[{"t":"cont","v":"Cam0","vt":"str"},{"t":"cont","v":"Cam1","vt":"str"},{"t":"cont","v":"Cam2","vt":"str"},{"t":"cont","v":"Cam3","vt":"str"},{"t":"cont","v":"Cam4","vt":"str"},{"t":"cont","v":"Cam5","vt":"str"},{"t":"cont","v":"Cam6","vt":"str"},{"t":"cont","v":"Cam7","vt":"str"},{"t":"nnull"}],"checkall":"false","repair":false,"outputs":9,"x":655,"y":370,"wires":[["43a331b6.b07c4"],["69fe597.086a7a8"],["9b642427.e60008"],["afc22664.c38fe8"],["345e0c7e.3d6eb4"],["a524c923.607d68"],["c7b185b9.480338"],["d1172ccc.421ee"],["26038262.243d5e"]],"outputLabels":["driveway","patio","","","","","","",""]},{"id":"43a331b6.b07c4","type":"change","z":"f0607957.14d578","name":"Cam0","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Zero'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":225,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"8e3b37fc.1e4cd8","type":"delay","z":"f0607957.14d578","name":"","pauseType":"rate","timeout":"5","timeoutUnits":"seconds","rate":"1","nbRateUnits":"4","rateUnits":"second","randomFirst":"1","randomLast":"5","randomUnits":"seconds","drop":true,"x":1065,"y":365,"wires":[["766406ad.3a2248","d410603e.5fe27"]]},{"id":"69fe597.086a7a8","type":"change","z":"f0607957.14d578","name":"Cam1","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera One'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":265,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"9b642427.e60008","type":"change","z":"f0607957.14d578","name":"Cam2","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Two'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":305,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"afc22664.c38fe8","type":"change","z":"f0607957.14d578","name":"Cam3","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Three'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":345,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"a524c923.607d68","type":"change","z":"f0607957.14d578","name":"Cam5","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Five'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":425,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"345e0c7e.3d6eb4","type":"change","z":"f0607957.14d578","name":"Cam4","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Four'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":385,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"d9044091.2496b","type":"mqtt in","z":"f0607957.14d578","name":"AmAlive","topic":"AmAlive","qos":"0","broker":"f7f213b3.fdec4","x":150,"y":1265,"wires":[["8a4384ce.a939e8"]]},{"id":"8a4384ce.a939e8","type":"trigger","z":"f0607957.14d578","op1":"","op2":"Watchdog Timeout Onvif AI Restart!","op1type":"nul","op2type":"str","duration":"30","extend":true,"units":"s","reset":"","bytopic":"all","name":"watchdog","x":290,"y":1265,"wires":[[]]},{"id":"e670596a.88f908","type":"ui_button","z":"f0607957.14d578","name":"Reboot","group":"1f34e858.de7018","order":2,"width":"3","height":"1","passthru":false,"label":"Reboot","tooltip":"","color":"","bgcolor":"#808080","icon":"","payload":"","payloadType":"date","topic":"","x":150,"y":1065,"wires":[[]]},{"id":"b8a58ba9.741ac8","type":"exec","z":"f0607957.14d578","command":" sudo /home/ai/ncs/RebootAI_mt.sh","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"reboot","x":280,"y":1065,"wires":[["9963798f.d3a498"],["9963798f.d3a498"],["9963798f.d3a498"]]},{"id":"9963798f.d3a498","type":"debug","z":"f0607957.14d578","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"false","x":460,"y":1065,"wires":[]},{"id":"b778d12d.c4eec","type":"inject","z":"f0607957.14d578","name":"Daily Cleanup","topic":"","payload":"","payloadType":"date","repeat":"","crontab":"00 01 * * *","once":false,"onceDelay":0.1,"x":190,"y":1135,"wires":[[]]},{"id":"5ca1f2e6.17f8cc","type":"exec","z":"f0607957.14d578","command":"/home/ai/ncs/cleanup.sh ","addpay":false,"append":"","useSpawn":"false","timer":"","oldrc":false,"name":"disk cleanup","x":370,"y":1135,"wires":[["f980da24.b99748"],["f980da24.b99748"],["f980da24.b99748"]]},{"id":"f980da24.b99748","type":"debug","z":"f0607957.14d578","name":"","active":true,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":520,"y":1135,"wires":[]},{"id":"6b993b9e.04eaa4","type":"ui_dropdown","z":"f0607957.14d578","name":"Set UI View","label":"","tooltip":"Enable/Disable Camera Live View","place":"Camera Viewing","group":"fab579a8.55a868","order":1,"width":0,"height":0,"passthru":true,"options":[{"label":"Disable","value":"0","type":"str"},{"label":"Live","value":"1","type":"str"},{"label":"Detections","value":"2","type":"str"},{"label":"All Detections","value":"3","type":"str"}],"payload":"","topic":"Alarm/UImode","x":360,"y":865,"wires":[["70af69c5.fcd368"]]},{"id":"70af69c5.fcd368","type":"mqtt out","z":"f0607957.14d578","name":"Alarm/UImode","topic":"Alarm/UImode","qos":"2","retain":"true","broker":"a12f6f63.c32c7","x":540,"y":865,"wires":[]},{"id":"c7b185b9.480338","type":"change","z":"f0607957.14d578","name":"Cam6","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Six'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":464,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"d1172ccc.421ee","type":"change","z":"f0607957.14d578","name":"Cam7","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Camera Seven'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":840,"y":504,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"26038262.243d5e","type":"change","z":"f0607957.14d578","name":"unspecified","rules":[{"t":"set","p":"payload","pt":"msg","to":"\"'Person Detected on New Camera'\"","tot":"str"},{"t":"set","p":"topic","pt":"msg","to":"Espeak","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":860,"y":545,"wires":[["8e3b37fc.1e4cd8"]]},{"id":"2efb95b4.af859a","type":"ui_dropdown","z":"f0607957.14d578","name":"Select Camera","label":"","tooltip":"Select Camera for Live View or Detection","place":"Select Camera","group":"fab579a8.55a868","order":2,"width":0,"height":0,"passthru":true,"options":[{"label":"Cam 0","value":"0","type":"str"},{"label":"Cam 1","value":"1","type":"str"},{"label":"Cam 2","value":"2","type":"str"},{"label":"Cam 3","value":"3","type":"str"},{"label":"Cam 4","value":"4","type":"str"},{"label":"Cam 5","value":"5","type":"str"},{"label":"Cam 6","value":"6","type":"str"},{"label":"Cam 7","value":"7","type":"str"},{"label":"Cam 8","value":"8","type":"str"},{"label":"Cam 9","value":"9","type":"str"},{"label":"Cam 10","value":"10","type":"str"},{"label":"Cam 11","value":"11","type":"str"},{"label":"Cam 12","value":"12","type":"str"},{"label":"Cam 13","value":"13","type":"str"},{"label":"Cam 14","value":"14","type":"str"}],"payload":"","topic":"ViewCamera","x":390,"y":920,"wires":[["523d28b6.3838f8"]]},{"id":"523d28b6.3838f8","type":"mqtt out","z":"f0607957.14d578","name":"Alarm/ViewCamera","topic":"Alarm/ViewCamera","qos":"2","retain":"true","broker":"a12f6f63.c32c7","x":590,"y":920,"wires":[]},{"id":"baf1818c.b3147","type":"mqtt in","z":"f0607957.14d578","name":"ImageBuffer/","topic":"ImageBuffer/#","qos":"0","broker":"a12f6f63.c32c7","x":115,"y":745,"wires":[["8c69bf5d.522ad","a2450f33.4037"]]},{"id":"98eac8e0.b34058","type":"debug","z":"f0607957.14d578","name":"Audio","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":655,"y":465,"wires":[]},{"id":"9ff33b90.794b88","type":"comment","z":"f0607957.14d578","name":"Flow Docuentation","info":"## **MQTT Input Topics from Python AI code:**\n - The Python AI code \"publishes\" these topics.\n - Other systems cam subscribe to these topics.\n1. **AI/Detection**: Buffer with topic containing camera info of most recent AI detection image.\n2. **ImageBuffer**: MQTT buffer of current image from UI selected camera, used by \"Dashboard\" web page. Depending on **Alarm/MODE** and **Alarm/UImode** either live images or only the last detection will be sent.\n3. **AmAlive**: \"watchdog\" timer, can trigger restart of Python code, or system if necessary.\n4. **AI/Status**: AI Startup, Shutdown, and \"Will\" status messages.\n\n## **MQTT Outputs to control Python AI code:**\n - These modes affect the flow operation as well as what the PythonAI code does\n - The Python AI code \"subscribes\" to these topics from broker set by -mqtt command line arguement, defaults to localhost.\n1. **Alarm/MODE**: payload Idle, Audio, or Notify: sets mode, affects flow and Python AI code. \n2. **Alarm/UImode**: payload 0, 1, 2: 0-Disable, 1-Live, 2-Detections sending of ImageBuffer to Dashboard \n3. **Alarm/ViewCamera**: payload N: Select camera N to be viewed on \"Dashboard\" UI webpage.\n - The Python code subscribes to broker set by -camMQTT command line arguement, defaults to localhost.\n4. **MQTTcam/N**: payload jpeg image: fed to AI as input from camera N\n \n## **Espeak**: MQTT Input Iopic lets node-red host be remote speaker for others if so equipped.\n - Payload is text to speak: \"' This is only a test. '\" note quotes within quotes\n\n# **Dashboard UI** Controls\n1. **AI Mode**: payload: Idle Audio Notify, sets **Alarm/MODE** for the Python AI code.\n2. **Set UI View**: payload 0, 1, 2: sets **Alarm/UImode** viewing mode.\n - 0 **Disable**: Don't show images on Dasboard webpage.\n - 1 **Live**: display real-time images from selected camera.\n - 2 **Detections**: Only show frames with AI detections.\n - 3 **All Detections**: Display the most recent detection from any camera, select camera is ignored.\n3. **Select Camera**: sets **AlarmViewCamera** to choose the camera to display on the Dashboard webpage.\n\n# **System Controls**\n - These generally are only useful if the node-red controller is running on the Python AI host.\n - Ideal for controlling a \"headless\" host remotely with a web browser.\n1. **Rebbot**\n2. **Shutdown**\n\n# **General Setup**:\n - Edit **Email Notification** for the username@gmail.com and password for Emailed notifications.\n - gmail account must have \"less secure access\" enabled in account settings for this node to work.\n - Edit Espeak \"Cam Num\" strings to something more useful, pay attension to the single quotes within the double quotes!\n\n# **Setup**: Only if viewer/controller is running on the Python AI Linux host.\n1. Wire **Daily Cleanup** to **disk cleanup**, edit exec command and script as necessary.\n2. Wire **Launch AI** and **watchdog** to **Autostart**, edit exec command and script as necessary.\n","x":140,"y":25,"wires":[]},{"id":"c2061c74.31138","type":"rbe","z":"f0607957.14d578","name":"loop breaker","func":"rbe","gap":"","start":"","inout":"out","property":"payload","x":455,"y":260,"wires":[["5738cf5c.29578","8b98ffdb.f2563"]]},{"id":"86befae8.562ff8","type":"mqtt in","z":"f0607957.14d578","name":"Get Alarm/MODE","topic":"Alarm/MODE","qos":"2","broker":"a12f6f63.c32c7","x":125,"y":425,"wires":[["ead5388c.5278f8"]]},{"id":"76a79a64.c81574","type":"inject","z":"f0607957.14d578","name":"Alarm/UImode","topic":"Alarm/UImode","payload":"3","payloadType":"num","repeat":"","crontab":"","once":true,"onceDelay":"1.1","x":180,"y":865,"wires":[["6b993b9e.04eaa4"]]},{"id":"bdae669b.b14da8","type":"inject","z":"f0607957.14d578","name":"Alarm/ViewCamera","topic":"Alarm/ViewCamera","payload":"0","payloadType":"num","repeat":"","crontab":"","once":true,"onceDelay":"1.2","x":190,"y":920,"wires":[["2efb95b4.af859a"]]},{"id":"d410603e.5fe27","type":"debug","z":"f0607957.14d578","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":1275,"y":310,"wires":[]},{"id":"8c69bf5d.522ad","type":"change","z":"f0607957.14d578","name":"","rules":[{"t":"move","p":"payload","pt":"msg","to":"image","tot":"msg"},{"t":"move","p":"topic","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":315,"y":745,"wires":[["64303452.db106c"]]},{"id":"64303452.db106c","type":"csv","z":"f0607957.14d578","name":"","sep":"!","hdrin":"","hdrout":"","multi":"one","ret":"\\n","temp":"topic,filename","skip":"0","x":475,"y":745,"wires":[["5cd55d69.bc6f44"]]},{"id":"5cd55d69.bc6f44","type":"change","z":"f0607957.14d578","name":"","rules":[{"t":"move","p":"payload.filename","pt":"msg","to":"filename","tot":"msg"},{"t":"move","p":"image","pt":"msg","to":"payload","tot":"msg"}],"action":"","property":"","from":"","to":"","reg":false,"x":630,"y":745,"wires":[["f8c2fbe1.8f8318","48e30ccd.4f91a4"]]},{"id":"a2450f33.4037","type":"debug","z":"f0607957.14d578","name":"","active":false,"tosidebar":true,"console":false,"tostatus":false,"complete":"true","x":280,"y":705,"wires":[]},{"id":"585ab982.05a2c8","type":"csv","z":"f0607957.14d578","name":"","sep":"!","hdrin":"","hdrout":"","multi":"one","ret":"\\n","temp":"topic,filename,startX,startY,endX,endY,Xcenter,Ycenter,Xlength,Ylength","skip":"0","x":446,"y":74,"wires":[["77b49268.0ae41c"]]},{"id":"77b49268.0ae41c","type":"change","z":"f0607957.14d578","name":"","rules":[{"t":"move","p":"payload.filename","pt":"msg","to":"filename","tot":"msg"},{"t":"move","p":"payload.startX","pt":"msg","to":"startX","tot":"msg"},{"t":"move","p":"payload.startY","pt":"msg","to":"startY","tot":"msg"},{"t":"move","p":"payload.endX","pt":"msg","to":"endX","tot":"msg"},{"t":"move","p":"payload.endY","pt":"msg","to":"endY","tot":"msg"},{"t":"move","p":"payload.Xcenter","pt":"msg","to":"Xcenter","tot":"msg"},{"t":"move","p":"payload.Ycenter","pt":"msg","to":"Ycenter","tot":"msg"},{"t":"move","p":"payload.Xlength","pt":"msg","to":"Xlength","tot":"msg"},{"t":"move","p":"payload.Ylength","pt":"msg","to":"Ylength","tot":"msg"},{"t":"move","p":"image","pt":"msg","to":"payload","tot":"msg"},{"t":"set","p":"topic","pt":"msg","to":"AIdetection","tot":"str"}],"action":"","property":"","from":"","to":"","reg":false,"x":606,"y":74,"wires":[["717dd0d5.d5ca8"]]},{"id":"b5381f24.da8dd","type":"mqtt-broker","z":"","name":"localhost","broker":"localhost","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthPayload":"","closeTopic":"","closeQos":"0","closePayload":"","willTopic":"","willQos":"0","willPayload":""},{"id":"a12f6f63.c32c7","type":"mqtt-broker","z":null,"name":"localhost:1883","broker":"localhost","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closePayload":"","willTopic":"","willQos":"0","willRetain":"false","willPayload":""},{"id":"fab579a8.55a868","type":"ui_group","z":"","name":" AI Mode","tab":"cbe60e1a.85145","order":1,"disp":true,"width":"4","collapse":false},{"id":"ce7f592b.fc44c8","type":"ui_group","z":"","name":"Camera Viewer","tab":"cbe60e1a.85145","order":2,"disp":true,"width":"13","collapse":false},{"id":"1f34e858.de7018","type":"ui_group","z":"","name":"System Controls","tab":"cbe60e1a.85145","order":3,"disp":true,"width":"5","collapse":true},{"id":"313c03c1.be782c","type":"mqtt-broker","z":"","name":"localhost:1883","broker":"localhost","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closePayload":"","willTopic":"","willQos":"0","willRetain":"false","willPayload":""},{"id":"f7f213b3.fdec4","type":"mqtt-broker","z":null,"name":"localhost:1883","broker":"localhost","port":"1883","clientid":"","usetls":false,"compatmode":true,"keepalive":"60","cleansession":true,"birthTopic":"","birthQos":"0","birthRetain":"false","birthPayload":"","closeTopic":"","closePayload":"","willTopic":"","willQos":"0","willRetain":"false","willPayload":""},{"id":"cbe60e1a.85145","type":"ui_tab","z":"","name":"AI Controller","icon":"dashboard","disabled":false,"hidden":false}]
I'm perfectly willing to help you get it running, and in the process we can develop instructions to make it more accessible. Post an issue to my GitHub and I'll either zip my code up and send it to you, or post the upgraded code.
I've scratched my itch with it and think it would be more generally useful with good instructions on setting it up. You can add it to an existing system for ~$200 for ~6-8 cameras.
My initial goal was to make it an add-on to my Lorex security DVR, but in many ways it works better talking directly to netcams, but if you want/need 24/7 recordings adding it to a security DVR/NVR is better than re-inventing the recorder, they are available at very reasonable prices and solve the 24/7 or scheduled recording problem pretty completely, but I've not much good to say about them beyond this.
