Home Surveillance System (HSS) - UPDATE and UPGRADE

dynamicdave · 4 February 2023 15:51

UPDATE to HSS (for existing users)

David (@dynamicdave), Paul (@zenofmud and Walter (@krambriw) are pleased to announce an update to their Home Surveillance System (HSS) project based on Node-RED running on a Raspberry Pi (recommended hardware is... Raspberry Pi-3B or 4B). Here's a link to the original project-thread.

Please note: This update is aimed at existing users who have a system that is up-and-running.

This update offers two enhancements...
(1) Replaces Single File Gallery (SFG) with version 4.11.0 (the latest as of 6th October 2022)
(2) Handles the situation where the username for the Raspberry Pi may be something other than 'pi'

The update affects the HSS Setup SAS and the HSS Housekeeping tabs on the Node-RED flow, so to take advantage of this update you need to...

(1) Import the two json files (as shown on the Downloads page of our website HSS- home ) into your copy of HSS. DO-NOT download the HSS complete flow file as this is specically for NEW users.
a_two_downloads

(2) Disable the old versions of the two tabs whilst you try out the functionality.

(3) Make a copy of your current config file especially if you are using hss_config_basic_nf.txt or hss_config_sample_nf.txt (as these files will be overwritten with new content).

(4) Click the inject nodes labelled STEP 1 and STEP 2 on the HSS_Setup_SAS-v2 tab
This action will download a new script file called org-sas-v4.sh into your Pi and then execute the bash-script to implement the above changes.

The update should be a fairly automatic process. However, if you do encounter any problems please report them here so a member of the HSS Team can try to sort it out for you.

The HSS Team hope you enjoy using HSS.

dynamicdave · 4 February 2023 16:20

UPGRADE to HSS (for new users)

David (@dynamicdave), Paul (@zenofmud and Walter (@krambriw) are pleased to announce release 1.2.2 of their Home Surveillance System (HSS) project based on Node-RED running on a Raspberry Pi (recommended hardware is... Raspberry Pi-3B or 4B). Here's a link to the original project-thread.

Please note: This release is aimed at NEW users who would like to install HSS.

The difference between release v1.2.1 and v1.2.2 is that new versions of Kevin Godell's 'ffmpeg' and 'mp4frag' nodes and Bart's 'multipart-stream-decoder' node have been packged into the flow. The updates listed in the posting above (i.e. SFG and use of a username that is not pi) have also been included.

If you are a NEW user, you just need to visit the Downloads page of our website HSS- home and download the HSS complete flow file then follow the Install instructions on our website.
a_upgrade

Please note:
Existing users who want to take advantage of the new versions of these nodes will need to remove the old nodes and replace them with new versions. If you are familiar with the mechanics inside Node-RED, then this shouldn't be a problem. However, for inexperienced users the HSS Team are currently looking at the feasibility of producing a script or a Node-RED flow to handle this task.

The HSS Team hope you enjoy using HSS.

kevinGodell · 5 February 2023 01:18

Wow. Nice work guys. It looks like you put much effort into that.

One little thing I noticed was on the installation instructions page:

The ffmpeg and mp4frag nodes should have the @ symbol in front and be lower case because it was published as a scoped packaged. Currently, the instructions would have somebody installing it directly from github, which is probably not what you want.

npm install @kevingodell/node-red-ffmpeg
npm install @kevingodell/node-red-mp4frag

dynamicdave · 5 February 2023 07:41

Thanks for your positive comments. I've updated the website with your suggestion. Thanks again.

Digital-Thor · 10 May 2023 05:43

Hello @dynamicdave.

I am trying HSS for the first time using. It looks very impressive. I'm not sure if this is the right place for questions, but please advise if I should be going elsewhere.

If I press the record button in the main app (at http://myPiAddress:1880/ui), I get an mp4 created at /var/www/html/hss_media/recordings/09/, but with a size of 0 bytes so something isn't working. Folder permission is 775.

When I create a snapshot, I get a jpeg in the ..hss_media/snapshots/09/ folder, so that part seems okay.

When I select "Snapshots" in the apps drop-down menu, below the large video area, I am shown a 403 Forbidden error. Same as when I select "Recordings" or "Motions".

As background, I followed all the instructions here. I imported hss_complete_v1.2.2.json into the Node-Red editor. I succeeded in getting the folders called default, motions, recordings, and snapshots.

I did use the standard "pi" user for my RPi4. I did not change the NGINX root. The hss_media folder is in /var/www/html/ as indicated above.

I found that the HSS website's instructions in the tab titled "3. Install", diverged from the HSS flow screenshot -- instead of an inject button named "get the configuration file" there was one named "Load... hss_config_basic_nf.txt" which I pressed.

Any thoughts on why I get "Forbidden" when I pick Snapshots, Recordings, or Motions?

Thanks,
-Thor

dynamicdave · 10 May 2023 05:56

Its been quite a while since I last set-up a new installation - so I'm a bit rusty.
One quick thing you could check is the ownership of the 'recordings' area. It should look like this...

ownership

If it's incorrect, you can adjust with this command.

sudo chown -R pi:www-data /var/www/html/hss_media

Digital-Thor · 10 May 2023 15:07

Thanks @dynamicdave,

Here's what I see after the chmod command:

pi@nr-vid-sec:/var/www/html/hss_media/recordings/09 $ ls -lt
total 400
-rw-r--r-- 1 pi pi       47940 May 10 07:22 07-22-45.mp4.jpg
-rw-r--r-- 1 pi pi           0 May 10 07:22 07-22-45.mp4
-rwxrwxr-x 1 pi www-data 38175 May 10 07:15 07-15-02.mp4.jpg
-rwxrwxr-x 1 pi www-data     0 May 10 07:15 07-15-02.mp4

You can see that before the chmod command, both pi and www-data had ownership, but now when the app saves the files, only pi has ownership. Regardless, I still get the 403 Forbidden error. Screenshot below.

Group membership looks okay:

pi@nr-vid-sec:/var/www/html/hss_media/recordings/09 $ members www-data
www-data pi

Perhaps related, the HSS site instructions say:

"After a few seconds (approx 12-secs) the background will revert back to green - indicating the recording has been completed."

but unfortunately, that didn't work for me. After I press the "Recording" button I almost instantly see a "Recoding Done" message (not in 12 seconds) and a file is created in ../recordings/09, but the button stays red, as shown in the screenshot.

Not sure if it's related, but my RPi is running headless and I'm running the app from a browser on a different computer.

Thoughts?

dynamicdave · 10 May 2023 15:35

That's fine - that's how I run my installation.

After I press the "Recording" button I almost instantly see a "Recoding Done" message (not in 12 seconds) and a file is created in ../recordings/09, but the button stays red, as shown in the screenshot.

That doesn't sound/look right. It will take some time for the recording to be performed (depending on the frame-rate of your camera) - so you need to be patient.

I think @zenofmud did some "installs" recently, so maybe he might "chip-in" with some suggestions.

Have you got a local camera on your network you could try or just the one in the bakery in Paris?

Digital-Thor · 10 May 2023 18:15

Thanks again @dynamicdave.

I'm definitely being patient after pressing the recording button, but it stays permanently red. Within a second of pressing it I get a "Recording Done" message and when I look at the folders in the pi, I can see a new mp4 with a size of zero bytes. I need to press the button again to turn it back to green.

I found an error in the HSS Recording flow. The existing Catch node didn't seem to show the error anywhere in the app's UX, so I added some debug nodes and saw the command going to the Camera recording exec node was:

"ffmpeg -y -i http://212.30.240.7/mjpg/video.mjpg -t 5 -c:v h264_omx -pix_fmt yuv420p -vb 20M -r 10 /var/www/html/hss_media/recordings/10/10-46-13.mp4"

and the error directly from the Camera recording exec node was:

10 May 10:46:15 - [info] [debug:Log Camera Rec Exec on HSS Rec Tab ]
ffmpeg version 4.3.6-0+deb11u1+rpt1 Copyright (c) 2000-2023 the FFmpeg developers
  built with gcc 10 (Debian 10.2.1-6)
  configuration:  --- Thor: I omitted these due to their length, but can post if needed. ---
  libavutil      56. 51.100 / 56. 51.100
  libavcodec     58. 91.100 / 58. 91.100
  libavformat    58. 45.100 / 58. 45.100
  libavdevice    58. 10.100 / 58. 10.100
  libavfilter     7. 85.100 /  7. 85.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  7.100 /  5.  7.100
  libswresample   3.  7.100 /  3.  7.100
  libpostproc    55.  7.100 / 55.  7.100
Input #0, mpjpeg, from 'http://212.30.240.7/mjpg/video.mjpg':
  Duration: N/A, bitrate: N/A
    Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 640x480, 25 tbr, 25 tbn, 25 tbc
Stream mapping:
  Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (h264_omx))
Press [q] to stop, [?] for help
[swscaler @ 0x55805ceba0] deprecated pixel format used, make sure you did set range correctly
[h264_omx @ 0x558056c6d0] libOMX_Core.so not found
[h264_omx @ 0x558056c6d0] libOmxCore.so not found

A possible clue to this error is that during the initial RPi setup, I had some troubles installing ffmpeg.

sudo apt install -y ffmpeg git

It flagged a handful of missing files and said to use --fix-missing. So I reinstalled with:

sudo apt install -y --fix-missing ffmpeg git

I didn't see any errors that time. I will look into the the libOmxCore.so not found errors.

Regarding your question about camera feeds, the only sample feeds that work are those from the French Bakery and Reykjavik Iceland.

I just created a hss_config.txt to use a local Tapo c100 camera. I modified the HSS initialization tab to use the new config file, but got no video in the app. (The URL is correct since it works in VLC.)

Does HSS support RTSP? If so, is the correct setting in hss_config.txt file "type": "IP cam" or is that just a free text field?

Also, from my previous post, does it seem correct that the mp4 files are being created by the app with only -rw-r--r-- 1 pi pi permissions / group?

Digital-Thor · 10 May 2023 18:55

I got it recording! According to this long post on h264_omx, there was a suggestion to use the h264_v4l2m2m codec. I updated the build recording command node in the "HSS Recording" tab as follows.

function get_decoder(platform) {
    var decoder = {
        'macOS': 'libx264',
        // 'RPI': 'h264_omx',
        'RPI': 'h264_v4l2m2m',
        'Windows': 'libx264'
    };
    return (decoder[platform]);
}

Not sure if that is the best codec choice, but working for now. It looks like there are some other tabs that will need the update too.

Outstanding Challenges

Question from my previous post still holds about whether RTSP cameras are supported by HSS. Looks like there is a "BigBuckBunny" sample in tab named IP Cam14, but not working and adjusting URL to my local RTSP cam didn't work either.
Still getting 403 Forbidden when trying to view Recording, Snapshots, etc, from UX. Maybe it's an apache config issue.

dynamicdave · 11 May 2023 05:09

I think you may have more than one issue with your installation. I noted in one of your messages you said you had problems installing ffmpeg git. That is not correct. You can use the following commands or use Manage palette to install these nodes.
Could you confirm if the nodes listed in your Palette match the screen-shot below?

npm install @kevingodell/node-red-ffmpeg
npm install @kevingodell/node-red-mp4frag

kevin_nodes

These are fairly new versions of Kevin's nodes that we are using for 'new' installs of HSS.
I assume you are a new user rather than a user who is performing an upgrade to v1.2.2 ?

Not sure why you had to change the contents of the function node 'build recording command'.
I'm using the following in my set-up and it works fine.

function get_decoder(platform) {
    var decoder = {
        'macOS': 'libx264',
        'RPI': 'h264_omx',
        'Windows': 'libx264'
    };
    return (decoder[platform]);
}

Last question. On the Node-RED tab... HSS Setup SAS are a couple of Inject nodes that will download a couple of scripts to your Pi and then run the 'org-sas-v4.sh' script. I assume you have done that?

Digital-Thor · 11 May 2023 22:59

Thanks. I had to use the new codec to get ffmpeg recording to work. It solved the error:

Yes, I did the Step 1 and Step 2 injections per the HSS website instructions.

I also already installed the latest of Kevin's libraries following the HSS site instructions. They included the @ symbol which he noted was important in the initial HSS project announcement page. Here's a text-paste from my palette:

@kevingodell/node-red-ffmpeg 0.1.1-beta.3
@kevingodell/node-red-mp4frag 0.1.1-beta.1

Not sure what you mean by:

Are you saying that the instructions to install ffmpeg sudo apt install -y ffmpeg git listed on the HSS instructions website are no longer valid?

dynamicdave · 12 May 2023 07:07

I was just saying what you posted here, didn't look right.

I see what you mean about the instructions on our website. Thanks for bringing it to our attention.

This conflicts with step-B.
I suggest you ignore all the instructions in step-A.

Just follow step-B

I'll check through the instructions on our website at the weekend and bring it up-to-date.

dynamicdave · 12 May 2023 08:10

One quick question. Are you using 32-bit or 64-bit operating system on your RPi-4 ??
There are many reports of issues running ffmpeg on 64-bit.

Digital-Thor · 13 May 2023 17:15

Thanks again for looking into this. I'm sure it's not easy.

I'm using a 64 bit RPi 4. The new h264_v4l2m2m codec is recording video and snapshots from only the French Bakery and Reykjavik Iceland default links. The key issues right now are:

RTSP

Do RTSP cameras work in HSS? It looks like there is an RTSP "BigBuckBunny" sample in tab named IP Cam14, but it is not working. I also tried the BigBuckBunny RTSP link in VLC, but it didn't work either. (I got this RTSP link from the injection node in the tab named "IP CAM14".)

Then, I changed the URL in the injection node to us my local RTSP cam and updated the hss_config file to use this camera, but the video still didn't show. Not sure if updating the hss_config is needed along with doing the injection manually from the CAM14 tab, but I figured I would do both.

403 Forbidden Error

I'm still getting 403 Forbidden when trying to view Recording, Snapshots, etc, from UX. According to Mozilla, that error means the server understands the request but refuses to authorize it. So it seems to be an apache config issue.

dynamicdave · 13 May 2023 18:00

I think the main issue is probably that you are using a 64-bit OS.
I seem to remember @zenofmud did an install with a RPi running 64-bit and had various problems.

My IoT students start their exams next week, so I'm pushed for time this weekend - as I need to support them in their last-minute revision sessions. Once this panic is over I'll flash a brand-new card with a 32-bit OS and go through the install instructions on our website.

Looking at our website logs it would appear nearly 1000 visitors have gone to our website and downloaded the various HSS files without raising any serious errors, so I'm sure we will be able to get you up-and-running with HSS (eventually).

Thanks for bearing with us.
Regards David.

zenofmud · 13 May 2023 20:16

@Digital-Thor as David said I did try with the 64 bit and had issues so reflashed with the 32 bit version and it works

Digital-Thor · 14 May 2023 06:09

Thanks @dynamicdave and @zenofmud. I wasn't aware of the 32-bit constraint -- I'll take your word for it. Unfortunately, I won't have time to do a whole installation again.

Nice to see the project and I hope that you continue to make progress with it.

-Thor

dynamicdave · 14 May 2023 06:15

Thanks for your response.

I'll check-out the 'install sequence' later this week and if there is a problem with 64-bit OS (which I suspect there is) I'll put a notice on our web site to warn potential users about it and make sure they use the 32-bit.

krambriw · 14 May 2023 15:23

I for sure hope so. I do not have any myself, therefore using public sources playlists (.m3u8). But you have to start them before you can view them. Like IP CAM10 (Red Bull), IP CAM11 and IP CAM12 all still seems to work when I just tested them. When using external sources, it cannot be guaranteed they will work "forever"

Recording & replaying them also worked fine

Topic		Replies	Views
Get node-red to host video stream from esp-eye's General	2	297	19 June 2022
How to build a video surveillance system from scratch? General	37	3999	10 December 2021
Setting up CCTV system and storing videos directly to onedrive General	5	509	30 September 2022
Gstreamer stream capture General	4	883	9 May 2022
AI enhanced video security system with Node-RED controller/viewer Share Your Projects	5	1752	12 April 2020

Home Surveillance System (HSS) - UPDATE and UPGRADE

Related Topics