Using the Pi with a CCTV system (DVR)

In the very old days, CCTV recorders were based on VHS video tape. These could record a single TV resolution image data stream at full resolution and (in the UK) 25fps. Since the CCTV cameras were of very low resolution anyway, the CCTV recorder typically divided the TV frame into quarters. This allowed 4 camera to be recorded at the same time. Then the digital DVD age arrived and the consumer moved to the DVR, the Digital Video Recorder. These digitised the TV signal, compressed it to MPG and recorded it to hard disk. At first the resolutions, frame rates and data rates just about allowed one TV program to be recorded at once. The CCTV industry adopted the DVR and recorded 4 quarter resolution image (320x240) streams in much the same way as before, except now they could record 8 cameras at once by interleaving the frames - in other words, instead of 25fps, they recorded at 12.5 fps. If you had 16 cameras, it was 6.25 fps. The recording quality was somewhat better but the resolution was not, so you ended up with much the same result. Of course customers were soon demanding that the CCTV industry use the higher resolution cameras being used by the computer industry, ie. VGA, 640x480. VGA, of course, was the NTSC standard by another name. Lucky for the UK, NTSC fitted within the 720x576 frame of the PAL standard (just add a bit of blank space), and whilst NTSC was 30fps, and PAL only 25 fps, since CCTV typically recorded no audio they simply dropped every 6th frame, 30 became 25 and no-one noticed. Four VGA/NTSC or 'D1' cameras as they were known, to hide the fact that the CCTV industry was charging at least 10 times what the computer industry charged for the same VGA camera, could be recorded at the same time if the frame rate was dropped to 6.25fps. Which they did. Soon domestic DVR's became available that could play back at the same time as recording, or you could record two programs at once. So the CCTV industry had access to machines that could record 8 cameras, or, if you dropped the frame rate to 3fps, which they did, 16 cameras. Then consumers and the computer industry moved to Hi-Def, Blueray and HDMI. Customers of exceedingly expensive CCTV systems were looking at the cost and quality of computer HD web-cams and comparing this to their CCTV VGA quality cameras. They were soon starting to complain. It was not so much the astronomical cost of CCTV, but the fact that VGA resolution was next to useless at identifying the criminal. Now one of the main costs of an industrial CCTV installation was the cabling. This consisted of a single co-ax cable- 24 volts was sent down the cable to the camera and an analogue TV signal sent back to the DVR. Usually BNC connectors were used, although some vendors took to using an RCA connector or other types to maintain incomparability and vendor 'lock in'. Some older cameras used the PL-259 connector. Plugging in a HD resolution camera was not a problem - it was getting the signal to the DVR. Of course customers eventually baulked at paying ten times the cost for co-ax cable that they paid for Ethernet and the CCTV industry was forced to move over to standard TCP/IP Ethernet and PoE (Power Over Ethernet). In the interim, more than one CCTV vendor came up with a way to send analogue HD over coaxial cable. Needless to say, every vendor had his own high definition analogue system so they could charge $$ thousands more than the computer industry. When UHD came along, to cope with double the data rate (8 Mpixel) they just dropped the camera frame rate (to 12.5fps) and shoved it down the same analogue coaxial cable. It didn't take long for customers to wake up to the fact that the computer industry was achieving bandwidths of 100MHz down a UTP cable. So when the CCTV industry found themselves using computer HD cameras, they also started using UTP cable. However existing installations continued to use their coaxial cable, which typically allowed double the distance, at least until digital transmission became common D1, 'digital 1', aka 720H When the VGA cameras became common, the CCTV industry used them in analogue mode to produce basic TV signals to which they added a time stamp and recorded using a domestic DVR inside a fancy box. D1 PAL was 576 lines by 720 pixels, whilst D1 NTSC was 480 lines by 720 pixels. It should be noted that, just as the computer industry took to using the monitor diagonal as the 'size' and, when it came to hard disks, calling 1,000 bytes a 'mega byte', so the CCTV industry took to using the 'effective' width or pixel count as a mark of the resolution. Hence D1 would be '720H' or '720 line' resolution, which sounded a lot better than VGA. The camera, of course, is 640 x 480, that is 480 lines of 640 pixels width. The lines are padded to 576 for PAL, the width is converted to analogue and then re-sampled at the DVR recorder to give '720' pixels wide. The Raspberry Pi 'TV out' generates PAL 576 by 720 from the Pi Camera. The quality should be at least equal to the VGA web-cams used by the CCTV industry as 'D1' cameras. The Pi TVout can be fed direct to the CCTV DVR if set to PAL 'progressive' mode, which sets the feed to 25fps. If set to 4:3 aspect, then the full (square) Pi camera frame will be used to create each PAL frame. However, if set to 14:9, or 16:9, a 'slice' is taken out of the centre of the PAL camera and down-converted to the PAL frame. The nujber of lines doesn't change, but more pixels will be used to generate the analogue line. If recorded by a standard DVR, then the letterbox format line will be sampled to create the standard 720 pixels width. During playback, this can be expanded to fill a wide aspect ratio display by stretching each pixel sideways. This is the same trick that is used by DVD's to store letterbox aspect ratio films. CVBS, 'Composite Video Blanking and Sync', aka 960H This is the same as D1 PAL or NTSC but letterbox aspect ratio. So PAL was 576 lines high x 960 pixels wide, NTSC 480 x 960. The DVR supposedly samples each line 960 times, however there was nothing to stop them using the same trick as the DVD industry. Take the 720 samples and simply stretch them sideways during playback to get the letterbox, 16:9, wide-screen AHD, Analog High Definition, 720P, 1080P, de-facto standard Data from a real HD camera, generating either 720 lines or 1080 lines, would be converted into analogue and sent down the coaxial cable. Since consumer standard HD DVR's are used, most probably this is sampled back into a standard 720P or 1080P video data stream. Audio requires a separate cable. HD-CVI, 720P/1080P (4Mpixel), 4k UHD (8Mpixel), Dahua 720P / 1280H is 1280×720, aka 'half HD', 1920H, full HD 1080P is 1920×1080. Again, converted into analog and transmitted over-coaxial-cable. Audio is supported down the same cable. HD-TVI, High DefinitionTransport Video Interface, 720P1080P (3,4,5Mpixel), 4k UHD (8Mpixel), Hikvision HD 720P digital data stream converted into a 3MHz (or higher) analog signal. HDTVI supports a cable length of up to 500 meters. Audio is supported down the same cable. Digital transmission standards HD-SDI, 1080i, open EX-SDI, 4k UHD, Eyenix (open) Eventually consumers started building their own CCTV systems for home security. They didn't need 300 foot cables and had no intention of paying $$$ thousands for 'specialist DVR's'. Instead they plugged a camera into their computers and recorded this to the computers hard disk. The CCTV industry was not interested in these potential customers and jealously guarded their propitiatory control and retrieval software with exorbitant licence fees and onerous annual 'support' contracts. Every vendor had their own software, with their own unique non-intuitive user interface. Each vied with the other to introduce new features whilst utterly failing to fix the legions of bugs that plagued their offerings. Each attempted to ensure their software was utterly incapable of operating with any other vendors cameras. This was achieved by the use of unique command and control codes to operate the pan, tilt and zoom motors (PTZ) of the cameras. Since the software was embedded within their DVR's, it ran on small embedded computers with no operating system or with some flavour of the Open Source Linux. So the open source community slimly wrote their own. The best Open Source CCTV system is iSpy. This can be run on Windows Mac and Linux computers. It remains free for home users, but has moved to an advertising supported model with features such as remote upload becoming changeable on a monthly subscription model. The common CCTV camera PTZ control codes have been reverse engineered allowing most 'professional' CCTV cameras to be added to a home system. Most that become available are going to be of low resolution, however those with a zoom capability can be used to get decent images if aimed at your front gate and left 'zoomed up'. Soon consumers were demanding cameras that could be plugged straight into their home Ethernets, then it was WiFi and finally cameras that could communicate across the Internet. Commercial CCTV vendors continue to operate, however they can no longer charge $$ thousands for the hardware. Some operate on a subscription model, charging for storing CCTV video feeds 'in the cloud' but the real money has moved into services and software. Services such as providing shed loads of people to view the CCTV feeds in real time, looking for a crime being committed, and advanced software including real-time number-plate and face recognition. WARNING Almost all CCTV systems that support 'remote access' (including iSpy) do so by storing 'your' video feed on 'their' servers. In fact, the data is stored 'in the cloud'. Few companies host their own cloud storage. Most use one of the world's biggest server farms. Many are sited outside the control of western governments who impose pesky privacy laws. In other words, China. In 2017 the Alibaba Cloud reached position 3 in the world. Whilst most of Google's data is in the USA, they have storage farms world wide, including places like Chile, India and Taiwan. Since it's a long established rule that local laws apply to these farms, your data can be copied over and 'mined' in ways that might not be allowed elsewhere. The advantage is that the home user does not need to run Internet Server software on their own computer. Many ISP's prohibit home users from doing so - it soaks up bandwidth - and to 'discourage' this they randomly change your IP address. Another advantage is that you don't have to find space on your own local hard disk for lots of, or even any, CCTV recording files. The disadvantage is obvious. You only get access to your camera feeds when their servers are running - and then only when they allow it (so stop paying the subscription and bye bye access). But more important your footage is under their control. So they decide how long to keep it and it's up to them to keep it secure. They can also decide what to use it for. Whilst you might not mind that they are using it to 'train' their AI software to look for people (rather than cats) you might be less happy that they are also using it train the AI for face recognition - and that means auto-identification of everyone who visits or enters your home, including you and your family. After a while, the AI will learn to identify you and common visitors, such as the postman, and highlight the unexpected or new. They will then try to sell you the AI software which has been developed using your video feeds that you paid them to store. In short, it's the same trick that Amazon and Google is pulling. They monitor your shopping and browsing habits and then shove adverts into your face in an effort to sell you more stuff. Most think this is an acceptable trade off in exchange for cheap goods and free searching. However I expect few feel it's appropriate for ghe companies you paid to look after your data to then mine it to find or create new stuff to sell to you. Few, if any, read the agreement that comes with their storage contract, however if you look hard enough you will find a clause in most that says, if effect, that they can do what they like with your data. IP, as in IP Cameras. The CCTV industry calls this 'Onvif' to justify their excessive charges for 'compatible' cameras IP cameras feed industry standard digital video, typically h264, using standard TCP/IP Ethernet protocols over UPT cable or WiFi. Many include embedded computers allowing motion detection and direct feed to the internet. The only components that might be non-standard are the camera controls. This includes things like contrast, day / night mode control, pan, tilt and zoom (PTZ) protocol and how the motion detection or other triggering operates. Most CCTV vendors camera protocols have been reverse engineered which means any number of third party cameras can be found for a fraction of the price of the CCTV vendors own offerings. Interestingly, there are any number of CCTV DVR systems aimed at the consumer. Most are manufactured and sold by companies based in China, where they can avoid any problems with Lawyers from the traditional CCTV companies waving Patents and arguing about Intellectual Property Rights. Yet instead of embedding iSpy or some other Open Source software, they all insist on using their propitiatory systems in their own doomed efforts to lock the customer into 'their' product. If you are going to use IP cameras, I suggest you avoid the dedicated bug ridden 'CCTV' recorders and, for much the same price, buy an old second hand computer, load it with Linux and use that together with iSpy. Most of us will have an old WiFi Router left over from some previous Internet provider contract. This can be used to set up a private wired and WiFi LAN that is NOT connected to the Internet. You can then put your whole CCTV system onto it's own dedicated LAN where it will not impact your normal browsing or 'smart' TV viewing. This will ensure your CCTV system stays under your control and not under control of some hacker group in Beijing If your normal Internet LAN is 192.168.0.xx, then I suggest setting the CCTV LAN to something like 172.16.0.xx (or 10.168.0.xx) so it's easier to remember. If you are going to use WiFi enabled IP cameras, then don't forget to set the CCTV Router WiFi 'group' to something that isn't going to clash with your main Internet LAN. Most routers default to 6, so set the CCTV router to 1 or 12. You will also need an SSID that's easy to remember and a password thats not easy to crack To avoid any 'cross contamination', you need to lock down the WiFi of both your Internet router and CCTV router. This means as soon as the cameras have been set up, go into the Rouiter Admin pages and set 'MAC Address Lock', fix the IP address for each camera in turn and shrink or disable the DHCP address range. NB. One thing that IP cameras allowed to connect to the Internet have in common is 'automatic updating'. This is the process whereby the vendor is able to send new firmware to the device by some means that you invariably can't prevent and will almost certainly result in it no longer working.

[top]

Turning the Pi into an IP camera

Using 'motion' On the Pi run 'motion' to serve frames from the PiCamera to your LAN on URL http:\\:port (default port is 8080 however that's usually used to serve web pages :-) ) With iSpy 5.7.4.0 (Win 64 bit), frames can be 'imported' into iSpy by adding the Pi as a 'mjpeg URL' camera on address http://:port/ (= note Windows '/')

Add, "Other Video Source"
then select "MJPEG URL" tab
on MJPEG URL page, in the field enter the URL to your motion video stream
(for example: http:mypi.ddns.net:8581 )

Using mjpeg-streamer Motion on the Pi Zero is abysmally slow. The best bet is mjpeg-streamer, which which only uses some 2-3% of the Pi's cpu and can stream 25fps low resolution (640x320) with very low latency. NB. mjpeg-streamer works for both a usb webcam and the raspberry pi cam. At the time of writing, however, you have to compile mjpeg-streamer yourself. From this guide :- lsusb sudo apt-get update sudo apt-get upgrade sudo apt-get install libjpeg8-dev imagemagick libv4l-dev sudo ln -s /usr/include/linux/videodev2.h /usr/include/linux/videodev.h sudo apt-get install subversion cd ~ svn co https://svn.code.sf.net/p/mjpg-streamer/code/mjpg-streamer/ mjpg-streamer cd mjpg-streamer make mjpg_streamer input_file.so input_uvc.so output_http.so sudo cp mjpg_streamer /usr/local/bin sudo cp output_http.so input_file.so input_uvc.so /usr/local/lib/ sudo cp -R www /usr/local/www sudo vi ~/.bashrc export LD_LIBRARY_PATH=/usr/local/lib/ source ~/.bashrc /usr/local/bin/mjpg_streamer -i "input_file.so -f /tmp/stream -n pic.jpg"-o "output_http.so -w /usr/local/www" sudo vi /usr/sbin/livestream.sh #!/bin/bash /usr/local/bin/mjpg_streamer -i "/usr/local/lib/input_uvc.so" -o "/usr/local/lib/output_http.so -w /var/www/stream -c username:password" sudo chmod 755 /etc/init.d/livestream.sh sudo update-rc.d livestream.sh defaults

Next page :- Using Pi RCA AV output - (TVout)

[top]