We have addressed the problem of object detection and tracking for video surveillance systems. For this, we have proposed the integration of an Edge TPU based Coral Dev Board within a Hikvision video surveillance system. In this way, video analytics capabilities are strongly enhanced, e.g., perform object detection at 75 FPS using Tensorflow detection API and deep learning MobileNet SSD models. This preliminary evaluation is encouraging future developments that include multiple modules performing crowd density monitoring, facial recognition, people counting, behaviour analysis and detection or vehicle monitoring.

THE PROPOSED SYSTEM

The components of the experimental setup are presented in fig. 3. They are briefly described below along with the justification for including the Google Edge TPU based “Coral” Development Board (and not other embedded Machine Learning (ML) accelerators) in a video surveillance system.

Fig. 3. The System Overview.

A. The Hardware

• Hikvision DS-7608NI-K2-8P, an 8 Channel Network Video Recorder (NVR), with 8 built-in PoE ports, can support eight IP cameras up to 8MP, supporting H.265/H.264/MPEG4 video formats (less storage and bandwidth) having 2 spaces for HDD that can hold a maximum capacity of 20TB [11].
• Four Hikvision DS-2CD2T32-I5 IP Outdoor Bullet Cameras capable of shooting 30 frames per second at full HD (up to 3-megapixel, 2048 x 153 resolution) and providing clear night-time images at up to 50 meters due to EXIR technology. These features are making it suitable for monitoring larger open areas [12].
• D-Link DGS-1100-08, an 8-Port Gigabit PoE Smart Managed Switch providing 8 x 10/100/1000BASE-T ports, PoE budget 64W [13].
• Machine Learning Accelerator Google Edge TPU (Coral Dev Board) having the following most important tech specs: NXP i.MX 8M SoC (quad Cortex-A53, Cortex-M4F) CPU, Integrated GC7000 Lite Graphics GPU, Google Edge TPU coprocessor as and 1 GB LPDDR4 RAM [14].

Although many high-speed ML embedded solutions are recently available on the market [15], e.g. Movidius Neural Compute Stick provided by Intel or Tegra SoC development platform NVIDIA Jetson Nano, the Edge TPU coprocessor presents certain advantages over competitors:

• it is very efficient, been capable of performing 4 trillion operations per second (TOPS) using just 0.5W for each TOPS.
• the other components of the SoC can be used for traditional image and video processing (the GPU) or to talk to other sensors like temperature sensor, ambient light sensor etc. (the Cortex-M4F core).
• built-in support for Wi-Fi and Bluetooth.

Moreover, according to a NVIDIA study [16], for the particular case of object detection, Google Edge TPU Dev Board outperforms the competition (tab. 1).

Tab. 1. A comparison of ML accelerator platforms for the object detection task.

B. The Software

• TensorFlow Lite including Object Detection API. This framework facilitates low latency inference on mobile/embedded devices along with a small binary size. Moreover, just TensorFlow Lite models can be compiled to run on the Edge TPU. The second component, the Object Detection API, enable us to define, train and deploy object detection models.
• Pre-compiled models for image classification, object detection and on-device retraining (last fully-connected layer removed), as depicted in tab. 2.

Tab. 2. Pre-compiled models available for Edge TPU.

• OpenCV 4.1.1 (last currently available version), mainly for reading, writing and displaying video streams.
• Python 3.7, virtualenv and virtualenvwrapper for code design and project management

The experimental part aimed, on one hand, to assess the performance contribution brought by TPU coprocessor over the classical embedded ARM application processors. This comparison has been done using the “Town Centre Dataset” processed using SSD v1 with MobileNet for running the Tensorflow Object Detection API. The Edge TPU appeared to be 30 times faster than the ARM CPU, providing an average inference time of 10ms – see below.

CPU vs TPU benchmark using “Town Centre” dataset.

The second part of the experiment involved the usage of the Coral Dev Board in processing live video stream provided by NVR+IP surveillance cameras. The shown results (fig. 5) are obtained using MobileNet SSD v2 pre-compiled model. It behaved better, in term of detection accuracy, than the MobileNet SSD v1. As we had point out previously, the model was pre-trained using the COCO dataset [19] and is able to detect in real time the location of 90 different objects.

Coral Dev Board integrated in a video surveillance system (just detections over 40% are displayed).