intelligent video analysis system based on gpu and...
TRANSCRIPT
Intelligent Video Analysis System
Based on
GPU and Distributed Architecture
Dr. Shiliang PU Hikivision Research Institute
High Resolution VS storage
Complexity VS Accuracy
Mass data VS efficiency
Mid-size City, about 22,000 cameras
316PB/year Precious video content service under complex situation
Challenge in Video Surveillance
Surveillance video content analysis
Object detection
• Human
• Vehicle
• others
Feature
• Human feature
• Vehicle feature
Identification
• Human body
• Face
• Vehicle
Surveillance video content understanding framework
Challenge in Video Surveillance
Traditional algorithm can understand simple or standard scene content
车型
Sun blade closed
Phone calling
White
Safe Belt
Car
Ford Fiesta
皖A??66R
Glass worn
Male
Clothes color
Teenage
Height
……
Traditional algorithm fails in such complex scene content, which is very common in public surveillance.
Revolution By Deep Learning in
Surveillance
Deep Learning in Surveillance
Traditional algorithm Deep learning
Deep Learning in Surveillance
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
overall passenger
channel
indoor public
area
sunny
day
rainny
day
winter summer
Pedestrian detection Recall rate, fppi = 0.1
Traditional Deep learning
Deep Learning in Surveillance
Clothes type Riding Safe belt
not fastened Phone calling
backpack Hat Hanging bag Mask
Deep Learning in Surveillance
70
75
80
85
90
95
100
vehicle color brand model sun blade safe belt phone calling
Vehicle feature accuracy increased by Deep Learning
traditional algorithm deep learning
Deep Learning in Surveillance
Identity?
Deep Learning in Surveillance
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
Rank1 Rank10 Rank20 Rank30 Rank40 Rank50
Face Recognition Rank in 1 million enroll dataset
Traditional Deep Learning
Deep Learning in Surveillance
Vehicle retrieval based on image comparison
Deep Learning in Surveillance
0.0000
0.1000
0.2000
0.3000
0.4000
0.5000
0.6000
0.7000
0.8000
0.9000
1.0000
Rank10 Rank25 Rank50 Rank100
Vehicle Image retrieval
Traditional Deep Learning
These are what we need!
BUT……
Limitation on Deep Learning
High computing performance Objects detection in surveillance video require 2T Flops/sec, which needs support from high-performance computing hardware.
High cost Price of GPU-based server is significant higher than general server.
High energy consumption General server costs around 9000KWh per channel every year.
GPU solution based on distributed
architecture
Tegra
Hikvision-Blade Server Base on GPUs
Advantage of Hikvision Blade Server
System stability meets industry requirement based on low-cost chip, based on distributed-computing architecture.
1
1-1 1-2 1-3
1-1-1 1-1-2 1-1-3
Advantage of Hikvision Blade Server
16,000
14,000 14,000
300 550
8050
0
2,000
4,000
6,000
8,000
10,000
12,000
14,000
16,000
18,000
Blade Tesla M40*2 General Server
Performance/power ratio
Performance(Gflops) Power(W)
High performance Low cost Low power consumption
Advantage of Hikvision Blade Server
Flexibility-for different product forms
Smart Server
Smart IPC
Smart NVR
Sensing
Storage
Application
Intelligent Video compressing standard
Surveillance Video Compressing Standard
General Video Compressing Standard
Background frame
IVA
Bit rate equalization
Smart 264
Intelligent Video compressing standard
24 hours typical surveillance scene contrast rate at a consistent subjective
quality case
outdoor busy free
H.264 1855Kbps 1245Kbps
Smart264 419Kbps 164Kbps
Promotion 4.43 7.57
indoor busy free
H.264 3448Kbps 1715Kbps
Smart264 945Kbps 315Kbps
Promotion 3.65 5.45
Intelligent Video compressing standard H
.264/H
.265
Sm
art
264
100%
H.264-3830kbps
Smart264-683kbps
17.8%
H.2651920kbps
50%
Video structured description
Human:
male female wear glasses
riding backpack handbag
Vehicle:
driver
driver’s sun visor
copilot
copilot’s sun visor
safe belt fastened/not
phone calling
Security Big Data framework
Non-structured data Structured data
Cloud storage
01
High speed data bus
High speed data bus
Distributed file database
Memory computing
Data mining
Fulltext database
Police Traffic
Other market
Big data manager platform
Big data service
Collecting mass data(video, image, alarm, GPS). Extracting structured data from video and images.
Offering high speed service, like data searching, analyzing and statistics.
Cloud analysis
Advantages from Security Big Data
Small size
Million data level
Low speed
Slow feature
extraction Low accuracy
Long time cost.
>10 nins
Issues on traditional system
01 Cloud analysis handles mass-data computing problem
02 Big data architecture handles above billions level data
03 Spark memory computing offers second degree service
04 Deep learning increases computing accuracy
Smart traffic
Police City
management
…… Smart city
Statistic Alarm Analysis
Q O S D ! f
F 8 D 6 A 1 0
F 5 4 F j u
* K 1 Y ^ g
Data inquiry
Security Big Data application
Security Big Data depth application
Case study – Billion-level image search engine
Image search based on image feature extraction and comparison,
based on billion-level vehicle images.
Case study- Face recognition system
Lost elder found in 5 seconds.
Future
Multi-sensor increases data dimensions.
Unsupervised learning in video surveillance
Optimized neural network framework
THE END
HIKVISION Internal