human detection[1]

8/6/2019 Human Detection[1]

1/33

Human DetectionHuman Detection

Mikel RodriguezMikel Rodriguez


2/33

OrganizationOrganization

1.1. Moving Target IndicatorMoving Target Indicator

(MTI)(MTI) Background modelsBackground models

Moving region detectionMoving region detection

Target chip generationTarget chip generation

ResultsResults

InputFrame

ObjectDetectio

n

TargetChips

WaveletFeatures

SVMClassifier

MTI Classification

2. Target Classification

(Human Detection) Target features

Support vector machines

Results


3/33

Moving Target IndicatorMoving Target Indicator

Moving target indicator (MTI) identifies movingMoving target indicator (MTI) identifies moving

objects which can be potential targetsobjects which can be potential targets


4/33

MTI MotivationMTI Motivation

Becoming increasingly important in militaryBecoming increasingly important in military

and civilian applicationsand civilian applications

To minimize human involvementTo minimize human involvement

ExpensiveExpensive

Short attention spansShort attention spans

Computerized monitoring systemComputerized monitoring system

Real-time capabilityReal-time capability 24/724/7


5/33

MTI ChallengesMTI Challenges

Different sensor modalitiesDifferent sensor modalities

LADAR, IR, EOLADAR, IR, EO

Targets with different dynamicsTargets with different dynamics

Small targetsSmall targets

Weather conditionsWeather conditions

Illumination changes, shadowsIllumination changes, shadows


6/33

Input Video

MTIMTI

Background

Modeling

Intensity

models

Gradient

models

Moving Target

Detection

Background

Subtraction

dynamic update

Targets

Chips Position


7/33

Hierarchical Approach to BackgroundHierarchical Approach to Background

ModelingModeling

Pixel levelPixel level

Region levelRegion level

Frame levelFrame level


8/33

Pixel LevelPixel LevelBackground FeaturesBackground Features

Intensity, heat indexIntensity, heat index

GradientGradient 2D: magnitude, orientation2D: magnitude, orientation

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

160

20 40 60 80 100 120 140 160 180 200

20

40

60

80

100

120

140

160

IREO

Magnitude Orientation


9/33

Pixel LevelPixel Level

Background FeaturesBackground Features

Intensity, heat indexIntensity, heat index Per-pixel mixture ofPer-pixel mixture of

Gaussians.Gaussians.

Gradient based subtractionGradient based subtraction Gradient feature vectorGradient feature vector

=[=[mm,, dd

dd]]

22 yxm ff +=

)(tan 1

x

y

df

f

=


10/33

Pixel LevelPixel LevelMoving Region DetectionMoving Region Detection

Mark pixels that are different from the background intensityMark pixels that are different from the background intensity

modelmodel

Mark pixels that are different from the background gradientMark pixels that are different from the background gradient

modelmodel


11/33

Color basedImage Gradient

Region LevelRegion LevelFusion of Intensity & Gradient ResultsFusion of Intensity & Gradient Results

For each color based region, presence ofedge difference

pixels at the boundaries is checked.

Regions with small number of edge difference pixel are removed,

color model is updated.

Final


12/33

Frame LevelFrame LevelModel UpdateModel Update

Performs a high level analysis of the scenePerforms a high level analysis of the scene

componentscomponents

If more > 50% of the

intensity basedbackground subtractedimage becomes

foreground.

Frame level processingissues an alert

Intensity basedsubtraction resultsare ignored


13/33

Structure of the MTI ClassStructure of the MTI Class

MTI

Background

ConnectedComponents()BoundaryEdges()

SetNumGaussians()

SetAlpha()

SetRhoMean()

SetWeightThresh()

SetActiveRegion()

GetNumGaussians()

GetAlpha()GetRhoMean()

GetWeightThresh()

GetActiveRegion()

Object

SetBoundingBox()

SetRhoLocation()

SetCentroid()

GetBoundingBox()

GetRhoLocation()

GetCentroid()

IsFalseDetection()

Chips

Centroid()ObjectArea()

Height()

Width()


14/33

ResultsResults


15/33

Target ClassificationTarget Classification

Classification of objects into two classes:Classification of objects into two classes: humanshumans

andand othersothers, from target chips generated by MTI, from target chips generated by MTI


16/33

ChallengesChallenges

Small sizeSmall size

Obscured targetsObscured targets

Background clutterBackground clutter

Weather conditionsWeather conditions


17/33

Classifier FlowClassifier Flow

Feature

Extraction

Wavelet

Testing

MTI Chips

Nega

tiv

e

Pos

itiv

eTraining

SVM

Support

Vectors

DecisionDecision


18/33

Wavelet Based Target FeaturesWavelet Based Target Features

Blurred

Vertical

Horizontal

Diagonal


19/33

Feature ExtractionFeature Extraction

Apply 2D Wavelet TransformApply 2D Wavelet Transform

Daubechies waveletsDaubechies wavelets

Apply Inverse 2D Wavelet Transform to eachApply Inverse 2D Wavelet Transform to each

of the coefficient matrices individuallyof the coefficient matrices individually

Rescale and vectorize output matricesRescale and vectorize output matrices


20/33

Why Wavelets?Why Wavelets?

Separability among samplesSeparability among samples Humans can be separated fromcan be separated from cars and background

Correlation using gray levels Correlation using gradient mag.


21/33

Why Wavelets?Why Wavelets?

Person 11 - DB3 Wavelet Correlation


22/33

Support Vector Machines (SVM)Support Vector Machines (SVM)

Classification of data into two classesClassification of data into two classes

NNdimensional data.dimensional data.

Linearly separableLinearly separable

If not transform data into a higher dimensionalIf not transform data into a higher dimensionalspacespace

Find separatingFind separatingNNdimensional hyperplanedimensional hyperplane


23/33

SVMSVMLinear ClassifierLinear Classifier

0+bTxw

0=+bTxwhyperplane equation

Ndimensional data point xi

w

xw br

T+

=

Sample distance to hyperplane


24/33

SVMSVMBest Hyperplane?Best Hyperplane?

Infinite number of hyperplanes.Infinite number of hyperplanes. MinimizeMinimize rr

iiover sample setover sample set xx

ii

Maximize marginMaximize margin around hyperplanearound hyperplane

Samples inside the margin are the support vectorsSamples inside the margin are the support vectors

0

0

0

22

11

=+

=+

=+

b

b

b

T

T

T

xw

xw

xw

w

2=


25/33

SVMSVMTraining SetTraining Set

LetLet =1=1,A training set is a set,A training set is a setof tuples:of tuples: {({(xx11,,yy11),(),(xx22,,yy22),(),(xxmm,,yymm)})}..

For support vectors inequalityFor support vectors inequality

becomes equalitybecomes equality

Unknowns areUnknowns are ww andand bb

1if11if1=+

=+

jj

Tii

T

ybyb

xw

xw


26/33

SVMSVMLinear SeparabilityLinear Separability

Linear programming,Linear programming,

Separator line in 2DSeparator line in 2D ww11xxii,1,1+w+w22xxii ,2,2++bb=0=0..

FindFind ww11,,ww22,, bb such thatsuch that is maximizedis maximized

FindFind ww11,,ww22,, bb such thatsuch that ((ww)=)=wwTTww is minimizedis minimized

( ) 1+by iT

i xw


27/33

SVMSVMSolutionSolution

Has the following form:Has the following form:

Non-zeroNon-zero iiindicatesindicates xx

iiis support vectoris support vector

Classifying function is:Classifying function is:

== iT

iiiii yby xwxw and

+= byfT

iii xxx )(


28/33

Classification ClassClassification ClassHuman Classification

TrainingFunction

ReadPositiveImages()

ReadNegativeImages()

AssemblePositive()AssembleNegative()

AssembleMatrices()

TestingFunction

LoadSVM()

ReadImages()

ExtractFeatures

ConvertToGray()ApplyWaveletFilter()

ApplyInverseTrans()

ResizeInverse()

VectorizeInverse()

Concatenate()TrainSVM

LIBSVM

TestSVM

LIBSVM


29/33

Classification Baseline AnalysisClassification Baseline Analysis

Run time for 3.0GHz dualcore, 2GB RAMRun time for 3.0GHz dualcore, 2GB RAM

Training:Training: 276 training samples 8.015 seconds276 training samples 8.015 seconds

Testing:Testing: 24.087 chips (25 by 25) per second24.087 chips (25 by 25) per second

Classifier sizeClassifier size Depends on diversity of imagesDepends on diversity of images

For 276 training samples of 25x25, classifierFor 276 training samples of 25x25, classifier

size is 1.101 MBsize is 1.101 MB


30/33

Classification Baseline AnalysisClassification Baseline Analysis

Memory requirementsMemory requirements

Requires entire set of support vectorsRequires entire set of support vectors

Current classifierCurrent classifier

( ) ( )( ) ( )pixelsof#bits32tscoefficienwavelet4SVof# =M

( ) MbbitsbitM 3.22400000252512830 ===


31/33

ExperimentsExperiments

Vivid DatasetVivid Dataset UCF DatasetUCF Dataset


32/33

ResultsResults

Training setTraining set

300 target chips300 target chips

TestingTesting

3872 human chips3872 human chips 5605 vehicle and background chips5605 vehicle and background chips

PerformancePerformance

2.4% false positive2.4% false positive (others classified as pedestrians)(others classified as pedestrians) 3.2% false negative3.2% false negative (pedestrian classified as others)(pedestrian classified as others)


33/33

Future directionsFuture directions

MTIMTI Detection by partsDetection by parts Motion clusteringMotion clustering

ClassificationClassification Various kernels for SVMVarious kernels for SVM Better target featuresBetter target features

Motion, steerable pyramids, shape featuresMotion, steerable pyramids, shape features

(height, width)(height, width) Local wavelet coefficientsLocal wavelet coefficients

AdaboostAdaboost

human detection[1]

Documents