Jonathan Dinger
1
Traffic footage example
2
Important step in video analysis Background subtraction is often used
3
Uses pixelwise computations Performance could be better Better segmentation = better traffic
detection
4
Use interaction between neighboring pixels Keep objects segmented together Better segmentations
5
G = (V,E) Two vertices in V
called source and sink ◦ s and t, respectively
Remaining vertices called M
Vertices in M connected to both s and t (T-links)
Vertices in M connected to neighboring vertices (N-links)
6
T-links are uni-directional N-links are bi-directional Each edge has a weight
◦ also known as a capacity Each pixel has its own
vertex
7
Path◦ List of vertices connected by edges
s-t cut◦ Removal of edges such that all vertices have a
path to either the source or the sink, but not both Flow
◦ Each edge has a capacity◦ That much flow can be pushed through each edge◦ Flow through a graph is the cumulative amount of
flow going from the source to the sink
8
Minimum cut/maximum flow◦ A cut with the smallest cost (weight)◦ Maximum flow that can be pushed from source to
sink (capacity)◦ By max-flow min-cut theorem,
these are equal Graph cuts
◦ Minimum cut on a graph
9
Method to find maximum flow Augmenting path
◦ Path where flow can be increased in all edges between vertices in path
Run search1.Find augmenting path from source to sink2.Add more flow to that path3.Loop back to 1.
10
Step 1. Step 2.
Step 3. Step 4.
11
Step 5. Step 6.
Step 7. Step 8.
12
Two separate implementations◦ Our implementation◦ Kolmogorov’s implementation
http://www.cs.ucl.ac.uk/staff/V.Kolmogorov/software/maxflow-v3.01.src.tar.gz
13
Downsampled images Cut performed over smaller area Then upsample, and perform cut over band Faster than graph cuts Less detailed than graph cuts
14
15
Downsampling images loses information Use Laplacian pyramid to store lost data
Computes differencebetween image and imagegained by downsamplingand upsampling again
Add back some of the lost detail◦ Resegment in areas where detail was lost
16
17
BGC faster, less accurate GC slightly slower, more accurate
Graph cut Banded graph cut
18
3 or more labels α-β swap
◦ Loop through label pairs◦ Run graph cut on current pair of labels
If current label of a pixel is not one of the pair, do not use pixel in graph cut
◦ Graph cuts will swap some pixels with label α to label β and vice versa
19
Graph cut on each image Background image computed per pixel
◦ N is the number of images, xi is the grayscale value of the current pixel, and μ is the average grayscale value over all image frames
N
iixN 1
1
20
Find variance of each pixel
N, xi, and μ are as above σ2 is the variance Threshold the variance If variance is below the threshold, do not
include pixel in graph cut◦ Assume non-varying pixels are background pixels◦ Avoid divide-by-zero errors in weights
N
iixN 1
22 )(1
21
Exponentials
x is the grayscale value of the current pixel◦ μ and σ2 are as above
β is a constant that forces the two functions to be equal at α standard deviations
22
Absolute differences
x and μ are as above L is the maximum possible distance
between x and μ, so for grayscale images
K is a shift constant that forces f3 and f4 to be equal at α standard deviations
23
Simple Grayscale Differences
x is the grayscale value of the pixel. Cm and Cn are two grayscale values used as a basis for segmentation
24
Distance
◦ Euclidean distance between two neighboring pixels with coordinates (x1, y1) and (x2, y2)
Smoothing (similarity)
◦ x and y are the grayscale values of two neighboring pixels.
◦ is the maximum possible difference between the pixel values
◦ γ is a modifier that defines the amount of smoothing that takes place
25
Image Graph cut Banded graph cut
Augmented BGC Our graph cut Multi-way cut
26
Computing times for cut results in milliseconds (ms)
27
Image Background Variance
Exponential cut Absolute differencecut
28
ExponentialImage Absolute difference
29
ExponentialImage Absolute difference
30
ExponentialImage Background subtraction
31
ExponentialImage Absolute difference
32
ExponentialImage Absolute difference
33
Absolute differenceImage Background subtraction
34
35
Image Graph cut BackgroundSubtraction
Segmentation performance without smoothing comparable to background subtraction◦ Background subtraction is faster, easier
Smoothing model ◦ Segments larger pieces of vehicles into one
section◦ Vehicle segmentations more “solid”
Absolute difference T-link weights combined with smoothing N-link weights give best results
36
Use multi-way cuts to add shadow segmentation
Extend to RGB
37