spatiotemporal saliency detection and its applications in static and dynamic scenes
DESCRIPTION
Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes. IEEE TCSVT 2011 Wonjun Kim Chanho Jung Changick Kim. Outline. Introduction Proposed Method Experiment Result Application Conclusion. Introduction. Problem occurs when background is highly textured. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/1.jpg)
Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes
IEEE TCSVT 2011Wonjun Kim
Chanho JungChangick Kim
![Page 2: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/2.jpg)
OutlineIntroductionProposed MethodExperiment ResultApplicationConclusion
![Page 3: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/3.jpg)
IntroductionProblem occurs when background is highly textured
![Page 4: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/4.jpg)
Proposed Method
Feature RepresentationEdge orientation histogram (EOH)Color orientation histogram (COH)Temporal Feature
Self-ordinal MeasureSaliency MapScale-invariant Saliency Map
![Page 5: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/5.jpg)
Edge Orientation Histogram (EOH)
1. Compute the edge orientation of every pixel in the local region center at the pixel
2. Quantized into K angle in the range of [,]3. Compute the histogram of edge orientation
m(x,y,n):edge magnitude(x,y,n):quantized orientation
𝑖
local region
![Page 6: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/6.jpg)
Color Orientation Histogram (COH)
1. Quantize the angle in HSV color space in the range of [,] into H angles
2. Compute the histogram of color orientation
s(x,y,n):saturation value(x,y,n):quantized hue value
![Page 7: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/7.jpg)
Temporal FeatureCompute the intensity differences between frames
Feature at the pixel of frame
P :total number of pixels in local regionj :index of those pixels in P :user-defined latency
![Page 8: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/8.jpg)
Self-ordinal MeasureDefine a 1(K+1) rank matrix by ordering the
elements of EOH(COH) ex:
![Page 9: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/9.jpg)
Self-ordinal Measure
![Page 10: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/10.jpg)
Saliency Map of Edge and ColorCompute the distance from the rank matrix of
center region to surrounding regions
Saliency Map of Edge Saliency Map of Color
N :total number of local regions in a center-surround window
, :maximum distance between two rank matrices
![Page 11: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/11.jpg)
Spatial Saliency MapCombine the edge and color saliency
![Page 12: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/12.jpg)
Combining with Temporal SaliencyCompute the SAD of temporal gradients between
center and the surrounding regions
Combine the spatial and temporal saliency
![Page 13: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/13.jpg)
Scale-invariant Saliency MapCombine 3 different scales of saliency Map
(3232, 6464, 128128)
3232 1281286464
![Page 14: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/14.jpg)
Algorithm
![Page 15: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/15.jpg)
Experiment Result
Static ImagesVideo Sequences
![Page 16: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/16.jpg)
Experiment ResultStatic Image
Local region = 55center-surround window = 77K = 8, H= 6 = 40, = 24
Video Sequence = 49Speed: 23ms per frame (43 fps)
![Page 17: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/17.jpg)
Static Images
![Page 18: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/18.jpg)
Static Images
![Page 19: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/19.jpg)
Video Sequences
![Page 20: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/20.jpg)
Video Sequences
![Page 21: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/21.jpg)
Application
Image RetargetingMoving Object Extraction
![Page 22: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/22.jpg)
Image Retargeting
![Page 23: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/23.jpg)
Image Retargeting
![Page 24: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/24.jpg)
Moving Object DetectionG:the set of salient pixels in the ground truth imageP:salient pixels in the binarized object mapCard(A):the size of the set A
![Page 25: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/25.jpg)
Moving Object Detection
![Page 26: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/26.jpg)
![Page 27: Spatiotemporal Saliency Detection and Its Applications in Static and Dynamic Scenes](https://reader036.vdocuments.us/reader036/viewer/2022062323/56816187550346895dd121a0/html5/thumbnails/27.jpg)
ConclusionOrdinal signature can tolerate more local feature
distribution than sample values.The proposed scheme performs in real-time and
can be extended in both static and dynamic scenes.