Download - Vision & Graphics Research in UCSD CSE
![Page 1: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/1.jpg)
Vision & Graphics ResearchVision & Graphics Researchin UCSD CSEin UCSD CSE
David KriegmanDavid Kriegman
Computer Science & EngineeringComputer Science & EngineeringUniversity of California, San DiegoUniversity of California, San Diego
![Page 2: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/2.jpg)
The Pixel LabThe Pixel Lab
Serge Belongie Henrik Wann Jensen David Kriegman
Sameer Agarwal
Kristin Branson
Piotr Dollar
Craig DonnerJeff Ho
Wojciech Jarosz
Arash KeshmirianTobias KlugAnders Wang Kristensen
Kuang-chih Lee
Jongwoo Lim
Satya Mallick
Ben Ochoa
Vincent Rabaud
Andrew Rabinovich
John Rapp
Geoffrey RomerSteve Rotenberg
Josh Wills
![Page 3: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/3.jpg)
Computer Vision ResearchComputer Vision Research
Computer Vision Research Threads
Segmentation Recognition
Reconstruction Motion
![Page 4: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/4.jpg)
A failed outlineA failed outlineI. Theoretical Contributions:
A. Object trackingB. Measurement and segmentation of motionC. Unsupervised learning (clustering) of objects from imagesD. Object recognition and categorizationE. Helmholtz reciprocity stereopsisF. Structure from motionG. Illumination and reflectance modelingH. Optical flow through refractive objects
II. Vision meets GraphicsA. Refractive Optical flow for video compositingB. Image-based renderingC. Efficient rendering using environment mapsD. Modeling of rough dielectric surfacesE. Texture synthesis on surfaces
III. Applications of VisionA. Face recognition in images and videoB. Person tracking for video monitoring and ubiquitous vision systemsC. Visual monitoring of animal health and welfare (Smart Vivarium)D. Tissue microarray analysis for cancer detection.E. Protein structure reconstruction from cryo-electron micrographs
![Page 5: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/5.jpg)
Face RecognitionThe challenge caused by lighting
variability
Same Person or
Different People
![Page 6: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/6.jpg)
![Page 7: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/7.jpg)
Same Person
or
Different People
![Page 8: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/8.jpg)
![Page 9: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/9.jpg)
Same Person
or
Different People
![Page 10: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/10.jpg)
![Page 11: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/11.jpg)
N-dimensional Image Space
The Illumination ConeThe Illumination Cone
Theorem:: The set of images of any object in fixed posed, but under all lighting conditions, is a convex cone in the image space. (Belhumeur and Kriegman, IJCV, 98)
Single light source images lie on cone boundary
2-light source image
x1
x2
xn
Illumination Cone
![Page 12: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/12.jpg)
Face Detection: First Online StepFace Detection: First Online Step
![Page 13: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/13.jpg)
3-D Face Modeling for 3-D Face Modeling for RecognitionRecognition
Training Images
[Georghiades, Belhumeur, Kriegman]
3-D GenerativeModel
![Page 14: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/14.jpg)
Face DatabaseFace Database
64 Lighting Conditions 9 Poses => 576 Images per Person Variable lighting
0-12o
12-25o
25-50o
50-77o
![Page 15: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/15.jpg)
Face Recognition Error Face Recognition Error RatesRates
0
10
20
30
40
50
60
70
80
0-25 degrees 25-50 degrees 50-77 degrees
Light Direction
Err
or
Rat
e
Eigenfaces Eigenfaces (w/o 1st 3)
Linear Subspace Illumination Cones
Our Method
![Page 16: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/16.jpg)
What Went Where?What Went Where?
Segmenting images into regions with different motion
![Page 17: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/17.jpg)
What Went Where: Motion SegmentationWhat Went Where: Motion Segmentation
[J. Wills, S. Agarwal, S. Belongie ]
![Page 18: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/18.jpg)
1. Compute point correspondences by comparing filter response vectors
2. Partition these correspondences into groups using RANSAC and estimate the represented motion layers
3. Densely assign pixels to one of the detected motion layers using a Markov Random Field method
Motion SegmentationMotion Segmentation
[J. Wills, S. Agarwal, S. Belongie ]
![Page 19: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/19.jpg)
Helmholtz StereoHelmholtz Stereo
Reconstructing shape of surfaces with arbitrary BRDF
![Page 20: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/20.jpg)
Helmholtz Stereo: Arbitrary BRDFHelmholtz Stereo: Arbitrary BRDF
Bi-directional Reflectance Distribution Function
(in, in ; out, out)
n̂(in,in)
(out,out)
LEFT RIGHT
Non-Matte surfaces:
A challenge for conventional stereo
[ Zickler, Kriegman, Belhumeur ]
![Page 21: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/21.jpg)
Helmholtz Reciprocity & StereoHelmholtz Reciprocity & Stereo
n̂
in, in
out, outn̂
in, in
out, out
((inin, , in in ; ; outout, , outout) = ) = ((outout, , out out ; ; inin, , inin))
[Helmholtz, 1910], [Minnaert, 1941], [ Nicodemus et al, 1977]
![Page 22: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/22.jpg)
Using Multiple Helmholtz Stereo PairsUsing Multiple Helmholtz Stereo Pairs
il1wl1T
– ir1wr1T
n̂
wl1
wr1
il2wl2T
– ir2wr2T
…
n̂ = 0
}A
• Multiple views (at least three pairs) yield a matrix constraint equation.
• For correct match, matrix A must be Rank 2.• Surface normal lies in null space of A
[ Zickler, Kriegman, Belhumeur ]
![Page 23: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/23.jpg)
Helmholtz StereoHelmholtz Stereo
[ Zickler, Kriegman, Belhumeur ]
![Page 24: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/24.jpg)
Experimental AparatusExperimental Aparatus
[ Zickler, Kriegman, Belhumeur ]
![Page 25: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/25.jpg)
Subtle Surface Geometry: WrinklesSubtle Surface Geometry: Wrinkles
![Page 26: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/26.jpg)
Helmholtz Stereo: Depth & NormalsHelmholtz Stereo: Depth & Normals
![Page 27: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/27.jpg)
Reconstructed GeometryReconstructed Geometry
[ Zickler, Kriegman, Belhumeur ]
![Page 28: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/28.jpg)
Comparison to other methodsComparison to other methods
Method
Assumed Reflectance
Surface Information Recovered
Recovers Constant Intensity
Active/ Passive
Depth Discontinuities
Handles Half-Occlusion
Robust to Cast Shadows
Photometric Stereopsis
Lambertian or Known
Surface Normals
Surface Normals
Active No NA No
Multinocular Stereopsis
Lambertian Depth Nothing Passive Sometimes Sometimes Yes
Helmholtz Stereopsis
Arbitrary Depth + Surface Normals
Surface Normals
Active Yes Yes Yes
[ Zickler, Kriegman, Belhumeur ]
![Page 29: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/29.jpg)
II. Vision meets GraphicsII. Vision meets Graphics
A. Refractive optical flow for compositing.B. Modeling of rough dielectric surfacesC. Efficient rendering using environment mapsD. Image-based renderingE. Texture synthesis on surfaces
![Page 30: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/30.jpg)
Refractive Optical FlowRefractive Optical Flow
We use motion of the background to recover how light rays are transformed from the background to the foreground as they travel through a refractive medium.
[ Agarawal, Mallick, Belongie, Kriegman ]
![Page 31: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/31.jpg)
The SetupThe Setup
Theory Experiment
[ Agarawal, Mallick, Belongie, Kriegman ]
![Page 32: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/32.jpg)
ResultsResults
Real Recovered
[ Agarawal, Mallick, Belongie, Kriegman ]
![Page 33: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/33.jpg)
Refractive Optical Flow MoviesRefractive Optical Flow Movies
With Water Without Water
[ Zickler, Kriegman, Belhumeur ]
![Page 34: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/34.jpg)
Henrik’s SlidesHenrik’s Slides
![Page 35: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/35.jpg)
Rendering with Environment MapsRendering with Environment Maps
[ Agarwal, Jensen, Belongie ]
![Page 36: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/36.jpg)
Area ImportanceArea Importance
[ Agarwal, Jensen, Belongie ]
![Page 37: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/37.jpg)
Illumination ImportanceIllumination Importance
[ Agarwal, Jensen, Belongie ]
![Page 38: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/38.jpg)
Structured Importance SamplingStructured Importance Sampling
A novel importance metric that combines area and illumination importance.
A new hierarchical stratification algorithm. [Hochbaum Shmoys]
–Fast
–Stable
–Quality guarantees.
[ Agarwal, Jensen, Belongie ]
![Page 39: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/39.jpg)
Lambertian BRDFLambertian BRDF
Area Illumination LightGenStructuredImportance
[ Agarwal, Jensen, Belongie ]
![Page 40: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/40.jpg)
Glossy BRDFGlossy BRDF
1 10 100 300
Illumination importance sampling
Structured importance sampling
[ Agarwal, Jensen, Belongie ]
![Page 41: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/41.jpg)
Texture Synthesis on SurfacesTexture Synthesis on Surfaces
Shape Model
Texture Sample [ Magda, Kriegman ]
![Page 42: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/42.jpg)
Making it fastMaking it fast
Run Time: 1-2 seconds
Bunny Model: 8192 faces
1.3Ghz Athlon
Blending to hide seamsFast methods for selecting
Texture Triangles
Textons
[ Magda, Kriegman ]
![Page 43: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/43.jpg)
![Page 44: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/44.jpg)
![Page 45: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/45.jpg)
![Page 46: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/46.jpg)
![Page 47: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/47.jpg)
![Page 48: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/48.jpg)
Image-based modeling & RenderingImage-based modeling & Rendering
Goal: To render, under arbitrary pose & lighting images of both natural and man-made objects.
Rendered Rendered
ImagesImages
![Page 49: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/49.jpg)
Model objects with many imagesModel objects with many images
![Page 50: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/50.jpg)
Real Objects in Synthetic ScenesReal Objects in Synthetic Scenes
![Page 51: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/51.jpg)
Textures that vary with Textures that vary with viewpointviewpoint
[ Koudelka, Magda, Belhumeur, Kriegman ]
![Page 52: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/52.jpg)
Data AcquisitionData Acquisition
![Page 53: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/53.jpg)
[ Koudelka, Magda, Belhumeur, Kriegman ]
![Page 54: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/54.jpg)
Compositing Real Objects in VideoCompositing Real Objects in Video
Movie with soundMovie with sound
![Page 55: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/55.jpg)
III. Applications of VisionIII. Applications of Vision
A. Person tracking for video monitoring and ubiquitous vision systems
B. Face recognition in images and video
C. Visual monitoring of animal health and welfare (Smart Vivarium)
D. Tissue microarray analysis for cancer detection.
E. Reconstruction of protein structure from cryo-electron micrographs
![Page 56: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/56.jpg)
Visual Tracking with Visual Tracking with Learned Linear SubspacesLearned Linear Subspaces
![Page 57: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/57.jpg)
Visual Tracking with Learned Linear SubspacesVisual Tracking with Learned Linear Subspaces
Main Challenges
1. 3-D Pose Variation
2. Occlusion of the target
3. Illumination variation
4. Camera jitter
5. Expression variation etc.
Approach
1. Initialize tracker with a single window
2. Target state: Affine warp to align with subspace
3. Model variation in target’s appearance as linear subspace
4. Learn linear subspace online while tracking
[ Ho, Lee, Kriegman ]
![Page 58: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/58.jpg)
Representations of ObjectsRepresentations of Objects
N-dimensional Image Space
x1
x2
xn
Linear
Subspace
3-D Representations
Shape
Reflectance
Appearance-Based
Representations
![Page 59: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/59.jpg)
Linear subspace of image space based on recent observed images
Problem Formulation: Given a collection of k Data points, {x1, x2, . . ., xk}. Find a linear subspace L that best approximates the data points.
Solution: Find a subspace L such that
max { dist (L, x1), dist (L, xs), . . . dist (L, xk) } < ε,
For some ε > 0.
Learned Representation of AppearancesLearned Representation of Appearances
![Page 60: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/60.jpg)
State hypothesis is a location, size, orientation of tracking window.
Tracking Algorithm Outline
1. Generate multiple state hypotheses using previous state.
2. Evaluate each hypothesis using linear subspace model .
3. Choose best state hypothesis.
4. Update subspace model (adapt to changes)
Tracking loopTracking loop
[ Ho, Lee, Kriegman ]
![Page 61: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/61.jpg)
Tracking Humans with Shape and AppearanceTracking Humans with Shape and Appearance
Goal : locate, identify and monitor people
Challenges– Lighting variation, full range of pose variation,
articulation
Identity : Jon Doe
Dimension : 5.1 x 3.2 x 9.7Planar position : (5.3, -10.1)Activity : Moving (-1.3, 2.0)……
[ Lim, Kriegman ]
![Page 62: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/62.jpg)
Tracking Humans with Shape & AppearanceTracking Humans with Shape & Appearance
Offline: train shape model from training vidoes
Online: Model the background Detect & track a person using background model,
shape model, appearance model
Online: learn person’s appearanceRaw Image Foreground region Estimated State
… K-meansClustering
[ Lim, Kriegman ]
![Page 63: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/63.jpg)
Tracking Humans with Shape & AppearanceTracking Humans with Shape & Appearance
Initial and learned appearance
Frame 88 117 225 312
[ Lim, Kriegman ]
![Page 64: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/64.jpg)
Face Tracking & Face Tracking & RecognitionRecognition Using Using Probabilistic Appearance ManifoldsProbabilistic Appearance Manifolds
Challenges– Pose variation– Misalignment – Partial occlusion– Illumination
change– Facial expression
[ Lee, Ho, Yang, Kriegman ]
![Page 65: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/65.jpg)
Representing an PersonRepresenting an Person
• Offline: Learn subspaces using clustering• Lean transition probabilities between subspaces
from training videos
Represent all appearance as union of subspaces
N-dimensional Image Space
x1
x2
x3
![Page 66: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/66.jpg)
Online ProcessOnline Process
Tracking Recognition
Cropped Sub-window
Pose Subspace
![Page 67: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/67.jpg)
Tracking ResultTracking Result
Red rectangle represents ground truth
Compare with two-frame based tracker:
![Page 68: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/68.jpg)
Recognition ResultRecognition Result
Testing on 30 videos of 20 people
![Page 69: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/69.jpg)
Smart Vivarium Project GoalSmart Vivarium Project Goal
Side-view video of multiple,
identical mice
Activity recognition statistics
Animal health analysis
![Page 70: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/70.jpg)
The First Challenge …The First Challenge …
To track multiple, indistinguishable mice through severe occlusions:
Incorporate information from before and after the current frame using an acausal reasoning algorithm.
before
after
![Page 71: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/71.jpg)
D IAGNOSIS
1 Field of View
1 Core
TMA
Region of Interest Detection
Spectral Decomposition
e.g., DAB Hematoxylin
densitometry
Tissue MicroarraysTissue Microarrays
[A. Rabinovich, S. Belongie]
![Page 72: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/72.jpg)
Particle Detection in Cryo-EMParticle Detection in Cryo-EM
[ Mallick, Zhu, Kriegman ]
![Page 73: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/73.jpg)
InspirationInspiration
P. Viola and M. Jones. Robust real-time object detection. In ICCV Workshop on Statistical and Computation Theories of Vision, Vancouver Canada, July 2001.
[ Mallick, Zhu, Kriegman ]
![Page 74: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/74.jpg)
Algorithm: AdaBoostAlgorithm: AdaBoost
Training using AdaBoost* Learning Algorithm. Using a cascade to speed up the detection
process.
* Y. Freund and R. Schapire .A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence,14(5),771-780,1999.
[ Mallick, Zhu, Kriegman ]
![Page 75: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/75.jpg)
ResultsResults
[ Mallick, Zhu, Kriegman ]
![Page 76: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/76.jpg)
NSF IGERT: Vision and Learning in NSF IGERT: Vision and Learning in Humans and MachinesHumans and Machines
Garrison W. Cottrell (CSE, PI)Geoff Boynton (Salk, co-PI)Virginia de Sa (CogSci, co-PI)
Senior PersonnelSenior Personnel Tom Albright (Salk) Serge Belongie (CSE) Leslie Carver (Psych,HDP) Sanjoy Dasgupta (CSE) Gedeon Deak (Cog Sci) Charles Elkan (CSE) Ione Fine (Psych) Don MacLeod (Psych)
Javier Movellan (INC) Vilayanur Ramachandran (Psych) Marty Sereno (Cog Sci) Joan Stiles (Cog Sci, HDP) Emo Todorov (Cog Sci) Jochen Triesch (Cog Sci) Terry Sejnowski (Salk)
Karen Dobkins (Psych, co-PI)David Kriegman (CSE, co-PI)
$3.4m over five years to support for 17 grad students per year
![Page 77: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/77.jpg)
FWGrid: NSF Research Infrastructure GrantFWGrid: NSF Research Infrastructure Grant
High Bandwidth to campus
GigE Switch
Compute + DataCluster “Brick”
High BW Wireless Cell
1st Floor
2nd Floor
3rd Floor
4th FloorHead mounted displayw/ video cameras wireless laptop
Handheld w/ camera & wireless
IBM T2213840 x 2200 pixels
Plasmadisplays
Special purpose:Range scanner
Eye tracker
Stereo cameraVideo camera
OmnicamNetwork camera
IEEE 1394
GPU
![Page 78: Vision & Graphics Research in UCSD CSE](https://reader035.vdocuments.us/reader035/viewer/2022062422/5681336a550346895d9a8004/html5/thumbnails/78.jpg)
Domain Independent Vision-Based NavigationDomain Independent Vision-Based Navigation
• Color camera• SICK laser scanner • Fiber optic gyroscope• Pan-tilt head• GPS• Digital compass• Ultrasonic• Contact bumpers
Goal: Robust Vision-based exploration, mapping, and Goal: Robust Vision-based exploration, mapping, and navigation for mobile robots over large environments.navigation for mobile robots over large environments.