computer vision - cs.ucf.edu
TRANSCRIPT
![Page 1: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/1.jpg)
1
CAP5415 Computer Vision• Instructor: Dr. Mubarak Shah, [email protected], office:238 CSB,
http://www.cs.ucf.edu/class/cap6411• Office Hours:
– 2PM to 3PM Mon, 4PM-5PM Tu, 3PM-4PM Thurs
• Grading– Mid term 20%, Final 30%, homework 10%, programs 30%, term
paper 10%
• Class notes– Fundamental of Computer Vision, Mubarak Shah, available on the
webpage
• Text Book– Introductory Techniques for 3D computer vision, E. Trucco and A.
Verri, Prentice Hall
• Other suggested Books– Machine Vision, R. Jain et al, Mc Graw Hill.
Computer Vision
• Image Analysis• Image Understanding• Video Analysis• Video Understanding
![Page 2: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/2.jpg)
2
Image Formation
• Light Source• Camera• Surface reflectance• Surface shape
Perspective Projection
(X,Y,Z)
LensImage Plane
image Zy
f
ZfX
xZfY
y
Zf
Yy
−=−=
=−
Worldpoint
![Page 3: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/3.jpg)
3
Orthographic Projection
(X,Y,Z)Image Plane
image
y
Worldpoint
Xx
Yy
==
Image• 2-D array of numbers (intensity values, gray
levels)• Gray levels 0 (black) to 255 (white)• Color image is 3 2-D arrays of numbers
– Red– Green– Blue
• Resolution (number of rows and columns)– 128X128– 256X256– 512X512– 640X480
![Page 4: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/4.jpg)
4
Video
• Sequence of frames• 30 frames per second
Digitization
• TV camera is analog, need – A to D converter– Frame grabber
• Digital Cameras do not need digitization – JVC (MPEG through fire wire)– Sony (MPEG through fire wire)
![Page 5: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/5.jpg)
5
Image Formats
• TIF• PGM• PBM• GIF• JPEG• MPEG• Quick Time
Digital TV
• Networks started broadcasting limited DTV programs in Nov 98.
• All commercial stations are supposed to switch to DTV by 2002
• All stations are supposed to switch to DTV by 2003
• Govt wants broadcasters’ NTSC channels returned by 2006 for auctioning!
![Page 6: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/6.jpg)
6
Digital TV
• CBS carried few NFL games last year• CBS and ABC plans
– evening news– movies– rest of the day up-convert standard TV
• NBC– no broadcast yet– plans for “Tonight Show” this fall!
Digital TV
• CBS and NBC use 1080i (1920X1080), which is 995Mb/s at 30 fps
• ABC and Fox use 720p (1280X720), which is 424Mb/s at 30 fps
• 6 MHz channel assigned to each network can carry 19.4Mb/s
• Need 50:1 compression ratio!
![Page 7: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/7.jpg)
7
Computer Vision
• Shape from X (Recover 3-D shape from 2-D image(s))– Stereo– Motion– Shading– Texture– Contours
![Page 8: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/8.jpg)
8
Stereo
http://www.vision3d.com/stereo.html
![Page 9: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/9.jpg)
9
Renault Stereo Pair
Depth Map
![Page 10: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/10.jpg)
10
Stereo Pair
Candy
![Page 11: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/11.jpg)
11
Dinosaur
Shark
![Page 12: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/12.jpg)
12
Shape from Shading
Lambertian Model
S=L, light source
),,).(,,(.),( zyxzyx lllnnnLnyxf ==),,)).(1,,(
1
1(.),(
22 zyx lllqpqp
Lnyxf −−++
==
![Page 13: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/13.jpg)
13
Sphere
( )
),,(1
),,(
222
zyxR
nnn
zy
yz
q
zx
xz
p
yxRz
zyx =
−=∂∂=
−=∂∂=
−−=
Sphere
![Page 14: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/14.jpg)
14
Vase
(1, 0, 1) (-1, 1, 1) (-1,-1, 1)
Visual Motion
![Page 15: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/15.jpg)
15
Image from Hamburg Taxi seq
optical flow
![Page 16: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/16.jpg)
16
Video Mosaic
Video Mosaic
![Page 17: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/17.jpg)
17
Video Mosaic
mosaic
Sprite
![Page 18: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/18.jpg)
18
JPEG
Original 64K 13K 5K
Difference
Model-Based Image Coding
![Page 19: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/19.jpg)
19
Synthesizing Realistic Facial Expressions
![Page 20: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/20.jpg)
20
Compression
Original 400 kbps 200 kbps
FACIAL EXPRESSIONS
RAISE EYE BROWS SMILE
![Page 21: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/21.jpg)
21
FACIAL EXPRESSIONS
DISGUSTANGER
Lipreading
![Page 22: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/22.jpg)
22
Human Behavior Recognition
Key Frames Sequence 1 (350 frames), Part 1
![Page 23: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/23.jpg)
23
Detecting Driver Alertness
Detecting Driver Alertness
..\..\..\d drive\STUDENTS\PAUL\Ehtml.html
..\..\..\d drive\STUDENTS\PAUL\Fhtml.html
![Page 24: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/24.jpg)
24
Eye Tracking
Tamara Miller
Results..\..\..\d drive\STUDENTS\TAMARAM\2latest.html
..\..\..\d drive\STUDENTS\TAMARAM\3latest.html
..\..\..\d drive\STUDENTS\TAMARAM\8latest.html
..\..\..\d drive\STUDENTS\TAMARAM\9latest.html
![Page 25: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/25.jpg)
25
Determining 3D Face Orientation
Alper Yilmaz
Determining Face Orientation
![Page 26: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/26.jpg)
26
Discriminating human and animal motion
Nan Li
Discriminating human and animal motion
..\..\..\d drive\STUDENTS\LINAN\PED\PED-1\RESULT.HTM
..\..\..\d drive\STUDENTS\LINAN\PED\PED-4\RESULT.HTM
..\..\..\d drive\STUDENTS\LINAN\PED\PED-2\RESULT.HTM
![Page 27: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/27.jpg)
27
Discriminating human and animal motion
D:\students\linan\horses\horse-1\resutl.htm
..\..\..\d drive\STUDENTS\LINAN\HORSES\HORSE-2\RESULT.HTM
A method is presented to:
• Remove commercials from interview videos
• Segment interviews into host and guest shots
A clip of Larry King interview
![Page 28: Computer Vision - cs.ucf.edu](https://reader031.vdocuments.us/reader031/viewer/2022013022/61d18483de5e95165d23889d/html5/thumbnails/28.jpg)
28
A Short Connectivity GraphStart
End
One story
Another story
Commercials
Shots detected as ‘Host’
Shots detected as ‘Guest’