announcements class web site – handouts –class info –lab access/accounts –survey readings

Announcements• Class web site

– http://www.cs.washington.edu/homes/seitz/course/590SS/v4g.htm

• Handouts– class info

– lab access/accounts

– survey

• Readings for Monday (via web site)– Paul Heckbert, Survey of Texture Mapping, IEEE Computer Graphics and

Applications, 6(11), November 1986, 56-67.

– Beier, T. and Neely, S., Feature-Based Image Metamorphosis, ACM Computer Graphics (SIGGRAPH'92), 26(2), July 1992, 35-42

CSE 590 “Vision for Graphics”

Today

Intro• Admin• Survey• Introductions• Course overview

2D image processing• Blending• Filtering• Pyramids

on Monday (1/8)• image warping, morphing• image enhancement

Vision for Graphics—Why?

Vision and Graphics are inverse problems

Computervision

World model

Computergraphics

World model

Intersection of Vision and Graphics

modeling- shape- light- motion- optics- images IP

animation

rendering

user-interfaces

surface design

Computer Graphics

shape estimation

motion estimation

recognition

2D modeling

modeling- shape- light- motion- optics- images IP

Computer Vision

Cross Fertilization

Vision impacts graphics• image-based rendering

• model acquisition

• motion capture

• perceptual user interfaces

• special effects

• image editing

Graphics impacts vision• reflectance

• transparency

• shape modeling

Course Objectives

What to expect• Knowledge of vision that is relevant to graphics

• How to apply your expertise in image analysis to synthesis

• Fundamentals

• Explore new avenues for research

What not to expect• Not a graphics course

• Not a complete vision course

Administrative StuffWeb Site

• http://www.cs.washington.edu/homes/seitz/course/590SS/v4g.htm

Grading• 2 programming projects• 1 final research project• Class presentation• Class participation

Software and Hardware• Programming projects in C/C++• Support code for Windows and Linux• Lab: 228 Sieg Hall (Win2K PC’s)

– Fill out forms to get key access, CSE class account– You’re welcome to use your own machines instead

• Digital still and video cameras, tripods, etc.

Prerequisites

Prior course on vision OR graphics

Assume• Familiarity with image representations• Basic image processing (linear filtering, transforms, etc.)• Differential equations, linear algebra• Camera modeling and projection• Ability to read research articles, fill in gaps

Questions? See Steve or Rick

Image Processing

Elder, J. H. and R. M. Goldberg. "Image Editing in the Contour Domain," Proc. IEEE: Computer Vision and Pattern Recognition, pp. 374-381, June, 1998.

http://www.fearthis.com/warpimages/pres.shtml

Motion Estimation

Interview with a Vampire, Courtesy Doug Roble, Digital Domain

mosaic demo

Pose Estimation

Ascending Stairs,Eadweard Muybridge, 1884-85

3D Shape Reconstruction

Debevec, Taylor, and Malik, SIGGRAPH 1996

Image-Based Rendering

View Morphing, Seitz and Dyer, SIGGRAPH 96

Modeling light

"Interface", courtesy of Lance Williams, 1985

Environment Matting and Compositing, Zongker, Werner, Curless, and Salesin. SIGGRAPH 99

Image Blending

Feathering

01

01

+

=

Encoding transparency

I(x,y) = (R, G, B, )

Iblend = Ileft + Iright

See Blinn reading (CGA, 1994) for details

Affect of Window Size

0

1 left

right0

1

Affect of Window Size

0

1

0

1

Good Window Size

0

1

“Optimal” Window: smooth but not ghosted

What is the Optimal Window?

To avoid seams• window = size of largest prominent feature

To avoid ghosting• window <= 2*size of smallest prominent feature

Natural to cast this in the Fourier domain• largest frequency <= 2*size of smallest frequency• image frequency content should occupy one “octave” (power of two)

FFT

What if the Frequency Spread is Wide

Idea (Burt and Adelson)• Compute Fleft = FFT(Ileft), Fright = FFT(Iright)

• Decompose Fourier image into octaves (bands)– Fleft = Fleft

1 + Fleft2 + …

• Feather corresponding octaves Flefti with Fright

i

– Can compute inverse FFT and feather in spatial domain

• Sum feathered octave images in frequency domain

Better implemented in spatial domain

FFT

Octaves in the Spatial Domain

Bandpass Images

Lowpass Images

Image Pyramids

Pyramid Creation

“Laplacian” Pyramid• Created from Gaussian

pyramid by subtractionLl = Gl – expand(Gl+1)

filter mask

“Gaussian” Pyramid

PyramidsAdvantages of pyramids

• Faster than Fourier transform• Avoids “ringing” artifacts

Many applications• small images faster to process• good for multiresolution processing• compression• progressive transmission

Known as “mip-maps” in graphics communityPrecursor to wavelets

• Wavelets also have these advantages

Pyramid Blending

laplacianlevel

4

laplacianlevel

2

laplacianlevel

0

left pyramid right pyramid blended pyramid

Blending Regions

Other applications• Removing block artifacts in compressed images

Limitations?

Related Topics

Matting• Given image and background(s), estimate foreground• What if foreground object is refractive?

– Environment matting

Hole filling• Remove scratches, holes in an image

Texture synthesis

Environment Matting and Compositing, Zongker, Werner, Curless, and Salesin. SIGGRAPH 99