computer graphics inf4/msc computer graphics lecture notes #16 image-based modelling, rendering and...
TRANSCRIPT
Computer Graphics Inf4/MSc
Computer Graphics
Lecture Notes #16
Image-Based Modelling, Rendering
and Lighting
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 2
Global Illumination and Image-Based Lighting
• Traditional Computer Graphics involves:– Modelling with matter: geometry with reflectance
properties.
• Image-Based Modelling & Rendering is:– Modelling and rendering with Light, often deriving
geometry and materials in the process.
• Image-Based Lighting allows:– Combination of real and synthetic graphics with
consistent illumination, using images as light sources.
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 3
Pursuit of Photorealism
• For complex models, tools can require lot of effort and rendering is very slow.
• Choice: do usual modelling or just take a photo (that would provide photorealism!)
• IBMR is about using these photos and transcending their limitations (eg. Light).
• Don’t need anything like a complete model, but may need a number of photographs…
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 4
Pursuit of Photorealism
• Need enough photographs to give coverage.
• Major result is - rendering is faster.
• So Image-Based Modelling and Rendering (IBMR) starts with the answer!
• Research inspired by film industry.
• Related to techniques in Computer Vision.
• New so no taxonomy, just techniques.
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 5
IBMR Techniques
• Panoramas – movement within and between panoramic cylinders, eg. Quicktime VR.– Panoramas can be real or synthetic.
• View interpolation –from a few photos, interpolate view from any position (morphing).
• * Lighting – adding real or synthetic objects to a photographed scene, illuminated with scene lights.
• Modelling – extracting models from photos.
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 6
“Traditional graphics”
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 7
Computer Vision
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 8
Vision meets graphics
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 9
However ...
• Vision falls short
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 10
And ...
… so does graphics
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 11
Image-Based Rendering (IBR)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 12
What is an image?
• Collection of radiance values
• radiance along a ray
• 3D position
• 2D direction
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 13
Plenoptic function• Radiance value for all
possible rays = plenoptic function.
• All possible images are a subset of this plenoptic function.
• Too much stuff!• Goal of IBR is generate a
continuous representation of the plenoptic function.
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 14
Plenoptic function
• Radiance is constant along a ray (line)
• 2D position
• 2D direction
Once we know one ‘origin’, we know them all
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 15
What is an image?• Image = rays going through one point
• usually restricted to viewing frustum, but can also be panoramic
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 16
What is an object?• Image = rays going through one point + image
plane
• 2D function (position on image plane)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 17
What is an object?• Outgoing radiance field of an object
• 2D function (position on surface)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 18
What is an object?• All light leaving the object
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 19
What is an object?• All light leaving the object
• 4D function (2D position + 2D direction)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 20
What is an object?• All possible images of an object
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 21
What is an object?• All possible images of an object
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 22
What is an object?• We don’t really need the object
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 23
What is an object?• We don’t really need the object
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 24
Lumigraph / Light Field• Object is only defined by its radiance field
stuff4D function
(Levoy - Cohen et al 96)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 25
Lumigraph - capture
(Stanford - Levoy et al.)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 26
Layered Depth Images
(McMillan)
Problem with one photo is “holes” when view point moved:
- this is why a number of photos are generally required
- fill the holes using intensities of neighbouring pixels
- interpolation …
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 27
Image Based Lighting
• Add models or objects to scenes and allow them to be manipulated in the scenes.
• Modelling with light allows added objects to be illuminated consistent with image existent lighting - photorealistic inclusions.
• We start with the answer by finding the scene illumination.
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 28
Image Based Lighting
• Real Scene
• Goal: place synthetic objects on table
(all pictures P. Debevec 98-99)
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 29
Extracting scene lighting
• Capture illumination using illumination sphere
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 30
Image Based Lighting
Real scene
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 31
Image Based Lightingcaptured illumination field
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 32
Image Based Lighting
Real scene
light based model
local scene
synthetic objects
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 33
Image Based Lighting• Use renderer - compute effects of synthetic
objects on local scene
local scene (brdf estimated)synthetic objects (brdf known)
light based model
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 34
Image Based Lighting
• Render into the scene
background
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 35
Image Based Lighting
• Render synthetic objects
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 36
Image Based Lighting• Effect of local scene on real scene
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 37
Image Based Lighting• Add differences to image
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 38
Reconstruction from images?
• Computer Vision related.• Constructs new views and extracts models.• Epipolar geometry
– expresses relationships between points in different images.
• Difficult to predict full impact in CG.• State of the Art
– 3D reconstruction from uncalibrated images.
Computer Graphics Inf4/MSc
13/11/2007 Lecture Notes #16 39
References
• You won’t find this material in the major graphics texts since it is new and not yet mainstream but I recommend the following:– The Computer Image, Watt & Policarpo, Addison-
Wesley 1998.
– SIGGRAPH courses for the past few years, we have some on CD rom and on-line within EdVEC and not immediately available to Informatics machines – ask me if you have a strong interest and need.