Incident Light FieldThe distribution of light in a static scene with a fixed illumination

Dr. Kurt DebattistaProf. Alan Chalmers

Debmalya Sinha

Courtesy: J. Unger / Linköping University

Synthetic objects on a physical table with real life lighting

Acquisition

Dynamic or StaticLarge Camera Array, Mirror sphere, Plane Mirror, HDR video, SfM (structure from motion)

Physical Scene

Representation

2PP

PDP

2SP

DPP

Rendering

Storage

Virtual Scene

1. Image Based Lighting

2. Light FieldIncident Light Field

We require an Environment Map

A Panoramic image of the environment that envelops the object space we’re about to Render.

1. Image Based Lighting

2. Light FieldIncident Light Field

Courtesy: nVidia

1. Image Based Lighting

2. Light FieldIncident Light Field

We take several pictures from spatially varying points on the camera plane.

Each of the pictures differ a little by angle

Courtesy: Pat Hanrahan / Stanford University

1. Image Based Lighting

2. Light FieldIncident Light Field

1. Image Based Lighting

2. Light FieldIncident Light Field

1. Image Based Lighting

2. Light FieldIncident Light Field

The distribution of light in a static scene with afixed illumination

We take separate environment map of eachlittle segments of the space to record theincident light rays on the entire object spacewe are about to render.

Courtesy: J. Unger / Linköping University

Acquisition

Dynamic or StaticLarge Camera Array, Mirror sphere, Plane Mirror, HDR video, SfM (structure from motion)

Physical Scene

Representation

2PP

PDP

2SP

DPP

Rendering

Storage

Virtual Scene

Representation

Represented by 7 dimension Plenoptic function on a point P

θ,φ -> direction of light ray from P, λ -> wavelength of incident light, t -> timeVx,Vy,Vz -> Cartesian coordinate of P.

P = P(θ,φ,λ,t,Vx,Vy,Vz)

Which can be reduced to a 4D function

P = P(θ,φ,u,v)

Assuming, the incident light only comes from the upper hemisphere of the surface [Vx,Vy] ([u,v]).

0 ≤ φ ≤ π/2 and -π ≤ θ ≤ π

Plenoptic parameterization

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene

Representation

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene

2PP

PDP

2SP

DPP

Line Space parameterization

Two Points on Two Planes

A Point and a Direction

Two Points on a Sphere

A Direction and a Point

2PP

2SP

PDP

DPP

Courtesy: Camahort et al.

Acquisition

Dynamic or StaticLarge Camera Array, Mirror sphere, Plane Mirror, HDR video, SfM (structure from motion)

Physical Scene

Representation

2PP

PDP

2SP

DPP

Rendering

Storage

Virtual Scene

Acquisition

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene Light Probes

A Light Probe is an omni-directional (360° panoramic) high-dynamic range image. (Debevec et al.)

• Typically a highly reflective silvered ball is used.• An HDR Image is taken of the ball.

Problems:• The image has the reflection of the camera itself.• Has to take two images from different direction to get rid

of the reflection of camera.• The edges are stretched badly and needs efficient warping.

Acquisition

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene Light Probes for ILF

Techniques

• Single HDR image of Light probe• HDR video of a moving Light Probe• HDR video from fisheye lens• Light Probe Array• Plain Mirror (edge coded)

Edge coded plain mirror - Courtesy: Ihrke et al. / INRIA Labs

Mirror Sphere array - Courtesy: Dr. J. Unger

HDR video with fisheye lens - Courtesy: Dr. J. Unger

Acquisition

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene Light Probes for ILF

Problems of mirror sphere array

• Reflection from other Mirrors• Camera reflection

Problems of HDR Video setup

• Impractical for big space

AcquisitionProposals

1. Bowls are better than Balls:Concave light probes to eliminate mutual reflection

2. 3D space coverage:45* angle concave probe plane to capture uneven surface such as race tracks

Far fetched idea:probe fitted in a Quadraptor!

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene

Blocked for obvious reasons !

AcquisitionProposals

1. Bowls are better than Balls:Concave light probes to eliminate mutual reflection

2. 3D space coverage:45* angle concave probe plane to capture uneven surface such as race tracks

Far fetched idea:probe fitted in a Quadraptor!

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene

1 2 3 4 5 6 7

Move Direction

Blocked for obvious reasons !

AcquisitionProposals

1. Bowls are better than Balls:Concave light probes to eliminate mutual reflection

2. 3D space coverage:45* angle concave probe plane to capture uneven surface such as race tracks

Far fetched idea:probe with a gimbal fitted in a Quadraptor!

Acquisition

Physical Scene

Representatio

n

Rendering

Storage

Virtual Scene

Little inefficient for covering large space

provides much more mobility than a fixed camera setup

Blocked for obvious reasons !

AcquisitionPhysical Scene

Representation

Rendering

Storage

Virtual Scene

ToDoin the next months

• Re-creating the ILF acquisition with spherical probe.• Implementing it into PARC• Comparing to original results in terms of Rendering

Artifacts• Seam• Hair• Pixelation• Ghosting• Disparity problem

Questions?