estimating natural illumination from a single outdoor scene final

Estimating natural illumination from a single outdoor image12/8/09

Estimating natural illumination from a single outdoor scene

Sagnik Dhar (107219686)Debaleena Chattopadhay (107224101)

Course Project for CSE – 591Fall ‘09

Based on, “Estimating Natural Illumination from a Single Outdoor Image” by Jean-

Francois Lalonde, Alexei A. Efros, and Srinivasa G. NarasimhanCVPR ‘09


Estimation of illumination


Why do people care about illumination ?

• Insertion of objects

• Retrieval of objects from a database


Approach

Natural Outdoor Daytime Scene

Extract Plausi-ble Cues

Output a Probable Distribution of Sun’s

Zenith angle

Sky Cue

Shadow Cue

Perez Modeling of

Skies

Simulate Normal

Distribution of sun positions

Determining Best Linear

Shadow

Vertical Surface Cue

Intensity Gradi-ents Vote For Pos-

sible Sun positions

Intensity Of The Vertical Surfaces

Weighed By Their Size Vote For Possible

Sun Directions

Compute Size Ratio Of Surfaces

To The Scene


Sky Cue

Determine Horizon Line Extract Sky Segment

Classify into Clear, Patchy or Overcast

category and Process(KNN)

Simulate Perez Sky Model to create 360 skies (each

for a sun zenith angle)(Video)

Taking our sky segment as the mean, find a probability

distribution of sun zenith angle


KNN Classification of skies

Sky

Clear Patchy Overcast

Binary segmentation to remove clouds Discard Cue

Continue


Shadow Cue

Extract ground Segment

Detect Longest Linear Shadow

Shadow pixel intensities vote for

probable sun directions

Convert to CIE L*a*b Color

Space

Detect Shadow Pixels in b channel

Find Intensity orientation of the shadow

pixels

Cluster Using K-means

Top Cluster votes for Sun

Directions


Vertical Surface Cue

Extract Vertical Surfaces

Compute surface size proportionality with respect to the scene.

Find intensity of each surfaces

Surface size weighted by intensity vote for their preferred Sun direction


Sun Zenith angle


Cue Combination

1ST

2ND3RD

4TH

271°Sky

14°Shadow

BehindVertical Surfaces


Results- Best Case


Results- Best Case

1ST

2ND3RD

4TH

100°Sky

8°Shadow

RightVertical Surfaces


Results- Best Case


Results- Best Case

1ST

2ND3RD

4TH

181°Sky189°

Shadow

LeftVertical Surfaces


Results- Sky + Vertical Cue


Results- Sky + Vertical Cue

1ST

2ND3RD

4TH

271°Sky



Results- Shadow + Vertical Cue



1ST

2ND3RD

4TH

14°Shadow




1ST

2ND3RD

4TH

9°Shadow



Results- Failure Cases



1ST

2ND3RD

4TH

5°Shadow


292°Sky



1ST

2ND3RD

4TH

16°Shadow


0°Sky


Human Validation

No. of Humans: 3

No. of Images: 58

Human Consensus

Human Contradiction

Accuracy

27 31 46%

Match with Human

Mismatch Accuracy

22 36 37%


Cue Weightage Investigation


Conclusion

• Accurate estimation of sun position – Tough problem for human visual system too.

• The photographic eye doesn’t bother to capture “informative” cues for visual systems to analyze.

• The weak cues are strongly coupled and highly susceptible to incorrect conclusion.

• Shadow detection from a single image is still in the need of better algorithms.


Thank You

To end, a quote I came across recently,

“All I see is the sun; And even when it's hiding behind a smoke of cloud, the shadow tells me all that there is...”

estimating natural illumination from a single outdoor scene final

Entertainment & Humor

results shadow

results sky

shadow detection

cue weightage investigation

intensity vote

natural illumination

estimation of illumination

single image