practical aspects of hdr capture and acquisition (deconstructing hdr) francisco imai
DESCRIPTION
Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai. CREATE 2010 June 08 2010. Disclaimer Only representing self Views in this presentations are not of any current or past employer. Image as representation. Real. Ideal. Tone editing. Ansel Adams. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/1.jpg)
CREATE 2010
June 08 2010
CREATE 2010
June 08 2010
Practical aspects of HDR capture and acquisition(Deconstructing HDR)
Francisco Imai
Practical aspects of HDR capture and acquisition(Deconstructing HDR)
Francisco Imai
![Page 2: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/2.jpg)
Disclaimer
Only representing selfViews in this presentations are not of
any current or past employer
![Page 3: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/3.jpg)
Image as representation
Ideal Real
![Page 4: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/4.jpg)
Tone editing
Ansel Adams
![Page 5: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/5.jpg)
What about color HDR?
Merced River and El Capitan in Winter, Yosemite Valley [in Nature]
![Page 6: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/6.jpg)
Image manipulation
![Page 7: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/7.jpg)
![Page 8: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/8.jpg)
![Page 9: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/9.jpg)
Definition Dynamic Range
![Page 10: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/10.jpg)
maximum non-saturated signal
DR = ------------------------------- minimum signal
![Page 11: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/11.jpg)
Sensor technologies
www.pixim.com
![Page 12: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/12.jpg)
HDR sensor technology:1. Multiple gains sensor
from Boyd Fowler [HDRI workshop 2009]
Advantages:
Good low light performanceGood linearity
Drawbacks:
Limited DR extensionAdditional silicon areaAdditional power dissipation
e.g. Dual column Level amp and ADC [Fowler 09], LOFIC [Adachi 05]
![Page 13: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/13.jpg)
HDR capture technology:2. Non-linear pixel response sensor
from Boyd Fowler [HDRI workshop 2009]
Advantages: Wide DR > 120 dBInstantaneous measurementHigh fill factorSimple operation
Drawbacks:
No low-light videoLarger FPNDifferent sensor architectureNo solution for color sensors
e.g. Logarithmic sensor [Dierickx 95], Dynamic well capacity adjustment [Sayag 91, Decker 98]
![Page 14: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/14.jpg)
HDR capture technology:3. Well-capacity recycling sensor
from Boyd Fowler [HDRI workshop 2009]
Advantages: Good linearityGood color imagingLarge DR > 100 dBBest SNR
Drawbacks:
Large pixel sizePoor low light performanceHigh power dissipationNeeds high speed read-out
e.g. Sigma delta pixel level ADC [Fowler 94], Asynchronous self-reset with multiple capture [Liu 02]
![Page 15: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/15.jpg)
HDR capture technology:4. Time to saturation sensor
from Boyd Fowler [HDRI workshop 2009]
Advantages: Very HDR > 150 dB
Drawbacks:
High power dissipationLarge pixel
e.g. Time to saturation [Brajovic 96]
![Page 16: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/16.jpg)
HDR capture technology:5. Time varying exposures
Advantages: Use conventional sensors
Drawbacks:
Motion blur
e.g. Two sample CID [Nakamura 97], Two sample APS [Yadid-Peckt 97]
![Page 17: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/17.jpg)
Blended exposures
![Page 18: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/18.jpg)
HDR capture technology:6. Spatially varying exposures
from Boyd Fowler [HDRI workshop 2009] and www.pixim.com
Advantages: Excellent linearityColor ImagingDR > 100 dB
Drawbacks:
High power dissipationLarge pixelPoor low light performanceNeeds high speed read-out
e.g. DPS pixel [Yang 99], Fuji Super CCD
![Page 19: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/19.jpg)
Pixim DPS
![Page 20: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/20.jpg)
There are plenty of publication on HDR imaging sensors but
most of cameras still do not use them.
WHY?
![Page 21: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/21.jpg)
Example
20 f-stops in HD-video
Spherican 180-360 degrees26 f-stops, 50 Mpixel
20 s to 1 minute
![Page 22: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/22.jpg)
Example
F200 EXR, F70EXR, F80EXR and S200 EXR
FujiFim SuperCCD EXR
![Page 23: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/23.jpg)
What to do with captured HDRI?
-Tone map to display in conventional displays-Global tone mapping-Local tone mapping-iCAM-Retinex
-Build an HDR display
![Page 25: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/25.jpg)
25
SIGGRAPH 2005 - Seetzen et al.
14 bits
1300 cd/m2
Off-the-shelf components HDR display
![Page 26: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/26.jpg)
26
HDR display modeling
![Page 27: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/27.jpg)
27
HDR display forward model
• All possible sextuplets combination is 6256 = 281 trillion
• Inaccurate colorimetric measurement in dark region (resulting in green cast in dark areas when model is used)
• Problem with cross-terms between LCD and DLP
• More robust physical based model is necessary
![Page 28: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/28.jpg)
28
Radiance based Forward model
Decompose 12 Component XYZ and perform
a priori eigenvector analysis
![Page 29: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/29.jpg)
29
LCD display spectral radiance
Red, green and blue ramps
![Page 30: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/30.jpg)
30
LCD additivity
Comparison sum of red, green and blue Channels with gray measurements
![Page 31: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/31.jpg)
31
Spectral radiance measurements
Photo Research PR-650
![Page 32: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/32.jpg)
32
Spectral radiance measurement
DLP light reflected on MgO2
Measurement Red, Green, Blue Ramps
Close-up measured areafor green ramp
![Page 33: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/33.jpg)
33
Additivity DLP channels?White is not active White is active
White channel
![Page 34: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/34.jpg)
34
DLP Radiance
LCD Radiance
Transmittance LCD
Transmittance LCDTransmittance LCD
![Page 35: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/35.jpg)
35
Compare LCD radiance between measurement and estimation
usingforward model
Blue Green Red
![Page 36: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/36.jpg)
36
Comparison measured and estimated LCD radiance
![Page 37: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/37.jpg)
37
Estimation Final RadianceTraining set
![Page 38: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/38.jpg)
38
luminance x chroma Hue angle x chroma
Verification 100 “random” samples (20 samples with white
on white on DLP)
![Page 39: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/39.jpg)
39
Inverse HDR model
![Page 40: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/40.jpg)
40
Colorimetric performance100 Random Samples
![Page 41: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/41.jpg)
41
Extra Verification set (280 samples)
Neutral ramp (14 levels)4 octaves of Color Checker
170 objects (Vrhel)
Mean CIEDE2000 = 2.4Maximum CIDE2000 = 7.7
Excluding 7 outliers (out-of-gamut samples)
![Page 42: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/42.jpg)
42
HDR rendering accuracy assessment
![Page 43: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/43.jpg)
Veiling glare
Scene-dependent scatter in optics/camera and in the human eye
reduce the HDR
TalvalaSIGGRPAH 2007
![Page 44: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/44.jpg)
Other IQ aspect: SNRfrom Boyd Fowler [HDRI workshop 2009]
![Page 45: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/45.jpg)
Other IQ aspect: Resolution
Quick MTF
![Page 46: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/46.jpg)
Other IQ aspect: Colour
EyeRex – SW by Media Chance
![Page 47: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/47.jpg)
Evocative HDR look
Fantoft stavkirke –www.pappafrezzo.com
Nina Aldin Thune
Tunliweb
![Page 48: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/48.jpg)
Hype Cycle HDRI
![Page 49: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/49.jpg)
Acknowledgments
![Page 50: Practical aspects of HDR capture and acquisition (Deconstructing HDR) Francisco Imai](https://reader035.vdocuments.us/reader035/viewer/2022062301/56815971550346895dc6b319/html5/thumbnails/50.jpg)
Unanswered questions
1. When are we going to have HDR cameras on cell-phone cameras?
2. How to deal with white balance in HDR?
3. What set of metrics is appropriate for HDR display quality?