vision enhancement - tumcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · vision...
TRANSCRIPT
![Page 1: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/1.jpg)
Vision Enhancement:Defocus Correction via
Optical See-Through Head-Mounted Displays
TU Munich, AH2015, 9th Mar 2015
Yuta Itoh and Gudrun Klinker
1/24
![Page 2: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/2.jpg)
Non Optical See-Through
Head-Mounted Displays
Optical See-Through (OST-HMD)
2/24
INDIRECT
DIRECT
![Page 3: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/3.jpg)
Vision Enhancement in Science Fictions…
From Ironman, 2008 ©Marvel Entertainment
3/24
![Page 4: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/4.jpg)
Vision Enhancement in Science Fictions…
World sensing
Eye tracking
4/24
![Page 5: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/5.jpg)
Vision Enhancement in Science Fictions…
From Mission: Impossible - Ghost Protocol , 2011 ©Paramount Pictures
5/24
![Page 6: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/6.jpg)
World sensing
Eye tracking
Color replication
Fine resolution
Vision Enhancement in Science Fictions…6/24
![Page 7: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/7.jpg)
Given an ultimate OST-HMD….
What’s unique in theoretical formulation?
Current limitations and open issues?
Vision Enhancement7/24
![Page 8: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/8.jpg)
Related Work | Low-vision devices8/24
Peli, Eli. "Vision multiplexing: an engineering approach to vision rehabilitation device
development." Optometry & Vision Science 78.5 (2001): 304-315.
Peli, Eli. "Treating with spectacle lenses: a novel idea!?." Optometry & Vision Science 79.9 (2002): 569-580.
![Page 9: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/9.jpg)
Related Work | Color enhancement
[AmanoBG09] Toshiyuki Amano, Oliver Bimber, and Anselm Grundhöfer. "Appearance
Enhancement for Visually Impaired with Projector Camera Feedback.“
TECHNICAL REPORT, BAUHAUS-UNIVERSITY WEIMAR, JANUARY 2009
9/24
![Page 10: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/10.jpg)
Broecker, Markus, Ross T. Smith, and Bruce H. Thomas. "Adaptive substrate for enhanced
spatial augmented reality contrast and resolution." transport 3.6 (2011): 22.
10/24
Related Work | Contract enhance.
![Page 11: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/11.jpg)
Huang, Fu-Chung, et al. "Computational light field display for correcting visual
aberrations." ACM SIGGRAPH 2013 Posters. ACM, 2013.
Related Work | De-blurring displays11/24
![Page 12: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/12.jpg)
Related Work | Eye-HMD calibration
Interaction-free Display Calibration
12/24
Itoh, Yuta, and Gudrun Klinker. "Interaction-free calibration for optical see-through head-mounted
displays based on 3D Eye localization." 3D User Interfaces (3DUI), 2014 IEEE Symposium on. IEEE, 2014.
![Page 13: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/13.jpg)
Related Work | Digital human models
Individual Observer Model of human
eyesight
13/24
Bennett, Mike, and Aaron Quigley. "Creating personalized digital human models of perception for visual
analytics." User Modeling, Adaption and Personalization. Springer Berlin Heidelberg, 2011. 25-37.
![Page 14: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/14.jpg)
Related Work | Edge Enhancement14/24
Luo, Gang, and Eli Peli. "Use of an augmented-vision device for visual search by patients with tunnel
vision." Investigative ophthalmology & visual science47.9 (2006): 4152-4159.
![Page 15: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/15.jpg)
Related Work | Edge Enhancement
Hwang, Alex D., and Eli Peli. "An augmented-reality edge enhancement application for Google Glass."
Optometry & Vision Science 91.8 (2014): 1021-1030.
15/24
![Page 16: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/16.jpg)
Eye aberration
(defocus)
Problem: Eye aberration16/24
𝒇abr
![Page 17: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/17.jpg)
Defocus Correction via OST-HMDs
Compensation
layer
17/24
Eye aberration
(defocus)
𝒇abr
![Page 18: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/18.jpg)
Hardware setup
User-perspective
Camera
World Camera
Optical See-Through HMD (Lumus DK32)
18/24
![Page 19: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/19.jpg)
Conceptual Evaluation
Enhanced
Compensation
layer
19/24
𝒇abr
![Page 20: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/20.jpg)
ResultsB
ett
er
Original
World Cam.
Defocus
Enhanced
20/24
![Page 21: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/21.jpg)
Open problems
User-perspective rendering
Eye aberration estimation
Display resolutions
Negative color display
Color replication
…
21/24
![Page 22: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/22.jpg)
Can ultimate OST-HMDs enhance vision?
Investigated defocus correction in a
conceptual OST-HMD setup
Provided open problems toward
a practical vision enhancement with
OST-HMDs
Summary22/24
![Page 23: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/23.jpg)
Any questions?
This project has received funding from
the European Union’s Seventh
Framework Programme for research,
technological development and
demonstration under PITN-GA-2012-
316919-EDUSAFE.
23/24
![Page 24: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/24.jpg)
Appendix
24/24
![Page 25: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/25.jpg)
Dynamic range
(sunglasses, night vision goggles)
Focal length (corrective glasses)
Field of view
(tele/micro-scopes, low-vision glasses)
Spectrum range (thermal goggles)
Exposure (stroboscopic/shutter glasses)
Vision Enhancement Applications25/24
![Page 26: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/26.jpg)
Related Work | AR X-ray vision
Avery, Ben, Christian Sandor, and Bruce H. Thomas. "Improving spatial perception for augmented reality
x-ray vision." Virtual Reality Conference, 2009. VR 2009. IEEE. IEEE, 2009.
26/24
![Page 27: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/27.jpg)
Related Work | Super-resolution
displays
Lanman, Douglas, et al. "Cascaded displays: spatiotemporal
superresolution using offset pixel layers." ACM SIGGRAPH 2014 Emerging Technologies. ACM, 2014.
27/24
![Page 28: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/28.jpg)
Related Work | HDR microscopy
Bimber, Oliver, et al. "Closed-loop feedback illumination for optical inverse tone-mapping
in light microscopy." Visualization and Computer Graphics, IEEE Transactions on 17.6
(2011): 857-870.
28/24
![Page 29: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/29.jpg)
Related Work | Field of view enhan.
Treating with Spectacle Lenses: a Novel Idea!?
PELI, ELI MSc, OD, FAAO
29/24
![Page 30: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/30.jpg)
Related Work | User-perspective
image rendering
Baricevic, Domagoj, et al. "A hand-held AR magic lens with user-
perspective rendering." Mixed and Augmented Reality (ISMAR), 2012
IEEE International Symposium on. IEEE, 2012.
30/24
![Page 31: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/31.jpg)
Related Work | HDR by projector
Bimber, Oliver, and Daisuke Iwai. "Superimposing dynamic range." ACM Transactions on Graphics (TOG). Vol. 27. No. 5. ACM, 2008.
31/24
![Page 32: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/32.jpg)
Related Work | temporal super
resolution
Sajadi, Behzad, M. Gopi, and Aditi Majumder. "Edge-guided resolution enhancement in projectors
via optical pixel sharing." ACM Transactions on Graphics (TOG) 31.4 (2012): 79.
32/24
![Page 33: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/33.jpg)
Related Work | de-blurring
Oyamada, Yuji, and Hideo Saito. "Defocus blur correcting projector-
camera system." Advanced Concepts for Intelligent Vision Systems.
Springer Berlin Heidelberg, 2008.
33/24
![Page 34: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/34.jpg)
Huang, Fu-Chung, et al. "Correcting for optical aberrations using
multilayer displays." ACM Transactions on Graphics (TOG) 31.6
(2012): 185.
Correct visual aberrations
Pamplona, Vitor F., et al. "Tailored
displays to compensate for visual
aberrations." ACM Trans. Graph. 31.4
(2012): 81.
Related Work | de-blurring displays34/24
![Page 35: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/35.jpg)
A military OST-HMD system
BAE Systems, Q-warrier
http://www.telegraph.co.uk/finance/newsbysector/industry/defence/1
0640078/BAE-Systems-launches-high-tech-Iron-Man-helmet.html
35/24
![Page 36: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/36.jpg)
Zernike Polynomial36/24
Corrective
glasses
![Page 37: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/37.jpg)
Related Work | Wave-front sensor37/24
二宮さゆり. "波面センサーによる眼球の波面収差測定と網膜像推定." Medical imaging
technology 23.1 (2005): 47-51.
Emmetropia
![Page 38: Vision Enhancement - TUMcampar.in.tum.de/pub/itoh2015ah/itoh2015ah.slides.pdf · Vision Enhancement: Defocus Correction via Optical See-Through Head-Mounted Displays TU Munich, AH2015,](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f675c6fb935b04a5664ee8b/html5/thumbnails/38.jpg)
Related Work | Wave-front sensor38/24
二宮さゆり. "波面センサーによる眼球の波面収差測定と網膜像推定." Medical imaging
technology 23.1 (2005): 47-51.
Keratoconus/円錐角膜