johnny lee, scott hudson, paul dietz carnegie mellon university mitsubishi electric research labs...
TRANSCRIPT
![Page 1: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/1.jpg)
Johnny Lee, Scott Hudson, Paul DietzCarnegie Mellon University
Mitsubishi Electric Research LabsUIST 2007 – Newport, RI
Hybrid Infrared and Visible Light Projection for Location Tracking
![Page 2: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/2.jpg)
Hybrid Projection
infrared visibleone projector
But, why?
![Page 3: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/3.jpg)
Projector-Based Location Discovery [Lee, UIST’04]
Light sensors
Electronics & computer
Projector
![Page 4: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/4.jpg)
Projector-Based Location Discovery
calibration free: - no computer vision - no alignment - no manual input
Scalable and robust
![Page 5: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/5.jpg)
Moveable Surfaces
[Lee, UIST 2005]
- calibration free- no external tracker- interactive content
![Page 6: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/6.jpg)
Drawbacks
Location Discovery [‘04] Moveable Surfaces [‘05]
Caustic B&W patternsMomentary movement
Incremental tracking
![Page 7: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/7.jpg)
Ideally
Full-screenLocation Patterns
Full-screenApplication Content
![Page 8: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/8.jpg)
Hybrid Projection
infrared visible
for the computer for the human
one projector
![Page 9: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/9.jpg)
Infrared & Visible Projection
Infrared Visible
9
![Page 10: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/10.jpg)
Infrared & Visible Projection
Infrared Visible
10
![Page 11: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/11.jpg)
Infrared & Visible Projection
Infrared Visible
11
![Page 12: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/12.jpg)
Infrared & Visible Projection
Infrared Visible
12
![Page 13: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/13.jpg)
Infrared & Visible Projection
Infrared Visible
13
![Page 14: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/14.jpg)
Infrared & Visible Projection
Infrared Visible
14
![Page 15: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/15.jpg)
Infrared & Visible Projection
Infrared Visible
15
![Page 16: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/16.jpg)
Infrared & Visible Projection
Infrared Visible
16
![Page 17: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/17.jpg)
Infrared & Visible Projection
Infrared Visible
17
![Page 18: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/18.jpg)
Infrared & Visible Projection
Infrared Visible
18
![Page 19: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/19.jpg)
How?
![Page 20: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/20.jpg)
Light Source: Lamps
Xenon Arc Lamp
![Page 21: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/21.jpg)
Light Technologies
![Page 22: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/22.jpg)
More Efficient and Better Lifespan
Products emerging on market this year, 2007
![Page 23: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/23.jpg)
IR and Visible Light LEDs 1000 LEDs/mm2
University of Strathclyde, Institute of Photonics
![Page 24: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/24.jpg)
DMD LED ArrayLens
Projection optics
![Page 25: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/25.jpg)
Our Dev Kit: 180 binary images/sHigh-Speed Dev Kit: 16,500 binary images/sProduction Unit: +50,000 binary images/s
1024x768 area = 20 binary images60Hz tracking = 2.4% duty cycle of production DMDRequired changes to commercial designs would be minimal.
![Page 26: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/26.jpg)
Demo of CapabilityUses a second projector for visible content
![Page 27: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/27.jpg)
Inherent Multi-Stylus Tracking
![Page 28: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/28.jpg)
Non-Planar and Discontinuous Surfaces
![Page 29: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/29.jpg)
Static IR Patterns
![Page 30: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/30.jpg)
Concept ApplicationsSimulated using external tracking (calibration)
![Page 31: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/31.jpg)
[Siggraph 2004]
Hand-held projection with photosensitive tags
![Page 32: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/32.jpg)
Foldable Interactive Displays [submitted to CHI]
![Page 33: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/33.jpg)
Acknowledgements
Johnny Chung Lee
Funded in part by the National Science Foundation under grants IIS-0121560 and IIS-0325351
Funded in part by Mitsubishi Electric Research Labs
![Page 34: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/34.jpg)
Other ways to make invisible patterns
Other non-visible wavelengthsSteganography
– Color shifting– Noise encoding
Bit Timing– Synchronization may be difficult
![Page 35: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/35.jpg)
Camera-Based TrackingRequires calibrationRequires markers for segmentation
– IR sensors + transmitter is less power than 4 IR LEDs
Does not provide IDLimitation on the number of pointsLimitation on tracking rateLimitation on scene/target complexityResolution is not as scalableLess optically robust
– Optical path geometry and variable illumination
![Page 36: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/36.jpg)
Projector vs Camera Tracking
• Sensors provide point ID• Independent of scene/surface complexity
![Page 37: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/37.jpg)
Space-Labeling Projectors
11 Infrared LED slide projectors
Potentially Low-cost
Per axis: 500Hz tracking at 10-bits
Outdoor motion tracking
37
![Page 38: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/38.jpg)
Binary Gray
0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
0000
0001
0011
0010
0110
0111
0101
0100
1100
1101
1111
1110
1010
1011
1001
1000
![Page 39: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/39.jpg)
Binary Gray
0000
0001
0010
0011
0100
0101
0110
0111
1000
1001
1010
1011
1100
1101
1110
1111
0000
0001
0011
0010
0110
0111
0101
0100
1100
1101
1111
1110
1010
1011
1001
1000
![Page 40: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/40.jpg)
Binary Gray
![Page 41: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/41.jpg)
Binary Gray
![Page 42: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/42.jpg)
Binary Gray
![Page 43: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/43.jpg)
Binary Gray
![Page 44: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/44.jpg)
Binary Gray
![Page 45: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/45.jpg)
Binary Gray
![Page 46: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/46.jpg)
Binary Gray
![Page 47: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/47.jpg)
Binary Gray
![Page 48: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/48.jpg)
Binary Gray
![Page 49: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/49.jpg)
Binary Gray
![Page 50: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/50.jpg)
Binary Gray
![Page 51: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/51.jpg)
Binary Gray
![Page 52: Johnny Lee, Scott Hudson, Paul Dietz Carnegie Mellon University Mitsubishi Electric Research Labs UIST 2007 – Newport, RI Hybrid Infrared and Visible Light](https://reader035.vdocuments.us/reader035/viewer/2022081414/5513d51455034646298b52f2/html5/thumbnails/52.jpg)
Pixart Chip
1024x768 resolution100Hz tracking4 points
Only 4 pointsHigh power LED points