powerpoint presentation...•named after the brazilian city, meaning “relating to birth” •...
TRANSCRIPT
![Page 1: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/1.jpg)
![Page 2: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/2.jpg)
![Page 3: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/3.jpg)
![Page 4: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/4.jpg)
![Page 5: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/5.jpg)
![Page 6: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/6.jpg)
![Page 7: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/7.jpg)
Project Natal
Kinect for Xbox 360
Kinect SDK from Microsoft Research
The Kinect Effect video
Kinect for Windows 1.0 product
Kinect for Windows 1.5 product
![Page 8: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/8.jpg)
![Page 9: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/9.jpg)
• Named after the Brazilian city, meaning “relating to birth”
• “The birth of the next generation of home entertainment“ (Alex Kipman)
• Not just the device. The sensor provides “eyes and ears”… but it needs a brain
• Raw data from that sensor is just “a whole bunch of noise that someone needs to take and turn into signal” that is what our software does: find the signal
Source: Wikipedia
![Page 10: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/10.jpg)
![Page 11: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/11.jpg)
• Find the people in the scene, ignore background
• Find their limbs and joints, which person is which
• Find and track their gestures
• Map the gestures to meaning and commands
• Also: recognize faces
• Also: recognize voices and commands
![Page 12: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/12.jpg)
• Machine Learning
• Effectively…
• Evaluate trillions of the possible body configurations of 32 body (skeletal) segments
• Every video frame
• 30 times a second
•On <10% of the CPU
![Page 13: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/13.jpg)
![Page 14: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/14.jpg)
Real-Time Human Pose Recognition in Parts from a Single Depth Image Jamie Shotton, Andrew Fitzgibbon, Mat Cook, Toby Sharp, Mark Finocchio, Richard Moore, Alex Kipman, Andrew Blake http://research.microsoft.com/apps/pubs/default.aspx?id=145347
http://cvpr2011.org
Paper Supplementary Video
![Page 15: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/15.jpg)
![Page 16: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/16.jpg)
• Collect training data – thousands of visits to global households, filming real users, the Hollywood motion capture studio generated billions of images
• Apply state-of-the-art object recognition research
• Apply state-of-the-art real-time semantic segmentation
• Build a training set – classify each pixel’s probability of being in any of 32 body segments, determine probabilistic cluster of body configurations consistent with those, present the most probable
• Millions of training images Millions of classifier parameters
• Hard to parallelize New algorithm for distributed decision-tree training
• Major use of DryadLINQ (large-scale distributed cluster computing)
![Page 17: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/17.jpg)
http://channel9.msdn.com/events/MIX/MIX11/RES01
Source: Wikipedia
x4
![Page 18: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/18.jpg)
![Page 19: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/19.jpg)
![Page 20: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/20.jpg)
![Page 21: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/21.jpg)
![Page 22: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/22.jpg)
THERAPY HEALTHCARE TRAINING RETAIL
![Page 23: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/23.jpg)
![Page 24: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/24.jpg)
![Page 25: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/25.jpg)
![Page 26: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/26.jpg)
Mexico
Brazil
![Page 27: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/27.jpg)
MULTI-ARRAY MIC RGB CAMERA
3D DEPTH SENSORS
![Page 28: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/28.jpg)
10 joint
40cm near
3D mesh
Lots of good things!
Record, playback, debug
4+11 language regions
![Page 29: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/29.jpg)
http://go.microsoft.com/fwlink/?LinkID=247735
![Page 30: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/30.jpg)
![Page 31: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/31.jpg)
![Page 32: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/32.jpg)
DISPLAY SPECTRUM
D
D
MAP INPUT AND OUTPUTS TO SCREEN SIZE AND ORIENTATION
![Page 33: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/33.jpg)
PROXIMITY SPECTRUM
0’-3’
3’-6’
6’-10’
OUTSIDE
FOV
INPUT/OUTPUT FIDELITY IS PROPORTIONAL TO DISTANCE
![Page 34: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/34.jpg)
POSTURE SPECTRUM
MAP INPUTS/OUTPUTS TO ACTIVITY LEVEL
SEATED
STANDING
ACTIVE
![Page 35: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/35.jpg)
ORIENTATION SPECTRUM
MAP OUTPUTS TO ORIENTATION
AUDIOVISUAL AUDIOVISUAL AUDIO
![Page 36: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/36.jpg)
ENGAGEMENT SPECTRUM
ENGAGED
ENGAGED
NOT ENGAGED
NOT ENGAGED
ENGAGED
TRACKING
![Page 37: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/37.jpg)
![Page 38: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/38.jpg)
How do we make
computing always,
instantly, intuitively
available?
![Page 39: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/39.jpg)
http://research.microsoft.com/en-us/um/redmond/groups/cue/skinput/
![Page 40: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/40.jpg)
http://research.microsoft.com/en-us/um/redmond/groups/cue/humantenna/
![Page 41: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/41.jpg)
http://research.microsoft.com/en-us/groups/sendev/
![Page 42: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/42.jpg)
http://research.microsoft.com/en-us/groups/natural/
![Page 43: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/43.jpg)
http://research.microsoft.com/en-us/groups/vision/
![Page 44: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/44.jpg)
![Page 45: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/45.jpg)
![Page 46: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/46.jpg)
![Page 47: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/47.jpg)
![Page 48: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/48.jpg)
![Page 49: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/49.jpg)
[email protected] http://research.microsoft.com/~stansley @dswtan #KinectWindows
![Page 50: PowerPoint Presentation...•Named after the Brazilian city, meaning “relating to birth” • “The birth of the next generation of home entertainment“ (Alex Kipman) • Not](https://reader033.vdocuments.us/reader033/viewer/2022043019/5f3b19911a9e70474a43ebc1/html5/thumbnails/50.jpg)