chuck norris had to destroy the periodic table… he only recognizes the element of surprise!
TRANSCRIPT
![Page 1: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/1.jpg)
![Page 2: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/2.jpg)
Kinect Interactions: Get a Grip in C# or HTML5/JavascriptBen LowerProgram Manager – Kinect for Windows3-705
![Page 3: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/3.jpg)
Periodic TableInteractionStreamSneak Peek of 1.8 SDKNew Kinect?Q&A
Agenda
![Page 4: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/4.jpg)
Periodic Table
![Page 5: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/5.jpg)
![Page 6: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/6.jpg)
CHUCK NORRISHAD TO DESTROYTHE PERIODIC TABLE…
HE ONLY RECOGNIZES THEELEMENT OF SURPRISE!
![Page 7: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/7.jpg)
AAudio
Waveform
Stream Symbol
Stream Name
Data Type
The Periodic Table of Kinect for Windows
![Page 8: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/8.jpg)
DDepth
DepthImagePixel[]
AAudio
Waveform
IRInfrared
byte[]
CColor
byte[]
ACAcceleromete
rVector4
Data Streams (Eyes & Ears)
![Page 9: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/9.jpg)
IInteractionsInteractionFrame
(UserInfo[])
KFKinect Fusion
Point Cloud/Mesh
FTFace Tracking
FaceTrackFrame
SSpeech
Recognition Events
STSkeleton TrackingSkeleton[]
Recognition Streams (Brains)
![Page 10: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/10.jpg)
Demo: Controls Basics +Interaction Gallery
![Page 11: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/11.jpg)
InteractionStream
![Page 12: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/12.jpg)
Why InteractionStream?
Moving to world that is multi-modal
Have experiences feel natural
Consistency of interactions
![Page 13: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/13.jpg)
Interaction Stream
Translates hand position to screen coordinates• Physical Interaction Zone (PhIZ) is shaped to be ergonomic for an
individual independent of position relative to the sensor
Track and recognize press action• Press action based on arm extension• Press distance dependent on starting arm position• Press adjusted to improve accuracy over buttons
Recognize hand grip and release gesture• Computer vision and machine learning algorithm recognize grip gesture• Grip recognized for a wide variety of hand poses and orientations
![Page 14: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/14.jpg)
The PhIZ
Physical Interaction Zone (PhIZ)The area in front of the user where there is a spatial mapping relationship between the user’s hand movements in physical space and the cursor movement on the screen.
Features• Ergonomics and Human Movement• Scaling to fit user• Separate PhIZ for each hand• Consistent experience
![Page 15: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/15.jpg)
Targeting & Selection
Cursor Default (L)
Cursor Default (R)
New WayTarget button with hand cursorPress toward button, then releaseCenter “pulls hand in” during press
Old WayTarget button with hand cursorHover, hold, wait (for action)
![Page 16: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/16.jpg)
Press Cursors
Roll Over Press Half Full Press Press Complete
Research showed people like visual cues &
feedback
Hand “fills up” to indicate press progress
![Page 17: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/17.jpg)
Scrolling & Direct Manipulation
Cursor Default
Grip Over
New WayTarget anywhere w/hand cursorGrip to manipulate
Old WayTarget scroll button w/hand cursorHold and wait for scroll
![Page 18: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/18.jpg)
InteractionStream APIs
Native COM based InteractionStream API• Core implementation• Usable with any UI frameworks that can call C functions and COM
objects
Managed .NET Wrapper• Wraps the InteractionStream API for .NET languages, e.g. C#, VB etc.
WPF Adapter – KinectRegion, Cursor and Controls
• Implements IInteractionClient for hit-testing WPF controls• Translates Interaction Frame to WPF events• Implements Cursor to provide feedback for: handedness, press progress
and grip state
![Page 19: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/19.jpg)
Interactions Architecture
NUI_INTERACTION STREAM
KinectInteraction170_*.dll (32 or 64)
DirectXKinect
Support InteractionStream
Microsoft.Kinect.Toolkit.Interaction.dll
.NET/WPF
Kinect-Enabled App
WPF Kinect Controls
Microsoft.Kinect.Toolkit.Controls.dll
XNA Kinect
SupportC++/Win32 Kinect
Controls
Windows Forms Kinect
Controls
C++/DirectX Kinect-Enabled
App
C++/Win32 Kinect-Enabled
App
XNAKinect-Enabled
App
Windows FormsKinect-Enabled
App
![Page 20: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/20.jpg)
Working with InteractionStream
Application
InteractionStream
UI Framework Adapter:IInteractionClient
ProcessSkeleton(skeleton, accelerometer, timestamp)
ProcessDepth(depthData, timestamp)
GetInteractionInfoAtLocation(id, handType, x, y)
new (sensor, interactionClient)
return InteractionInfo{ isGripTarget, isPressTraget,PressAttractionPointX, PressAttractionPointY,PressControlId}
InteractionFrameReady
![Page 21: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/21.jpg)
Integration Considerations
Primary User and Primary Hand• InteractionStream tracks 2 users and 4 hands• Primary User and Primary Hand implemented by KinectRegion• UX Guidance: Avoid 2 handed actions – difficult to learn & master
Cursor – press and grip visualization• Continuous press progress feedback critical to user experience• Press initiates action on release – similar to button press with mouse
Pressable button affordance• Button provides visual feedback when cursor is over pressable button
![Page 22: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/22.jpg)
Sneak Peek: v1.8 SDK
![Page 23: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/23.jpg)
Demo: Background Removal
![Page 24: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/24.jpg)
Demo: Photobombr
![Page 25: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/25.jpg)
Interactions Architecture (1.8)
NUI_INTERACTION STREAM
KinectInteraction170_*.dll (32 or 64)
DirectXKinect
Support InteractionStream
Microsoft.Kinect.Toolkit.Interaction.dll
HTML5/Javascript
Kinect-Enabled AppHTML5/JS Controls
Microsoft.Samples.
Kinect.Webserver.dll
XNA Kinect
SupportC++/Win32 Kinect
Controls
Windows Forms Kinect
Controls
C++/DirectX Kinect-Enabled
App
C++/Win32 Kinect-Enabled
App
XNAKinect-Enabled
App
Windows FormsKinect-Enabled
App
.NET/WPF
Kinect-Enabled App
WPF Kinect Controls
Microsoft.Kinect.Toolkit.Controls.dl
l
![Page 26: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/26.jpg)
Announcing: v1.8 SDK (September 2013)Background Removal API (green screening w/out the screen)
HTML5/Javascript support for Interactions
Kinect Fusion Color
Plus more…stay tuned
![Page 27: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/27.jpg)
What’s up with the new Kinect?
![Page 28: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/28.jpg)
New Kinect for Windows Sensor
• Wider field of view• HD resolution• Track six skeletons• Better ST, audio, infrared• Coming next year
![Page 29: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/29.jpg)
Announcing: Developer Kit Program
Direct access to our engineering teamExclusive access to pre-release SDKsPrivate access to all API & sample docsPre-release/alpha sensorNew RTM sensor next year
$399 (coming November 2013)
Apply now: http://
aka.ms/k4wdevkitapplication
![Page 30: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/30.jpg)
Who should apply?
Already working today to build & deploy NUI
experiences
You are interested in or experimenting with NUI
You want to help push the boundaries of NUI
Experienced developer who wants to jump in
You have an awesome idea for a touch-free app
![Page 31: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/31.jpg)
Thank you!
Get 1.7 SDK now
Follow our blog to know when 1.8 SDK is released
Apply today for new dev kit program
![Page 32: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/32.jpg)
Kinect for Windows @ Build 2013
All Week• Kinect for Windows Booth (demos, 3D printing with Fusion)
Wednesday, June 26th
• 2:00 PM Real-world Machine Learning: How Kinect Gesture Recognition Works, 3-704Alisson Sol, Bing (formerly Kinect for Windows), North Hall: 123
Thursday, June 27th
• 2:00 PM Kinect Interactions: Get a Grip in C# or HTML5/Javascript, 3-705Ben Lower, Kinect for Windows, North Hall: 131
• 5:00 PM Scanning & Printing the Real-World using Kinect for Windows and Preview of vNext, 3-702
Chris White, Kinect for Windows, North Hall: 131
![Page 33: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/33.jpg)
Kinect for Windows Resources
Developer Kit Program Applicationhttp://aka.ms/k4wdevkitapplication
Developer Center (SDKs, Samples, Videos)http://aka.ms/k4wdev
Product Infohttp://aka.ms/k4w
![Page 34: CHUCK NORRIS HAD TO DESTROY THE PERIODIC TABLE… HE ONLY RECOGNIZES THE ELEMENT OF SURPRISE!](https://reader035.vdocuments.us/reader035/viewer/2022062800/56649de45503460f94adc32a/html5/thumbnails/34.jpg)
© 2013 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.