using multitouch and sensors in java
DESCRIPTION
TRANSCRIPT
Using multi-touch and sensors in Java
Felipe Pedroso
2
Agenda
•The future of PCs• JavaFX and Touch Support•Using JNI to work with sensors
Mobile Computing Lines are Blurring
Ultrabook™ Convertible & Detachable
Ultrabook™TabletSmartphone
Consumption Usages Creation/Productivity
Notebook
4
Reshaping the PC Experience
Ambient Light
Sensor
GPSCompass
Near Field Communicati
on
Gyroscope
Ultrabook™
Accelerometer
Multi-Touch
Context Aware Sensors
OK, they have all those features…
… but how can we implement them using Java?
6
To allow users to TOUCH my Java App...
• Should I use...• AWT?• Swing?• SWT?
• Actually, you can use them but...• They aren’t made for multi-touch (OK, there’s a way to do it)• You must optimize your UI controls to be more ‘touch friendly’
• So, how to do it in a simple and easy way?
7
8
Actions supported
• Touch events: Down, Move and Up• Tap (Down and Up) / Double tap• Drag and Drop (Down, Move and Up)• Hold
• Gestures• Swipe: Up, Down, Left and Right• Zoom: Pinch / Spread• Rotate
9
What I need to do to manipulate components?
• Your components must extend the Node class or any of its subclasses (StackPane, ImageView, etc)
• Set the proper EventHandler to handle the action.
10
What I need to do to manipulate components?
• Apply a transform to the component (Translation, Rotation and Scale):
• Let’s dive into some code!
What about the sensors?
JNI, I choose you!
12
Available sensors on Ultrabooks
• Accelerometer• Proper acceleration in three axis (x, y, z)
• Gyrometer• Device orientation
• Magnetometer• Strenght and direction of magnetic fields
• GPS (Global Positioning System)• Location and Time information
• NFC• Near Field Communication
• Ambient Light Sensor• Ambient Light Level
13
API Windows – Sensor Fusion
14
API Windows – Namespaces• Windows.Sensors.* Common sensors
• Accelerometer• Gyrometer• Inclinometer• OrientationSensor• SimpleOrientationSensor• Compass• LightSensor
• Windows.Devices.Geolocation GPS• Geolocator
• Windows.Networking.Proximity NFC• ProximityDevice
15
Windows API – How to
• Get the default object of your sensor using the GetDefault method
• You can call the GetCurrentReading() to get the current value of the sensors or...
• ... work with the ReadingChanged event• Set the attribute ReportInterval (please, respect the MinimumReportInterval to avoid problems)• Delegate a method to handle the event (something like setting a method as a listener)• Handle the event!
• This procedure is valid only for sensors from Windows.Sensors namespace
16
How can I access that?
17
Using JNI to access the sensors from Windows.Sensors
1. [Java] Create a native method to register the object that will handle the events that come from the sensor
2. Generate the header file using javah
3. [C++] Create a DLL Project in Visual Studio with the DLLs and namespaces of the Windows 8 APIS
4. [C++] Use a variable to keep a reference to the object and the ID (GetMethodID) of the method that will handle the event
18
Using JNI to access the sensors from Windows.Sensors
5. [C++] Initialize the sensor and delegate a C++ method to handle the event
6. [C++] Redirect the delegate method call to the Java method using the function CallVoidMethod with the following parameters: the Java Object, the method ID and it’s parameters
7. [Java] Handle the event!
A lot of text? Please, show me the code!
19
What about Linux?
• Touch works fine, but there aren’t clear APIs to read sensors. If you know how to do it, let’s work together!
20
Wants to know more about Intel Software?http://software.intel.com/pt-br/
21
Windows Community
http://software.intel.com/en-us/windows
22
www.javafxcommunity.com
23
JFXtras.org
Take aways
• The PC experience is being reshaped
• JavaFX + Touch = AWESOME
• Using JNI to access the sensors can be LEGENDARY!
Any doubts?(if yes, the answer is 42)
Thanks a lot!
Felipe Pedrosofacebook.com/felipe.a.pedroso
@felipeapedroso