mole: motion leaks through smartwatch sensors
Post on 21-Jan-2017
424 Views
Preview:
TRANSCRIPT
MoLe: Motion Leaks through Smartwatch Sensors
Master’s course 29th, Park Joon Young
Contents• Attack concept / Contributions • Previous works • First look • System overview / Assumption • Design details / Evaluation • Discussion • Related works • Future works
Attack concept
Accelerometer
Gyroscope
NO PERMISSIONS
Attack concept
Identifying the leakage - key-press detection - handmotion tracking- cross-user data matching- Bayesian inference
Developing the system - Samsung Gear Live smart watch- experimenting with real users- revealing accuracy
Contributions
Previous Works
Key logging based on side-channels
Attacks using sensors on smartphone
Previous Works
Keyboard Acoustic Emanations (2004) - neural network
TouchLogger (2011) - accelerometer
Timing Analysis SSH (2001) - Hidden Markov Model
KeySweeper (2015) - RF signal
Compromising Electromagnetic Emanations (2009) - electromagnetic
ACCessory (2012) - accelerometer
On the Practicality - (2012) - gyroscope, accelerometer
TapPrints (2012) - gyroscope, accelerometer
(sp)iPhone (2011) - accelerometer
Key logging based on side-channels
Attacks using sensors on smartphone
Previous Works
Keyboard Acoustic Emanations (2004) - neural network
TouchLogger (2011) - accelerometer
Timing Analysis SSH (2001) - Hidden Markov Model
KeySweeper (2015) - RF signal
Compromising Electromagnetic Emanations (2009) - electromagnetic
ACCessory (2012) - accelerometer
On the Practicality - (2012) - gyroscope, accelerometer
TapPrints (2012) - gyroscope, accelerometer
(sp)iPhone (2011) - accelerometer
MoLe (2015)- gyroscope, accelerometer
First look
First look
• Tested with computer vision techniques. (NOT accel / gyro data)
• Left hand only
• “F” is home position
X axis displacements
watch’s X axis
time(sec)
First look
First look
System overview
• MoLe app installed on smartwatch
• Sensor data receiving at the server
System overview
System overview
!
"
#$
%&
Assumptions
• One word at a time
• Only on English
• Only on Samsung smart watch (can compute CPC for other model)
• Appropriate typing fingers
Design details
Design details
Keystroke detector
Point cloud fitting
Bayesian inference
Design details
• Z axis of the watch
• FP / FN occurs
• Bagged decision tree
- Keystroke detector -
Bagging decision tree
• Decision tree
• Bootstrap aggregating-> Bagging
• Attempt again and again, average each samples
Design details
• Z axis of the watch
• FP / FN occurs
• Bagged decision tree
- Keystroke detector -
Design details- Keystroke detector -
Pressed-or-not accuracy
Design details- Keystroke detector -
MoLe against Android API
• Find / Remove gravity
• Calculate displacement
• Kalman smoothing
Design details- Keystroke detector -
Design details
Keystroke detector
Point cloud fitting
Bayesian inference
Design details- Point cloud fitting -
Generate convex hulls of CPC / UPC
Calculate centroids
Rotate & Scale
Design details
Keystroke detector
Point cloud fitting
Bayesian inference
Design details- Bayesian inference -
- Bayesian inference -
• : candidate word(dictionary)
• : observation motion data
• : posterior probability
• : probability word W based on the observed motion data
• : prior probability, captures the word’s occurrence frequency
• : probability of the observation
Design details- Bayesian inference -
same for all possible words
assume, equal among words
Key Goal : obtaining high values
Bayesian inference
Design details- Bayesian inference : Step 1 -
Design details- Bayesian inference : Step 1 -
* example * “apple” -> ap, ap, al, ae, pp, pl, pe, pl, pe, le
t(O) h(O) e(X)
t(O) h(X) e(O)
t(X) h(O) e(O)
Design details- Bayesian inference : Step 2 -
• Consecutive characters
• “er”, “re”, “ea”, “fa”
• Treat as one key
🤔
Design details- Bayesian inference : Step 3 -
• 2D displacements
• Point cloud fitting makes better predict
• Gaussian distribution
Design details- Bayesian inference : Step 3 -
• 2D displacements
• Point cloud fitting makes better predict
• Gaussian distribution
Design details- Bayesian inference : Step 3 -
• 2D displacements
• Point cloud fitting makes better predict
• Gaussian distribution
Design details- Bayesian inference : Step 3 -
• 2D displacements
• Point cloud fitting makes better predict
• Gaussian distribution
Design details- Bayesian inference : Step 3 -
• 2D displacements
• Point cloud fitting makes better predict
• Gaussian distribution Probability density of given character
• Detect sequential movements
• Considers previous character
Design details- Bayesian inference : Step 4 -
• Detect sequential movements
• Considers previous character
Design details- Bayesian inference : Step 4 -
• Missing keys from right hand
• Check time interval every possible character-sequence
• Compensates speed bias between attacker and attackee with a factor
Design details- Bayesian inference : Step 5 -
• Missing keys from right hand
• Check time interval every possible character-sequence
• Compensates speed bias between attacker and attackee with a factor
Design details- Bayesian inference : Step 5 -
🤔
Evaluation
Evaluation• Gyroscope readings at 200Hz with timestamps
• 8 subjects, 5 native English speakers, 3 females
• 300 words randomly selected from 5000 most frequently used words
• word-length ranged from 1 to 14
• re-enter if incorrectly typed
• Between each word, hand position initialized on “F” and “J”
• Two attackers, trained Top-500 longest words in the dictionary on same keyboard
Evaluation
30% for 5 possible words
50% for 24 possible words
🤔(1) How well can MoLe guess each word?
Evaluation
Better results
(1) How well can MoLe guess each word?
Evaluation
(2) What factors affect the rank?
Evaluation
(3) Impact of each Bayesian opportunity
Evaluation
(4) Impact of sampling rate
Evaluation
(5) Keyboard variant
Evaluation
(6) Recovery via human observation
Evaluation
(6) Recovery via human observation
are
Discussion
Discussion
Confined to separate words
Applying nature language processing
Typing activity classifier
Conclusion
Identifying the leakage - key-press detection - handmotion tracking- cross-user data matching- Bayesian inference
Developing the system - Samsung Gear Live smart watch- experimenting with real users- revealing accuracy
Conclusion
Sensor data can leak informations
Diminishing the sampling rate of the sensors can alleviate the attack
Wearable devices could be “double edged sword”
Conclusion
Related works
Related works- (Smart)Watch Your Taps -
NHHT HHT
Related works
• Classification algorithms- Simple linear regression - Random forest - K-nearest neighbors
- (Smart)Watch Your Taps -
Related works
• Classification algorithms- Simple linear regression - Random forest - K-nearest neighbors
- (Smart)Watch Your Taps -
Related works- We can track you .. Metro -
• Tracking metro riders using accelerometers on smartphones
• boosted Naive Bayesian (AdaBoost)
• Decision trees (Random forest)
• Naive Bayesian - family of algorithms based on a common principle - a particular feature is independent of any other feature
• AdaBoost(Adaptive Boosting) - machine learning meta-algorithm - can be used in conjunction with many other types of algorithms- ‘weak learners’ can boost classify
• Decision trees (Random forest)
Related works- We can track you .. Metro -
Related works- We can track you .. Metro -
Question & Answer
top related