do you hear what i hear? research paper
Post on 19-Feb-2017
129 Views
Preview:
TRANSCRIPT
DO YOU HEAR WHAT I HEAR? FINGERPRINTING SMART DEVICES THROUGH EMBEDDED ACOUSTIC
COMPONENTSCSS2014 -Anupam Das, Nikita Borisov , Matthew Caesar
Presenter: Harshitha Chidananda
Introduction• Mobile devices, including smartphones, PDAs, and tablets, are quickly
becoming widespread in modern society
• 1 in every 5 people in the world owns a smartphone• 1 in every 17 own a tablet
2012 2013 2014 20150
250,000
500,000
750,000
1,000,000
1,250,000
1,500,000
1,750,000
2,000,000
2,250,000PC(Desktop & Notebook) Tablet Mobile Phone
Year
Thou
sand
s of
Uni
ts
Introduction• 75% of the mobile phones are smartphones and highly-featured
phones
• Motion detection• Gesture detection• Audio Genre detection
• Location detection• Interaction with nearby devices
• CompassSensors could be potential source for unique fingerprints.
Main problem• Fingerprinting smart devices can
jeopardize privacy• Software strength is not enough• Hardware level idiosyncrasies are used to
fingerprint physical devices:• Microphones• Speakers
Why Fingerprint Smartphones?• Smartphones can be fingerprinted for:
• Targeted Advertisement• Secondary Authentication factor
Related Work: Fingerprint techniquesSoftware based approaches• Browser based
• Cookies• Analyze Bing and Hotmail• JavaScript and Flash• Device drivers
Hardware based approaches• Radio Transmitter
• Accelerometer
• Microphones tested for fingerprinting(no variations)
Main points and contributions Study the feasibility of using microphones and speakers
embedded in smartphones. Exploit fabrication imperfections to fingerprint smartphones
through playback and recording of audio samples. Explore different acoustic features and analyze their ability to
successfully fingerprint smartphones. Examines the effect of background noise and other variables on
fingerprinting accuracy
Source of fingerprinting• Microphone• Speaker
1 Storage : modify/delete USB storage and SD card contents2 Network communication : full Internet access3 Network communication : view network state4 System tools : prevent device from sleeping5 Phone calls : read phone state and identity6 Hardware controls : control vibrator7 System tools : automatically start at boot8 Network communication : view Wi−Fi state9 Your location : fine (GPS) location
10 Your location : coarse (network−based) location11 System tools : retrieve running applications12 Your personal information : read contact data13 Your messages : read SMS or MMS14 Your messages : receive SMS15 Hardware controls : take pictures and videos16 Hardware controls : record audio17 System tools : modify global system settings
Top 17 permissions out of 173 total permissions
• Scenario 2: Stealthy App
App
1.Play audio
2. Audio signal
3.Record
audio4. Extract Fingerprint
Need access to only:1. Microphone2. Internet
connection
• Scenario 1: External attacker locally present1. Attacker records audio signal from distance2. Create a fingerprint of the recorded audio and link the fingerprint to a unique smartphone
Requires: microphones
Real World Apps
CHALLENGES• Determining acoustic features and audio analysis techniques are
effective in identifying unique signatures of devices.• Audio properties that could be used includes:
• Spectral entropy• Zero crossing
• Analysis algorithms that could be used are:• Principle component analysis• Linear discriminant analysis• Feature selection
OverviewKey observation:• Imperfections in smart device hardware induce unique signatures
on received and transmitted audio streams• These unique signatures are used to fingerprint device.
3 finger printing scenarios
Speaker Microphone Speaker + Microphone
Microphones• Micro Electro Mechanical System(MEMS)• Noise and echo canceling capabilities
Sound Wave
Mechanical Energy
Capacitive Change
Voltage Change
Microphone• The sensitivity of the microphone depends on:
• How well the diaphragm deflects to acoustic pressure• Gap between back-plate and flexible diaphragm
• Imperfections can arise due to: • Slight variations in the chemical composition of components from one
batch to the next • Wear and tear in the manufacturing machines • Changes in temperature and humidity
Speaker• Sound waves are produced whenever electrical current flows
through the voice coil.• Each speaker component can introduce into the generated sound.
Electrical current
Magnetic field around voice coil Mechanical
EnergySound Wave
Fingerprinting Microphones
•Convince the user to install an app on their phone•Observer inputs from the device’s microphone
Fingerprinting Speakers
This scenario is suited for the case where the attacker has deployed nearby microphones to capture audio signal (e.g., ringtone) from user device.
Fingerprinting both speakers and Microphones
The attacker could potentially play a theme song at the start of the game and at the same time make a recording of the audio clip
ApproachListener module• Responsible for receiving and recording
audioAnalyzer module• Identify device signatures
Fingerprint
Mechanism:1)Extract auditory fingerprints2)Efficiently search for matching fingerprints
Acoustic Features
Classification Algorithms• Training step
• Collect a no. of observations from a set of devices• Data Point
• Each observation corresponds to a set of featuresApproaches• K-nearest neighbors
• Associates an incoming data point with the device corresponding to the nearest “learned” data points
• Gaussian mixture models• Computes a probability distribution for each device and determines the
maximal likely association
Feature Selection• Are there any dominant features?• Identifying the dominant feature set benefits us in two ways:
• Less computation • Improve accuracy as there might be conflicting features
Feature Reduction (Dimensionality reduction)
Feature Extraction[new features=function(old
features)]e.g., PCA, LDA
Feature Selection[subset of old features]
EVALUATION
Setup• 266 square foot (14’x19’) office room• Nine-foot dropped ceilings• The room also receives a minimal amount
of ambient noise from • Air conditioning• Desktop computers• florescent lighting.
• Laptop(ACER Aspire 5745 ) is used to emulate the attacker
Experimental SetupInvestigate:• Different maker-model• Same maker-model• Combination Duration of the audio clips varies from 3 to 10 seconds
Evaluation Metrics• 2 classification algorithms:• K-NN(k nearest neighbors)• GMM(Gaussian Mixture Model)
TP: True PositiveFP: False PositiveFN: False Negative
Evaluation Algorithms
Dataset• 50% Training, 50% testing• Varying classification
algorithms from 1 to 5
DIFFERENT MAKER AND MODEL
Feature Exploration
MFCCs
• Mel-Frequency Cepstral Coefficients of the same audio sample take from:• 3 different handsets • Same vendor
• Coefficients vary significantly• Exploit this feature.
Fingerprinting using Microphone
• F1-score of 97%• Smartphones can be
fingerprinted using microphones
Fingerprinting using Microphone and Speaker
• Android App only• F1-score of 100%• Malicious app having access to
microphone and speaker can successfully fingerprint smartphones.
Fingerprinting using Speaker
• F1-score of 100%• Smartphones can be
fingerprinted using Speaker
Instrument Human Speech Song0123456789
10
0 0 00
2.59999999999999
00 0 0
Speaker Microphone
Genre
1 - F
1_sc
ore
(%)
So we can accurately distinguish smartphones of different make and model.
Different Make & Model SetsMaker Model #Apple iPhone 5 1
HTC Nexus One
14
Samsung
Nexus S 8Galaxy
S33
Galaxy S4
10
Motorola
Droid A855
15
Sony Ericsson
W518 1
Total 52
SAME MAKER AND MODEL
Motorola Droid A855 handsets (15)
Feature Exploration
MFCCs are the dominant features for all categories of audio excerpt.
Fingerprinting using Microphone
• F1-score of 95%• Feasible option
Fingerprinting using Microphone and Speaker
• Android App only• F1-score of 100%• App developed by authors - Best
Fingerprinting using Speaker
• F1-score of 94%• Viable Option
Instrument Human Speech Song0123456789
10
1.7 1.7
5.54.7
0
3.90000000000001
0 0 0
Speaker Microphone
Genre
1 - F
1_sc
ore
(%)
Fingerprinting both the speaker and microphone seems to provide better results.
Same Make & Model SetsMaker Model #Apple iPhone 5 1
HTC Nexus One
14
Samsung
Nexus S 8Galaxy
S33
Galaxy S4
10
Motorola
Droid A855
15
Sony Ericsson
W518 1
Total 52
COMBINATIONAL: MAKER AND MODEL
50 Android Phones
Feature Exploration
MFCCs are the dominant features for all categories of audio excerpt.
Fingerprinting using Microphone and Speaker
• Android App only• F1-score of 98%• Malicious app having access
to microphone and speaker can successfully fingerprint smartphones
Instrument Human Speech Song0123456789
10
1.7 0.700000000000003
0
Genre
1- F
1_sc
ore
(%)
Maker Model #Apple iPhone 5 1
HTC Nexus One
14
Samsung
Nexus S 8Galaxy
S33
Galaxy S4
10
Motorola
Droid A855
15
Sony Ericsson
W518 1
Total 52
Heterogeneous Smartphones
SENSITIVITY ANALYSIS
Sensitivity AnalysisImpact of Sampling Rate• Record Ringtone• 3 Different frequencies• As Sampling frequency
decreases, precision/recall also decreases.
• Higher sampling frequency includes more fine-tuned information
Sensitivity AnalysisImpact of Varying Training Size• Expected results• Performance increases as
training size increases• 3 samples per class
• F1-score of 90%• Too many training samples are
not required.
Sensitivity AnalysisImpact of Varying Distance between Speaker and Recorder• As distance increases, F1-score
decreases.• Works only up to a certain
distance0.1 1 2 3 4 50
102030405060708090
100
k-NN GMM
Distance (in meters)
F1 S
core
Sensitivity AnalysisImpact of Ambient Background Noise• Crowded environment• F1-score of 91%
STRENGTHS AND WEAKNESSESSTRENGTHS• Requires small amount of data• Very Good evaluation• Good results• Diverse
WEAKNESS• Repetitive• Unstructured
OPEN ISSUES• Privacy concerns in smartphones• Consequences of fingerprinting smart devices through acoustic
channels.• More experiments on Apple• Find out if this is applicable to laptops as well
MY THOUGHTS AND CONCLUSIONMy thoughts• Easy to understand• Extensive evaluation• Good illustrations• Good graphical representation• Good results• The application developed by the
authors results in good performance
ConclusionIt is feasible to fingerprint smart-devices through on-board acoustic components like
• microphones • speakers
Able to accurately attribute ~98% of all recorded audio excerpts from 50 different Android devices.
THANK YOUQuestions?
top related