E0-245: ASP
Lecture 19: Multimedia and Android Components
Dipanjan Gope
2
Dipanjan Gope
• Location Sensors - Theory of location sensing
- Package android.location
• Physical Sensors - Sensor Manager
- Accelerometer
- Gyroscope
- Magnetometer
- Sensor fusion
• NFC
• Multimedia - Camera
- Microphone
Module 2: Android Sensor Applications
3
Dipanjan Gope
Activity
Views
Intent
ContentProvider
BroadcastReceiver
Service
Coverage
4
Dipanjan Gope
• Greg Milette, Adam Stroud: Professional Android Sensor Programing, 2012, Wiley India
• Reto Meier: Professional Android 4 application development. John Wiley & Sons, 2012.
References
5
Dipanjan Gope
Content Provider
https://bestandroidtraining.wordpress.com/2013/03/30/android-project-structure/
6
Dipanjan Gope
android.database.sqlite
class SQLiteDatabase - insert - query - delete
- ….
abstract class SQLiteOpenHelper - onCreate
- onUpgrade
class ContentValues class Cursor
7
Dipanjan Gope
Quick Guide Database Usage
-http://www.androidhive.info/2011/11/android-sqlite-database-tutorial/
8
Dipanjan Gope
Step 0: Database content
….
9
Dipanjan Gope
Step 1: Schema and SQLiteOpenHelper
10
Dipanjan Gope
Step 2: Add to DB
11
Dipanjan Gope
Step 3: Read from DB
12
Dipanjan Gope
Step 4: Update DB
13
Dipanjan Gope
Step 5: Delete DB
14
Dipanjan Gope
DB Flow
15
Dipanjan Gope
• Wrapper over DB for decoupling data and UI apps
ContentProvider
Interface ContentProvider:
16
Dipanjan Gope
• Design DB, schemas, URIs
• Extend ContentProvider
• Implement interface functions
• Register in manifest
Steps: ContentProvider
17
Dipanjan Gope
ContentProvider: Demo
18
Dipanjan Gope
• Permissions .. Permissions .. Permissions
• Same query
Usage From Another App
19
Dipanjan Gope
Permissions: CP Creator App
1
2
3
20
Dipanjan Gope
Permissions: CP Access App
1
2
21
Dipanjan Gope
Native Android Content Providers
Media Store
Browser
Contacts
Calendar
Call Log
22
Dipanjan Gope
• Application continues to run when UI is invisible
• Has higher priority than inactive Activities - lesser chance of getting killed when system requires resources
• Can be configured to restart when resources are re-available
• Service priority can be raised to foreground Activity
Service
23
Dipanjan Gope
Service
http://developer.android.com/guide/components/services.html
24
Dipanjan Gope
Service Class
Ref: [2]
25
Dipanjan Gope
Start/Stop a Service
Ref: [2]
26
Dipanjan Gope
Multimedia
27
Dipanjan Gope
• Camera - photos and videos
- other applications: barcode, face recognition, pulse rate detection
• Microphone - record audio
- other applications: Text-to-speech, voice activation
• Speaker - play audio
- other applications: audiogram
Multimedia: Components
28
Dipanjan Gope
Audio: Speaker
29
Dipanjan Gope
Speaker Application: Audiogram
30
Dipanjan Gope
Audio: Microphone
31
Dipanjan Gope
Heart Sounds: Phonocardiography
After basic Noise Filtering
Recorded Heart Sound:
S1 S2
Mobile Phone Sensor: Microphone
32
Dipanjan Gope
Heart Rate Detection
0 0.5 1 1.5 2 2.5
x 105
-0.5
0
0.5
0 0.5 1 1.5 2 2.5
x 105
0
0.005
0.01
0 20 40 60 80 100 120 140 160 180 2000
200
400
80.7495 Hz
33
Dipanjan Gope
Heart Sounds: Phonocardiography
S3
S4
Healthy Heart Sound
Third Heart Sound
Fourth Heart Sound
Mp3 from Littmann Heart Sounds
34
Dipanjan Gope
Video/Image: Camera
Ref: [2]
35
Pulse Detection from Mobile Phone
Video Recording Face Recognition RGB Components
ICA Analysis FFT Pulse From Spectrum
Pre-processing - Detrend - Zscore
Post-processing - Smooth - Bandpass Filter - Spline
M.Z. Poh, D.J. McDuff, R.W. Picard, Opt Express. 18(10) pp. 10762–10774, 2010.
36
Pulse Detection from Mobile Phone
(a) RGB components extracted from 1280x720 resolution video capture from a Samsung Galaxy S3 mobile phone at 30 frames per second over a period of 30 seconds. (b) The FFT of (a) lacks prominent peaks in spectrum. (c) The 3 components extracted from the RGB traces using JADE ICA analysis algorithm (d) The FFT of (c) contains distinct peaks in the spectrum from which pulse-rate is obtained
37
Accuracy Worsens With Distance
0.4 0.6 0.8 1 1.2 1.4 1.6 1.8
x 104
-2.5
-2
-1.5
-1
-0.5
0
0.5
Total Pixel Size of Detected Face
log
10
(Err
or)
WebCam Results
MTLB_dtrnd_schm1
MTLB_dtrnd_schm2
MTLB_dtrnd_schm3
TRVN_dtrnd_schm1
TRVN_dtrnd_schm2
TRVN_dtrnd_schm3
38
Dipanjan Gope
Activity
Views
Intent
ContentProvider
BroadcastReceiver
Service
Coverage
39
Dates
• Final Exam: 30th April 10-1pm
• Final Project submission: same day
- Codes or apk file