mobile operating systems - application benchmarking

Mobile Operating Systems: Android

Nicos Demetriou

Operating Systems for New Architectures Fall Semester 2012

Outline

• Application Development

• Experiments

– Emulator, VM, Real device

– Metrics

– Applications

– Results

• Conclusions

2

Android OS - Application development

• Tools needed: – Android SDK – Java JDK, Eclipse – Android Development Tools plugin

• Application components: – Activities: Interactive Tasks – Services: Non-interactive Tasks – Content providers: Database Server – Broadcast receivers: Capture event responses – Intents: Component activation mechanism

• Device Emulator inside Android SDK

3


4


• Graphical User Interface

– Layouts and Views (Widgets)

– Only the main thread can directly call View methods

– Call-back mechanism for other threads to command the execution of a View method in the main thread

• Manifest File (AndroidManifest.xml)

– Describes the application and components used

5

Android OS - Application development <manifest xmlns:android="http://schemas.android.com/apk/res/android"

package="com.example.android.notepad" >

<application android:icon="@drawable/app_notes“ android:label="@string/app_name" >

<provider android:name="NotePadProvider"

android:authorities="com.example.notepad.provider.NotePad" />

<activity android:name="NoteEditor"

android:theme="@android:style/Theme.Light"

android:configChanges="keyboardHidden|orientation">

<intent-filter android:label="@string/resolve_edit">

<action android:name="android.intent.action.VIEW" />

<action android:name="android.intent.action.EDIT" />

<action android:name="com.android.notes.action.EDIT_NOTE" />

<category android:name="android.intent.category.DEFAULT" />

<data android:mimeType="vnd.android.cursor.item/vnd.google.note" />

</intent-filter>

<intent-filter>

<action android:name="android.intent.action.INSERT" />

<category android:name="android.intent.category.DEFAULT" />

<data android:mimeType=“vnd.android.cursor.dir/vnd.google.note" />

</intent-filter>

</activity>

</application>

<uses-sdk android:minSdkVersion="3" android:targetSdkVersion="4"/>

</manifest>

6


• Bin/

– classes/ holds the compiled Java classes

– classes.dex holds the executable created from the compiled Java classes

– nameapp.ap_ holds your application’s resources, packaged as a ZIP file

– nameapp.apk is the actual Android application (where nameapp is the name of your application)

7


• src: It contains all source code file (.java file) of android application

• res/

– drawable: it contains images or pictures

– layout: contains XML files that contain the UI layout used in Project or view window of an application

– values: Arrays, colors, dimensions, strings, and styles are declared in XML files

– raw: it contains non-complied file (e.g. Audio file .mp3, video file .mpg)

8


• assets: External content reference

• libs: Holds any third-party Java JARs your application requires

• gen:

– The R.java file is an index (id) into all the resources defined in the file

– This class files are used in source code file or it gives for reference/location user interface object that is used in the source code

– Automatically generated files

9


10


• Deploying an Android application to apk file

11

Focus on • Testing applications on:

– SDK Emulator

– Real device

– Virtual Machine running Android

• Share experience

• Compare metrics: – CPU usage

– Memory

• Using low level commands (adb)

12

Experimenting – SDK Emulator

• Supported in all operating systems

• Android Emulator

• Supports various versions of Android

• Functionality of real device (buttons, keyboard)

• Android Virtual Device manager – Customize device

• Use shell commands through terminal

• Used along with Eclipse to debug applications

• Device used: Nexus One with Android 2.3/4.0

13


• Android Virtual Device manager

14


• How to execute it

1. Run as an Android Application in Eclipse

2. AVD Manager

3. Through terminal: ..\Android\android-sdk\platform-tools>adb devices

..\Android\android-sdk\tools>emulator -avd AndroidEmulator

• Basic Commands

a) Load .apk files in android-sdk\platform-tools b) ..\Android\android-sdk\platform-tools>adb install [apk]

c) adb shell

d) adb pull <file>

e) adb push <file>

15


16

Experimenting – Virtual Machine

• VMWare Player with Android 2.2

• Android OS Image free for various hardware – http://www.android-x86.org/download

• Live CD or installation on disk

• Minor changes to add network support

• Little complicate to add USB flash as SDCard

• Install applications from SDCard or internet

• Customizable hardware

17

Experimenting – Virtual Machine

18

Experimenting – Real Mobile Device

• Model: Huawei Sonic U8650

• Android v2.3.3 (Gingerbread)

• Install applications from Google Play or SDCard

• More stable

19

Experimenting – Real Mobile Device

20

Experiments – Specifications

HUAWEI Emulator VM CPU 600MHz (ARMv6) 204MHz 1,60GHz

Memory 177MB 504MB 882MB Disk 160MB 197MB 2GB

Display Resolution 320x480 480x720 800x600 Android OS v2.3 v2.3/4.0 v2.2

21

Experiments - Metrics

• CPU

• Memory

• Filesystem / IO

• Graphics

• Network utilization

• Evaluation

– Benchmark Applications

22

Experiments - Metrics

• Benchmark Applications

23

Application CPU Memory I/O Graphics Network

Quadrant Standard Edition x x x x

Linpack for Android x

RealPi BenchMark x

BenchMark x x x x

0xBench x x x x x

MemBench x

An3DBench x

SpeedTest x

Experiments - Quadrant Standard Edition

• CPU, I/O, 3D graphics Benchmark

• Compares your device with other brand devices

24

Experiments - Quadrant Standard Edition

25

0

2000

4000

6000

8000

10000

12000

CPU Memory IO Graphics

Quadrant Standard Edition

VM

Emulator

Emulator 4.0

Real Device

Experiments – Linpack for Android

26

• CPU benchmark

• Measure of system’s floating point computing power

– N x N linear equations Ax = b

• Reflection of the Android Dalvik

• Results in MFLOPS

Experiments – Linpack for Android

27

0

10

20

30

40

50

60

70

80

MFLOPS Time (s) Norm Res

Single Thread

VM

Emulator

Android 4.0

Real Device

0

20

40

60

80

100

120

140

160

180

MFLOPS Time (s) Norm Res

MultiThread

VM

Emulator

Android 4.0

Real Device

Experiments – RealPi Benchmark

• CPU and memory performance

• Calculates the value of Pi to many decimal places

• AGM + FFT formula (Arithmetic Geometric Mean

28

Experiments – RealPi Benchmark

29

0,00

500,00

1000,00

1500,00

2000,00

2500,00

3000,00

3500,00

Time for 1000000 digits (s)

RealPi Benchmark

VM

Emulator

Android 4.0

Real Device

Experiments - Benchmark

• 2D Graphic performance

• CPU performance

• Memory performance

• File system performance

30

Experiments - Benchmark

31

0,00

500,00

1000,00

1500,00

2000,00

2500,00

3000,00

3500,00

Totalgraphics

score

Total CPUscore

Totalmemory

score

Total filesystemscore

Benchmark

VM

Emulator

Android 4.0

Real Device

Experiments – 0xBench

• Comprehensive benchmark suite

• C library and system call – LinPack

– SciMark2

• OpenGL

• 2D Canvas

• Garbage collection in Dalvik

• JavaScript engine

• Open source

32

Experiments – 0xBench

33

0

10

20

30

40

50

60

VM

Emulator

Android 4.0

Real Device

0

5000

10000

15000

20000

25000

30000

35000

40000

45000

GarbageCollection

VM

Emulator

Android 4.0

Real Device

Experiments - MemBench

• Simple Memory Benchmark

• Integer array copy and sum

• Runs 4 times

34

Experiments - MemBench

35

0

10

20

30

40

50

Copy MB/sec Add MB/sec

MemBench

VM

Emulator

Emulator v4.0

Real Device

Experiments - An3DBench

• Based on jPCT 3d engine

• Runs 7 tests from fill rate to complex scenes

36

Experiments - An3DBench

37

0

10

20

30

40

50

60

70

High ObjectCount (fps)

MultipleLihgt (fps)

HighPolygon

count (fps)

Keyframeanimation

(fps)

Game level(fps)

An3DBench

VM

Emulator

Emulator v4.0

Real Device

0

500

1000

1500

2000

2500

3000

3500

4000

4500

5000

Total

An3DBench

VM

Emulator

Emulator v4.0

Real Device

Experiments - SpeedTest

• Discover your Download, Upload, and Ping

• Real-time graphs show connection consistency

38

Experiments - SpeedTest

39

0

1000

2000

3000

4000

5000

6000

7000

8000

9000

Ping (ms) Download (kbps) Upload (kbps)

SpeedTest

VM

Emulator

Emulator v4.0

Real Device

Other benchmark applications for Android

40

Application CPU Memory I/O Graphics Network

AnTuTu Benchmark x x x x x

SmartBench 2012 x x x x

NBench x x

CPUBenchmark x

NenaMark x

BrowserMark x

Conclusions

• Android Application Development A bit tricky

• CPU is the major factor

• Newer versions of Android perform better

• Real devices are more stable

• Some applications are not supported by VM and Emulator

• But…good for testing No harm to OS

41

Other options – Blue Stacks

42

References

• Developing Applications – http://developer.android.com/tools/building/index.html

• Virtual Machine / Emulator – http://forum.xda-developers.com/showthread.php?t=1474956 – http://developer.android.com/tools/devices/emulator.html – http://tutorialfor-android.blogspot.com.es/2012/07/how-to-download-apk-from-google-play-

to.html – http://www.vladan.fr/how-to-install-android-in-vmware-workstation/ – http://quinxy.com/2011/03/12/running-android-2-2-in-vmware-in-less-than-5-minutes/

• Experimenting – http://www.phonearena.com/news/10-useful-Android-benchmark-apps_id26876 – http://www.techhive.com/article/255977/how_to_benchmark_your_android_device.html – http://www.howtogeek.com/111594/how-to-benchmark-your-android-device-5-free-apps/ – http://code.google.com/p/0xbench/

• Book & paper – Beginning Android Application Development – Wei-Meng Lee – Evaluating Android OS for Embedded Real-time Systems - Cl´audio Maia, Lu´ıs Nogueira, Lu´ıs

Miguel Pinho

43

Mobile Operating Systems: Android

Nicos Demetriou

Operating Systems for New Architectures Fall Semester 2012

mobile operating systems - application benchmarking

Technology