iphone lecture

7/29/2019 iPhone Lecture

1/63

iPhone ProgrammingCS4347 Sound and Music Computing

Zhou Yinsheng ([email protected])

Jan 22nd, 2011

Total 67 Pages 1

Acknowledgement:

Some materials are borrowed from the course of iPhone ApplicationDevelopment in Stanford University.


2/63

Tools to create Apps in iPhone

Total 67 Pages 2

Xcode Interface Builder

Instruments

Simulator/device

Mac


3/63

Sound and Music Computing

We use iPhone to make music!

Mobile Music Group

MOGFUN

MOGCLASS

MOGHEALTH

Total 67 Pages 3


4/63

MOGCLASS

MOGCLASS DEMO

Total 67 Pages 4


5/63

MOGCLASS

Do you want to create your own iPhoneApp or develop novel user interface forMOGCLASS?

Total 67 Pages 5


6/63

You need to know

Language: Objective-C and C/C++

Objective Oriented Programming

Programming and testing in Xcode

Digital Signal Processing basics

Assembly Code (ARM)

Computer Network

Etc

Total 67 Pages 6


7/63

Today we will cover

Objective-C basics Building an Application

Sensor programming (accelerometer, GPS,

microphone) Audio Libraries

There are a lot of materials about iphoneonline. What we will cover today is moreoriented for your final project.

Total 67 Pages 7


8/63

How to learn iPhone Programming

Recommended Book: None! Well useApple documentation

iOS Developer Center:http://developer.apple.com/devcenter/ios/index.action

Download the sample code and learn.

Write the code by yourself Trial and

error.

Total 67 Pages 8
http://developer.apple.com/devcenter/ios/index.actionhttp://developer.apple.com/devcenter/ios/index.action


9/63

iPhone SDK TechnologyArchitecture

Multi-Touch Events Alerts

Multi-Touch Controls Web View

Accelerometer People Picker

View Hierarchy Image Picker

Localization Camera

CocoaTouch

Core Audio JPG, PNG, TIFF

OpenAL PDF Audio Mixing Quartz (2D)

Audio Recording Core Animation

Video Playback OpenGL ES

Media

Collections Core Location

Address Book Net Services

Networking Threading

File Access Preference SQLite URL utilities

Core

Services OS X Kernel Power Mgmt

Mach 3.0 Keychain

BSD Certificates

Sockets File System

Security Bonjour

Core OS

Total 67 Pages 9


10/63

Outlines

Objective-C basics

Building an Application


microphone)

Audio Libraries

Total 67 Pages 10


11/63

OOP Vocabulary

Class: defines the grouping of data andcode, the type of an object.

Instance: a specific allocation of a class.

Method: a function that an object knowshow to perform.

Instance Variable (or ivar): a specific

piece of data belonging to an object.

Total 67 Pages 11


12/63

OOP Vocabulary

Encapsulation Keep implementing private and separate from

interface

Polymorphism Different objects, same interface

Inheritance

Hierarchical organization, share code,customize or extend behaviors

Total 67 Pages 12


13/63

Objective-C

Strict superset of C Mix C with ObjC Or even C++ with ObjC (usually referred to as

ObjC++)

A very simple language, but some newsyntax Single inheritance, classes inherit from one

and only one superclass

Protocols define behavior that cross classes Dynamic runtime Loosely typed, if youd like

Total 67 Pages 13


14/63

Class and Instance Methods

Instance respond to instance methods-(id)init;

-(float)height;

-(void)walk;

Classes respond to class methods+(id)alloc;

+(id)person;

+(Person*)sharedPerson;

Total 67 Pages 14


15/63

Message Syntax

[receiver message];

[receiver message:argument];

[receiver message:arg1 andArg: arg2]

Total 67 Pages 15


16/63

Terminology

Message expression[receiver method:argument]

Message

[receiver method:argument]

Selector[receiver method:argument]

Method

The code selected by a message.

Total 67 Pages 16


17/63

Dot Syntax

Objective-C 2.0 introduced dot syntax Convenient shorthand for invoking accessor

methodsfloat height = [person height];

float height = person.height;

[person setHeight: newHeight];

person.height = newHeight;

Follows the dots[[person child] setHeight:newHeight];

//exact the same as

person.child.height = newHeight;

Total 67 Pages 17


18/63

Dynamic and static typing

Dynamically-typed objectid anObject

just id

Not id * (unless you really, really mean it)

Statically-typed objectPerson *anObject

Objective-C provides compile-time, not

runtime, type checking

Objective-C always uses dynamic binding

Total 67 Pages 18


19/63

Selectors identify methods by name

A selector has type SELSEL action = [button action];

[button setAction: @selector(start:)];

Conceptually similar to function pointer Selectors include the name and all colons,

for example:-(void) setName:(NSString*)name age:(int)age;

would have a selector:SEL sel = @selector(setName:age:);

Total 67 Pages 19


20/63

Working with selectors

You can determine if an object responds to agiven selectorid obj;

SEL sel = @selector(start:)

if ([obj respondsToSelector:sel]){[obj performSelector: sel withObject:self];

}

This sort of introspection and dynamic

messaging underlies many Cocoa designpatterns-(void)setTarget:(id)target;

-(void)setAction:(SEL)action;

Total 67 Pages 20


21/63

Working with Classes

You can ask an object about its classClass myClass = [myObject class];

NSLog(@"My class is %@", [myObject className]);

Testing for general class membership (subclasses

included):if ([myObject isKindOfClass:[UIControl class]]) {// something

}

Testing for specific class membership (subclasses

excluded):if ([myObject isMemberOfClass:[NSString class]]) {

// something string specific

}

Total 67 Pages 21


22/63

Working with Objects

Identity v.s. Equality Identitytesting equality of the pointer

valuesif (object1 == object2) {

NSLog(@"Same exact object instance");

}

Equalitytesting object attributesif ([object1 isEqual: object2]) {

NSLog(@"Logically equivalent, but may

be different object instances");

}

Total 67 Pages 22


23/63

-description

NSObject implements -description- (NSString *)description;

Objects represented in format strings using %@

When an object appears in a format string, it is

asked for its description[NSString stringWithFormat: @The answer is: %@,

myObject];

You can log an objects description with:

NSLog([anObject description]);

Your custom subclasses can override descriptionto return more specific information

Total 67 Pages 23


24/63

Foundation Framework

Foundation Classes NSObject String

NSString / NSMutableString

Collection NSArray / NSMutableArray

NSDictionary / NSMutableDictionary

NSSet / NSMutableSet

NSNumber Others

NSData / NSMutableData

NSDate / NSCalendarDate

Total 67 Pages 24


25/63

More OOP Info

Tons of books and articles on OOP

Objective-C 2.0 Programming Language

Total 67 Pages 25


26/63

Outlines

Objective-C basics

Building an Application


microphone)

Audio Libraries

Total 67 Pages 26


27/63

Anatomy of an Application

Compiled code Your code

Frameworks

Nib files UI elements and other objects

Details about object relations

Resources (images, sounds, strings, etc)

Info.plist file (application configuration)

Total 67 Pages 27


28/63

App Lifecycle

Total 67 Pages 28


29/63

App Lifecycle

Main function

UIApplicationMain which Info.plist tofigure out what nib to load.

MainWindow.xib contains theconnections for our application.

AppDelegate

ViewController.xib

Viewhandle UI events

Total 67 Pages 29


30/63

Model-View-Controller

Total 67 Pages 30


31/63

Communication and MVC

Total 67 Pages 31


32/63

Model

Manages the app data and state Note concerned with UI or presentation

Often persists somewhere

Same model should be reusable,unchanged in different interfaces

Total 67 Pages 32


33/63

View

Present the Model to the user in anappropriate interface

Allows user to manipulate data

Does not store any data (except to cache state)

Easily reusable & configurable to display

different data

Total 67 Pages 33


34/63

Controller

Intermediary between Model & View Updates the view when the model

changes

Updates the model when the usermanipulates the view

Typically where the app logic lives

Total 67 Pages 34


35/63

Outlines



microphone, )

Audio Libraries

Total 67 Pages 35


36/63

Accelerometer

What are the accelerometers? Measure changes in force

What are the uses?

Physical Orientation vs. InterfaceOrientation

Ex: Photos & Safari

Total 67 Pages 36


37/63

Orientation-Related Changes

Getting the physical orientation UIDevice class

UIDeviceOrientationDidChangeNotification

Getting the interface orientation UIApplication class

statusBarOrientation property

UIViewController class interfaceOrientation property

Total 67 Pages 37


38/63

ShakeUndo! UIEvent type

@property(readonly) UIEventType type;

@property(readonly) UIEventSubtype

subtype; UIEventTypeMotion

UIEventSubtypeMotionShake

Total 67 Pages 38


39/63

Getting raw Accelerometer Data

3-axis data Configurable update frequency (10-100Hz)

Sample Code (AccelerometerGraph)

Class

UIAccelerometer

UIAcceleration

Protocol

UIAccelerometerDelegate

Total 67 Pages 39


40/63

Getting raw Accelerometer Data

-(void)initAccelerometer{[[UIAccelerometer sharedAccelerometer] setUpdateInterval: (1.0 / 100)];

[[UIAccelerometer sharedAccelerometer] setDelegate: self];

}

-(void)accelerometer: (UIAccelerometer*)accelerometer didAccelerate:

(UIAcceleration*) acceleration {double x, y, z;

x = acceleration.x;

y = acceleration.y;

z = acceleration.z;

//process the data

}

-(void)disconnectAccelerometer{

[[UIAccelerometer sharedAccelerometer] setDelegate: nil];

}

Total 67 Pages 40


41/63

Filtering Accelerometer Data

Low-pass filter Isolates constant acceleration

Used to find the device orientation

High-pass filter Shows instantaneous movement only

Used to identify user-initiated movement

Total 67 Pages 41


42/63


Total 67 Pages 42


43/63


Total 67 Pages 43


44/63

Applying Filters

Simple low-pass filter example

Simple high-pass filter example

Total 67 Pages 44

#define FILTERFACTOR 0.1

Value = (newAcceleration * FILTERFACTOR) + (previousValue *

(1.0FILTERFACTOR));

previousValue = value;

lowpassValue = (newAcceleration * FILTERFACTOR) +

(previousValue * (1.0FILTERFACTOR));previousLowPassValue = lowPassValue;

highPassValue = newAccelerationlowPassValue;


45/63

GPS

Classes CLLocationManager

CLLocation Protocol

CLLocationManagerDelegate

Total 67 Pages 45


46/63

Getting a Location

Starting the location service

Using the event data

Total 67 Pages 46

-(void)initLocationManager{

CLLocationManager* locManager = [[CLLocationManager alloc] init];

locManager.delegate = self;

[locManager startUpdatingLocation];

}

-(void)locationManager: (CLLocationManager*)manager didUpdateToLocation:

(CLLocation*)newLocation fromLocation: (CLLocation*)oldLocation{

NSTimerInterval howRecent = [newLocation.timestamp timeIntervalSinceNow];

if(howRecent < -10) return;

if(newLocation.horizontalAccuracy > 100) return;

//Use the coordinate data.

double lat = newLocation.coordinate.latitude;

double lon = newLocation.coordinate.longitude;

}


47/63

Getting a Heading

Geographic North CLLocationDirection trueHeading

Magnetic North

CLLocationDirection magneticHeading

Total 67 Pages 47

-(void)locationManager: (CLLocationManager *)manager

didUpdateHeading:(CLHeading*)newHeading{

//Use the coordinate data.CLLocationDirection heading = newHeading.trueHeading;

CLLocationDirection magnetic = newHeading.magneticHeading;

}


48/63

Microphone

We will cover it in the audio library

Total 67 Pages 48


49/63

Outlines



microphone) Audio Libraries

Total 67 Pages 49


50/63

Audio Libraries

System Sound APIshort sounds AVAudioPlayerObjC, simple API

Audio Session - Audio Toolbox

Audio Queue - Audio Toolbox

Audio Units

OpenAL

MediaPlayer Framework

Total 67 Pages 50


51/63

AVAudioPlayer

Play longer sounds (> 5 seconds) Locally stored files or in-memory (no network

streaming) Can loop, seek, play, pause

Provides metering Play multiple sounds simultaneously Cocoa-style API

Initialize with file URL or data

Allows for delegate

Supports many more formats Everything the AudioFile API supports

Sample Code: avTouch

Total 67 Pages 51


52/63

AVAudioPlayer

Create from file URL or dataAVAudioPlayer *player;

NSString *path = [[NSBundle mainBundle] pathForResource: ofType:];

NSURL *url = [NSURL fileURLWithPath:path];

player = [[AVAudioPlayer allow] initWithContentsOfURL:url];

[player prepareToPlay];

Simple methods for starting/stoppingIf(!player.playing){

[player play];}else{

[player pause];

}

Total 67 Pages 52


53/63

Audio Sessions

Handles audio behavior at the application,inter-application, and device levels

Ring/Silent switch?

iPod audio continue? Headset plug / unplug?

Phone call coming?

Sample Code: avTouch & aurioTouch

Total 67 Pages 53


54/63

Audio Sessions

Six audio session categories Ambient

Solo Ambient (Default session)

Media playback Recording

Play and record

Offline audio processing

Total 67 Pages 54


55/63

Audio Queue

Audio File Stream Services & Audio QueueServices

Supports wider variety of formats

Finer grained control over playback Streaming audio over network

Cf: AVAudioPlayer(local)

Allows queueing of consecutive buffers for

seamless playback Callback functions for reusing buffers

Sample code: SpeakHere

Total 67 Pages 55


56/63

Audio Units

For serious audio processing Graph-based audio

Rate or format conversion

Real time input/output for recording & playback

Mixing multiple streams

VoIP (Voice over Internet Protocol).

Very, very powerful.

Sample code: aurioTouch

Total 67 Pages 56


57/63

Audio Units

Lowest programming layer in iOS audiostack

Real-time playback of synthesized sounds

Low-latency I/O

Specific audio unit features

Otherwise, look first at the Media Player,

AV Foundation, OpenAL, or AudioToolbox!

Total 67 Pages 57


58/63

Audio Units

Ex: Karaoke app in iPhone real-time input from mic

Real-time output to speaker

Audio Unit provides excellent responsiveness Audio Unit controls audio flow to do pitch

tracking, voice enhancement, iPod

equalization, and etc.

Total 67 Pages 58


59/63

OpenAL

High level, cross-platform API for 3D audio mixing Great for games Mimics OpenGL conventions

Models audio in 3D space Buffers: Container for Audio Sources: 3D point emitting Audio Listener: Position where Sources are heard

Sample code: oalTouch

More information: http://www.openal.org/

Total 67 Pages 59


60/63

MediaPlayer Framework

Tell iPod app to play music Access to entire music library

For playback, not processing

Easy access throughMPMediaPickerController

Deeper access through Query APIs

Sample code: AddMusic

Total 67 Pages 60


61/63

Others

Accelerate Framework C APIs for vector and matrix math, digital

signal processing large number handling, andimage processing

vDSP programming guide

Bonjour and NSStream

The Synthesis ToolKit in C++ (STK) andOSC

Total 67 Pages 61
http://developer.apple.com/library/ios/https://ccrma.stanford.edu/software/stk/http://www.audiomulch.com/~rossb/code/oscpack/http://www.audiomulch.com/~rossb/code/oscpack/https://ccrma.stanford.edu/software/stk/http://developer.apple.com/library/ios/http://developer.apple.com/library/ios/http://developer.apple.com/library/ios/


62/63

More Info for Beginners

iPhone Application Programming(Standford University - iTunes University)

Dan Pilone & Tracey Pilone. Head First

iPhone Development.

Total 67 Pages 62


63/63

Thank you and good luck to your finalprojects!

iphone lecture

Documents