• Next week (July 21)– No class

– No office hour

Problem

• Multiple tasks for computer– Draw & display images on screen

– Check keyboard & mouse input

– Send & receive data on network

– Read & write files to disk

– Perform useful computation (editor, browser, game)

• How does computer do everything at once?– Multitasking

– Multiprocessing

Multitasking (Time-Sharing)

• Approach– Computer does some work on a task

– Computer then quickly switch to next task

– Tasks managed by operating system (scheduler)

• Computer seems to work on tasks concurrently• Can improve performance by reducing waiting

Multitasking Can Aid Performance• Single task

• Two tasks

Multiprocessing and Multithreading

• Approach – Multiple processing units (multiprocessor)

– Computer works on several tasks in parallel

– Performance can be improved

4096 processor Cray X1

32 processor Pentium Xeon

Dual-core AMD Athlon X2

Perform Multiple Tasks Using…

1. Process– Definition – executable program loaded in memory

– Has own address space• Variables & data structures (in memory)

– Each process may execute a different program

– Communicate via operating system, files, network

– May contain multiple threads

Perform Multiple Tasks Using…

2. Thread– Definition – sequentially executed stream of instructions

– Shares address space with other threads

– Has own execution context • Program counter, call stack (local variables)

– Communicate via shared access to data

– Multiple threads in process execute same program

– Also known as “lightweight process”

Motivation for Multithreading

1. Captures logical structure of problem– May have concurrent interacting components

– Can handle each component using separate thread

– Simplifies programming for problem

• Example

Web Server uses threads to handle …

Multiple simultaneous web browser requests

Motivation for Multithreading

2. Better utilize hardware resources– When a thread is delayed, compute other threads

– Given extra hardware, compute threads in parallel

– Reduce overall execution time

• Example

Multiple simultaneous web browser requests…

Handled faster by multiple web servers

Multithreading Overview

• Motivation & background • Threads

– Creating Java threads

– Thread states

– Scheduling

• Synchronization– Data races

– Locks

– Wait / Notify

Programming with Threads

• Concurrent programming– Writing programs divided into independent tasks

– Tasks may be executed in parallel on multiprocessors

• Multithreading– Executing program with multiple threads in parallel

– Special form of multiprocessing

Creating Threads in Java

• Two approaches– Thread subclass

public class Thread extends Object { … }

– Implement Runnable interfacepublic interface Runnable {

public void run(); // work thread

}

Thread Class

public class Thread extends Object implements Runnable {

public Thread(); public Thread(String name); // Thread name public Thread(Runnable R); // Thread R.run() public Thread(Runnable R, String name);

public void run(); // if no R, work for thread public void start(); // begin thread execution ...}

More Thread Class Methods

public class Thread extends Object { … public static Thread currentThread() public String getName() public void interrupt() public boolean isAlive() public void join() public void setDaemon() public void setName() public void setPriority() public static void sleep() public static void yield()}

Creating Threads in Java

1. Thread subclass– Creating threads: extend Thread class and override the run method

• Examplepublic class MyThread extends Thread { public void run() { … // work for thread }}MyThread T = new MyThread () ; // create threadT.start(); // begin running thread… // thread executing in parallel

Creating Threads in Java

2. Implement Runnable interface– Create object implementing Runnable interface

– Pass it to Thread object via Thread constructor

• Examplepublic class MyRunnable implements Runnable {

public void run() {

… // work for thread

}

}

Thread T = new Thread(new MyRunnable ); // create threadT.start(); // begin running thread… // thread executing in parallel

Creating Threads in Java

• Note– Thread starts executing only if start() is called

– Runnable is interface• So it can be multiply inherited

• Required for multithreading in applets

Threads – Thread States

• Java thread can be in one of these states– New – thread allocated & waiting for start()

– Runnable – thread can begin execution

– Running – thread currently executing

– Blocked – thread waiting for event (I/O, etc.)

– Dead – thread finished

• Transitions between states caused by– Invoking methods in class Thread

• new(), start(), yield(), sleep(), wait(), notify()…

– Other (external) events• Scheduler, I/O, returning from run()…

Threads – Thread States

• State diagram

runnable

scheduler

new

dead

running blocked

new start

terminateIO, sleep,wait, join

yield,time slice

notify, notifyAll,IO complete,

sleep expired,join complete

Daemon Threads

• Java threads types– User

– Daemon• Provide general services

• Typically never terminate

• Call setDaemon() before start()

• Program termination1. All user threads finish

2. Daemon threads are terminated by JVM

3. Main program finishes

Terminating Threads

• A thread terminates when its run method terminates

• Do not terminate a thread using the deprecated stop method

– If the thread being stopped was modifying common data, that common data remains in an inconsistent state.

• Instead, notify a thread that it should terminate:

t.interrupt();

• interrupt does not cause the thread to terminate – it sets a boolean variable in the thread data structure

Terminating Threads (2)

• The run method checks occasionally whether it has been interrupted:

– Use the interrupted method

– An interrupted thread should release resources, clean up, and exit:

public void run(){ for (int i = 1;

i <= REPETITIONS && !Thread.interrupted(); i++) {

Do work } Clean up}

Terminating Threads (3)

• Java does not force a thread to terminate when it is interrupted

• It is entirely up to the thread what it does when it is interrupted

• Interrupting is a general mechanism for getting the thread’s attention

Threads – Scheduling

• Scheduler– Determines which runnable threads to run

– Can be based on thread priority

– Part of OS or Java Virtual Machine (JVM)

• Scheduling policy– Nonpreemptive (cooperative) scheduling

– Preemptive scheduling

Threads – Non-preemptive Scheduling

• Threads continue execution until – Thread terminates

– Executes instruction causing wait (e.g., IO)

– Thread volunteering to stop (invoking yield or sleep)

Threads – Preemptive Scheduling

• Threads continue execution until– Same reasons as non-preemptive scheduling

– Preempted by scheduler

Java Thread Example

public class ThreadExample extends Thread { public void run() { for (int i = 0; i < 3; i++) System.out.println(i); try { sleep((int)(Math.random() * 5000)); // 5 secs } catch (InterruptedException e) { } } public static void main(String[] args) { new ThreadExample().start(); new ThreadExample().start(); System.out.println("Done"); }}

Java Thread Example – Output

• Possible outputs– 0,1,2,0,1,2,Done // thread 1, thread 2, main()

– 0,1,2,Done,0,1,2 // thread 1, main(), thread 2

– Done,0,1,2,0,1,2 // main(), thread 1, thread 2

– 0,0,1,1,2,Done,2 // main() & threads interleaved

thread 1: println 0, println 1, println 2

main (): thread 1, thread 2, println Done

thread 2: println 0, println 1, println 2

Data Races

public class DataRace extends Thread { static int x; public void run() { for (int i = 0; i < 1000; i++) { x = x + 1; try{ sleep(1); } catch (InterruptedException e){} x = x – 1; } } public static void main(String[] args) { x = 0; for (int i = 0; i < 10; i++) new DataRace().start(); System.out.println(x); // x not always 0! }}

Thread Scheduling Observations

• Order thread is selected is indeterminate– Depends on scheduler

• Thread can block indefinitely (starvation)– If other threads always execute first

• Thread scheduling may cause data races– Modifying same data from multiple threads

– Result depends on thread execution order

• Synchronization– Control thread execution order

– Eliminate data races

Race Conditions

• When threads share a common object, they can conflict with each other

• Sample program: multiple threads manipulate a bank account

– Create two sets of threads:

• Each thread in the first set repeatedly deposits $100

• Each thread in the second set repeatedly withdraws $100

Sample Program (1)

•run method of DepositRunnable class:

public void run(){ try { for (int i = 1; i <= count; i++) { account.deposit(amount); Thread.sleep(DELAY); } } catch (InterruptedException exception) { }}

• Class WithdrawRunnable is similar – it withdraws money instead

Sample Program (2)

• Create a BankAccount object, where deposit and withdraw methods have been modified to print messages:

public void deposit(double amount){ System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is " + newBalance); balance = newBalance;}

Sample Program (3)

• Normally, the program output looks somewhat like this:

Depositing 100.0, new balance is 100.0Withdrawing 100.0, new balance is 0.0 Depositing 100.0, new balance is 100.0Withdrawing 100.0, new balance is 0.0...Withdrawing 100.0, new balance is 0.0

• The end result should be zero, but sometimes the output is messed up, and sometimes end result is not zero:

Depositing 100.0Withdrawing 100.0, new balance is 100.0, new balance is -100.0

Sample Program (4)

• Scenario to explain problem:

1. A deposit thread executes the lines:

System.out.print("Depositing " + amount);double newBalance = balance + amount;

The balance variable is still 0, and the newBalance local variable is 100

– The deposit thread reaches the end of its time slice and a withdraw thread gains control

– The withdraw thread calls the withdraw method which withdraws $100 from the balance variable; it is now -100

– The withdraw thread goes to sleep

Sample Program (5)

• Scenario to explain problem (cont.):

5. The deposit thread regains control and picks up where it was iterrupted. It now executes:

System.out.println(", new balance is " + newBalance);balance = newBalance;

The balance variable is now 100 instead of 0 because the deposit method used the old balance to compute the value of its local variable newBalance

Race Condition

• Occurs if the effect of multiple threads on shared data depends on the order in which they are scheduled

• It is possible for a thread to reach the end of its time slice in the middle of a statement

• It may evaluate the right-hand side of an equation but not be able to store the result until its next turn:

public void deposit(double amount){ balance = balance + amount; System.out.print("Depositing " + amount + ", new balance is " + balance);}

• Race condition can still occur:

balance = the right-hand-side value

An example with bank accounts

// A bank account has a balance that can be changed by // deposits and withdrawals.

public class BankAccount{ public BankAccount() { balance = 0; }

public void deposit(double amount) { System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is " + newBalance); balance = newBalance; }

public void withdraw(double amount) { System.out.print("Withdrawing " + amount); double newBalance = balance - amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } public double getBalance() { return balance; } private double balance;}

/** A deposit runnable makes periodic deposits to a bank account.*/public class DepositRunnable implements Runnable{ public DepositRunnable(BankAccount anAccount, double anAmount, int aCount) { account = anAccount; amount = anAmount; count = aCount; }

public void run() { try { for (int i = 1; i <= count; i++) { account.deposit(amount); Thread.sleep(DELAY); } } catch (InterruptedException exception) {} }

private static final int DELAY = 1; private BankAccount account; private double amount; private int count;}

/** A withdraw runnable makes periodic withdrawals from a bank account.*/public class WithdrawRunnable implements Runnable{ public WithdrawRunnable(BankAccount anAccount, double anAmount, int aCount) { account = anAccount; amount = anAmount; count = aCount; }

public void run() { try { for (int i = 1; i <= count; i++) { account.withdraw(amount); Thread.sleep(DELAY); } } catch (InterruptedException exception) {} }

private static final int DELAY = 1; private BankAccount account; private double amount; private int count;}

/** This program runs four threads that deposit and withdraw money from the same bank account. */public class BankAccountThreadTester{ public static void main(String[] args) { BankAccount account = new BankAccount(); final double AMOUNT = 100; final int REPETITIONS = 5;

DepositRunnable d1 = new DepositRunnable( account, AMOUNT, REPETITIONS); WithdrawRunnable w1 = new WithdrawRunnable( account, AMOUNT, REPETITIONS); DepositRunnable d2 = new DepositRunnable( account, AMOUNT, REPETITIONS); WithdrawRunnable w2 = new WithdrawRunnable(account, AMOUNT, REPETITIONS);

Thread t1 = new Thread(d1); Thread t2 = new Thread(w1); Thread t3 = new Thread(d2); Thread t4 = new Thread(w2);

t1.start(); t2.start(); t3.start(); t4.start(); }}

Synchronizing Object Access

• Lock object: used to control threads that manipulate shared resources

• In Java library: Lock interface and several classes that implement it

– ReentrantLock: most commonly used lock class

– Locks are a feature of Java version 5.0

– Earlier versions of Java have a lower-level facility for thread synchronization

Locks (Java)

• Only one thread can hold a lock at once– Other threads that try to acquire it block (or become

suspended) until lock becomes available

• Reentrant lock can be reacquired by same thread– As many times as desired

– No other thread may acquire lock until has been released same number of times has been acquired

interface Lock { Void lock(); Void unlock(); …//some other stuff}

Synchronization Using Locks

public class Example extends Thread {

private static int cnt = 0;

private Lock lock;

public Example(){

lock = new ReentrantLock();

}

public void run() {

lock.lock();

int y = cnt;

cnt = y + 1;

lock.unlock();

}

...

}

Creating a lock, for protecting the shared state

Acquires the lock; Only succeeds if not held by another thread

Releases the lock

Using Locks to Synchronize Threads

• Step 1: add a lock to the class that owns the shared resource

• Step 2: surround any code that uses the shared resource by a call to lock ( ) and unlock ( )

public class BankAccount{ public BankAccount() {

balanceChangeLock = new ReentrantLock();// the rest of your code here

} … // more code here private Lock balanceChangeLock;}

balanceChageLock.lock ( );try{ // Code that changes the balance}finally{ balanceChangeLock.unlock ( );}

Use finally block to ensure that the lock is released even if exception happens

finally

• The finally block always executes when the try block exits (whether an exception was thrown or not)try{

// try to execute this statements

}catch(Exception e){

// if an exception was thrown, handle it here

}finally{

// code always executes

}

• Putting cleanup code in a finally block is always a good practice, even when no exceptions are anticipated.

Deadlocks

• Deadlock occurs when a thread acquires a lock and then must wait for another thread to do some work before proceeding, but where the second thread needs the lock to proceed

• Example: Let’s assume we want to prevent a withdrawal if there is not enough balance– In a single threaded world, we might code:

• if (account.getBalance ( ) >= amount) account.withdraw (amount);

– For multi-threading:public void withdraw(double amount){ balanceChangeLock.lock(); try { while (balance < amount) // Wait until the balance becomes //sufficient (i.e., until a deposit is made) } finally{

balanceChangeLock.unlock(); }}

Call sleep() to wait?

Avoiding Deadlocks

• A deadlock occurs if no thread can proceed because each thread is waiting for another to do some work first

• How can we wait for the balance to grow?

• We can’t simply call sleep inside withdraw method; thread will block all other threads that want to use balanceChangeLock

• In particular, no other thread can successfully execute deposit

• Other threads will call deposit, but will be blocked until withdraw exits

• But withdraw doesn’t exit until it has funds available

• DEADLOCK

Condition Objects

• To overcome this problem, use a condition object

• Condition objects allow a thread to temporarily release a lock, and to regain the lock at a later time

• Each condition object belongs to a specific lock object

Waiting and Signaling Using a Condition

• A thread waits on a condition (e.g., balance > 0) and another thread signals when the condition becomes true

• To create a condition– Condition sufficientFundsCondition =

balanceChangeLock.newCondition();

• To start waiting on a condition– sufficientFundsCondition.await();

• To signal that a condition has become true– sufficientFundsCondition.signalAll() or

sufficientFundsCondition.signal();

Waiting and Signaling Using a Condition

• Waiting threads are blocked and will not be considered for execution until the condition is signaled

• The lock must still be released before they can run• Recall: Can also be done with an object’s built-in

lock and condition: wait() to wait and notifyAll() or notify() to signal

The BankAccount class with locks and conditions

import java.util.concurrent.locks.Condition;import java.util.concurrent.locks.Lock;import java.util.concurrent.locks.ReentrantLock;

// A bank account has a balance that can be changed by // deposits and withdrawals.public class BankAccount{ public BankAccount() { balance = 0; balanceChangeLock = new ReentrantLock(); sufficientFundsCondition = balanceChangeLock.newCondition(); }

public void deposit(double amount) { balanceChangeLock.lock(); try { System.out.print("Depositing " + amount); double newBalance = balance + amount; System.out.println(", new balance is " + newBalance); balance = newBalance; sufficientFundsCondition.signalAll(); } finally { balanceChangeLock.unlock(); } }

public void withdraw(double amount) throws InterruptedException { balanceChangeLock.lock(); try { while (balance < amount) sufficientFundsCondition.await(); System.out.print("Withdrawing " + amount); double newBalance = balance - amount; System.out.println(", new balance is " + newBalance); balance = newBalance; } finally { balanceChangeLock.unlock(); } } public double getBalance() { return balance; } private double balance; private Lock balanceChangeLock; private Condition sufficientFundsCondition;}