CS 2200

Presentation 19

Threads

Questions?

Our Road Map

Processor

Networking

Parallel Systems

I/O Subsystem

Memory Hierarchy

Recall• Process

– Program Counter– Registers– Stack– Code (Text)– Data– Page Table– etc.

• Processes must be protected from one another.

Memory

Recall• Context Switching

– Requires considerable work

• What about a single users application?– Is there a way to make it more efficient– In effect, allow the user to have multiple

processes executing in the same space?– Yes, solution: Threads or Multithreading

What is Multithreading?• Technique allowing program to do multiple

tasks– Example: Java GUI's

Java Event Handling

import java.awt.*;public class HelloGUI { public static void main (String[ ] arg) { System.out.println (“About to make GUI”); Frame f = new Frame (“Hello GUIs”); f.setSize( 200, 200 ); f.show(); System.out.println (“Finished making GUI”); }// main}// class HelloGUI

callbackcallback

Our eventhandling

code

The code trappingthis event appears

in the graphics thread

What is Multithreading?• Technique allowing program to do multiple

tasks– Example: Java GUI's

• Is it a new technique?– has existed since the 70’s (concurrent Pascal,

Ada tasks, etc.)

• Why now?– “Time has come for this technology”– Emergence of SMP’s in particular

SMP?• What is an SMP?

– Multiple CPUs in a single box sharing all the resources such as memory and I/O

• Is a dual-processor SMP more cost effective than two uniprocessor boxes?– Yes, (roughly 20% more for a dual processor

SMP compared to a uniprocessor).– Modest speedup for a program on a dual-

processor SMP over a uniprocessor will make it worthwhile.

What is a Thread?• Basic unit of CPU utilization• A lightweight process (LWP)• Consists of

– Program Counter– Register Set– Stack Space

• Shares with peer threads– Code– Data– OS Resources

• Open files• Signals

Threads• Can be context switched more easily

– Registers and PC– Not memory management

• Can run on different processors concurrently in an SMP

• Share CPU in a uniprocessor

• May (Will) require concurrency control programming like mutex locks.

active

Process

Threads in a Uniprocessor?

• Allows concurrency between I/O and user processing even in a uniprocessor box

Threads and OSTraditional OS

• DOS– Memory layout

user

kernel Data

– Protection between user and kernel?

DataProgram

DOS Code

User

Kernel Kernel code and data

– Protection between user and kernel?

Process code and data

PCB

P1

Process code and data

PCB

P2

Threads and OS• Unix - memory layout

Threads and OS• Programs in a traditional OS are single

threaded– One PC per program (process), one stack,

one set of CPU registers– If a process blocks (say disk I/O, network

communication, etc.) then no progress for the program as a whole

MultiThreaded Operating Systems• How widespread is support for threads in

OS?– Digital Unix, Sun Solaris, Win9x, Win NT,

Win2k, Linux, Free BSD, etc

• Process vs. Thread?– In a single threaded program, the state of the

executing program is contained in a process– In a multithreaded program, the state of the

executing program is contained in several ‘concurrent’ threads

What is this?

1. Crack in slide2. Hair stuck on lens3. Symbol for thread4. San Andreas Fault

Process Vs. Thread

user

kernel Kernel code and data

t1

– Computational state (PC, regs, …) for each thread

– How different from process state?

code data code data

t2 t3 t1

PCB PCB

P1 P2

Memory Layout

• Multithreaded program has a per-thread stack

• Heap, static, and code are common to all threads

MT program

stk1 stk2 stk3 ...

Heap

Static

Code

ST program

stack1

Heap

Static

Code

Threads• Threaded code different from non-

threaded?– Protection for data shared among threads

• Mutex

– Synchronization among threads• Way for threads to talk (signal) to one another

– Thread-safe libraries

NOTESstrtok is unsafe for multi-thread applications.strtok_r is MT-Safe and should be used instead.

Typical Operation• Main programs creates (or spawns)

threads

• Threads may – perform one task and die– last for duration of program

• Threads must– be able to synchronize activity– communicate with one another

Programming Support for Threads• Creation

– pthread_create(top-level procedure, args)

• Termination– return to top-level procedure– explicit cancel

• Rendezvous– creator can wait for children

• pthread_join(child_tid)

• Synchronization– pthread_mutex_...– condition variables (pthread_cond_...)

Programming with Threads

• Synchronization– For coordination of the threads

• Communication– For inter-thread sharing of data– Threads can be in different processors– How to achieve sharing in SMP?

• Software: accomplished by keeping all threads in the same address space by the OS

• Hardware: accomplished by hardware shared memory and coherent caches

Synchronization Primitives• lock and unlock

– mutual exclusion among threads– busy-waiting vs. blocking– pthread_mutex_trylock: no blocking (rare)– pthread_mutex_lock: blocking– pthread_mutex_unlock

• condition variables– pthread_cond_wait: block for a signal– pthread_cond_signal: signal one waiting

thread– pthread_cond_broadcast: signal all waiting

threads

Example

lock(mutex);

while (resource_state == BUSY)

//spin;

resource_state = BUSY;

unlock(mutex);

use resource;

lock(mutex);

resource_state = FREE;

unlock(mutex);

Initially mutex is unlocked

resource_state is FREE

Will this work?1 - Yes2 - No3 - Maybe

Example

lock(mutex);

while (resource_state == BUSY)

//spin;

resource_state = BUSY;

unlock(mutex);

use resource;

lock(mutex);

resource_state = FREE;

unlock(mutex);

Thread 1

Example

lock(mutex);

while (resource_state == BUSY)

//spin;

resource_state = BUSY;

unlock(mutex);

use resource;

lock(mutex);

resource_state = FREE;

unlock(mutex);

Thread 1

Thread 2

Example

lock(mutex);

while (resource_state == BUSY)

//spin;

resource_state = BUSY;

unlock(mutex);

use resource;

lock(mutex);

resource_state = FREE;

unlock(mutex);

Thread 1

Thread 2

Example with cond-var

lock(mutex)

while(resource_state == BUSY)

wait(cond_var); /* implicitly give up mutex */

/* implicitly re-acquire mutex */

resource_state = BUSY

unlock(mutex)

/* use resource */

lock(mutex)

resource_state = FREE;

unlock(mutex)

signal(cond_var);

Example with cond-var

lock(mutex)

while(resource_state == BUSY)

wait(cond_var); /* implicitly give up mutex */

/* implicitly re-acquire mutex */

resource_state = BUSY

unlock(mutex)

/* use resource */

lock(mutex)

resource_state = FREE;

unlock(mutex)

signal(cond_var);

T1

Example with cond-var

lock(mutex)

while(resource_state == BUSY)

wait(cond_var); /* implicitly give up mutex */

/* implicitly re-acquire mutex */

resource_state = BUSY

unlock(mutex)

/* use resource */

lock(mutex)

resource_state = FREE;

unlock(mutex)

signal(cond_var);

T1

T2

pthreads• Mutex

– Must create mutex variablespthread_mutex_t padlock;

– Must initialize mutex variablepthread_mutex_init(&padlock, NULL);

• Condition Variable (used for signaling)– Must create condition variables

pthread_cond_t non_full;

– Must initialize condition variablespthread_cond_init(&non_full, NULL);

Classic CS ProblemProducer Consumer

• ProducerIf (! full)

Add item to buffer

empty = FALSE

if(buffer_is_full)

full = TRUE

• ConsumerIf (! empty)

Remove item from buffer

full = FALSE

if(buffer_is_empty)

empty = TRUE

full

empty

buffer

...

Example Producer Threads Programwhile(forever){

// produce item

pthread_mutex_lock(padlock);

while (full)

pthread_cond_wait(non_full, padlock);

// add item to buffer;

buffercount++;

if (buffercount == BUFFERSIZE)

full = TRUE;

empty = FALSE;

pthread_mutex_unlock(padlock);

pthread_cond_signal(non_empty);

}

Example Consumer Threads Programwhile(forever) {

pthread_mutex_lock(padlock);

while (empty)

pthread_cond_wait (non_empty, padlock);

// remove item from buffer;

buffercount--;

full = false;

if (buffercount == 0)

empty = true;

pthread_mutex_unlock(padlock);

pthread_cond_signal(non_full);

// consume_item;

}

// Producerwhile(forever){

// produce itempthread_mutex_lock(padlock);while (full)

pthread_cond_wait(non_full, padlock);// add item to buffer;buffercount++;if (buffercount == BUFFERSIZE)

full = TRUE;empty = FALSE;pthread_mutex_unlock(padlock);pthread_cond_signal(non_empty);

}// Consumerwhile(forever) {

pthread_mutex_lock(padlock);while (empty)

pthread_cond_wait (non_empty, padlock);// remove item from buffer;buffercount--;full = false;if (buffercount == 0)

empty = true;pthread_mutex_unlock(padlock);pthread_cond_signal(non_full);

// consume_item;}

Using ThreadsServers

• dispatcher model

Mailbox

dispatcher

Workers

• Team model

• Pipelined model

Mailbox

Mailbox

Threads Implementation• User level threads

– OS independent– Scheduler is part of the runtime system– Thread switch is cheap (save PC, SP, regs)– Scheduling customizable, i.e., more

application control– Blocking call by thread blocks process

• Solution to blocking problem in user level threads– Non-blocking version of all system calls

• Switching among user level threads– Yield voluntarily– How to make preemptive?

• Timer interrupt from kernel to switch

• Kernel Level– Expensive thread switch– Makes sense for blocking calls by threads– Kernel becomes complicated: process vs.

threads scheduling– Thread packages become non-portable

• Problems common to user and kernel level threads– Libraries– Solution is to have thread-safe wrappers to

such library calls

Questions?

Solaris Threads• Three kinds

– user, lwp, kernel

• User: Any number can be created and attached to lwp’s

• One to one mapping between lwp and kernel threads

• Kernel threads known to the OS scheduler

• If a kernel thread blocks, associated lwp, and user level threads block as well

Solaris Terminology

More Conventional Terminology

Thread kernel thread

(user-level view)

(Inside the kernel)

Processes

P1 P2 P3

Questions?• Does kernel see user threads?

• Are all threads from P3 on same CPU?

• Are all threads in kernel attached to lwp’s?

• If the left-most thread of P3 issues a blocking system call does P3 block?

• Who schedules lwp’s?

• Who schedules threads?

Kernel Threads vs. User Threads• Advantages of kernel threads

– Can be scheduled on multiple CPU’s– Can be preempted by CPU– Kernel scheduler knows their relative priorities

• Advantages of user threads– (Unknown to kernel)– Extremely lightweight: No system call to

needed to change threads.

Questions?

Things to know?1. The reason threads are around?

2. Benefits of increased concurrency?

3. Why do we need software controlled "locks" (mutexes) of shared data?

4. How can we avoid potential deadlocks/race conditions.

5. What is meant by producer/consumer thread synchronization/communication using pthreads?

6. Why use a "while" loop around a pthread_cond_wait() call?

Things to know?7. Why should we minimize lock scope (minimize

the extent of code within a lock/unlock block)?

8. Do you have any control over thread scheduling?

Questions?