04/19/23 104/19/23 1

CPU Scheduling• ICS 240: Operating Systems

– William Albritton• Information and Computer Sciences

Department at Leeward Community College– Original slides by Silberschatz, Galvin, and Gagne

from Operating System Concepts with Java, 7th Edition with some modifications

– Also includes material by Dr. Susan Vrbsky from the Computer Science Department at the University of Alabama

04/19/23 2

Class Feedback• On an anonymous (don’t write your name)

sheet of paper, answer the following questions1. Average time working on each assignment

2. Total time studying for the exam

3. List several things that help you learn

4. List several things that make learning difficult

5. List several suggestions for improving the class

04/19/23 3

Basic Concepts

• Maximum CPU utilization obtained with multiprogramming

• CPU–I/O Burst Cycle– Process execution consists of a cycle of CPU

execution and I/O wait• CPU burst: a time interval when a process uses the

CPU only (few long CPU bursts)

• I/O burst: a time interval when a process uses I/O devices only (more short CPU bursts)

• CPU burst distribution (next two slides)

04/19/23 4

Alternating Sequence of CPU And I/O Bursts

04/19/23 5

Histogram of CPU-burst Times•This histogram (a graphical display of a frequency distribution) shows many, short CPU bursts and a few, long CPU bursts.

04/19/23 6

CPU Scheduler• Short-term scheduler

– Selects from among the processes in memory (in the ready queue) that are ready to execute, and allocates the CPU to one of them

• Note that the ready queue is not necessarily a first-in first-out (FIFO) queue, as different algorithms other than first-in first-out (FIFO) might be used to manage the queue

– For example, a queue at a club is not necessarily first-in first-out (FIFO), as VIPs usually jump to the front of the queue

– A queue is a fancy word for line

04/19/23 7

Preemptive vs. Non-preemptive• Non-preemptive scheduling scheme

– The short-term scheduler cannot forcibly take the CPU back from a process

• Once the CPU has been allocated to a process, the process keeps the CUP until the process releases the CPU either by terminating or by switching to the waiting state

– Also called cooperative scheduling scheme

04/19/23 8

Preemptive vs. Non-preemptive• Preemptive scheduling scheme

– The short-term scheduler takes the CPU from the process

– Can take the CPU back due to• Time slice

• Higher priority process arrived

• Interrupts

04/19/23 9

Process State• As a process executes, it changes state

1. new: The process is being created2. running: Instructions are being executed3. waiting: The process is waiting for some event

to occur• For example, I/O completion or reception of a signal

4. ready: The process is waiting to be assigned to a processor

5. terminated: The process has finished execution

04/19/23 10

Diagram of Process State

04/19/23 11

CPU Scheduler• CPU scheduling decisions (by the short-

term scheduler) may take place when a process

1. Switches from running to waiting state• For example, when the process has an IO request

• This is non-preemptive

2. Switches from running to ready state• For example, when an interrupt occurs

• This is preemptive

04/19/23 12

CPU Scheduler• CPU scheduling decisions (by the short-

term scheduler) may take place when a process

3. Switches from waiting to ready state• For example, at the completion of I/O

• This is preemptive

4. Terminates– For example, the process (running program) is done

– This is non-preemptive

04/19/23 13

Dispatcher

• Dispatcher module gives control of the CPU to the process selected by the short-term scheduler

– This function involves the following steps1. switching context

2. switching to user mode

3. jumping to the proper location in the user program to restart that program

04/19/23 14

Dispatcher

• Because the dispatcher is used during every process switch, the dispatcher should be as fast as possible

– Dispatch latency• time it takes for the dispatcher to stop one process and

start another running

04/19/23 15

Scheduling Criteria• The following 5 factors are used as Scheduling

Criteria (Performance Evaluation), which are used to compare CPU scheduling algorithms1. CPU utilization

• Fraction of time CPU busy (through multiprogramming)• Range from 40% (light load) to 90% (heavily used)

– goal: maximize

2. Throughput• Number of processes that complete their execution per

time unit (measure of work)• Range from one process per hour to ten processes per

second– goal: maximize

04/19/23 16

Scheduling Criteria• The following 5 factors are used as Scheduling

Criteria (Performance Evaluation), which are used to compare CPU scheduling algorithms3. Turnaround time

• Amount of time to execute a particular process• In other word, the delay between job submission and job

completion• The sum of the time waiting to get into memory, waiting

in the ready queue, executing on the CPU, and doing I/O

– goal: minimize

04/19/23 17

Scheduling Criteria• The following 5 factors are used as Scheduling

Criteria (Performance Evaluation), which are used to compare CPU scheduling algorithms4. Waiting time

• Amount of time a process has been waiting in the ready queue

• Note that a CPU scheduling algorithm cannot change the execution time or I/O– goal: minimize

5. Response time• Amount of time it takes from when a request was

submitted until the first response is produced• In other words, the amount of time from request until the

process starts to respond– goal: minimize

04/19/23 18

Optimization Criteria• Max CPU utilization

• Max throughput

• Min turnaround time

• Min waiting time

• Min response time

04/19/23 19

FCFS Scheduling• FCFS - First Come First Serve

– The process that requests the CPU first is allocated to the CPU first

– Simple and non-preemptive– Better to schedule shorter processes before longer

ones• Convoy effect happens when short process get stuck

behind long process

– Unfair to processes with short bursts– Not usually used alone, but as a part of a more

complex algorithm

04/19/23 20

Process Burst Time

P1 24

P2 3

P3 3

• Suppose that the processes arrive in the order: P1 , P2 , P3 all at time 0. The Gantt Chart for the schedule is:

– Throughput: 3 processes/30 milliseconds = 1/10 – Turnaround time for P1 = 24; P2 = 27; P3 = 30– Average turnaround time: (24 + 27 + 30)/3 = 81/3 = 27– Waiting time for P1 = 0; P2 = 24; P3 = 27– Average waiting time: (0 + 24 + 27)/3 = 51/3 = 17

P1 P2 P3

24 27 300

FCFS Scheduling

04/19/23 21

FCFS SchedulingSuppose that the processes arrive in the order

P2 , P3 , P1 all at time 0

• The Gantt chart for the schedule is:

– Throughput: 3 processes/30 milliseconds = 1/10– Turnaround time for P1 = 30; P2 = 3; P3 = 6– Average turnaround time: (30 + 3 + 6)/3 = 39/3 = 13

– Waiting time for P1 = 6; P2 = 0; P3 = 3

– Average waiting time: (6 + 0 + 3)/3 = 9/3 = 3

• Much better than previous case!

P1P3P2

63 300

04/19/23 22

Shortest-Job-First (SJF) Scheduling

• Associate with each process the length of its next CPU burst. – Use these lengths to schedule the process with

the shortest time first

• Non-preemptive– once CPU given to the process it cannot be

preempted until completes its CPU burst

• SJF is optimal non-preemptive algorithm– Gives minimum average waiting time for a

given set of processes

04/19/23 23

Process Arrival Time Burst Time

P1 0.0 7

P2 0.0 4

P3 0.0 1

P4 0.0 4

• Suppose that the processes arrive in the order: P1 , P2 , P3 , P4 all at time 0. The Gantt Chart for the schedule is

| P3 | P2 | P4 | P1 |

0 1 5 9 16

– Throughput: 4 processes/16 milliseconds = 1/4– Turnaround time for P1 = 16; P2 = 5; P3 = 1; P4 = 9– Average turnaround time: (16 + 5 + 1 + 9)/4 = 31/4 = 7.75

– Waiting time for P1 = 9; P2 = 1; P3 = 0 ; P4 = 5

– Average waiting time: (9 + 1 + 0 + 5)/4 = 15/4 = 3.75

Example of SJF

Process Arrival Time Burst Time

P1 0.0 7

P2 2.0 4

P3 4.0 1

P4 5.0 4

– Throughput: 4 processes/16 milliseconds = 1/4– Turnaround time for P1 = 7-0 = 7; P2 = 12-2 = 10;

P3 = 8-4 = 4; P4 = 16-5 = 11– Average turnaround time: (7 + 10 + 4 + 11)/4 = 32/4 = 8

– Waiting time for P1 = 0-0 = 0; P2 = 8-2 = 6;

P3 = 7-4 = 3; P4 = 12-5 = 7

– Average waiting time: (0 + 6 + 3 + 7)/4 =16/4 = 4

Another Example of SJF

P1 P3 P2

73 160

P4

8 12

04/19/23 25

SRTF - Shortest Remaining Time First

• Preemptive version of Shortest-Job-First (SJF) Scheduling

• Preemptive– If new process arrives with CPU burst less than

time left of process currently executing, then new process gets the CPU

• SRTF is optimal preemptive algorithm– Gives minimum average waiting time for a

given set of processes

04/19/23 26

Example of SRTF Process Arrival Time Burst Time

P1 0.0 7 P2 2.0 4 P3 4.0 1 P4 5.0 4

– Throughput: 4 processes/16 milliseconds = 1/4– Turnaround time for P1 = 16-0 = 16; P2 = 7-2 = 5;

P3 = 5-4 = 1; P4 = 11-5 = 6– Average turnaround time: (16 + 5 + 1 + 6)/4 = 28/4 = 7– Waiting time for P1 = 11-2 = 9; P2 = 5-4 = 1;

P3 = 4-4 = 0; P4 = 7-5 = 2– Average waiting time: (9 + 1 + 0 +2)/4 = 12/4 = 3

P1 P3P2

42 110

P4

5 7

P2 P1

16

04/19/23 27

Determining Length of Next CPU Burst

• For the SJF and SRTF scheduling algorithms, knowing the exact length of the next CPU burst is difficult– For long-term scheduling, the user-defined

process time limit can be used• The user has the responsibility for estimating the

process time limit accurately

– The short-term scheduler does not know the exact length of the next CPU burst

• Instead, the next CPU burst is estimated

04/19/23 28

Determining Length of Next CPU Burst

• For short-term scheduling, the length of next CPU burst is estimated by using the length of previous CPU bursts, using exponential averaging

:Define 4.

10 , 3.

burst CPU next the for value predicted 2.

burst CPU of length actual 1.

1n

thn nt

.1 1 nnn t

04/19/23 29

Determining Length of Next CPU Burst

04/19/23 30

Examples of Exponential Averaging =0

n+1 = n

– Recent history does not count =1

– n+1 = tn

– Only the actual last CPU burst counts• If we expand the formula, we get:

n+1 = tn+(1 - ) tn -1 + …

+(1 - )j tn -j + …

+(1 - )n +1 0

• Since both and (1 - ) are less than or equal to 1, each successive term has less weight than its predecessor

04/19/23 31

Priority Scheduling• A priority number (integer) is associated with

each process

• The CPU is allocated to the process with the highest priority – In this class, assume that the smallest integer has

the highest priority• Can be either way, so we will stick with the smallest

number for the highest priority to try to avoid confusion

• Can be preemptive or non-preemptive

04/19/23 32

Priority Scheduling• Problem of Starvation

– Low priority processes may never execute

• Solution is Aging– As time progresses, increase the priority of the

process

• By the way, SJF and SRTF are also a type of priority scheduling algorithm– Priority is the predicted next CPU burst time

04/19/23 33

Process Priority Burst Time

P1 3 10

P2 1 1

P3 4 2

P4 5 1

P5 2 5• Suppose that the processes arrive in the order: P1 , P2 , P3 , P4 , P5 all at

time 0. The Gantt Chart for the schedule is

| P2 | P5 | P1 | P3 | P4 | 0 1 6 16 18 19 – Throughput: 5 processes/19 milliseconds = 5/19– Turnaround time for P1 = 16; P2 = 1; P3 = 18; P4 = 19; P5 = 6– Average turnaround time: (16 + 1 + 18 + 19 + 6)/4 = 60/5 = 12– Waiting time for P1 = 6; P2 = 0; P3 = 16; P4 = 18; P5 = 1– Average waiting time: (6 + 0 + 16 + 18 + 1)/4 = 41/5 = 8.2

Priority Scheduling (Non-preemptive)

04/19/23 34

Round Robin (RR)• Each process gets a small unit of CPU time

(time quantum or time slice), usually 10-100 milliseconds (ms)– After this time has elapsed, the process is preempted and

added to the end of the ready queue

• If there are n processes in the ready queue and the time quantum is q– Then each process gets 1/n of the CPU time in

chunks of at most q time units at once

04/19/23 35

Example of RRProcess Burst Time Arrival Time

P1 53 0

P2 17 0

P3 68 0

P4 24 0

• Time Quantum = 20 ms– Throughput: 4 processes/162 milliseconds = 2/81– Turnaround time for P1 = 134; P2 = 37; P3 = 162; P4 = 121– Average turnaround time: (134 + 37 + 162 + 121)/4 = 454/4 = 113.5– Waiting time for P1 = (77-20)+(121-97) = 81; P2 = (20-0)=20;

P3 = (37-0)+(97-57)+(134-117)=94; P4 = (57-0)+(117-77) = 97– Average waiting time: (81 + 20 + 94 +97)/4 = 292/4 = 73.0

P1 P2 P3 P4 P1 P3 P4 P1 P3 P3

0 20 37 57 77 97 117 121 134 154 162

04/19/23 36

RR with Different Arrival TimesProcess Burst Time Arrival Time

P1 53 0

P2 17 30

P3 68 50

P4 24 60

• Time Quantum = 20 ms– Throughput: 4 processes/162 milliseconds = 2/81– Turnaround time for P1 = 70-0 = 70; P2 = 57-30 = 27;

P3 = 162-50 = 112; P4 = 134-60 = 74– Average turnaround time: (70 + 27 + 112 + 74)/4 = 283/4 = 70.75– Waiting time for P1 = (57-40) = 37; P2 = (40-30) = 10;

P3 = (162-68-50)=44; P4 = (134-24-60) = 50– Average waiting time: (37 + 10+ 44 + 50)/4 = 141/4 = 35.25

P1 P1 P2 P1 P3 P4 P3 P4 P3 P3

0 20 40 57 70 90 110 130 134 154 162

04/19/23 37

Time Quantum• How to choose value for time quantum (time

slice)?– If time quantum (time slice) is 20 milliseconds

and context switch is 5 milliseconds • then 20% wasted on overhead to improve

– If time quantum is 500 milliseconds and context switch is 5 milliseconds

• then only 4% overhead

04/19/23 38

Time Quantum• If time quantum (slice) is too small

– Overhead of context switch becomes dominant

• If time quantum (slice) too large– RR degenerates to FCFS (First Come First Serve)– Typical to have 80% of CPU bursts shorter than

time slice

• If processes finish CPU burst in single time quantum– The turnaround time will be shorter

04/19/23 39

Time Quantum & Context Switches

•This figure shows how decreasing the time quantum increases the number of context switches

04/19/23 40

Turnaround Time Varies With The Time Quantum

•The average turnaround time does not necessarily improve as the time quantum size increases

04/19/23 41

Round Robin (RR)• Tradeoffs

– RR typically has a higher average turnaround time than SRTF (Shortest Remaining Time First)

– However, RR typically has better response time than SRTF

• For RR, the implicit assumption is that all processes are equally important

• According to Dr. Tanenbaum, RR is the "oldest, simplest, fairest, and most widely used" of the scheduling algorithms

04/19/23 42

Multilevel Queue Scheduling• For a multilevel queue scheduling algorithm,

the ready queue is partitioned into several separate queues– For example, two queues

1. Interactive/foreground processes at high priorities

2. Background/batch processes at low priorities

• With a multilevel queue scheduling algorithm, each queue has its own scheduling algorithm– Continuing from the above example

1. foreground – RR (Round Robin)

2. background – FCFS (First Come First Serve)

04/19/23 43

Multilevel Queue Scheduling• Scheduling must be done between the queues

– Here are some example choices1. Use fixed priority scheduling (serve all processes from

the foreground, and then from the background) – Possibility of starvation

2. Use time slicing (each queue gets a certain amount of CPU time which it can schedule amongst its processes)– 80% to foreground in RR

– 20% to background in FCFS

• A Multilevel Queue is good for time-sharing systems and favors short jobs

04/19/23 44

Multilevel Queue Scheduling

•In this figure, the ready queue is divided into five (5) separate queues

04/19/23 45

Multilevel Feedback Queue• In a multilevel queue scheduling algorithm, a

process is permanently assigned to a specific queue– A multilevel queue scheduling algorithm has low

overhead, but is inflexible

• In contrast, in a Multilevel Feedback Queue Scheduling Algorithm, a process moves between the various queues– Processes that wait for too long in a particular

queue can be moved to another queue• This kind of aging prevent starvation

04/19/23 46

Multilevel Feedback Queue• Dynamic priorities (priorities change) for each

process– Aging, which prevent starvation, can be

implemented in this manner

• Design decisions– Scheduling algorithm for each queue– Rule for upgrading/degrading process to

higher/lower queue– To which queue do new processes go?

04/19/23 47

Multilevel Feedback Queue• Typical behavior

– Higher priority queues have smaller time slices– If a process uses up its time slice, it moves down a

priority– If a process blocks before using time slice, it stays

at that priority– New processes go to the highest priority

04/19/23 48

Example of Multilevel Feedback Queue

• Three queues

1. Q0 – RR time quantum 8 milliseconds

2. Q1 – RR time quantum 16 milliseconds

3. Q2 – FCFS

04/19/23 49

Example of Multilevel Feedback Queue

• Scheduling

– A new job enters queue Q0 which is served FCFS

• When it gains CPU, job receives 8 milliseconds.

• If it does not finish in 8 milliseconds, job is moved to queue Q1

– At Q1 job is again served FCFS and receives 16 additional milliseconds

• If it still does not complete, it is preempted and moved to queue Q2

04/19/23 50

Multilevel Feedback Queues

Multiple-Processor Scheduling• CPU scheduling is more complex when

multiple CPUs are available

• N processors under control on one operating system– All homogeneous processors

• Use any processor for any job type

– Versus heterogeneous processors• Schedule according to which processor the job was

compiled on

04/19/23 51

Multiple-Processor Example• Assume homogeneous processors

– Any processor can handle any process

• Assume a common ready queue– Want to prevent the case of having one full

queue and all other queues empty

04/19/23 52

Multiple-Processor ExampleProcess Arrival Burst Time

  P1          0         2   P2          0            2   P3             0            4

• If use FCFS, assign to the processors as follows – CPU1: P1     P3 (6 ms)

– CPU2: P2 (2 ms)

• Alternative (better) way to assign to the processors– CPU1: P1     P2 (4 ms)

– CPU2: P3 (4 ms)

04/19/23 53

Multiple-Processor Example•  Which scheduling strategy is better?

– COMPLEX problem

• Optimal Schedule– Enumerate all possible schedules

• Choose the one with the least amount of processor idle times

• This problem is NP complete (non-polynomial), which means that the only solution is one that takes the longest time possible (check all combinations)

• In practice, use exponential averaging to estimate the optimal solution

04/19/23 54

04/19/23 55

Operating System Examples1. Solaris scheduling

2. Windows XP scheduling

3. Linux scheduling

Solaris Scheduling• Priority-based, preemptive scheduling

• Time slicing (RR) & multilevel feedback queues

• Four classes of scheduling (each class has different priorities and scheduling algorithms)1. Real time

2. System

3. Time sharing (default)

4. Interactive

04/19/23 56

04/19/23 57

Solaris Scheduling

Solaris Scheduling• Time Sharing & Interactive classes have 60

priority levels– Contain several different fields

1. Priority – higher number is higher priority

2. Time quantum – lowest priority has highest time quantum (inversely proportional to priority)

3. Time quantum expired – decrease in priority of a thread if its entire time quantum has been used up

4. Return from sleep – boost in priority if finished waiting for I/O

04/19/23 58

Solaris Dispatch Table

04/19/23 59

Windows XP Scheduling• Priority-based, preemptive scheduling

• Time slicing (RR)

• The dispatcher ensures that highest-priority process will always run– Has 32 priority levels, with higher number for

higher priority– Has two main priority classes

1. Real-time class – priorities 16 to 31

2. Variable class – priorities 1 to 15

3. Memory management – priority 0

04/19/23 60

Windows XP Scheduling• The dispatcher has a queue for each

scheduling priority

• The dispatcher traverses through all the queues from highest to lowest until it finds a thread that is ready to run

04/19/23 61

Windows XP Scheduling• After a process uses up its time quantum

– The priority is lowered

• After a process finishes a wait– The priority is boosted, depending on what the

process has been waiting for• Threads waiting for the mouse & keyboard get large

increases

• Threads waiting for a disk operation get smaller increases

04/19/23 62

Windows XP Priorities

04/19/23 63

•The far-left column is the relative priority (within each class), the 2nd column is the real-time class, and the next 5 columns are the variable class

04/19/23 64

Linux Scheduling• Priority-based, preemptive scheduling• Two separate priority ranges (lower

numbers have a higher priority)1. Real-time values: 0 to 99

• Once assigned, values never change

2. Nice values: 100 to 140 • If much I/O, up to -5 change; if CPU-bound, +5

• RR (Round Robin)– Unlike most other OS scheduling algorithms,

higher-priority processes are assigned a longer time quantum and vice-versa

04/19/23 65

Relationship Between Priorities & Time-slice

• A higher priority process is assigned a longer time quantum (and vice-versa)– Lower integers signify higher priorities

04/19/23 66

Linux Scheduling• Each processor has two kinds of queues,

which are implemented as priority arrays1. Active priority array

• Contains all processes which still have time remaining in their time slices

• Short-term scheduler chooses the process with the highest priority to run on CPU first

2. Expired priority array• Contains all processes which have used up their

time slice

• Once the active priority array is empty, then the two priority arrays are switched

04/19/23 67

List of Tasks Indexed According to Priorities

• This shows the two priority arrays

04/19/23 68

Algorithm Evaluation• In order to evaluate the CPU scheduling

algorithms, we need to determine the evaluation criteria (what will be measured)

1. CPU utilization

2. throughput

3. turnaround time

4. waiting time

5. response time

04/19/23 69

Algorithm Evaluation• Analytic Techniques

1. Deterministic Scheduling Theory

2. Queuing theory

3. Simulations

4. Implementation

04/19/23 70

Deterministic Scheduling Theory

• Because processes characteristics are all known ahead of time, we can:– Evaluate each algorithm based on the

characteristics of the processes– Choose algorithm based on performance

criteria

• Deterministic modeling is simple & fast, but results only apply to specific cases– On the other hand, may indicate certain trends

04/19/23 71

Deterministic Scheduling Theory

• Processes characteristics Process Burst time Arrival time

P1 10 0

P2 29 0

P3 3 0

P4 7 0

P5 12 0

04/19/23 72

Deterministic Scheduling Theory

• FCFS: 28 ms average waiting time

• SJF: 13 ms average waiting time

• RR: 23 ms average waiting time

04/19/23 73

Queuing Theory• Models systems with arriving customers,

wait for service, get service, depart

• Operating system characteristics such as1. number of servers

2. capacity of queues

3. arrival time distribution

4. service time distribution

• Huge body of work to analyze these systems

04/19/23 74

Simulations• Write software to fake operation

– Feed simulation a bunch of jobs– Check performance of each scheduling

algorithm

• Two possible workload characteristics1.Randomly generated from probability

distributions

2.Or compiled from real samples (called a trace tape)

04/19/23 75

Simulations• Evaluation of CPU schedulers by

simulation

04/19/23 76

Implementation• Two steps for implementation

1. Implement a scheduling algorithm in an actual operating system

2. Monitor the performance of the operating system