queuing theory and stochastic service systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf ·...
TRANSCRIPT
![Page 1: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/1.jpg)
Queuing Theory and Stochastic Service Systems
Li Xia
Tsinghua University, 2015 Fall
![Page 2: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/2.jpg)
Syllabus • Instructor
– Li Xia 夏俐, FIT 3-618, 62793029, [email protected]
• Text book – Mor Harchol-Balter, Performance Modeling and Design of
Computer Systems—Queueing Theory in Action, Cambridge Press, 2013. (copy is provided)
• Reference books: – D. Gross, J.F. Shortle, J.M. Thompson, and C.M. Harris,
Fundamentals of Queueing Theory, 4th Edition, Hoboken: Wiley, 2008.
– Leonard Kleinrock, Queueing Systems, vol. 1: Theory, John Wiley, 1975.
– Caltech course (Prof. Adam Wierman): http://courses.cms.caltech.edu/cs147/
2 Li Xia, Tsinghua Univ.
![Page 3: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/3.jpg)
Syllabus • Grading
– Homework: 30% (4 assignments, 1 simulation task, in English) plagiary is prohibited
– Midterm: 20%
– Final Project: 40% (the 9th week)
– Course Interaction: 10%
• Lecture notes and assignments are available online (in English)
– http://cfins.au.tsinghua.edu.cn/personalhg/xiali/teaching/course_queues.htm
3 Li Xia, Tsinghua Univ.
![Page 4: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/4.jpg)
What’s your purpose to take this course
• What do you expect to learn from this course?
– Open discussion
• Let’s see some examples in practice
4 Li Xia, Tsinghua Univ.
![Page 5: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/5.jpg)
Telephone switch • A.K. Erlang studied the problem:
– No automatic switch, operator
– How many how many telephone operators were needed to handle a given volume of calls
– Erlang A / Erlang B formula
Li Xia, Tsinghua Univ. 5
Agner K. Erlang
(1878 –1929)
Danish engineer
Erlang (1909) "The Theory of Probabilities and Telephone Conversations" –
which proves that the Poisson distribution applies to random telephone traffic.
Erlang (1917) "Solution of some Problems in the Theory of Probabilities of Significance in
Automatic Telephone Exchanges"
![Page 6: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/6.jpg)
Beijing Subway
Throughput?
Safety?
More lines Increase buffer So what?
6 Li Xia, Tsinghua Univ.
![Page 7: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/7.jpg)
Railway ticket online booking in 2012 Chinese new year
• Crash of ticket booking system
– Large number of tickets for sale (4million)
– Huge visit requests 秒杀? (billion)
– System architecture is not optimal
• Bandwidth of network
• CPU/RAM of computer
• Business logic – Queue + Feedback, greatly reduce the repeating request
• Other factors…
7 Li Xia, Tsinghua Univ.
![Page 8: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/8.jpg)
Modeling and Analysis
• How to solve it?
– Performance analysis and optimization
– Queueing scheme, increase bandwidth…
Internet
Web server Application server
client Database server
IE browser data input interaction display…
passwd verify cookies/other application…
ticket data booking records… 8 Li Xia, Tsinghua Univ.
![Page 9: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/9.jpg)
Applications in daily life
• Supermarket
– How to define the express line (# of items)?
– How to determine the number of checkouts?
– How long customers have to wait at checkouts?
– Behavior of waiting time during peak-hours
• Line at bank counters
– Multiple lines v.s. single line
– Specialist purpose v.s. generalist purpose
– Number of counters?
9 Li Xia, Tsinghua Univ.
![Page 10: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/10.jpg)
Applications in engineering
• Computer/circuit architecture design – 1 fast disk v.s. 2 slow disks?
– Invest on large buffer v.s. fast CPU?
– Scheduling policy to improve performance
• Communication network design – Buffer size design of switch/router
– Data packet scheduling policy in sensor or mobile network
• …
10 Li Xia, Tsinghua Univ.
![Page 11: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/11.jpg)
List of applications areas
• Production system (machine, different products)
• Computer system (cpu, disk, RAM design)
• Communication network (buffer design, link capacity)
• Transportation system (traffic lights control)
• Bank branches operation (counter/type design)
• Airlines scheduling (takeoff/landing arrangement)
• Data center (optimal control, energy saving)
• Call center (optimize the operators, hotlines,…)
• Post office (multi-class, specialization)
• …
11 Li Xia, Tsinghua Univ.
![Page 12: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/12.jpg)
What’s queue? • A general queuing system
Customer arrival
waiting room
Service facility
Customer departure
r22
r11
r12
r23
r31 r21 r32
r13 r33 1
2
3
r20
r10
r30
γ1
γ2
γ3 Queuing
network:
A single
server queue:
12 Li Xia, Tsinghua Univ.
![Page 13: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/13.jpg)
Terminology in queuing theory
• Basic element in queue
– Arrival pattern, service pattern, number of servers, service discipline, system capacity, customer type,..
• Performance metrics
– Average number of customers
– Queue length, average number of queuing customers
– Throughput
– Response time, sojourn time, system time
– Waiting time, queuing time
13 Li Xia, Tsinghua Univ.
![Page 14: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/14.jpg)
Why we need queuing theory?
• Resource constraints
– Why queues appear? How to make them go away?
• Goal of queuing theory
– Predict the performance
– Design the architecture
– Optimize the parameter/policy
• Counter-intuitive
– Randomness is complicated
– Some examples
14 Li Xia, Tsinghua Univ.
![Page 15: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/15.jpg)
Example 1: CPU design
• A simple model of CPU
– Job arrival at rate λ=3/s, Poisson process
– Job mean size is 1/μ, exponential • i.e., service rate is μ=5/s
– FCFS(first come first serve), buffer is infinite
– assume λ < μ, [question]why?
CPU
buffer
Model of a cpu
λ μ
NOTE: modern CPU may have other features, multi-core/PS, etc. 15 Li Xia, Tsinghua Univ.
![Page 16: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/16.jpg)
CPU design, cont.
• If the arrival rate λ doubles, how to upgrade?
– If want to maintain the same delay of jobs, [question] what you choose?
• A. double μ
• B. less than double μ
• C. more than double μ
– Why? Double μ will cut the delay in half
• prove with M/M/1 queuing theory
• Physical intuition, time speeds up with scale 2
16 Li Xia, Tsinghua Univ.
![Page 17: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/17.jpg)
Example 2: Lines in bank
• Customer arrival in Poisson with rate λ
• Counter service rate is μ, exponential
• FCFS, infinite waiting capacity
3
3 lines 1 line
17 Li Xia, Tsinghua Univ.
![Page 18: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/18.jpg)
L1=1.5, L2=0.237
W1=0.5min, W2=0.079min
T1=1min, T2=0.579min
Lines in bank, cont.
• Assume μ = 2/min, λ = 1/min
– queue length,
– waiting time,
– response time,
• [question] how is the following queue?
33
L3 = 0.5
W3 = 0.1667
T3 = 0.3333
18 Li Xia, Tsinghua Univ.
![Page 19: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/19.jpg)
Example 3: many slow v.s. one fast
• CPU selection:
– 1 core CPU with 3GHz freq.
– 3 core CPU with 1GHz freq.
• Which one has a better mean response time?
33
3
19 Li Xia, Tsinghua Univ.
![Page 20: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/20.jpg)
• Depend on the variability of jobs
– Job size variability is high, choose many slow CPU
– Job size variability is low, choose one fast CPU
• Exponential distr.: coefficient of variation(cv) = 1; – For the case of M/M/c and M/M/1, the latter is better
• Uniform distr. or deterministic: cv < 1;
• Hyper-exponential distr. or other distr. (PH, MAP): cv > 1. (self-similarity of Internet traffic)
– If workload is low, one fast is preferred
– If jobs are preemptible (priority, stop, resume)
• One fast is preferred
many slow v.s. one fast, cont.
20 Li Xia, Tsinghua Univ.
![Page 21: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/21.jpg)
many slow v.s. one fast, cont. • many slow v.s. a few fast, widely exist in practice
– Power allocation in data center with server farm
• Fast freq., more power consumption, green data center
– Bandwidth partition in communication systems
• Small chunks of bandwidth, TDMA/FDMA/CDMA …
– Road network in transportation
• Few wide roads v.s. many small roads (Beijing’s problem) – Traffic is bursty with high variability, prefer many slow
– etc. Consider economic factors…
– Service rate control in Jackson network
• (Xia and Shihada, IEEE-TAC 2013) 21 Li Xia, Tsinghua Univ.
![Page 22: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/22.jpg)
Example 4: Closed queueing network
• Model the intensive traffic with N capacity of network
– Batch system, intensive queue with limited capacity, etc.
1
1
3
0.5
0.5
N=6 jobs
1
1
3
22 Li Xia, Tsinghua Univ.
![Page 23: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/23.jpg)
Closed queueing network, cont.
• If we double the speed of server 1
– How it effects the response time of job?
– How it effects the throughput? • [Answer] only change by a small amount
• Suppose N is very large, how is above question?
– Change 0, if N ∞
• What if N is very small
– If N=1, changed amount is large
23 Li Xia, Tsinghua Univ.
![Page 24: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/24.jpg)
Closed queueing network, cont.
• What if the queueing network is open?
– remarkable improvement of throughput and average response time
0.5
0.5
1
1
3
1
1
3
24 Li Xia, Tsinghua Univ.
![Page 25: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/25.jpg)
Example 5: Task assignment in a server farm
• Front-end dispatcher, web server farm, assign task among back-end servers
– used in engineering, Cisco/IBM network device
1
2
Arrivals Dispatcher
(Load Balancer)
25 Li Xia, Tsinghua Univ.
![Page 26: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/26.jpg)
• Task assignment policy
– Determine the task should go to which server
– Based on the system state, policy in MDP
• Different policy
– Random
– Shortest-Queue (SQ)
– Size-Interval-Task Assignment (SITA)
– Least-Work-Left (LWL)
– Central-Queue (CQ)
• Question: which one has best mean response time?
Task assignment, cont.
26 Li Xia, Tsinghua Univ.
![Page 27: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/27.jpg)
• Answer
– Depend on the property of job size • If job size is known, LWL is usually the best
• LWL = CQ ?
– If server discipline is processor-sharing (PS) • SQ is the near optimal
• Task assignment problem
– FCFS/PS, modeled as an MDP optimization problem
– Minimize variance of response time, rather than the mean response time
– Variance (fairness, risk) v.s. Mean (social welfare)
Task assignment, cont.
Chinese Proverb: 不患寡而患不均 27 Li Xia, Tsinghua Univ.
![Page 28: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/28.jpg)
Example 6: Scheduling
• How service disciplines affect response time?
– FCFS, first come first serve
– LCFS, last come first serve
– Random
– [answer] all the same
28 Li Xia, Tsinghua Univ.
![Page 29: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/29.jpg)
Scheduling, cont.
• What if PR-LCFS, preemptive-resumed LCFS?
– Depends on the randomness of job size
• High randomness, big improvement
• No randomness, twice worse
29 Li Xia, Tsinghua Univ.
![Page 30: Queuing Theory and Stochastic Service Systemscfins.au.tsinghua.edu.cn/.../lectures/lect_01.pdf · Queuing Theory and Stochastic Service Systems Li Xia Tsinghua University, 2015 Fall](https://reader034.vdocuments.us/reader034/viewer/2022042321/5f0b85c67e708231d430eef8/html5/thumbnails/30.jpg)
Summary of examples
• Why counter-intuition?
– Randomness of queuing
– Interactions among customers and servers
• Toy example, but many insights
– Models
– Analysis
– Design
– Optimization
– …
30 Li Xia, Tsinghua Univ.