quicksort - skkumodsim.skku.ac.kr/bbs/course/data/al/lecturenote/alg... · 2009. 3. 27. ·...
TRANSCRIPT
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.14
Quicksort
• Proposed by C.A.R. Hoare in 1962.• Divide-and-conquer algorithm.• Sorts “in place” (like insertion sort, but not
like merge sort).• Very practical (with tuning).• Can be viewed as a randomized Las Vegas
algorithm
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.15
Quicksort an n-element array:1. Divide: Partition the array into two subarrays
around a pivot x such that elements in lower subarray ≤ x ≤ elements in upper subarray.
2. Conquer: Recursively sort the two subarrays.3. Combine: Trivial.
Divide and conquer
xx
Key: Linear-time partitioning subroutine.
≤ x≤ x≤ x≤ x xx ≥ x≥ x
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.16
Pseudocode for quicksortQUICKSORT(A, p, r)
if p < rthen q ← PARTITION(A, p, r)
QUICKSORT(A, p, q–1)QUICKSORT(A, q+1, r)
Initial call: QUICKSORT(A, 1, n)
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.17
xx
Partitioning subroutinePARTITION(A, p, r) ⊳ A[p . . r]
x ← A[p] ⊳ pivot = A[p]i ← pfor j ← p + 1 to r
do if A[ j] ≤ xthen i ← i + 1
exchange A[i] ↔ A[ j]exchange A[p] ↔ A[i]return i
≤ x≤ x ≥ x≥ x ??p i rj
Invariant:
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.18
Example of partitioning
i j66 1010 1313 55 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.19
Example of partitioning
i j66 1010 1313 55 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.20
Example of partitioning
i j66 1010 1313 55 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.21
Example of partitioning
66 1010 1313 55 88 33 22 1111
i j66 55 1313 1010 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.22
Example of partitioning
66 1010 1313 55 88 33 22 1111
i j66 55 1313 1010 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.23
Example of partitioning
66 1010 1313 55 88 33 22 1111
i j66 55 1313 1010 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.24
Example of partitioning
66 1010 1313 55 88 33 22 1111
i j66 55 33 1010 88 1313 22 1111
66 55 1313 1010 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.25
Example of partitioning
66 1010 1313 55 88 33 22 1111
i j66 55 33 1010 88 1313 22 1111
66 55 1313 1010 88 33 22 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.26
Example of partitioning
66 1010 1313 55 88 33 22 1111
66 55 33 1010 88 1313 22 1111
66 55 1313 1010 88 33 22 1111
i j66 55 33 22 88 1313 1010 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.27
Example of partitioning
66 1010 1313 55 88 33 22 1111
66 55 33 1010 88 1313 22 1111
66 55 1313 1010 88 33 22 1111
i j66 55 33 22 88 1313 1010 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.28
Example of partitioning
66 1010 1313 55 88 33 22 1111
66 55 33 1010 88 1313 22 1111
66 55 1313 1010 88 33 22 1111
i j66 55 33 22 88 1313 1010 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.29
Example of partitioning
66 1010 1313 55 88 33 22 1111
66 55 33 1010 88 1313 22 1111
66 55 1313 1010 88 33 22 1111
66 55 33 22 88 1313 1010 1111
i22 55 33 66 88 1313 1010 1111
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.30
xx
Analysis of quicksort
• Assume all input elements are distinct.• In practice, there are better partitioning
algorithms for when duplicate input elements may exist.
• What is the worst case running time of Quicksort ?
≤ x≤ x ≥ x≥ x
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.31
Worst-case of quicksort
• Input sorted or reverse sorted.• Partition around min or max element.• One side of partition always has no elements.
)()()1(
)()1()1()()1()0()(
2nnnT
nnTnnTTnT
Θ=
Θ+−=Θ+−+Θ=Θ+−+=
(arithmetic series)
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.32
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.33
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
T(n)
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.34
cnT(0) T(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.35
cnT(0) c(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
T(0) T(n–2)
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.36
cnT(0) c(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
T(0) c(n–2)
T(0)
Θ(1)
O
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.37
cnT(0) c(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
T(0) c(n–2)
T(0)
Θ(1)
O
( )2
1nk
n
kΘ=
Θ ∑
=
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.38
cnΘ(1) c(n–1)
Worst-case recursion treeT(n) = T(0) + T(n–1) + cn
Θ(1) c(n–2)
Θ(1)
Θ(1)
O
( )2
1nk
n
kΘ=
Θ ∑
=
h = n
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.39
Nice-case analysis
If we’re lucky, PARTITION splits the array evenly:T(n) = 2T(n/2) + Θ(n)
= Θ(n lg n) (same as merge sort)
What if the split is always 109
101 : ?
( ) ( ) )()( 109
101 nnTnTnT Θ++=
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.40
Analysis of nice case)(nT
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.41
Analysis of nice casecn
( )nT 101 ( )nT 10
9
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.42
Analysis of nice casecn
cn101 cn10
9
( )nT 1001 ( )nT 100
9 ( )nT 1009 ( )nT 100
81
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.43
Analysis of nice casecn
cn101 cn10
9
cn1001 cn100
9 cn1009 cn100
81
Θ(1)
Θ(1)
… …log10/9n
cn
cn
cn
…
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.44
log10n
Analysis of nice casecn
cn101 cn10
9
cn1001 cn100
9 cn1009 cn100
81
Θ(1)
Θ(1)
… …log10/9n
cn
cn
cn
T(n) ≤ cn log10/9n + Ο(n)
…
cn log10n ≤
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.45
Randomized quicksort• Partition around a random element. I.e.,
around A[t] , where t chosen uniformly at random from {p…r}
• We will show that the expected time is O(n log n)
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.46
“Paranoid” quicksort
• Will modify the algorithm to make it easier to analyze:
• Repeat:• Choose the pivot to be a random element of the array• Perform PARTITION
• Until the resulting split is “lucky”, i.e., not worse than 1/10: 9/10• Recurse on both sub-arrays
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.51
Quicksort in practice
• Quicksort is a great general-purpose sorting algorithm.
• Quicksort is typically over twice as fast as merge sort.
• Quicksort can benefit substantially from code tuning.
• Quicksort behaves well even with caching and virtual memory.
• Quicksort is great!
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.52
More intuitionSuppose we alternate lucky, unlucky, lucky, unlucky, lucky, ….
L(n) = 2U(n/2) + Θ(n) luckyU(n) = L(n – 1) + Θ(n) unlucky
Solving:L(n) = 2(L(n/2 – 1) + Θ(n/2)) + Θ(n)
= 2L(n/2 – 1) + Θ(n)= Θ(n lg n)
How can we make sure we are usually lucky?Lucky!
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.53
Randomized quicksort analysis
Let T(n) = the random variable for the running time of randomized quicksort on an input of size n, assuming random numbers are independent.For k = 0, 1, …, n–1, define the indicator random variable
Xk =1 if PARTITION generates a k : n–k–1 split,0 otherwise.
E[Xk] = Pr{Xk = 1} = 1/n, since all splits are equally likely, assuming elements are distinct.
Expected valueExpected value = Outcome • Probability
E[Xk] = 1•Pr{Xk = 1} + 0•Pr{Xk = 0}= 1•(1/n) + 0 •((n-1)/n)= 1/n
Dice Example:Dice Example:
Xk : Indicator random variable of dice throwing
Xk = 1 if dice’s value is k0 otherwise
E[X5] = 1•Pr{X5 = 1} + 0•Pr{X5 = 1}= 1•(1/6) + 0•(5/6) = 1/6
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.54
Analysis (continued)
T(n) =
T(0) + T(n–1) + Θ(n) if 0 : n–1 split,T(1) + T(n–2) + Θ(n) if 1 : n–2 split,
MT(n–1) + T(0) + Θ(n) if n–1 : 0 split,
( )∑−
=Θ+−−+=
1
0)()1()(
n
kk nknTkTX .
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.55
Calculating expectation( )
Θ+−−+= ∑
−
=
1
0)()1()()]([
n
kk nknTkTXEnTE
Take expectations of both sides.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.56
Calculating expectation( )
( )[ ]∑
∑−
=
−
=
Θ+−−+=
Θ+−−+=
1
0
1
0
)()1()(
)()1()()]([
n
kk
n
kk
nknTkTXE
nknTkTXEnTE
Linearity of expectation.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.57
Calculating expectation( )
( )[ ]
[ ] [ ]∑
∑
∑
−
=
−
=
−
=
Θ+−−+⋅=
Θ+−−+=
Θ+−−+=
1
0
1
0
1
0
)()1()(
)()1()(
)()1()()]([
n
kk
n
kk
n
kk
nknTkTEXE
nknTkTXE
nknTkTXEnTE
Independence of Xk from other random choices.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.58
Calculating expectation( )
( )[ ]
[ ] [ ]
[ ] [ ] ∑∑∑
∑
∑
∑
−
=
−
=
−
=
−
=
−
=
−
=
Θ+−−+=
Θ+−−+⋅=
Θ+−−+=
Θ+−−+=
1
0
1
0
1
0
1
0
1
0
1
0
)(1)1(1)(1
)()1()(
)()1()(
)()1()()]([
n
k
n
k
n
k
n
kk
n
kk
n
kk
nn
knTEn
kTEn
nknTkTEXE
nknTkTXE
nknTkTXEnTE
Linearity of expectation; E[Xk] = 1/n .
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.59
Calculating expectation( )
( )[ ]
[ ] [ ]
[ ] [ ]
[ ] )()(2
)(1)1(1)(1
)()1()(
)()1()(
)()1()()]([
1
1
1
0
1
0
1
0
1
0
1
0
1
0
nkTEn
nn
knTEn
kTEn
nknTkTEXE
nknTkTXE
nknTkTXEnTE
n
k
n
k
n
k
n
k
n
kk
n
kk
n
kk
Θ+=
Θ+−−+=
Θ+−−+⋅=
Θ+−−+=
Θ+−−+=
∑
∑∑∑
∑
∑
∑
−
=
−
=
−
=
−
=
−
=
−
=
−
=
Summations have identical terms.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.60
Hairy recurrence
[ ] )()(2)]([1
2nkTE
nnTE
n
kΘ+= ∑
−
=
(The k = 0, 1 terms can be absorbed in the Θ(n).)
Prove: E[T(n)] ≤ an lg n for constant a > 0 .
Use fact: 21
2812
21 lglg nnnkk
n
k∑
−
=−≤ (exercise).
• Choose a large enough so that an lg ndominates E[T(n)] for sufficiently small n ≥ 2.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.61
Substitution method
[ ] )(lg2)(1
2nkak
nnTE
n
kΘ+≤ ∑
−
=
Substitute inductive hypothesis.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.62
Substitution method
[ ]
)(81lg
212
)(lg2)(
22
1
2
nnnnna
nkakn
nTEn
k
Θ+
−≤
Θ+≤ ∑−
=
Use fact.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.63
Substitution method
[ ]
Θ−−=
Θ+
−≤
Θ+≤ ∑−
=
)(4
lg
)(81lg
212
)(lg2)(
22
1
2
nannan
nnnnna
nkakn
nTEn
k
Express as desired – residual.
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.64
Substitution method
[ ]
nan
nannan
nnnnna
nkakn
nTEn
k
lg
)(4
lg
)(81lg
212
)(lg2)(
22
1
2
≤
Θ−−=
Θ+
−=
Θ+≤ ∑−
=
if a is chosen large enough so that an/4 dominates the Θ(n).
,
September 20, 2004 (c) Piotr Indyk & Charles Leiserson L4.66
Randomized Algorithms
• Algorithms that make decisions based on random coin flips.
• Can “fool” the adversary.• The running time (or even correctness) is a
random variable; we measure the expectedrunning time.
• We assume all random choices are independent .
• This is not the average case !