COMP3600/6466 – Algorithms More on Sorting Cont.

[CLRS ch. 8]

Hanna Kurniawati

https://cs.anu.edu.au/courses/comp3600/Comp_3600_6466@anu.edu.au

TopicsüLower bound on worst case time

complexity of comparison-based sorting• Non-sorting-based sortingüCounting sort• Radix sort• Bucket sort

Radix Sort• Sort d-digit numbers

• i=1 is the least significant digit –the last digit of the numbers • Usually, line 2 uses Counting Sort• Example:

• Complexity: Θ(𝑑(𝑛 + 𝑘)), assuming Counting Sort is used

325457657839431

Input431325457657839

i=1325431839457657

i=2325431457657839

i=3

TopicsüLower bound on worst case time

complexity of comparison-based sorting• Non-sorting-based sortingüCounting sortüRadix sort• Bucket sort

Bucket Sort • Assumption: Sorting numbers in the interval [0,1)

that are drawn independently from a uniform distribution • The idea is to transform the problem of sorting n

elements into n problems of sorting k elements, where k is much smaller than n. • It starts by assigning similar elements to the same

bucket, and then sort the elements within the bucket

Bucket Sort

Bucket Sort: Illustration

0.780.170.390.260.720.940.210.120.230.68

123456789

10

A0123456789

B

0.78

0.17

0.390.26

0.72

Bucket Sort: Illustration

Bucket Sort: Running-time• 𝑇 𝑛 = Θ 𝑛 + ∑()*+,-𝑂(𝑛(.)

• Average case:𝐸[𝑇 𝑛 ] = 𝐸[Θ 𝑛 + ∑()*+,-𝑂 𝑛(. ]

Using linearity of expectation, we have𝐸[𝑇 𝑛 ] = 𝐸[Θ 𝑛 ] + ∑()*+,-𝐸[𝑂 𝑛(. ]𝐸[𝑇 𝑛 ] = Θ 𝑛 + ∑()*+,-𝑂 𝐸[𝑛(

.]

Cost of Insertion Sort in bucket-𝑖

Bucket Sort: Illustration Running-time• Now, we need to compute 𝐸[𝑛!" ] where 𝑛! is the

expected number of elements in bucket- 𝑖• Use Indicator Random Variable

• 𝑋!,# = 𝐼 𝐴 𝑗 𝑖𝑛 𝑏𝑢𝑐𝑘𝑒𝑡 𝑖 = :1 𝐴 𝑗 𝑖𝑛 𝑏𝑢𝑐𝑘𝑒𝑡 𝑖0 𝑂𝑡ℎ𝑒𝑟𝑤𝑖𝑠𝑒

• Since the input are independent and uniformly distributed samples, 𝑃(𝑋!,# = 1) = $

%

Bucket Sort: Running-time• Now, we can compute: 𝐸[𝑛!

" ] = 𝐸 ∑#$%& 𝑋!,#"

= 𝐸 ∑#$%& ∑($%& 𝑋!,#𝑋!,(= 𝐸 ∑#$%& 𝑋!,#" +∑#$%& ∑%)()&,(*#𝑋!,#𝑋!,(= ∑#$%& 𝐸[𝑋!,#" ] + ∑#$%& ∑%)()&,(*#𝐸[𝑋!,#𝑋!,(]

𝐸 𝑋!,#" = 1". %&+ 0" 1 − %

&= %

&

𝐸 𝑋!,#𝑋!,( = 𝐸 𝑋!,# 𝐸 𝑋!,( = %&%&= %

&&

Bucket Sort: Running-time• Now, we can plug the expected values back to

𝐸[𝑛!" ] = ∑#$%& 𝐸[𝑋!,#" ] + ∑#$%& ∑%)()&,(*#𝐸[𝑋!,#𝑋!,(]

= ∑#$%& %&+∑#$%& ∑%)()&,(*#

%&&

= 𝑛 %&+ 𝑛 𝑛 − 1 %

&&= 1 + (&,%)

&= 2 − %

&

• Returning to 𝑇 𝑛 :𝑇 𝑛 = Θ 𝑛 +∑!$.&,%𝑂(𝑛!")

= Θ 𝑛 + 𝑛.𝑂 2 − %&

= Θ 𝑛

Notes-1• Comparison-based methods can be used to sort any

type of data, as long as there’s a comparison function• Its running-time complexity is asymptotically lower bounded

by 𝑛 log 𝑛 --that is, Ω(𝑛 log 𝑛), where n is the input size• Non comparison-based can run in linear time. But, it

assumes the data to be sorted is of a certain type.• Tips: More efficient algorithms can be developed by

focusing on a sub-class of a problem (in the sorting case, a particular data type of the input).

Notes-2• All the non-comparison based methods we

discussed are not in-place, which means it takes more memory• In-place Sorting Algorithm: Re-arrange the elements inside

the array, with only a constant number of input array being stored outside the array at any given time

• There are comparison-based algorithms that are not in-place (e.g., merge sort whose worst case run-time is Θ(𝑛 log 𝑛)• Insertion Sort, Bubble Sort, and Rand Quick Sort are in-

place. But, the worst case run-time of all these in-place algorithms are Θ(𝑛')

Notes-2• Tips: Pay attention to Speed vs Memory tradeoff• Usually we can make an algorithm faster by using more

memory and vice versa• Tips: Randomization sometimes help speed-up

while keeping memory requirement low• Example: Rand Quick Sort allows average case running-

time to be Θ(𝑛 log 𝑛) and in practice tend to be close to this average case rather than the worst case Θ(𝑛')

TopicsüLower bound on worst case time

complexity of comparison-based sortingüNon-sorting-based sortingüCounting sortüRadix sortüBucket sort

Next: Binary Search Tree