04/19/23 Tucker, Sec. 3.4 1

Applied Combinatorics, 4th Ed.Alan Tucker

Section 3.4

Tree Analysis of Sorting AlgorithmsPrepared by: Nathan Rounds and David Miller

04/19/23 Tucker, Sec. 3.4 2

“Big O” Notation

− If we have a function f (x) defined

f (x) = 5x³ + x – 4,

we can approximate its value as a constant multiple of x³ for large values of x, since 5x³ is the dominating term in the function (it grows the fastest as x gets large).

− For x = 5, f (x) = 626 and the constant multiple 5(x³) = 625 which is a very close estimate.

− Therefore, we say that f (x) in “Big O” notation is O(x³).

04/19/23 Tucker, Sec. 3.4 3

Theorem

• In the worst case, the number of binary comparisons required to sort n items is at least O(nlog2n).

• This number is a constant multiple of (nlog2n) which approximates the height of a binary testing tree (log2n!) for large values of n.

04/19/23 Tucker, Sec. 3.4 4

Bubble SortFor m from 2 to n do:

For i from n (step-1) to m do:

If Ai Ai-1, then interchange items Ai and Ai-1.

4

6

3

1

5

2

m = 2

i = 6

{

4

6

3

1

2

5

4

6

1

3

2

5

m = 2

i = 5

m = 2

i = 4

4

1

6

3

2

5

m = 2

i = 3

m = 2

i = 2

1

4

6

3

2

5

This technique moves the smallest number to the top, then the next smallest number below that until the set of numbers is in numerical order.

i = 1

i = 2

i = 3

i = 4

i = 5

i = 6

m = 3

i = 6

m = 3

i = 5

m = 3

i = 4

1

4

6

2

3

5

1

4

2

6

3

5

m = 3

i = 3

1

2

4

6

3

5

m = 3

i = 2

04/19/23 Tucker, Sec. 3.4 5

Bubble Sort Comparisons

• When m=2, I goes from n to 2, which means you have to do (n-1) comparisons

• When m=3, I goes from n to 3, which means you have to do (n-2) comparisons

• Continuing this trend, the total number of comparisons must be:

121 1

2 21

1

2

n

i

n ni n n

This shows that the Bubble Sort method will require comparisons, which is more then the theoretical bound of .

1 2 1n n

2O n 2logO n n

04/19/23 Tucker, Sec. 3.4 6

Merge Sort – Setup• Roughly splits the set in half, until only single elements are left.

5 4 0 9 2 6 7 1 3 8

5 4 0 9 2 6 7 1 3 8

5 4 0 9 267 1 3 8

0

5 49 2

6 7

1 3 8

5 4 6 7

04/19/23 Tucker, Sec. 3.4 7

Merge Sort - Ordering• The set then gets “merged” into numerical order.

5 4

0 9 2

6 71

3 8

4 5 6 72 9

3 8

0 4 5 1 6 7

0 2 4 5 9 1 3 6 7 8

0 1 2 3 4 5 6 7 8 9

Note: The root of the tree

is on the bottom

04/19/23 Tucker, Sec. 3.4 8

Complexity of Merging• Assume that every set has n = 2r elements

– There are r levels– There are 2k vertices on level k– On level k, each set will have 2r-k elements

n = 24

++++++++++++++++

++++++++ ++++++++

++++ ++++ ++++ ++++

++ ++ ++ ++ ++ ++ ++ ++

+ + + + ++ + + + + + + + + + +

Level 0

Level 1

Level 2

Level 3

Level 4

04/19/23 Tucker, Sec. 3.4 9

Merge Sort Comparisons

• In general, at each vertex on level k two ordered sublists of 2r-k-1 items are merged into an ordered sublist of 2r-k items

• This merging requires 2r-k-1 comparisons• The two ordered sublists on level one merge into the root• The number of comparisons needed at level k is 2k (2r-k – 1 ) = 2r - 2k

since there are 2k different vertices• Totaling the number of comparisons needed for a Merge Sort is:

• With n = 2r this becomes:• This shows that Merge Sort method requires

comparisons, which achieves the theoretical bound of a binary search.

1 1 1

0 0 0

2 2 2 2 2 2 1r r r

r k r k r r

k k k

r

2log 1n n n 2logO n n

04/19/23 Tucker, Sec. 3.4 10

QUIK Sort• Takes the first element and compares it with the other elements to

divide the list, putting smaller elements into a set on the left and the larger elements into a set on the right.

5 4 0 7 1 6 8 2 3 9

4 0 1 2 3 5 7 6 8 9

Puts the first element at the end of the left list

0 1 2 3 4 5 6 7 8 9Once an element is alone

there is no need to continue to divide it

0

2 1 3 4 6 7 8 9

1 2 3 4

1 2 3 4

04/19/23 Tucker, Sec. 3.4 11

Heap Sort

• A binary tree so that the parent is always bigger than its children

6

3

0 2 1

5

4

Put the root at the beginning of a list, then move up the next biggest grandchildren.

LIST:

04/19/23 Tucker, Sec. 3.4 12

Class Exercise

• Sort the following set of numbers using a bubble sort and then do it using a merge sort.

• Which sort technique is more efficient for this set?

5,3,8,23, 6,0,2,66S

04/19/23 Tucker, Sec. 3.4 13

Bubble Sort Solution5

3

8

23

-6

0

2

66

5

3

8

-6

23

0

2

66

5

3

-6

8

23

0

2

66

5

-6

3

8

23

0

2

66

-6

5

3

8

23

0

2

66 . . .

-6

0

2

5

3

8

23

66

-6

0

5

2

3

8

23

66

-6

0

5

3

2

8

23

66

-6

0

2

3

5

8

23

66

04/19/23 Tucker, Sec. 3.4 14

Merge Sort Solution

5 3 8 23 -6 0 2 66

5 3 8 23 -6 0 2 66

5 3 8 23-6 0 2 66

35 8 23 0 2 66-6

04/19/23 Tucker, Sec. 3.4 15

Merge Sort Solution cont’d

35 8 23 0 2 66-6

3 5 8 23 -6 0 2 66

3 5 8 23 -6 0 2 66

-6 0 2 3 5 8 23 66

04/19/23 Tucker, Sec. 3.4 16

Something to Ponder

• How does one create an initial heap to perform a heap sort on?

• This will be on the homework.