object-oriented programming 1 and data-structures summer ... · analyse their complexity this...

167
CS/ENGRD 2110 SUMMER 2018 Lecture 8: Sorting http://courses.cs.cornell.edu/cs2110/2018su Object-oriented programming and data-structures 1

Upload: others

Post on 08-Jul-2020

1 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

CS/ENGRD 2110SUMMER 2018Lecture 8: Sorting

http://courses.cs.cornell.edu/cs2110/2018su

Object-oriented programming and data-structures

1

Page 2: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

◻ Introduced a formal notation for analysing the runtime/space complexity of algorithms

◻ Went through examples of Big-O formulas/proofs

◻ Analysed complexity of ArrayList/LinkedList

Lecture 7 Recap 2

Page 3: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

◻ There will not be a prelim next week. Instead, future homeworks will include exam-style questions on the whole course to help you master the revision

◻ A3 will be released tomorrow evening and due next Tuesday

◻ HW5 has been released. It covers a lot of material and will get challenging at times. Start early!

◻ Please fill out the poll on Piazza. Thanks to those who already have.

Admin3

Page 4: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

◻ Introduce Sorting Algorithms

◻ Derive and implement⬜ Insertion sort⬜ Selection sort⬜ Merge sort⬜ Quick sort

◻ Analyse their complexity

This lecture4

Page 5: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Why Sorting?◻ Sorting is useful

⬜ Database indexing

⬜ Compressing data

⬜ Sorting TV channels, Netflix shows, Amazon products, etc.

◻ There are lots of ways to sort

⬜ There isn't one right answer

⬜ You need to be able to figure out the options and decide which one is right for your application.

⬜ Today, we'll learn about several different algorithms (and how to derive them)

5

Page 6: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Some Sorting Algorithms◻ Insertion sort

◻ Selection sort

◻ Binary Sort

◻ Bubble Sort

◻ Merge sort

◻ Quick sort

6

Page 7: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

◻ Worst-case⬜ Complexity in worst possible scenario. Gives an upper bound on

performance, but may only arise rarely◻ Average-case

⬜ Analyse for an 'average' input. Problem here is that need to somehow know what “average” means”

◻ Best-case⬜ What is the minimum number of operations that must be done in the

best case scenario◻ Amortized analysis

⬜ If expensive operation happens rarely, and lots of cheap operations happen frequently, may want to amortise total cost over all the operations to get average cost per operation.

7

Refining our analysis

Page 8: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Let’s begin by looking through an example

◻ Consider the following array

Let’s sort it!

8

3 6 4 5 1 2

Page 9: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

9

3 6 4 5 1 2

Page 10: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

10

3 6 4 5 1 2

Round 0: Position 0 of the array is already sorted.

Sorted Unsorted

Page 11: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

11

3 6 4 5 1 2

Round 0: Select element at position 1 array[i+1] and place to correct position in sorted array

Sorted Unsorted

Page 12: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

12

3 6 4 5 1 2

Round 0: Element does not move as it is greater than sorted array (6>3)

Sorted Unsorted

Page 13: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

13

3 6 4 5 1 2

Round 1: select item at index 2, and swap it to the correct place.

Sorted Unsorted

Page 14: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

14

3 6 4 5 1 2

Round 1: 6>4, so swap 6 and 4

Sorted Unsorted

Page 15: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

15

3 4 6 5 1 2

Round 1: 6>4, so swap 6 and 4

Sorted Unsorted

Page 16: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

16

3 4 6 5 1 2

Round 1: 3<4, and all the elements that precede 3 are sorted, so 4 is in the correct position

Sorted Unsorted

Page 17: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

17

3 4 6 5 1 2

Round 1: 3<4, and all the elements that precede 3 are sorted, so 4 is in the correct position

Sorted Unsorted

Page 18: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

18

3 4 6 5 1 2

Round 2: Select element at position 3 in the array

Sorted Unsorted

Page 19: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

19

3 4 6 5 1 2

Round 2: Try to place it in the right position

Sorted Unsorted

Page 20: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

20

3 4 6 5 1 2

Round 2: 6>5, so swap the two elements

Sorted Unsorted

Page 21: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

21

3 4 5 6 1 2

Round 2: 6>5, so swap the two elements

Sorted Unsorted

Page 22: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

22

3 4 5 6 1 2

Round 2: 4<5, and the array array[0..1] is sorted, so 5 is in the correct position

Sorted Unsorted

Page 23: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

23

3 4 5 6 1 2

Round 2: 4<5, and the array array[0..1] is sorted, so 5 is in the correct position

Sorted Unsorted

Page 24: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

24

3 4 5 6 1 2

Round 3: Select element at position 4 (i+1) in the array. The first

Sorted Unsorted

Page 25: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

25

3 4 5 6 1 2

Round 3: Place it at the appropriate position in the sorted array

Sorted Unsorted

Page 26: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

26

3 4 5 6 1 2

Round 3: 6>1, so swap

Sorted Unsorted

Page 27: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

27

3 4 5 1 6 2

Round 3: 6>1, so swap

Sorted Unsorted

Page 28: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

28

3 4 5 1 6 2

Round 3: 5>1, so swap

Sorted Unsorted

Page 29: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

29

3 4 1 5 6 2

Round 3: 5>1, so swap

Sorted Unsorted

Page 30: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

30

3 4 1 5 6 2

Round 3: 4>1, so swap

Sorted Unsorted

Page 31: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

31

3 1 4 5 6 2

Round 3: 4>1, so swap

Sorted Unsorted

Page 32: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

32

3 1 4 5 6 2

Round 3: 3>1, so swap

Sorted Unsorted

Page 33: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

33

1 3 4 5 6 2

Round 3: 3>1, so swap

Sorted Unsorted

Page 34: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

34

1 3 4 5 6 2

Round 3: Reached end of the array, so stop

Sorted Unsorted

Page 35: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

35

1 3 4 5 6 2

Round 3: Reached end of the array, so stop

Sorted Unsorted

Page 36: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

36

1 3 4 5 6 2

Round 4: Repeat process for 2

Sorted Unsorted

Page 37: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

37

1 2 3 4 5 6

Round 4: Repeat process for 2

Sorted Unsorted

Page 38: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort

◻ Insertion sort iterates over the array, swapping pairs of integers until they are in the correct position

◻ Maintains the following invariant: at round i, array[0,i] is sorted

38

1 2 3 4 5 6

Round 4: Array is fully sorted.

Sorted Unsorted

Page 39: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to implement Insertion Sort?39

// sort b[], an array of int// inv: b[0..i] is sortedfor (int i= 0; i < b.length - 1; i= i+1) { // Push b[i+1] down to its sorted // position in b[0..i]

}

while (k > 0 && b[k-1] > b[k]) {swap(b,k-1,k);k--;

}

int k= i+1;

Page 40: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort - Analysis40

◻ How many comparisons does each round of the algorithm do?

Page 41: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort - Analysis41

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, does at most 1

⬜ Round 1, at most 2. Round 2 at most 2, …

⬜ Round i does i+1 comparisons max

◻ How many rounds are there?

Page 42: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort - Analysis42

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, does at most 1

⬜ Round 1, at most 2. Round 2 at most 2, …

⬜ Round i does i+1 comparisons max

◻ How many rounds are there?

⬜ N-1 rounds

Page 43: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort - Analysis43

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, does at most 1

⬜ Round 1, at most 2. Round 2 at most 2, …

⬜ Round i does i+1 comparisons max

◻ How many rounds are there?

⬜ N-1 rounds

◻ How many comparisons in total?

⬜ n(n-1)/2

Page 44: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Insertion Sort - Analysis44

◻ Insertion sort is therefore O(n^2)

◻ In practice however, is it going to be expensive?

⬜ Can you think of scenarios where insertion sort is likely to perform well?

Page 45: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort45

◻ Selection sort has a similar invariant as insertion sort:

⬜ At round i, the positions before a[i] are already sorted

◻ Instead of swapping values, iterate over the unsorted array a[i..n-1] to find the minimum value, and place it in a[i].

sorted, smaller values larger valuesb0 i length

Each iteration, swap min value of this section into b[i]

Page 46: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Let’s begin by looking through an example

◻ Consider the following array

Let’s sort it!

46

3 6 4 5 1 2

Page 47: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

47

3 6 4 5 1 2

Page 48: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

48

3 6 4 5 1 2

Round 0: Find the minimum element in a[i,n-1]

Page 49: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

49

3 6 4 5 1 2

Round 0: Find the minimum element in a[i,n-1]

Page 50: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

50

3 6 4 5 1 2

Round 0: Now swap with a[0] (a[i])

Page 51: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

51

1 6 4 5 3 2

Round 0: Now swap with a[0] (a[i])

Page 52: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

52

1 6 4 5 3 2

Round 1: Consider a[1] ( = 6). Find the minimum in a[1,n-1]

Page 53: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

53

1 6 4 5 3 2

Round 1: Consider a[1] ( = 6). Find the minimum in a[1,n-1]

Page 54: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

54

1 6 4 5 3 2

Round 1: Now swap with a[1]

Page 55: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

55

1 2 4 5 3 6

Round 1: Now swap with a[1]

Page 56: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

56

1 2 4 5 3 6

Round 2: Look at a[2]. Find minimum

Page 57: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

57

1 2 4 5 3 6

Round 3: Look at a[2]. Find minimum

Page 58: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

58

1 2 3 5 4 6

Round 2: Swap with a[2]

Page 59: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

59

1 2 3 5 4 6

Round 3: Look at a[3]

Page 60: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

60

1 2 3 5 4 6

Round 3: Look at a[3]. Find minimum

Page 61: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

61

1 2 3 4 5 6

Round 3: Swap with a[3]

Page 62: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

62

1 2 3 4 5 6

Round 4: Look at a[4]. Find minimum

Page 63: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

63

1 2 3 4 5 6

Round 4: Swap with itself

Page 64: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

64

1 2 3 4 5 6

Round 5: Look at a[5].

Page 65: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort

◻ Selection sort over the array, placing the minimum element of the unsorted array in a[i]

◻ Maintains the following invariant: at round i, the positions before array[i] are sorted

65

1 2 3 4 5 6

Array is sorted!

Page 66: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to implement Selection Sort?66

// sort b[], an array of int// inv: positions before b[i] are sortedfor (int i= 0; i < b.length - 1; i= i+1) { // Find the smallest element in //a [i..end] and swap it into a[i]

}

while (k > 0 && b[k-1] > b[k]) {swap(b,k-1,k);k--;

}

int k= i+1;

Page 67: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort - Analysis67

◻ How many operations does each round of the algorithm do?

Page 68: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort - Analysis68

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, n ( +1 swap)

⬜ Round 1, n-1 (+ 1 swap)

⬜ Round i does n - i comparisons ( + 1 swap)

◻ How many rounds are there?

Page 69: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort - Analysis69

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, n ( +1 swap)

⬜ Round 1, n-1 (+ 1 swap)

⬜ Round i does n - i comparisons ( + 1 swap)

◻ How many rounds are there?

⬜ n

Page 70: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort - Analysis70

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, n ( +1 swap)

⬜ Round 1, n-1 (+ 1 swap)

⬜ Round i does n - i comparisons ( + 1 swap)

◻ How many rounds are there?

⬜ N

◻ How many comparisons in total?

Page 71: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort - Analysis71

◻ How many comparisons does each round of the algorithm do?

⬜ Round 0, n ( +1 swap)

⬜ Round 1, n-1 (+ 1 swap)

⬜ Round i does n - i comparisons ( + 1 swap)

◻ How many rounds are there?

⬜ N

◻ How many comparisons in total?

⬜ n(n+1)/2

Page 72: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Selection Sort - Analysis72

◻ Selection sort is therefore O(n^2)

◻ In practice however, is it going to be expensive?

◻ What is likely to be faster ?

⬜ Insertion sort?

⬜ Selection sort?

Page 73: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort73

◻ Could recursion save the day?

◻ Instead of processing one large big array, what if we recursively subdivided each array into smaller arrays, sorted those subarrays, and then merged the sorted arrays together at the end?

◻ What would be the base case of merge sort?

Page 74: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort74

◻ Could recursion save the day?

◻ Instead of processing one large big array, what if we recursively subdivided each array into smaller arrays, sorted those subarrays, and then merged the sorted arrays together at the end?

◻ What would be the base case of merge sort?

Page 75: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort75

◻ Let’s begin by looking through an example

◻ Consider the following array

Let’s sort it!

3 6 4 5 1 2

Page 76: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort76

3 6 4 5 1 2

Let’s first partition the array into two smaller arrays

Page 77: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort77

3 6 4 5 1 2

Let’s first partition the array into two smaller arrays

3 6 4 5 1 2

Page 78: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort78

3 6 4 5 1 2

Let’s first partition the array into two smaller arrays

3 6 4 5 1 2

And again

Page 79: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort79

3 6 4 5 1 2

Let’s first partition the array into two smaller arrays

3 6 4 5 1 2

And again

3 6 4 5 1 2

Page 80: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort80

3 6 4 5 1 2

Let’s first partition the array into two smaller arrays

3 6 4 5 1 2

And again

3 6 4 5 1 2

And again

3 6 4 5 1 2

Page 81: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort81

3 6 4 5 1 2

Let’s first partition the array into two smaller arrays

3 6 4 5 1 2

And again

3 6 4 5 1 2

And again

3 6 4 5 1 2

Arrays of size 1 are sorted. Base case!

Page 82: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge Sort

3 6 4 5 1 2

Now merge sorted arrays back together

3 6

3 64

51

51 2

3 64 51 2

Page 83: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?83

3

6

4

5

1

2

Page 84: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?84

3

6

4

5

1

2

Step 1: Create an array of size a.length + b.length

Page 85: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?85

3

6

4

5

1

2

Step 1: Create an array of size a.length + b.length

Step 2: Where can we find the smallest element of the new array?

Page 86: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?86

3

6

4

5

1

2

Step 1: Create an array of size a.length + b.length

Step 2: Where can we find the smallest element of the new array?

It is either the smallest element of a or the smallest element of b

Step 3: Where can we find the second smallest element of the new array?

Depending on what we chose last, it is either the first element of a/b or the second element of a/b

Page 87: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?87

3

6

4

5

1

2

Step 1: Create an array of size a.length + b.length

Step 2: Where can we find the smallest element of the new array?

It is either the smallest element of a or the smallest element of b

Step 3: Where can we find the second smallest element of the new array?

Depending on what we chose last, it is either the first element of a/b or the second element of a/b

Keep two indices headA and headB. When select an element of a, incremeant headA. When select an element of b, increment headB. Then test for a[headA]<=b[headB] at every iteration

Page 88: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?88

3

6

4

5

1

2

headA = 0;headB = 0;

Page 89: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?89

3

6

4

5

1

2

headA = 0;headB = 0;

Page 90: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?90

3

6

4

5

1

2

headA = 0;headB = 0;

a[headA] <= b[headB]?No -> result[0] = b[headB]; headB++

Page 91: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?91

3

6

4

5

2

headA = 0;headB = 1;

a[headA] <= b[headB]?No -> result[1] = b[headB]; headB++

1

Page 92: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?92

3

6

4

5

headA = 0;headB = 2;

a[headA] <= b[headB]?Yes -> result[2] = a[headA]; headA++

1

2

Page 93: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?93

6

4

5

headA = 1;headB = 2;

a[headA] <= b[headB]?Yes -> result[3] = a[headA]; headA++3

1

2

Page 94: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?94

6 5

headA = 2;headB = 2;

a[headA] <= b[headB]?Yes -> result[3] = a[headA]; headA++3

1

2

4

Page 95: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?95

6

headA = 1;headB = 2;

a[headA] <= b[headB]?No -> result[3] = b[headB]; headB++3

1

2

4

5

Page 96: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to merge two sorted arrays?96

headA = 2;headB = 2;

headB >= b.length.result[3] = a[headA]; headA++3

1

2

6

4

5

Page 97: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge sort complexity analysis97

◻ And more generally: how do we analyse the complexity of a recursive algorithm.

Page 98: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge sort complexity analysis98

◻ And more generally: how do we analyse the complexity of a recursive algorithm.

◻ First: what is the complexity of the function that merges two sorted arrays?

Page 99: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge sort complexity analysis99

◻ And more generally: how do we analyse the complexity of a recursive algorithm.

◻ First: what is the complexity of the function that merges two sorted arrays?

⬜ One pass over the arrays, so O(n)

Page 100: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge sort complexity analysis100

◻ And more generally: how do we analyse the complexity of a recursive algorithm.

◻ First: what is the complexity of the function that merges two sorted arrays?

⬜ One pass over the arrays, so O(n)

◻ When algorithm contains recursive call, can describe its running time by a recurrence equation that describes running time of a problem of size n in terms of running time on smaller inputs.

Page 101: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Recurrence Relations101

◻ Let T(n) be the running time on a problem of size n.

◻ If the problem is small enough (ex: base case), say n<= c for some constant c, solving the base case takes constant time O(1)

◻ Assume that the recursive call yields a subproblems, each of which is 1/b of the size of the original. It takes time T(n/b) to solve one subproblem of size n/b and so it takes aT(n/b) to solve a of them. If it takes D(n) to subdivide the arrays, and C(n) to combine them, then we have the following recurrence rel

T(n) = O(1) if n<= cT(n) = aT(n/b) + D(n) + C(n) otherwise

n/b n/b n/b n/ba=4

Page 102: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Recurrence Relation for Merge Sort102

◻ In the general case:⬜ T(n) = O(1) if n<= c

T(n) = aT(n/b) + D(n) + C(n) otherwise

◻ For merge sort:

Page 103: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Recurrence Relation for Merge Sort103

◻ In the general case:⬜ T(n) = O(1) if n<= c

T(n) = aT(n/b) + D(n) + C(n) otherwise

◻ For merge sort:

⬜ a = 2, b = 2

Page 104: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Recurrence Relation for Merge Sort104

◻ In the general case:⬜ T(n) = O(1) if n<= c

T(n) = aT(n/b) + D(n) + C(n) otherwise

◻ For merge sort:

⬜ a = 2, b = 2

⬜ D(n) is O(1) (computes middle of the subarray)

Page 105: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Recurrence Relation for Merge Sort105

◻ In the general case:⬜ T(n) = O(1) if n<= c

T(n) = aT(n/b) + D(n) + C(n) otherwise

◻ For merge sort:

⬜ a = 2, b = 2

⬜ D(n) is O(1) (computes middle of the subarray)

⬜ C(n) is O(n) (merge procedure of sorted arrays)

Page 106: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Recurrence Relation for Merge Sort106

◻ In the general case:⬜ T(n) = O(1) if n<= c

T(n) = aT(n/b) + D(n) + C(n) otherwise

◻ For merge sort:

⬜ a = 2, b = 2

⬜ D(n) is O(1) (computes middle of the subarray)

⬜ C(n) is O(n) (merge procedure of sorted arrays)

◻ T(n) = c if n = 1 T(n) = 2T(n/2) + cn if n > 1 (For simplicitly lets assume that n is a power of 2)

Page 107: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?107

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

Page 108: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?108

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

Each level in the tree has exactly cn cost:Level 0 has cnLevel 1 has cn/2 + cn/2 = cnLevel 2 has cn/4 + cn/4 + cn/4 + cn/4 = cnLevel i has (i+1) * cn/2^i = cnLevel d (where n = 2^d) has n * cn/2^d = n * c = cn

Page 109: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?109

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

How many levels are there?

Page 110: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?110

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

How many levels are there?

Remember that the recursion stops when the input size is 1

Page 111: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?111

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

How many levels are there?

Remember that the recursion stops when the input size is 1

So if n = 2^1, there would be 2 levels

Page 112: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?112

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

How many levels are there?

Remember that the recursion stops when the input size is 1

So if n = 2^1, there would be 2 levelsIf n = 2^2, there would be 3 levels

Page 113: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?113

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

How many levels are there?

Remember that the recursion stops when the input size is 1

So if n = 2^1, there would be 2 levelsIf n = 2^2, there would be 3 levelsIf n = 2^3, there would be 4 levels

Page 114: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?114

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

How many levels are there?

Remember that the recursion stops when the input size is 1

In general, there are lg(n) + 1 levels in the tree.

Page 115: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?115

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

Each level of the tree does cn work and there are lg(n) + 1 levels of the tree(cn)(lg(n) + 1) = cn* lg(n) + cn

Page 116: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How do we solve the recurrence?116

◻ Use a recurrence tree⬜ Graphically lay out the cost of each level of the recursion in a

tree-structure.

cn

cn/2 cn/2

cn/4 cn/4 cn/4 cn/4

cn/n cn/n cn/n cn/n

...cn/n cn/n...

Merge sort is O(nlgn)

Page 117: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Merge sort vs others117

◻ Insertion sort and selection sort have worse time complexity than merge sort.

◻ But, they have better space complexity as they can sort the data in-place whereas merge sort requires additional arrays (merge sort has O(n) space complexity)

◻ For small inputs, insertion sort is often faster!

Page 118: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Best of both worlds?118

◻ Can we design an algorithm that sorts arrays in-place but with O(nlgn) complexity?

Page 119: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Best of both worlds?119

◻ Can we design an algorithm that sorts arrays in-place but with O(nlgn) complexity?

◻ The answer is almost!

⬜ Quicksort sorts arrays in place and has O(nlgn) complexity in the best-case, but O(n^2) in the worst-case.

Page 120: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort120

◻ Can we design an algorithm that sorts arrays in-place but with O(nlgn) complexity?

◻ The answer is almost!

⬜ Quicksort sorts arrays in place and has O(nlgn) complexity in the best-case, but O(n^2) in the worst-case.

Page 121: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Cute Story121

Quicksort developed by Tony Hoare (he’s currently 83, still works at Microsoft Research)

Developed Quicksort in 1958. But he could not explain it to his colleage, so he gave up on it.

Later, he saw a draft of the new language Algol 58 (which became Algol 60). It had recursive procedures, for the first time, in a procedural programming language. “Ah!”. he said. “I know how to write it better now”. 15 minutes later, his colleague also understood it.

Fun fact: at university, we had a course called Hoare Logic, based on what he invented. He attended our lectures a few times :-). Nothing like attending an entire course named after you.

Page 122: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Key Idea122

◻ Quicksort is recursive like merge sort.

◻ Unlike merge sort, however, quicksort first processes the array before partitioning the array in two.

◻ This processing allows quicksort to have better space complexity

◻ Idea is to pick a pivot element and partition the array into those bigger than the pivot and those smaller than the pivot, calling quicksort recursively on each side of the array.

Page 123: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Partitioning123

◻ Pick a pivot (any element in the array) and partition the array sy such that all elements smaller than the pivot are to the left of the pivot, all the elements greater than the pivot are to the right.

Swap array values around until b[h..k] looks like this:

x ? h h+1 k

<= x x >= x

h j k

pre:

post:

x is called the pivot

Page 124: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example124

◻ Let’s sort this array (again)

3 6 4 5 1 2

Select 5 as the pivot

Page 125: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example125

◻ Let’s sort this array (again)

3 64 512

Partition the array

Page 126: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example126

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

Page 127: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example127

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 4 12 6

Page 128: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example128

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 4 12 6

Page 129: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example129

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 4 12 6

Partition Array

Page 130: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example130

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 41 2 6

Partition Array

Page 131: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example131

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 41 2 6

Run quicksort on two partitions

1 43

Page 132: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example132

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 41 2 6

Run quicksort on two partitions

1 43

4

Page 133: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort - Example133

◻ Let’s sort this array (again)

3 64 512

Run quicksort on the two partitions

3 41 2 6

Run quicksort on two partitions

1 43

4

When we reach the base case, no need to merge. The array is already sorted!

Page 134: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm134

◻ The secret sauce of quicksort is its partitioning algorithm that partitions the array in place

◻ Partition function as it executes, partitions the array into four regions:

<=x > x unrestricted x

p i j A[r]

Define several terms: p, the beginning index of the array that we want to partition. i is the start of the region for which >x. j is the end of the region for which >x. a[r] is the pivot x

Page 135: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm135

◻ The secret sauce of quicksort is its partitioning algorithm that partitions the array in place

◻ Partition function as it executes, partitions the array into four regions:

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

<=x > x unrestricted x

p i j A[r]

Page 136: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm136

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

Page 137: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm137

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

Initialise i to p-1, j to p, and select the last element as the pivot

i p,j r

Page 138: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm138

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

Now loop from j = p to j = r-1

i p,j r

Page 139: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm139

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

If A[j] <= x, increment a[i] to indicate that there is now one element that is < x. Then swap A[j] with A[i]

i p,j r

Page 140: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm140

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

If A[j] <= x, increment a[i] to indicate that there is now one element that is < x. Then swap A[j] with A[i]

p,j,i r

Page 141: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm141

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

If A[i]> x, then do not change i, and simply increment j. The partition between a[i+1] and a[j-1] denotes the values that are greater than the pivot

p,i rj

Page 142: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm142

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

If A[i]> x, then do not change i, and simply increment j. The partition between a[i+1] and a[j-1] denotes the values that are greater than the pivot

p,i rj

Page 143: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm143

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

If A[i]> x, then do not change i, and simply increment j. The partition between a[i+1] and a[j-1] denotes the values that are greater than the pivot

p,i rj

Page 144: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm144

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

If A[j] <= x, increment a[i] to indicate that there is now one element that is < x. Then swap A[j] with A[i]

p,i rj

Page 145: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm145

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 7 1 3 5 6 4

i = 0 + 1, so swap a[1] ( = 8 ) with a[j] = 1.

p,i rj

Page 146: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm146

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 871 3 5 6 4

i = 0 + 1, so swap a[1] ( = 8 ) with a[j] = 1.

p rj i

Page 147: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm147

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 871 3 5 6 4

If A[j] <= x, increment a[i] to indicate that there is now one element that is < x. Then swap A[j] with A[i]

p rj i

Page 148: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm148

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 871 3 5 6 4

i = 1 + 1, so swap a[2] ( = 7 ) with a[j] = 3.

p rj i

Page 149: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm149

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 71 3 5 6 4

i = 1 + 1, so swap a[2] ( = 7 ) with a[j] = 3.

p rj i

Page 150: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm150

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 71 3 5 6 4

i = 1 + 1, so swap a[2] ( = 7 ) with a[j] = 3.

p rj i

Page 151: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm151

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 71 3 5 6 4

If A[i]> x, then do not change i, and simply increment j. The partition between a[i+1] and a[j-1] denotes the values that are greater than the pivot

p rj i

Page 152: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm152

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 71 3 5 6 4

If A[i]> x, then do not change i, and simply increment j. The partition between a[i+1] and a[j-1] denotes the values that are greater than the pivot

p rj i

Page 153: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm153

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 8 71 3 5 6 4

When j = r, exchange a[i+1] (i=2, so 8) with A[r]

p r,j i

Page 154: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Back to the partition algorithm154

Goes in a loop from p to r-1, and maintains the following invariant for each element a[k] in the array

◻ If p <= k < =i, then a[k] <= x◻ If i+1 <=k<=j-1, then A[k]>x

◻ Let’s sort this array

2 871 3 5 64

Important point: the pivot is now in the correct location for the sorted array. Now recurse on a[p,i] and a[i+2, r]

p r,j i

Page 155: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort Pseudocode155

Quicksort(a,p,r): if p < r

q = partition(a,p,r)quicksort(a,p,q-1)quicksort(a,q+1,r)

Partition(a,p,r): x = a[r] i = p-1 For j = p to r-1

If a[j] <=xI = i+1swap(a[i],a[j])

Swap(a[i+1], a[r]) return i+1

Page 156: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis156

◻ What is the complexity of the partition function?

Page 157: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis157

◻ What is the complexity of the partition function?⬜ O(n)

Page 158: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis158

◻ What is the complexity of the partition function?⬜ O(n)

◻ What about for Quicksort?

Page 159: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis159

◻ What is the complexity of the partition function?⬜ O(n)

◻ What about for Quicksort?⬜ Let’s write the recurrence relation

◻ T(n) = c if n = 1

Page 160: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis160

◻ What is the complexity of the partition function?⬜ O(n)

◻ What about for Quicksort?⬜ Let’s write the recurrence relation

◻ T(n) = c if n = 1

◻ T(n) = T(n1) + T(n

2) + O(n) where n

1 is array size before pivot, n

2 after

Page 161: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis161

◻ T(n) = c if n = 1

◻ T(n) = T(n1) + T(n

2) + O(n) where n

1 is array size before pivot, n

2 after

◻ If choose pivot such that exactly in the middle of the array: n1 = n

2 = n/2

⬜ T(n) = 2T(n/2) + O(n)

Page 162: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis162

◻ T(n) = c if n = 1

◻ T(n) = T(n1) + T(n

2) + O(n) where n

1 is array size before pivot, n

2 after

◻ If choose pivot such that exactly in the middle of the array: n1 = n

2 = n/2

⬜ T(n) = 2T(n/2) + O(n)

⬜ Same as merge sort O(nlgn)

◻ But what if pivot is such that it is always the last element of the array?

⬜ T(n) = O(n) + T(n-1) + O(1)

Page 163: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Quicksort complexity analysis163

◻ If pivot is such that it is always the last element of the array?

⬜ T(n) = O(n) + T(n-1) + O(1)

⬜ N + N -1 + N-2 + … + 2 + 1

◻ Quicksort has O(n^2) complexity in the worst-case

⬜ Because we always choose the last element as the pivot, arises when input is sorted already

Page 164: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

How to choose pivot?164

Popular heuristics: Use

• first array value (not so good)

• middle array value (not so good)

• Choose a random element (not so good)

• median of first, middle, last, values (often used)!

x ? h h k

<= x x >= x h j k

b

b

pre:

post:

Choosing pivot

Ideal pivot: the median, since it splits array in half

But computing is O(n), quite complicated

Page 165: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Can we do better?

◻ Do algorithms with better than O(nlgn) complexity exist?

⬜ Yes and no

◻ For comparison based algorithms (ak: when you actually compare to elements in the array a[i]<a[j], O(nlgn) is actually optimal!

◻ But there are algorithms that are not comparison based!

165

Page 166: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Non-comparison based sorting

◻ Counting sort

⬜ Assumes that each of the n input elements is an integer in range (0,k)

⬜ Determines, for each input element x, the number of elements less than x. Uses this information to place element x directly into its position in the output array

◻ Radix sort

⬜ Used by the card-sorting machines you see in computer museums

⬜ Sorts integers by their digits (starting from the least significant one)

◻ Bucket sort

⬜ Assumes data is uniformly generated.

⬜ Creates different buckets and assumes buckets will be mostly empty

166

Page 167: Object-oriented programming 1 and data-structures SUMMER ... · Analyse their complexity This lecture 4. Why Sorting? Sorting is useful ⬜ Database indexing ⬜ Compressing data

Sorting in Java◻ Java.util.Arrays has a method Sort()

⬜ implemented as a collection of overloaded methods

⬜ for primitives, Sort is implemented with a version of quicksort

⬜ for Objects that implement Comparable, Sort is implemented with mergesort

◻ Tradeoff between speed/space and stability/performance guarantees

167