ADSA: Sorting/3 1

241-423 Advanced Data Structures and Algorithms

• Objective– examine popular sorting algorithms, with an

emphasis on divide and conquer

Semester 2, 2013-2014

3. Sorting Algorithms

ADSA: Sorting/3 2

Contents

1. Insertion Sort

2. Divide and Conquer Algorithms

3. Merge Sort

4. Quicksort

5. Comparison of Sorting Algorithms

6. Finding the kth Largest Element

ADSA: Sorting/3 3

1. Insertion Sort

• Each pass inserts an element (x) into a sorted sublist (sub-array) on the left.

• Items larger than x move to the right to make room for its insertion.

ADSA: Sorting/3 4

Insertion Sort Diagram

ADSA: Sorting/3 5

Outline Algorithm

• Assume the first array element is in the right position.

• In the ith pass (1 ≤ i ≤ n-1), the elements in the range 0 to i-1 are already sorted.

• Insert ith position target into correct position j by moving elements in the range [j, i-1] to the right until there is space in arr[j].

ADSA: Sorting/3 6

Simple Insertion Sort

public static void insertion_srt(int arr[]){

int n = arr.length;

for (int i = 1; i < n; i++) {

int j = i;

int target = arr[i]; // sort ith elem

while ((j > 0) && (arr[j-1] > target)){

arr[j] = arr[j-1]; // move right

j--;

}

arr[j] = target;

}

}

ADSA: Sorting/3 7

insertionSort()public static <T extends Comparable<? super T>>void insertionSort(T[] arr){ int n = arr.length; for (int i = 1; i < n; i++) { int j = i; T target = arr[i];

while (j > 0 && target.compareTo(arr[j-1]) < 0) {

arr[j] = arr[j-1];

j--;

}

arr[j] = target;

}

} // end of insertionSort()

ADSA: Sorting/3 8

Insertion Sort Efficiency

• Best case running time is O(n) – when the array is already sorted

• The worst and average case running times are O(n2).

• Insertion sort is very efficient when the array is "almost sorted".

ADSA: Sorting/3 9

2. Divide and Conquer Algorithms

• Divide a problem into smaller versions of the same problem, using recursion.

• Solve the smaller versions.

• Combine the small versions solutions together to get an answer for the big, original problem.

ADSA: Sorting/3 10

Examples

• Binary search

• Merge sort and quicksort (here)

• Binary tree traversal

ADSA: Sorting/3 11

3. Merge Sort• Sort an array with n elements by splitting it

into two halves. Keep splitting in half recursively.

• Sort the small elements.

• Merge the small elements recursively back together into a single sorted array.

ADSA: Sorting/3 12

Merge Sort Diagram

ADSA: Sorting/3 13

General Sort Methods

• Ford and Topp's Arrays class provides two versions of the merge sort algorithm.– one version takes an Object array arr[] as input;– the other version is generic and specifies arr[]

as an array of type T

• Both methods call msort() to carry out the merge sort.

ADSA: Sorting/3 14

sort() - with Object Array

public static void sort(Object[] arr){ // create a temporary array Object[] tempArr = arr.clone();

msort(arr, tempArr, 0, arr.length);}

sort the entire array

(the range 0-arr.length)

ADSA: Sorting/3 15

sort() - Generic Version

public static <T extends Comparable<? super T>>void sort(T[] arr){ // create a temporary array T[] tempArr = (T[])arr.clone();

msort(arr, tempArr, 0, arr.length);}

ADSA: Sorting/3 16

msort()

• Split into two lists by computing the midpoint of the index range:

int midpt = (last + first)/2;

• Call msort() recursively on the index range [first, mid) and on the index range [mid, last).

• When the resulting lists are small, start merging them back together into sorted order.

ADSA: Sorting/3 17

Tracing msort()

split

merge

ADSA: Sorting/3 18

msort()private static void msort(Object[] arr, Object[] tempArr, int first, int last){ // if sublist has more than 1 elem. if ((first + 1) < last){ int midpt = (last + first)/2;

msort(arr, tempArr, first, midpt); msort(arr, tempArr, midpt, last);

// if arr[] is now sorted, finish if (((Comparable)arr[midpt-1]).compareTo (arr[midpt]) <= 0) return; :

ADSA: Sorting/3 19

// indexA scans arr[] in range [first, mid) int indexA = first;

// indexB scans arr[] in range [mid, last) int indexB = midpt;

int indexC = first; // for merged temp list

/* while both sublists are not finished, compare arr[indexA] and arr[indexB]; copy the smaller into the temp list */ while (indexA < midpt && indexB < last) { if (((Comparable)arr[indexA]).compareTo (arr[indexB]) < 0) { tempArr[indexC] = arr[indexA]; indexA++; }

ADSA: Sorting/3 20

else { tempArr[indexC] = arr[indexB]; indexB++; } indexC++; }

// copy over what's left of sublist A while (indexA < midpt) { tempArr[indexC] = arr[indexA]; indexA++; indexC++; } :

ADSA: Sorting/3 21

// copy over what's left of sublist B while (indexB < last) { tempArr[indexC] = arr[indexB]; indexB++; indexC++; }

// copy temp array back to arr[] for (int i = first; i < last; i++) arr[i] = tempArr[i]; }

} // end of msort()

ADSA: Sorting/3 22

msort() Notes

• Continue only as long as first+1 < last

• Do not merge arr if arr[mid-1] < arr[mid]

ADSA: Sorting/3 23

Recursion Tree for Merge Sort

ADSA: Sorting/3 24

Efficiency of Merge Sort

• Total number of comparisons = no. of levels * no. of comparisons at a level

• msort() starts with a list of size n

• msort() recurses until the sublist size is 1

• Each level roughly halves the sublist size:– n, n/2, n/4, ..., 1

– no. of levels = log2n (roughly)

ADSA: Sorting/3 25

• No. of msort() calls at a level:– at level 0: 1 msort() call– at level 1: 2 calls– at level 2: 4 calls– ...– at level i: 2i calls

ADSA: Sorting/3 26

• No of comparisons in 1 msort call at a level:– at level 0: a msort() call compares n elements

– at level 1: n/2 comparisons

– at level 2: n/4 comparisons

– ...

– at level i: n/2i elements

• Total no. of comparisons at a level:– no. of calls at a level * comparisons in 1 msort()call

– 2i * n/2i = n

ADSA: Sorting/3 27

• Total number of comparisons = no. of levels * no. of comparisons at a level

= log2n * n

• So the worst case running time is = O(n log2n)

ADSA: Sorting/3 28

4. Quicksort

• Uses a divide-and-conquer strategy like merge sort.

• But, unlike merge sort, quicksort is an in-place sorting algorithm– elements are exchanged within the list without the need

for temporary lists/arrays– space efficient

ADSA: Sorting/3 29

Quicksort Steps

• Pick an element, called a pivot, from the list.

• Reorder the list so that all elements which are less than the pivot come before the pivot and so that all elements greater than the pivot come after it

ADSA: Sorting/3 30

• Recursively call quicksort on the sublist of lesser elements and the sublist of greater elements.

• The stopping case for the recursion are lists of size zero or one, which are always sorted.

ADSA: Sorting/3 31

Quicksort Diagram

pivot

ADSA: Sorting/3 32

Partitioning a List

• The pivot is the element at index– mid = (first + last)/2.

• Separate the elements of arr[] into two sublists, Sl and Sh.

– Sl contains the elements ≤ pivot (l = low)

– Sh contains the elements ≥ pivot (h = high)

ADSA: Sorting/3 33

• Exchange arr[first] and arr[mid]

• Scan the list with index range [first+1, last)– scanUp starts at first+1 and moves up the list,

finding elements for Sl.

– scanDown starts at position last -1 and moves down the list, finding elements for Sh.

ADSA: Sorting/3 34

• When arr[scanUp] pivot and arr[scanDown] pivot then the two elements are in the wrong sublists.

• Exchange the elements at the two positions and then resume scanning.

ADSA: Sorting/3 35

ADSA: Sorting/3 36

ADSA: Sorting/3 37

• scanUp and scanDown move toward each other until they meet or pass one another (scanDown scanUp).

ADSA: Sorting/3 38

• scanDown is at the place where the pivot should appear– exchange arr[0] and arr[scanDown] to correctly

position the pivot

ADSA: Sorting/3 39

pivotIndex()

• The methodpublic static <T extends Comparable<? super

T>> int pivotIndex(T[] arr, int first, int last)

takes array arr and index range [first, last) and returns the index of the pivot after partitioning arr[].

ADSA: Sorting/3 40

public static <T extends Comparable<? super T>>int pivotIndex(T[] arr, int first, int last){ int mid; // index for the midpoint T pivot;

if (first == last) // empty sublist return last; else if (first == (last-1)) // 1-element sublist return first; else { mid = (last + first)/2; pivot = arr[mid];

:

ADSA: Sorting/3 41

// exchange pivot and bottom end of range arr[mid] = arr[first]; arr[first] = pivot;

int scanUp = first + 1; // scanning indices int scanDown = last - 1;

while(true) { /* move up the lower sublist while scanUp is less than or equal to scanDown and the array value is less than pivot */ while ((scanUp <= scanDown) && (arr[scanUp].compareTo(pivot) < 0)) scanUp++;

ADSA: Sorting/3 42

/* move down upper sublist while array value is greater than the pivot */ while (pivot.compareTo(arr[scanDown]) < 0) scanDown--;

/* if indices are not in their sublists, partition is complete */ if (scanUp >= scanDown) break;

// found two elements in wrong sublists; exchange T temp = arr[scanUp]; arr[scanUp] = arr[scanDown]; arr[scanDown] = temp;

scanUp++; scanDown--; } :

ADSA: Sorting/3 43

// copy pivot to index posn (scanDown) that // partitions the sublists arr[first] = arr[scanDown]; arr[scanDown] = pivot;

return scanDown; }} // end of pivotIndex()

ADSA: Sorting/3 44

quicksort()

public static <T extends Comparable<? super T>>void quicksort(T[] arr){ qsort(arr, 0, arr.length); }

• quicksort() sorts a generic array arr[] by calling qsort() with the index range [0, arr.length).

ADSA: Sorting/3 45

qsort()

• Recursively partition the elements in the index range into smaller and smaller sublists, terminating when the size of a list is 0 or 1.

• For efficiency, handle a list of size 2 by comparing the elements and exchanging them if necessary.

ADSA: Sorting/3 46

• For larger lists, call pivotIndex() to reorder the elements and determine the pivot.

• Make two calls to qsort(): – the first call specifies the index range for the

lower sublist– the second call specifies the index range for the

upper sublist

ADSA: Sorting/3 47

qSort() Diagram

ADSA: Sorting/3 48

private static <T extends Comparable<? super T>>void qsort(T[] arr, int first, int last){ // if range is less than two elements if ((last – first) <= 1) return;

// if sublist has two elements else if ((last – first) == 2) {

:

ADSA: Sorting/3 49

/* compare arr[first] and arr[last-1] and exchange if necessary */ if (arr[last-1].compareTo(arr[first]) < 0) { T temp = arr[last-1]; arr[last-1] = arr[first]; arr[first] = temp; } return; } else { int pivotLoc = pivotIndex(arr, first, last); qsort(arr, first, pivotLoc); qsort(arr, pivotLoc +1, last); }

} // end of qsort()

ADSA: Sorting/3 50

Running Time of Quicksort

• The average case running time is O(n log2n).

• The best case occurs when the array is already sorted.

ADSA: Sorting/3 51

• Quicksort is efficient even when the array is in descending order.

ADSA: Sorting/3 52

• The worst-case occurs when the chosen pivot is always the largest or smallest element in its sublist. – the running time is O(n2)– highly unlikely

ADSA: Sorting/3 53

5. Comparison of Sorting Algorithms

• An inversion in an array, arr[], is an ordered pair (arr[i], arr[j]), i < j, where arr[i] > arr[j].

• When sorting in ascending order, arr[i] and arr[j] are out of order.

ADSA: Sorting/3 54

• The O(n2) sorting algorithms compare adjacent elements, generally remove one inversion with each iteration– e.g. selection and insertion sort

• The O(n log2n) sorting algorithms compare non-adjacent elements, and generally remove more than one inversion with each iteration.– e.g. quicksort and merge sort

ADSA: Sorting/3 55

Timing Sorts

import java.util.Random;import ds.util.Arrays;import ds.time.Timing;

public class TimingSorts{

public static void main(String[] args) { final int SIZE = 75000; Integer[] arr1 = new Integer[SIZE], arr2 = new Integer[SIZE], arr3 = new Integer[SIZE]; Random rnd = new Random(); :

ADSA: Sorting/3 56

/* load each array with the same sequence of random numbers in the range 0 to 999999 */ int rndNum; for (int i=0; i < SIZE; i++) { rndNum = rnd.nextInt(1000000); arr1[i] = arr2[i] = arr3[i] = rndNum; }

// call timeSort() for each sort timeSort(arr1, 0, "Merge sort"); timeSort(arr2, 1, "Quick sort"); timeSort(arr3, 2, "Insertion sort"); } // end of main()

ADSA: Sorting/3 57

public static <T extends Comparable<? super T>> void timeSort(T[] arr, int sortType, String sortName) { Timing t = new Timing(); t.start();

if(sortType == 0) Arrays.sort(arr); // merge sort in F&T else if (sortType == 1) Arrays.quicksort(arr); else Arrays.insertionSort(arr);

double timeRequired = t.stop();

outputFirst_Last(arr); System.out.print(" " + sortName + " time is " + timeRequired + "\n\n"); } // end of timeSort()

}

ADSA: Sorting/3 58

public static void outputFirst_Last(Object[] arr) // output first 3 elements and last 3 elements { for (int i=0; i < 3; i++) System.out.print(arr[i] + " ");

System.out.print(". . . ");

for (int i=n-3; i < arr.length; i++) System.out.print(arr[i] + " ");

System.out.println(); }

ADSA: Sorting/3 59

Output

26 38 47 . . . 999980 999984 999984 Merge sort time is 0.109

26 38 47 . . . 999980 999984 999984 Quick sort time is 0.078

26 38 47 . . . 999980 999984 999984 Insertion sort time is 100.611 O(nO(n22))

O(n logO(n log22n)n)

ADSA: Sorting/3 60

6. Finding the kth Largest Element

• Sort the array and then access the element at position k. – running time is O(n log2n) is we use quicksort

or merge sort

• For a more efficient solution, locate the position of the kth-largest value by partitioning the elements into two sublists.

ADSA: Sorting/3 61

• The lower sublist contains k elements that are ≤ the kth-largest.

• The upper sublist contains elements that are ≥ the kth-largest.

• The elements in the sublists do not need to be ordered.

values ≤ kth-largest values ≥ kth-largestkth-largest

k k+1 ... n-10 ... k-1

ADSA: Sorting/3 62

• Use the pivoting technique from the quicksort algorithm to create a partition.

• The algorithm is recursive: – index = pivotIndex() – If index == k, done, return arr[index]; – otherwise, call pivotIndex() with range [first, index) if k < index,

or with range [index+1, last) if k > index.• examine only one of the lists

ADSA: Sorting/3 63

public static <T extends Comparable<? super T>>int findKth(T[] arr, int first, int last, int k){ if (first > last) return -1; // partition range (first, last) in arr about the // pivot arr[index] int index = pivotIndex(arr, first, last);

// if index == k, we are done. kth largest is arr[k] if (index == k) return arr[index]; // return array value else if(k < index) // search in lower sublist (first, index) findKth(arr, first, index, k); else // search in upper sublist (index+1, last) findKth(arr, index+1, last, k);}

ADSA: Sorting/3 64

Running Time of findKth()

• The running time is O(n)– no of comparisons = n + n/2 + n/4 + n/8 + ...

= 2n

• This is faster than the O(n log2n) result for a sorted array– this is to be expected since findKth() only uses one of its

sublists at each recursive call compared to quicksort or merge sort which use both