Sorting AlgorithmsINSTRUCTOR: MUHAMMAD HAMMAD WASEEEM

Sorting Algorithm

A sorting algorithm is an algorithm that puts elements of a list in a

certain order. The most-used orders are numerical order and

lexicographical order.

Sorting Example:

Given a set (container) of n elements

E.g. array, set of words, etc.

Suppose there is an order relation that can be set across the elements

Goal Arrange the elements in ascending order

Start 1 23 2 56 9 8 10 100

End 1 2 8 9 10 23 56 100

2

Popular Sorting Algorithms

While there are a large number of sorting algorithms, in practical implementations a few algorithms predominate. Insertion sort is widely used for small data sets, while for large data sets an asymptotically efficient sort is used, primarily heap sort, merge sort, or quicksort.

Insertion sort

Bubble sort

Merge sort

Quick sort

Selection sort

Heap sort

3

Sorting AlgorithmsINSERTION SORT

4

Insertion Sort: Idea

Idea: sorting cards.

8 | 5 9 2 6 3

5 8 | 9 2 6 3

5 8 9 | 2 6 3

2 5 8 9 | 6 3

2 5 6 8 9 | 3

2 3 5 6 8 9 |

5

Insertion Sort: Idea

1. We have two group of items:

sorted group, and

unsorted group

2. Initially, all items in the unsorted group and the sorted group is

empty.

We assume that items in the unsorted group unsorted.

We have to keep items in the sorted group sorted.

3. Pick any item from unsorted, then insert the item at the right

position in the sorted group to maintain sorted property.

4. Repeat the process until the unsorted group becomes empty.

6

40 2 1 43 3 65 0 -1 58 3 42 4

2 40 1 43 3 65 0 -1 58 3 42 4

1 2 40 43 3 65 0 -1 58 3 42 4

40

Insertion Sort: Example 7

1 2 3 40 43 65 0 -1 58 3 42 4

1 2 40 43 3 65 0 -1 58 3 42 4

1 2 3 40 43 65 0 -1 58 3 42 4

Insertion Sort: Example 8

1 2 3 40 43 65 0 -1 58 3 42 4

1 2 3 40 43 650 -1 58 3 42 4

1 2 3 40 43 650 58 3 42 41 2 3 40 43 650-1

Insertion Sort: Example 9

1 2 3 40 43 650 58 3 42 41 2 3 40 43 650-1

1 2 3 40 43 650 58 42 41 2 3 3 43 650-1 5840 43 65

1 2 3 40 43 650 42 41 2 3 3 43 650-1 5840 43 65

Insertion Sort: Example 10

1 2 3 40 43 650 421 2 3 3 43 650-1 584 43 6542 5840 43 65

Insertion Sort: Code (1/2)

#include <iostream.h> #include <conio.h>

void main()

{

int Size, I_Array[50];

cout<<"Enter The Size Or Number Of Digits You Want To Enter=";

cin>>Size;

for(int i=0; i<Size; i++)

{

cout<<"\nEnter "<<(i+1)<<" Element=";

cin>>I_Array[i];

}

cout<<"\nBefore INSERTION SORT THE LIST IS:\n";

for(int i=0; i<Size; i++)

cout<<I_Array[i]<<"\t";

InsertionSort(I_Array,Size);

getch();

}

11

Insertion Sort: Code (2/2)

void InsertionSort (int I_Array[], int Size)

{

int Temp, Walker, Current = 1;

while (Current < Size)

{

Temp = I_Array [Current];

Walker = Current -1;

while (( Walker >= 0 )&&( Temp < I_Array [Walker] ))

{

I_Array[Walker +1] = I_Array[Walker];

Walker = Walker -1;

}

I_Array [Walker+1] = Temp;

Current = Current + 1;

}

cout<<"\nAfter INSERTION SORT THE LIST IS:\n";

for(int i=0; i<Size; i++)

cout<<I_Array[i]<<"\t";

}

12

Sorting AlgorithmsBUBBLE SORT

13

Bubble Sort: Definition

Bubble sort, sometimes referred to as sinking sort, is a simple sorting

algorithm that works by repeatedly stepping through the list to be

sorted, comparing each pair of adjacent items and swapping them if they are in the wrong order

14

Bubble Sort: Idea

Idea: bubble in water.

Bubble in water moves upward. Why?

How?

When a bubble moves upward, the water from above will move

downward to fill in the space left by the bubble.

15

Bubble Sort Example 16

9, 6, 2, 12, 11, 9, 3, 7

6, 9, 2, 12, 11, 9, 3, 7

6, 2, 9, 12, 11, 9, 3, 7

6, 2, 9, 12, 11, 9, 3, 7

6, 2, 9, 11, 12, 9, 3, 7

6, 2, 9, 11, 9, 12, 3, 76, 2, 9, 11, 9, 3, 12, 76, 2, 9, 11, 9, 3, 7, 12The 12 is greater than the 7 so they are exchanged.

The 12 is greater than the 3 so they are exchanged.

The twelve is greater than the 9 so they are exchanged

The 12 is larger than the 11 so they are exchanged.

In the third comparison, the 9 is not larger than the 12 so no

exchange is made. We move on to compare the next pair without

any change to the list.

Now the next pair of numbers are compared. Again the 9 is the

larger and so this pair is also exchanged.

Bubblesort compares the numbers in pairs from left to right

exchanging when necessary. Here the first number is compared

to the second and as it is larger they are exchanged.

The end of the list has been reached so this is the end of the first pass. The

twelve at the end of the list must be largest number in the list and so is now in

the correct position. We now start a new pass from left to right.

Bubble Sort Example 17

6, 2, 9, 11, 9, 3, 7, 122, 6, 9, 11, 9, 3, 7, 122, 6, 9, 9, 11, 3, 7, 122, 6, 9, 9, 3, 11, 7, 122, 6, 9, 9, 3, 7, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

Notice that this time we do not have to compare the last two

numbers as we know the 12 is in position. This pass therefore only

requires 6 comparisons.

First Pass

Second Pass

Bubble Sort Example 18

2, 6, 9, 9, 3, 7, 11, 122, 6, 9, 3, 9, 7, 11, 122, 6, 9, 3, 7, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12Second Pass

First Pass

Third Pass

This time the 11 and 12 are in position. This pass therefore only

requires 5 comparisons.

Bubble Sort Example 19

2, 6, 9, 3, 7, 9, 11, 122, 6, 3, 9, 7, 9, 11, 122, 6, 3, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12Second Pass

First Pass

Third Pass

Each pass requires fewer comparisons. This time only 4 are needed.

2, 6, 9, 3, 7, 9, 11, 12Fourth Pass

Bubble Sort Example 20

2, 6, 3, 7, 9, 9, 11, 122, 3, 6, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12Second Pass

First Pass

Third Pass

The list is now sorted but the algorithm does not know this until it

completes a pass with no exchanges.

2, 6, 9, 3, 7, 9, 11, 12Fourth Pass

2, 6, 3, 7, 9, 9, 11, 12Fifth Pass

Bubble Sort Example 21

2, 3, 6, 7, 9, 9, 11, 12

6, 2, 9, 11, 9, 3, 7, 12

2, 6, 9, 9, 3, 7, 11, 12Second Pass

First Pass

Third Pass

2, 6, 9, 3, 7, 9, 11, 12Fourth Pass

2, 6, 3, 7, 9, 9, 11, 12Fifth Pass

Sixth Pass2, 3, 6, 7, 9, 9, 11, 12

This pass no exchanges are made so the algorithm knows the list is

sorted. It can therefore save time by not doing the final pass. With

other lists this check could save much more work.

Bubble Sort Example 22

Quiz Time1. Which number is definitely in its correct position at the

end of the first pass?

Answer: The last number must be the largest.

Answer: Each pass requires one fewer comparison than the last.

Answer: When a pass with no exchanges occurs.

2. How does the number of comparisons required change as

the pass number increases?

3. How does the algorithm know when the list is sorted?

4. What is the maximum number of comparisons required

for a list of 10 numbers?

Answer: 9 comparisons, then 8, 7, 6, 5, 4, 3, 2, 1 so total 45

Bubble Sort: Example

Notice that at least one element will be in the correct position each

iteration.

23

40 2 1 43 3 65 0 -1 58 3 42 4

652 1 40 3 43 0 -1 58 3 42 4

65581 2 3 40 0 -1 43 3 42 4

1 2 3 400 65-1 43 583 42 4

1

2

3

4

1 0 -1 32 653 43 5842404

Bubble Sort: Example 24

0 -1 1 2 653 43 58424043

-1 0 1 2 653 43 58424043

6

7

8

1 2 0 3-1 3 40 6543 584245

Bubble Sort: Code (1/2)#include <iostream.h> #include <conio.h>

void BubbleSort(int *B_Array, int Size); //Function Declaration

void main()

{

int Size, B_Array[50];

cout<<"Enter The Size Or Number Of Digits You Want To Enter=";

cin>>Size;

for(int i=0; i<Size; i++) //Filling Array

{ cout<<"\nEnter "<<(i+1)<<" Element="; cin>>B_Array[i]; }

cout<<"\nBefore BUBBLE SORT THE LIST IS:\n";

for(int i=0; i<Size; i++)

cout<<B_Array[i]<<"\t";

BubbleSort(B_Array,Size); //Calling BubbleSort Function

cout<<"\nAfter BUBBLE SORT THE LIST IS:\n";

for(int i=0; i<Size; i++)

cout<<B_Array[i]<<"\t";

getch();

}

25

Bubble Sort: Code (2/2)

void BubbleSort(int *B_Array, int Size)

{

for(int i=0; i<Size; i++)

{

for(int j=0; j<Size-1; j++)

{

if(B_Array[j]>B_Array[j+1])

{

int Swap = B_Array[j]; //Swaping Element for Sorting

B_Array[j] = B_Array[j+1];

B_Array[j+1] = Swap;

}

}

}

}

26

Sorting AlgorithmsMERGE SORT

27

Merge Sort…. Divide And Conquer

Merging a two lists of one element each is the same as sorting them.

Merge sort divides up an unsorted list until the above condition is

met and then sorts the divided parts back together in pairs.

Specifically this can be done by recursively dividing the unsorted list

in half, merge sorting the right side then the left side and then

merging the right and left back together.

28

Merge Sort Algorithm

Given a list L with a length k:

If k == 1 the list is sorted

Else:

Recursively Sort the left side (0 thru k/2)

Recursively Sort the right side (k/2+1 thru k)

Merge the right side with the left side

29

Merge Sort Example 30

99 6 86 15 58 35 86 4 0

Merge Sort Example 31

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

Merge Sort Example 32

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

86 1599 6 58 35 86 4 0

Merge Sort Example 33

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

86 1599 6 58 35 86 4 0

99 6 86 15 58 35 86 4 0

Merge Sort Example 34

99 6 86 15 58 35 86 4 0

99 6 86 15 58 35 86 4 0

86 1599 6 58 35 86 4 0

99 6 86 15 58 35 86 4 0

4 0

Merge Sort Example 35

99 6 86 15 58 35 86 0 4

4 0Merge

Merge Sort Example 36

15 866 99 58 35 0 4 86

99 6 86 15 58 35 86 0 4

Merge Sort Example 37

6 15 86 99 0 4 35 58 86

15 866 99 58 35 0 4 86

Merge Sort Example 38

0 4 6 15 35 58 86 86 99

6 15 86 99 0 4 35 58 86

Merge Sort Example

Sorted List

Do it Yourself: Code for Merge sort

39

0 4 6 15 35 58 86 86 99

Mergesort

Mergesort (divide-and-conquer)

Divide array into two halves.

40

A L G O R I T H M S

divideA L G O R I T H M S

Mergesort

Mergesort (divide-and-conquer)

Divide array into two halves.

Recursively sort each half.

41

sort

A L G O R I T H M S

divideA L G O R I T H M S

A G L O R H I M S T

Mergesort

Mergesort (divide-and-conquer)

Divide array into two halves.

Recursively sort each half.

Merge two halves to make sorted whole.

42

merge

sort

A L G O R I T H M S

divideA L G O R I T H M S

A G L O R H I M S T

A G H I L M O R S T

auxiliary array

smallest smallest

A G L O R H I M S T

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

43

A

auxiliary array

smallest smallest

A G L O R H I M S T

A

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

44

G

auxiliary array

smallest smallest

A G L O R H I M S T

A G

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

45

H

auxiliary array

smallest smallest

A G L O R H I M S T

A G H

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

46

I

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

47

L

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

48

M

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L M

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

49

O

auxiliary array

smallest smallest

A G L O R H I M S T

A G H I L M O

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

50

R

auxiliary array

first half

exhausted smallest

A G L O R H I M S T

A G H I L M O R

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

51

S

auxiliary array

first half

exhausted smallest

A G L O R H I M S T

A G H I L M O R S

Merging

Merge.

Keep track of smallest element in each sorted half.

Insert smallest of two elements into auxiliary array.

Repeat until done.

52

T

Sorting AlgorithmsQUICK SORT

53

Quicksort

Quicksort is more widely used than any other sort.

Quicksort is well-studied, not difficult to implement, works well on a

variety of data, and consumes fewer resources that other sorts in nearly all situations.

Quicksort is O(n*log n) time, and O(log n) additional space due to

recursion.

54

Quicksort Algorithm

Quicksort is a divide-and-conquer method for sorting. It works by

partitioning an array into parts, then sorting each part

independently.

The crux of the problem is how to partition the array such that the

following conditions are true:

There is some element, a[i], where a[i] is in its final

position.

For all l < i, a[l] < a[i].

For all i < r, a[i] < a[r].

55

Quicksort Algorithm (cont)

As is typical with a recursive program, once you figure out how to divide your problem into smaller subproblems, the implementation is amazingly simple.

int partition(Item a[], int l, int r);

void quicksort(Item a[], int l, int r)

{ int i;

if (r <= l) return;

i = partition(a, l, r);

quicksort(a, l, i-1);

quicksort(a, i+1, r);

}

56

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

57

Q U I C K S O R T I S C O O L

partitioned

partition element left

right

unpartitioned

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

58

swap me

Q U I C K S O R T I S C O O L

partitioned

partition element left

right

unpartitioned

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

59

partitioned

partition element left

right

unpartitioned

swap me

Q U I C K S O R T I S C O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

60

partitioned

partition element left

right

unpartitioned

swap me

Q U I C K S O R T I S C O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

61

partitioned

partition element left

right

unpartitioned

swap me

Q U I C K S O R T I S C O O L

swap me

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

62

partitioned

partition element left

right

unpartitioned

C U I C K S O R T I S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

63

swap me

partitioned

partition element left

right

unpartitioned

C U I C K S O R T I S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

64

partitioned

partition element left

right

unpartitioned

swap me

C U I C K S O R T I S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

65

partitioned

partition element left

right

unpartitioned

swap me

C U I C K S O R T I S Q O O L

swap me

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

66

partitioned

partition element left

right

unpartitioned

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

67

partitioned

partition element left

right

unpartitioned

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

exchange

repeat until pointers cross

68

partitioned

partition element left

right

unpartitioned

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

69

partitioned

partition element left

right

unpartitioned

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

70

swap me

partitioned

partition element left

right

unpartitioned

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

71

partitioned

partition element left

right

unpartitioned

swap me

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

72

partitioned

partition element left

right

unpartitioned

swap me

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

73

partitioned

partition element left

right

unpartitioned

swap me

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

74

pointers crossswap with

partitioning

element

partitioned

partition element left

right

unpartitioned

C I I C K S O R T U S Q O O L

Partitioning in Quicksort

How do we partition the array efficiently?

choose partition element to be rightmost element

scan from left for larger element

scan from right for smaller element

Exchange and repeat until pointers cross

75

partitioned

partition element left

right

unpartitioned

partition is

complete

C I I C K L O R T U S Q O O S

Quick Sort

In order to get the sorted list, Sort the both partitions using any

sorting algorithm

76

Sorting AlgorithmsSELECTION SORT

77

Selection Sort: Idea

1. We have two group of items:

sorted group, and

unsorted group

2. Initially, all items are in the unsorted group. The sorted group is

empty.

We assume that items in the unsorted group unsorted.

We have to keep items in the sorted group sorted.

3. Select the “best” (eg. Smallest or largest) item from the unsorted

group, then put the “best” item at the end of the sorted group.

4. Repeat the process until the unsorted group becomes empty.

78

Selection Sort

5 1 3 4 6 2

79

This is starting state of an array

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

80

Start traversing or searching for the largest or smallest element in this array for sorting purpose.

In this example assume for larger one.

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

81

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

82

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

83

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

84

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

85

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 6 2

86

Largest

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 2 6

87

Unsorted

Comparison

Data Movement

Sorted

Swap the larger element with the last element in the unsorted list.

Selection Sort

5 1 3 4 2 6

88

Unsorted

Comparison

Data Movement

Sorted

Largest element is at its sorted position

Selection Sort

5 1 3 4 2 6

89

Unsorted

Comparison

Data Movement

Sorted

Again Start traversing or searching for the largest or smallest element in unsorted portion of

array for sorting purpose. In this example assume for larger one.

Selection Sort

5 1 3 4 2 6

90

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 2 6

91

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 2 6

92

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 2 6

93

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

5 1 3 4 2 6

94

Largest

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

95

Unsorted

Comparison

Data Movement

Sorted

Swap the larger element with the last element in the unsorted list.

Selection Sort

2 1 3 4 5 6

96

Unsorted

Comparison

Data Movement

Sorted

Largest element is at its sorted position

Selection Sort

2 1 3 4 5 6

97

Unsorted

Comparison

Data Movement

Sorted

Again Start traversing or searching for the largest or smallest element in unsorted portion of

array for sorting purpose. In this example assume for larger one.

Selection Sort

2 1 3 4 5 6

98

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

99

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

100

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

101

Largest

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

102

Unsorted

Comparison

Data Movement

Sorted

Largest element is already at sorted position therefore no data swapping is required.

Selection Sort

2 1 3 4 5 6

103

Unsorted

Comparison

Data Movement

Sorted

Largest element is at its sorted position

Selection Sort

2 1 3 4 5 6

104

Unsorted

Comparison

Data Movement

Sorted

Again Start traversing or searching for the largest or smallest element in unsorted portion of

array for sorting purpose. In this example assume for larger one.

Selection Sort

2 1 3 4 5 6

105

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

106

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

107

Largest

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

108

Unsorted

Comparison

Data Movement

Sorted

Largest element is already at sorted position therefore no data swapping is required.

Selection Sort

2 1 3 4 5 6

109

Unsorted

Comparison

Data Movement

Sorted

Largest element is at its sorted position

Selection Sort

2 1 3 4 5 6

110

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

111

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

2 1 3 4 5 6

112

Largest

Unsorted

Comparison

Data Movement

Sorted

Selection Sort

1 2 3 4 5 6

113

Unsorted

Comparison

Data Movement

Sorted

Swap the larger element with the last element in the unsorted list.

Selection Sort

1 2 3 4 5 6

114

DONE!

Unsorted

Comparison

Data Movement

Sorted

4240 2 1 3 3 4 0 -1 655843

40 2 1 43 3 4 0 -1 42 65583

40 2 1 43 3 4 0 -1 58 3 6542

40 2 1 43 3 65 0 -1 58 3 42 4

Selection Sort: Example 115

4240 2 1 3 3 4 0 655843-1

42-1 2 1 3 3 4 0 65584340

42-1 2 1 3 3 4 655843400

42-1 2 1 0 3 4 655843403

Selection Sort: Example 116

1

42-1 2 1 3 4 6558434030

42-1 0 3 4 6558434032

1 42-1 0 3 4 6558434032

1 420 3 4 6558434032-1

1 420 3 4 6558434032-1

Selection Sort: Example 117

Selection Sort: Code (1/2)

#include <iostream.h> #include <conio.h>

void SelectionSort (int A[], int Size);

void main()

{

int Size, S_Array[50];

cout<<"Enter The Size Or Number Of Digits You Want To Enter=";

cin>>Size;

for(int i=0; i<Size; i++)

{ cout<<"\nEnter "<<(i+1)<<" Element="; cin>>S_Array[i]; }

cout<<"\nBefore SELECTION SORT THE LIST IS:\n";

for(int i=0; i<Size; i++)

cout<<S_Array[i]<<"\t";

SelectionSort(S_Array, Size);

getch();

}

118

Selection Sort: Code (2/2)

void SelectionSort (int A[], int Size)

{

for (int i=0; i<Size; i++)

{

int min = i; //Letting first Element of the Array is Minimum

for (int j=i+1; j<Size; j++) //Loop for searching Min element

{

if (A[j]<A[min])

min = j;

}

int Swap = A[i]; //Swapping the Min element & the 1st element

A[i] = A[min];

A[min] = Swap;

}

cout<<"\nAfter SELECTION SORT THE LIST IS:\n";

for (int i=0; i<Size; i++)

cout<<A[i]<<"\t";

}

119

Sorting AlgorithmsHEAP SORT

120

Why study Heapsort?

It is a well-known, traditional sorting

algorithm you will be expected to know

Heapsort is always O(n log n)

Quicksort is usually O(n log n) but in the

worst case slows to O(n2)

Quicksort is generally faster, but Heapsort is

better in time-critical applications

121

What is a “heap”?

Definitions of heap:

1. A large area of memory from which the programmer can allocate blocks as needed, and deallocate them (or allow them to be garbage collected) when no longer needed

2. A balanced, left-justified binary tree in which no node has a value greater than the value in its parent

Heapsort uses the second definition

122

Balanced binary trees

Recall:

The depth of a node is its distance from the root

The depth of a tree is the depth of the deepest node

A binary tree of depth n is balanced if all the nodes at depths 0 through n-2 have two

children

123

Balanced Not balanced

n-2

n-1

n

The heap property

A node has the heap property if the value in the node is as large as

or larger than the values in its children

124

All leaf nodes automatically have the heap property

A binary tree is a heap if all nodes in it have the heap property

12

8 3

Blue node has

heap property

12

8 12

Blue node has

heap property

12

8 14

Blue node does not

have heap property

siftUp

Given a node that does not have the heap property, you can give it the heap property by exchanging its value with the value of the larger child

This is sometimes called sifting up

Notice that the child may have lost the heap property

125

14

8 12

Blue node has

heap property

12

8 14

Blue node does not

have heap property

Constructing a heap I

A tree consisting of a single node is automatically a heap

We construct a heap by adding nodes one at a time:

Add the node just to the right of the rightmost node in the deepest level

If the deepest level is full, start a new level

Examples:

126

Add a new

node here

Add a new

node here

Constructing a heap II

Each time we add a node, we may destroy the heap property of its parent

node

To fix this, we sift up

But each time we sift up, the value of the topmost node in the sift may

increase, and this may destroy the heap property of its parent node

We repeat the sifting up process, moving up in the tree, until either

We reach nodes whose values don’t need to be swapped (because

the parent is still larger than both children), or

We reach the root

127

Constructing a heap III 128

8 8

10

10

8

10

8 5

10

8 5

12

10

12 5

8

12

10 5

8

1 2 3

4

Other children are not affected

The node containing 8 is not affected because its parent gets larger, not smaller

The node containing 5 is not affected because its parent gets larger, not smaller

The node containing 8 is still not affected because, although its parent got smaller, its parent is still greater than it was originally

129

12

10 5

8 14

12

14 5

8 10

14

12 5

8 10

A sample heap Here’s a sample binary tree after it has been

heapified

Notice that heapified does not mean sorted

Heapifying does not change the shape of the binary tree; this binary tree is balanced and left-justified because it started out that way

130

19

1418

22

321

14

119

15

25

1722

Removing the root Notice that the largest number is now in the root

Suppose we discard the root:

How can we fix the binary tree so it is once again balanced and left-justified?

Solution: remove the rightmost leaf at the deepest level and use it for the new root

131

19

1418

22

321

14

119

15

1722

11

The reHeap method I Our tree is balanced and left-justified, but no longer a

heap

However, only the root lacks the heap property

We can siftUp() the root

After doing this, one and only one of its

children may have lost the heap property

132

19

1418

22

321

14

9

15

1722

11

The reHeap method II

Now the left child of the root (still the number

11) lacks the heap property

We can siftUp() this node

After doing this, one and only one of its

children may have lost the heap property

133

19

1418

22

321

14

9

15

1711

22

The reHeap method III Now the right child of the left child of the root

(still the number 11) lacks the heap property:

We can siftUp() this node

After doing this, one and only one of its children may have lost the heap property —but it doesn’t, because it’s a leaf

134

19

1418

11

321

14

9

15

1722

22

The reHeap method IV

Our tree is once again a heap, because

every node in it has the heap property

Once again, the largest (or a largest) value is in the root

We can repeat this process until the tree becomes empty

This produces a sequence of values in order largest to smallest

135

19

1418

21

311

14

9

15

1722

22

Sorting

What do heaps have to do with sorting an array?

Here’s the neat part:

Because the binary tree is balanced and left justified, it can be represented as an array

All our operations on binary trees can be represented as operations on arrays

To sort:

heapify the array;

while the array isn’t empty {

remove and replace the root;

reheap the new root node;}

136

Mapping into an array

Notice:

The left child of index i is at index 2*i+1

The right child of index i is at index 2*i+2

Example: the children of node 3 (19) are 7 (18) and 8 (14)

137

19

1418

22

321

14

119

15

25

1722

25 22 17 19 22 14 15 18 14 21 3 9 11

0 1 2 3 4 5 6 7 8 9 10 11 12

Removing and replacing the root

The “root” is the first element in the array

The “rightmost node at the deepest level” is the last element

Swap them...

...And pretend that the last element in the array

no longer exists—that is, the “last index” is 11 (9)

138

25 22 17 19 22 14 15 18 14 21 3 9 11

0 1 2 3 4 5 6 7 8 9 10 11 12

11 22 17 19 22 14 15 18 14 21 3 9 25

0 1 2 3 4 5 6 7 8 9 10 11 12

Reheap and repeat

Reheap the root node (index 0, containing

11)...

...And again, remove and replace the root node

Remember, though, that the “last” array index is changed

Repeat until the last becomes first, and the array is sorted!

139

22 22 17 19 21 14 15 18 14 11 3 9 25

0 1 2 3 4 5 6 7 8 9 10 11 12

9 22 17 19 22 14 15 18 14 21 3 22 25

0 1 2 3 4 5 6 7 8 9 10 11 12

11 22 17 19 22 14 15 18 14 21 3 9 25

0 1 2 3 4 5 6 7 8 9 10 11 12

Analysis I

Here’s how the algorithm starts:

heapify the array;

Heapifying the array: we add each of n nodes

Each node has to be sifted up, possibly as far as the root

Since the binary tree is perfectly balanced, sifting up a single node

takes O(log n) time

Since we do this n times, heapifying takes n*O(log n) time,

that is, O(n log n) time

140

Analysis II

Here’s the rest of the algorithm:

while the array isn’t empty {

remove and replace the root;

reheap the new root node;

}

We do the while loop n times (actually, n-1 times), because

we remove one of the n nodes each time

Removing and replacing the root takes O(1) time

Therefore, the total time is n times however long it takes the

reheap method

141

Analysis III

To reheap the root node, we have to follow one path from the root

to a leaf node (and we might stop before we reach a leaf)

The binary tree is perfectly balanced

Therefore, this path is O(log n) long

And we only do O(1) operations at each node

Therefore, reheaping takes O(log n) times

Since we reheap inside a while loop that we do n times, the total

time for the while loop is n*O(log n), or O(n log n)

142

Analysis IV

Here’s the algorithm again:

heapify the array;

while the array isn’t empty {

remove and replace the root;

reheap the new root node;

}

We have seen that heapifying takes O(n log n) time

The while loop takes O(n log n) time

The total time is therefore O(n log n) + O(n log n)

This is the same as O(n log n) time

143

THANKS!EMAIL: m.hammad.waseem@gmail.com

144