the art of data structures sortingrsarkis/csc162/_static/lectures/sorting.pdfbubble sort exchanging...
TRANSCRIPT
The Art of Data Structures Sorting
Richard E Sarkis CSC 162: The Art of Data Structures
Class Administrivia
Agenda
• To be able to explain and implement various sorting algorithms
• Bubble
• Selection
• Insertion
• Shell
• Merge
• Quick
Sorting
Sorting
• This is the process of organizing data in some particular order
• Numbers, increasing order
• Words, alphabetically
• etc
• Some algorithms benefit with pre-sorted data, e.g. binary search
Sorting
• Sorting is an important area of computer science
• Many sorting algorithms have been developed, and analyzed
• Sorting can take significant time, and is related to the number of items to process
Sorting
• Sorting requires two main operations:
• Comparisons of items to see if they are out of order; comparisons will be an important metric
• Exchange of items can be a costly operation, and also an important metric
Bubble Sort
Bubble Sort bubble_sort: The First Pass
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
bubbleSort: The First Pass
54 26 93 17 77 31 44 55 20
26 54 93 17 77 31 44 55 20
26 54 93 17 77 31 44 55 20
26 54 17 93 77 31 44 55 20
26 54 17 77 93 31 44 55 20
26 54 17 77 31 93 44 55 20
26 54 17 77 31 44 93 55 20
26 54 17 77 31 44 55 93 20
Exchange
No Exchange
Exchange
Exchange
Exchange
Exchange
Exchange
Exchange
26 54 17 77 31 44 55 20 9393 in place
after first pass
First pass
Algorithm Analysis
Bubble Sort Exchanging Two Values in Python
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
Exchanging Two Values in Python
93 44
i j
temp
1st 3rd
2nd
93 44
Most programming languages require a 3-stepprocess with an extra storage location.
In Python, exchange can be done astwo simultaneous assignments.
Algorithm Analysis
Bubble Sort Implementation
def bubble_sort(alist): for passnum in range(len(alist)-1,0,-1): for i in range(passnum): if alist[i]>alist[i+1]: temp = alist[i] alist[i] = alist[i+1] alist[i+1] = temp
Bubble Sort Modified Implementation (cont.)
def bubble_sorted(alist): exchanges = True passnum = len(alist)-1 while passnum > 0 and exchanges: exchanges = False for i in range(passnum): if alist[i] > alist[i+1]: exchanges = True temp = alist[i] alist[i] = alist[i+1] alist[i+1] = temp passnum = passnum-1
Selection Sort
Selection Sort
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
selectionSort
54 26 93 17 77 31 44 55 20
26 54 20 17 77 31 44 55 93
93 is largest
77 is largest
26 54 20 17 55 31 44 77 93 55 is largest
26 54 20 17 44 31 55 77 93 54 is largest
26 31 20 17 44 54 55 77 93 44 is largeststays in place
26 31 20 17 44 54 55 77 93 31 is largest
26 17 20 31 44 54 55 77 93 26 is largest
20 17 26 31 44 54 55 77 93 20 is largest
17 20 26 31 44 54 55 77 93 17 oklist is sorted
Algorithm Analysis
Selection Sort Implementation
def selection_sort(alist): for fillslot in range(len(alist)-1, 0, -1): position_of_max = 0 for location in range(1, fillslot+1): if alist[location] > alist[position_of_max]: position_of_max = location
temp = alist[fillslot] alist[fillslot] = alist[position_of_max] alist[position_of_max] = temp
Insertion Sort
Insertion Sort
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
insertionSort
54 26 93 17 77 31 44 55 20
26 54 93 17 77 31 44 55 20
26 54 93 17 77 31 44 55 20
17 26 54 93 77 31 44 55 20
17 26 54 77 93 31 44 55 20
17 26 31 54 77 93 44 55 20
17 26 31 44 54 77 93 55 20
17 26 31 44 54 55 77 93 20
17 20 26 31 44 54 55 77 93
Assume 54 is a sortedlist of 1 item
inserted 26
inserted 93
inserted 17
inserted 77
inserted 31
inserted 44
inserted 55
inserted 20
Algorithm Analysis
Insertion Sort insertion_sort: The Fifth Pass
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
insertionSort: Fifth Pass of the Sort
17 26 54 77 93 31 44 55 20
17 26 54 77 93 44 55 20
Need to insert 31
back into the sorted list
93>31 so shift it
to the right
17 26 54 77 93 44 55 20 77>31 so shift it
to the right
17 26 54 77 93 44 55 20 54>31 so shift it
to the right
17 26 31 54 77 93 44 55 20 26<31 so insert 31
in this position
Algorithm Analysis
Insertion Sort Implementation (cont.)
def insertion_sort(alist): for index in range(1, len(alist)): currentvalue = alist[index] position = index
while position > 0 and alist[position-1] > currentvalue: alist[position] = alist[position-1] position = position-1
alist[position] = currentvalue
Shell Sort
Shell Sort With Increments of Three
5.3. Sorting 215
1 def insertionSort(alist ):2 for index in range(1,len(alist )):3
4 currentvalue = alist[index]5 position = index6
7 while position >0 and alist[position -1]> currentvalue:8 alist[position ]=alist[position -1]9 position = position -1
10
11 alist[position ]= currentvalue
Listing 5.12: insertionSort
This can be seen in Figure 5.18. This list has nine items. If we usean increment of three, there are three sublists, each of which can be sortedby an insertion sort. After completing these sorts, we get the list shownin Figure 5.19. Although this list is not completely sorted, something veryinteresting has happened. By sorting the sublists, we have moved the itemscloser to where they actually belong.
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
sublist 1
sublist 2
sublist 3
Figure 5.18: A Shell Sort with Increments of Three
Figure 5.20 shows a final insertion sort using an increment of one; inother words, a standard insertion sort. Note that by performing the earliersublist sorts, we have now reduced the total number of shifting operationsnecessary to put the list in its final order. For this case, we need only fourmore shifts to complete the process.
Shell Sort After Sorting Each Sublist216 Chapter 5. Searching and Sorting
17 26 93 44 77 31 54 55 20
54 26 93 17 55 31 44 77 20
54 26 20 17 77 31 44 55 93
sublist 1 sorted
sublist 2 sorted
sublist 3 sorted
17 26 20 44 55 31 54 77 93 after sorting sublistsat increment 3
Figure 5.19: A Shell Sort after Sorting Each Sublist
17 26 20 44 55 31 54 77 93 1 shift for 20
17 20 26 44 55 31 54 77 93 2 shifts for 31
17 20 26 31 44 55 54 77 93 1 shift for 54
17 20 26 31 44 54 55 77 93 sorted
Figure 5.20: ShellSort: A Final Insertion Sort with Increment of 1
We said earlier that the way in which the increments are chosen is theunique feature of the shell sort. The function shown in Listing 5.13 uses adi↵erent set of increments. In this case, we begin with n
2 sublists. On thenext pass, n
4 sublists are sorted. Eventually, a single list is sorted with the
Shell Sort A Final Insertion Sort with Increment of 1
216 Chapter 5. Searching and Sorting
17 26 93 44 77 31 54 55 20
54 26 93 17 55 31 44 77 20
54 26 20 17 77 31 44 55 93
sublist 1 sorted
sublist 2 sorted
sublist 3 sorted
17 26 20 44 55 31 54 77 93 after sorting sublistsat increment 3
Figure 5.19: A Shell Sort after Sorting Each Sublist
17 26 20 44 55 31 54 77 93 1 shift for 20
17 20 26 44 55 31 54 77 93 2 shifts for 31
17 20 26 31 44 55 54 77 93 1 shift for 54
17 20 26 31 44 54 55 77 93 sorted
Figure 5.20: ShellSort: A Final Insertion Sort with Increment of 1
We said earlier that the way in which the increments are chosen is theunique feature of the shell sort. The function shown in Listing 5.13 uses adi↵erent set of increments. In this case, we begin with n
2 sublists. On thenext pass, n
4 sublists are sorted. Eventually, a single list is sorted with the
Shell Sort Initial Sublists for a Shell Sort
5.3. Sorting 217
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
sublist 1
sublist 2
sublist 3
54 26 93 17 77 31 44 55 20 sublist 4
Figure 5.21: Initial Sublists for a Shell Sort
basic insertion sort. Figure 5.21 shows the first sublists for our exampleusing this increment.
The following invocation of the shellSort function shows the partiallysorted lists after each increment, with the final sort being an insertion sortwith an increment of one.
>>> alist=[54,26,93,17,77,31,44,55,20]
>>> shellSort(alist)
After increments of size 4 the list is
[20, 26, 44, 17, 54, 31, 93, 55, 77]
After increments of size 2 the list is
[20, 17, 44, 26, 54, 31, 77, 55, 93]
After increments of size 1 the list is
[17, 20, 26, 31, 44, 54, 55, 77, 93]
At first glance you may think that a shell sort cannot be better thanan insertion sort, since it does a complete insertion sort as the last step. Itturns out, however, that this final insertion sort does not need to do verymany comparisons (or shifts) since the list has been pre-sorted by earlierincremental insertion sorts, as described above. In other words, each passproduces a list that is “more sorted” than the previous one. This makes thefinal pass very e�cient.
Although a general analysis of the shell sort is well beyond the scopeof this text, we can say that it tends to fall somewhere between O(n) andO(n2), based on the behavior described above. For the increments shown
Shell Sort Implementation
def shell_sort(alist): sublistcount = len(alist)//2 while sublistcount > 0: for startposition in range(sublistcount): gap_insertion_sort(alist, startposition, sublistcount)
print("After increments of size", sublistcount, "The list is", alist)
sublistcount = sublistcount // 2
Shell Sort Implementation
def gap_insertion_sort(alist, start, gap): for i in range(start+gap, len(alist), gap): currentvalue = alist[i] position = i
while position >= gap and \ alist[position-gap] > currentvalue: alist[position] = alist[position-gap] position = position-gap
alist[position] = currentvalue
Merge Sort
Merge Sort Splitting and Merging
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
Splitting and Merging in a Merge Sort
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
54 26 93 17 77 31 44 55 20
55 20
Algorithm Analysis
Merge Sort Splitting and Merging
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
Splitting and Merging in a Merge Sort
17 20 26 31 44 54 55 77 93
17 26 54 93
20 31 44 55 77
26 54 17 93 31 77
20 44 55
54 26 93 17 77 31 44
20 55
55 20
Algorithm Analysis
Merge Sort Implementation
def merge_sort(alist): print("Splitting ", alist) if len(alist) > 1: mid = len(alist)//2 lefthalf = alist[:mid] righthalf = alist[mid:]
merge_sort(lefthalf) merge_sort(righthalf)
i = 0 j = 0 k = 0
Merge Sort Implementation (cont.)
while i < len(lefthalf) and j < len(righthalf): if lefthalf[i] < righthalf[j]: alist[k] = lefthalf[i] i = i+1 else: alist[k] = righthalf[j] j = j+1 k = k+1
while i < len(lefthalf): alist[k] = lefthalf[i] i = i+1 k = k+1
while j < len(righthalf): alist[k] = righthalf[j] j = j+1 k = k+1 print("Merging ", alist)
Quick Sort
Quick Sort The First Pivot Value
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
The First Pivot Value for a Quick Sort
54 26 93 17 77 31 44 55 20 54 will be thefirst pivot value
Algorithm Analysis
Quick Sort Finding the Split Point for 54
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
Finding the Split Point for 54
54 26 93 17 77 31 44 55 20
leftmark rightmark
54 26 93 17 77 31 44 55 20
leftmark rightmark
54 26 93 17 77 31 44 55 20
leftmark rightmark
54 26 20 17 77 31 44 55 93
leftmark rightmark
54 26 20 17 44 31 77 55 93
leftmarkrightmark
26<54 move to right93>54 stop
now rightmark20<54 stop
54 26 20 17 77 31 44 55 93
leftmark rightmark
exchange 20 and 93
77>54 stop44<54 stop
exchange 77 and 44
77>54 stop31<54 stop
rightmark<leftmarksplit point found
exchange 54 and 31
leftmark and rightmark will converge on split point
now continue moving leftmark and rightmark
until they cross
Algorithm Analysis
Quick Sort Completing the Partition Process to Find
the Split Point for 54
What Is Algorithm Analysis?Searching
Sorting
The Bubble SortThe Selection SortThe Insertion SortThe Shell SortThe Merge SortThe Quick Sort
Completing the Partition Process to Find the SplitPoint for 54
31 26 20 17 44 54 77 55 93 54 is in place
<54 >54
31 26 20 17 44 77 55 93
quicksort left half quicksort right half
Algorithm Analysis
Quick Sort Implementation
def quick_sort(alist): quick_sort_helper(alist, 0, len(alist)-1)
def quick_sort_helper(alist, first, last): if first<last:
splitpoint = partition(alist,first,last)
quick_sort_helper(alist, first, splitpoint-1) quick_sort_helper(alist, splitpoint+1, last)
Quick Sort Implementation (cont.)
def partition(alist, first, last): pivotvalue = alist[first]
leftmark = first+1 rightmark = last
done = False while not done: while leftmark <= rightmark and \ alist[leftmark] <= pivotvalue: leftmark = leftmark + 1
while alist[rightmark] >= pivotvalue and \ rightmark >= leftmark: rightmark = rightmark -1
# Continued on next slide...
Quick Sort Implementation (cont.)
if rightmark < leftmark: done = True else: temp = alist[leftmark] alist[leftmark] = alist[rightmark] alist[rightmark] = temp
temp = alist[first] alist[first] = alist[rightmark] alist[rightmark] = temp
return rightmark
Analysis
Analysis
• A bubble sort, a selection sort, and an insertion sort are O(n2) algorithms
• A shell sort improves on the insertion sort by sorting incremental sub-lists
• It falls between O(n) and O(n2)
• A merge sort is O(n log n), but requires additional space for the merging process
Analysis
• A quick sort is O(n log n), but may degrade to O(n2) if the split points are not near the middle of the list
• It does not require additional space
Questions?