![Page 1: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/1.jpg)
Sorting Algorithms
CSE21 Winter 2017, Day 2 (B00), Day 1-2 (A00)
January 11, 2017
![Page 2: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/2.jpg)
Sorting (or Ordering)
vs.
* Assume elements of the set to be sorted have some underlying order
Section 3.1 in Rosen
![Page 3: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/3.jpg)
Why sort?
A TA facing a stack of exams needs to
input all 400 scores into a spreadsheet
where the students are listed in
alphabetical order.
OR
You want to find all the duplicate
values in a long list.
![Page 4: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/4.jpg)
General questions to ask about algorithms
1) What problem are we solving?
2) How do we solve the problem?
3) Why do these steps solve the problem?
4) When do we get an answer?
![Page 5: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/5.jpg)
General questions to ask about algorithms
1) What problem are we solving? PROBLEM SPECIFICATION
2) How do we solve the problem? ALGORITHM DESCRIPTION
3) Why do these steps solve the problem? CORRECTNESSS
4) When do we get an answer? RUNNING TIME PERFORMANCE
![Page 6: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/6.jpg)
Given a lista1, a2, ..., an
rearrange the values so that
a1 <= a2 <= ... <= an
Sorting: Specification: WHAT
Values can be any type (with underlying total order). For simplicity, use integers.
Rosen page 196
![Page 7: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/7.jpg)
Your approaches: HOW
https://en.wikipedia.org/wiki/Sorting_algorithm
• Selection (min) sort
• Bubble sort
• Insertion sort
• Bucket sort
• Merge sort
• Bogo sort
• Quick sort
• Binary search tree traversal
Discussion
section!
![Page 8: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/8.jpg)
Selection Sort (Min Sort)
"Find the first name alphabetically, move it to the front.
Then look for the next one, move it, etc.''
![Page 9: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/9.jpg)
Selection Sort (MinSort) Pseudocode
Rosen page 203, exercises 41-42
procedure selection sort(a1, a2, ..., an: real numbers with n >=2 )
for i := 1 to n-1
m := i
for j:= i+1 to n
if ( aj < am ) then m := j
interchange ai and am
{ a1, ..., an is in increasing order}
![Page 10: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/10.jpg)
Bubble Sort
"Compare the first two cards, and if the first is bigger,
keep comparing it to the next card in the stack until we
find one larger than it. Repeat until the stack is sorted.''
![Page 11: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/11.jpg)
Bubble Sort Pseudocode
procedure bubble sort(a1, a2, ..., an: real numbers with n >=2 )
for i := 1 to n-1
for j:= 1 to n-i
if ( aj > aj+1 ) then interchange aj and aj+1
{ a1, ..., an is in increasing order}
Rosen page 197
![Page 12: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/12.jpg)
Insertion Sort
"We passed the cards from right to left, each individual
inserting their own card in the correct position as they
relayed the pile."
![Page 13: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/13.jpg)
Insertion Sort Pseudocode
procedure insertion sort(a1, a2, ..., an: real numbers with n >=2 )
for j := 2 to n
i := 1
while aj > aii := i+1
m := ajfor k := 0 to j-i-1
aj-k := aj-k-1ai := m
{ a1, ..., an is in increasing order}
Rosen page 198
![Page 14: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/14.jpg)
Bucket Sort
"Call out from A to Z, collecting cards by first letter. If
there are more than one with the same first letter, repeat
with the second letter, and so on.''
![Page 15: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/15.jpg)
Bucket Sort – Pseudo pseudo code
• Create empty buckets that have an ordering.
• Put each of the elements of the list into the correct bucket.
• Sort within each bucket.
• Concatenate the buckets in order.
![Page 16: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/16.jpg)
Merge Sort
"We split into two groups and organized each of the groups, then
got back together and figured out how to interleave the groups in
order."
![Page 17: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/17.jpg)
Merge Sort – Pseudo pseudo code
• If the list has just one element, return.
• Otherwise,
• Divide list into two pieces: L1 = a1 ... an/2 and L2 = an/2+1 ... an
• M1 = Merge sort ( L1 )
• M2 = Merge sort ( L2 )
• Merge the two (sorted) lists M1 and M2
Rosen page 196, 367-370
![Page 18: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/18.jpg)
Why so many algorithms?
![Page 19: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/19.jpg)
Why so many algorithms?
Practice for homework / exam / job interviews.
Some algorithms are better than others. Wait, better?
![Page 20: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/20.jpg)
From "How" to "Why"
What makes this algorithm work?
How do you know that the resulting list will be sorted?
For loop-based algorithms:
What’s the effect of each loop iteration on the list?
Have we made progress?
![Page 21: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/21.jpg)
Loop Invariants
A loop invariant is a property that remains true after each time the
body of a loop is executed.
For an iterative algorithm:
Look for a loop invariant
• State the property precisely
Prove that it is invariant
• It must be true after any number of loop iterations
Use the invariant to prove correctness
• Show that when the loop is finished, the invariant guarantees
that we've reached a solution
1
2
3
![Page 22: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/22.jpg)
Selection Sort (MinSort)
"Find the first name alphabetically, move it to the front.
Then look for the next one, move it, etc.''
![Page 23: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/23.jpg)
Selection Sort (MinSort) Pseudocode
Rosen page 203, exercises 41-42
procedure selection sort(a1, a2, ..., an: real numbers with n >=2 )
for i := 1 to n-1
m := i
for j:= i+1 to n
if ( aj < am ) then m := j
interchange ai and am
{ a1, ..., an is in increasing order}
What is the loop invariant for this iterative algorithm?
![Page 24: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/24.jpg)
Selection Sort (MinSort) Correctness: WHY
Loop invariant: After the kth time through the outer
loop, the first k elements of the list are the k
smallest list elements in order.
1
![Page 25: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/25.jpg)
Selection Sort (MinSort) Correctness: WHY
Loop invariant: After the kth time through the outer
loop, the first k elements of the list are the k
smallest list elements in order.
How can we show that this loop invariant is true?
Once we do, why can we conclude that the program
is correct?
1
3
2
![Page 26: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/26.jpg)
Selection Sort (MinSort) Correctness: WHY
Loop invariant: After the kth time through the outer
loop, the first k elements of the list are the k
smallest list elements in order.
How can we show that this loop invariant is true?
Once we do, why can we conclude that the program
is correct?
1
3
2
![Page 27: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/27.jpg)
Selection Sort (MinSort) Correctness: WHY
Loop invariant: After the kth time through the outer
loop, the first k elements of the list are the k
smallest list elements in order.
Therefore,
How would you use the loop invariant to prove the correctness of MinSort?
3
![Page 28: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/28.jpg)
Loop invariant: After the kth time through the outer
loop, the first k elements of the list are the k
smallest list elements in order.
How can we show that this loop invariant is true?
Once we do, why can we conclude that the program
is correct?
Selection Sort (MinSort) Correctness: WHY
Induction
1
3
2
![Page 29: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/29.jpg)
Loop invariant: After the kth time through the outer
loop, the first k elements of the list are the k
smallest list elements in order.
How can we show that this loop invariant is true?
Selection Sort (MinSort) Correctness: WHY
2
Induction…… What variable do we induct on?
![Page 30: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/30.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It’s true when k=0 (before the loop.)
Induction Step: If it’s ever true, then going through the loop one more
time keeps it true.
OR
2
0
k k+1k-1 k
![Page 31: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/31.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It’s true when k=0 (before the loop.)
Induction Step: If it’s ever true, then going through the loop one more
time keeps it true.
ORTherefore: It’s true for all values of k>=0.
2
0
k k+1k-1 k
![Page 32: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/32.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It's true when k=0 (before the loop.)
Induction Step: If it's ever true, then going through the loop one more
time keeps it true.
ORTherefore: It's true for all values of k>=0.
2
k k+1
0
k-1 k
0
![Page 33: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/33.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It's true when k=0 (before the loop.)
Induction Step: If it's ever true, then going through the loop one more
time keeps it true.
ORTherefore: It's true for all values of k>=0.
2
k k+1
0 1
k-1 k
0
![Page 34: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/34.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It's true when k=0 (before the loop.)
Induction Step: If it's ever true, then going through the loop one more
time keeps it true.
ORTherefore: It's true for all values of k>=0.
2
k k+1
0 1
k-1 k
2
0
![Page 35: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/35.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It's true when k=0 (before the loop.)
Induction Step: If it's ever true, then going through the loop one more
time keeps it true.
ORTherefore: It's true for all values of k>=0.
2
k k+1
0 1
k-1 k
2 3
0
![Page 36: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/36.jpg)
Structure of the Induction Proof
Induction variable (k): the number of times through the loop.
Base Case: It's true when k=0 (before the loop.)
Induction Step: If it's ever true, then going through the loop one more
time keeps it true.
ORTherefore: It's true for all values of k>=0.
2
k k+1
0 1
k-1 k
2 3
0
![Page 37: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/37.jpg)
Statement: After the kth time through the outer loop, the first k
elements of the list are the k smallest list elements in order.
Induction variable (k): the number of times through the loop.
Base case: Need to show the statement holds for k=0, before the loop
Inductive step: Let k be a positive integer.
Induction hypothesis: Suppose the statement holds after k-1 times
through the loop.
Need to show that the statement holds after k times through the loop.
2
![Page 38: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/38.jpg)
Selection Sort (MinSort) Pseudocode
Rosen page 203, exercises 41-42
procedure selection sort(a1, a2, ..., an: real numbers with n >=2 )
for i := 1 to n-1
m := i
for j:= i+1 to n
if ( aj < am ) then m := j
interchange ai and am
{ a1, ..., an is in increasing order}
![Page 39: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/39.jpg)
Proving Loop Invariants
Induction variable (k): the number of times through the loop.
Base case: Need to show the statement holds for k=0, before the loop
Inductive step: Let k be a positive integer.
Induction hypothesis: Suppose the statement holds after k-1 times
through the loop.
Need to show that the statement holds after k times through the loop.
Need help?
See website handout for
extra practice problems on
invariants and induction.
2
![Page 40: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/40.jpg)
Why sort?
A TA facing a stack of exams needs to
input all 400 scores into a spreadsheet
where the students are listed in
alphabetical order.
OR
You want to find all the duplicate
values in a long list.
![Page 41: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/41.jpg)
Sorting helps with searching
Two searching algorithms:
One that works for any data, sorted or not
One that is much faster, but relies on the data being sorted
More on this next time!
![Page 42: Sorting Algorithms - vlsicad.ucsd.edu · • Selection (min) sort • Bubble sort • Insertion sort • Bucket sort • Merge sort • Bogo sort • Quick sort](https://reader031.vdocuments.us/reader031/viewer/2022012317/5b4fab477f8b9a2f6e8cd798/html5/thumbnails/42.jpg)
Announcements
Things to do:• Sign up on Gradescope
• Enroll in and attend your
discussion section
• Sign up on Piazza
On course website:• Guide to proofs and logic
• Guide to correctness proofs
• Extra practice problems
on invariants
and induction
HW1 due
Tuesday 1/17
at 11:59PM