madhavan mukund - cmi
TRANSCRIPT
Programming and Data Structures with Python
Madhavan Mukund
https://www.cmi.ac.in/~madhavan
25 January, 2021
Merge SortTo sort A[0:n] into B[0:n]
If n is 1, nothing to be done
Otherwise
Sort A[0:n//2] into L (left)
Sort A[n//2:n] into R (right)
Merge L and R into B
Divide & Conquer
7?Sort sort¥Merged
Merge Sort43 32 22 78 63 57 91 131.3 22 32 43 ft 63 It 91
gits jiyzz It! 1352639132 Irs 2278
*""ii:*. of
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2)+
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2)+
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
Tfn) - time to sort a list of20 length n
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2) +Merging cost
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
-Recursive sort of two halves
- mum
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2) +Merging cost
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2) +Merging cost
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
"
on- TEA←
D
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2) + n
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
O
Merge Sort: Analysis
Assume n = 2k
T (1) = 1
T (n) = 2T (n/2) + n
What is the cost of merge?
Merge lists of length m, n elements into a list of length m + n
Each comparison adds one element to output list
O(m + n) steps overall
If m = n, O(n) steps
Expand and solve, T (n) = O(n log n)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 2 / 7
O←
Tcn) = 2T(nlz) t nI
d
= 2(2Tlnl47t4z) t nZ' Tate) t 2n ②
- --
24%4123 ) the)t2n23 TIM 123) t 3h Ds
=
"
2kt (Yew) then Dk
TIN -- 2kT(n/zk) t kn
k -- Logan Edh
.mn/zk--1TCi7--lITG7--2b9?tCi:std.ogan)na--
n .I X n log n
- olnlogn)
kelogn
# ④
same-7D -I 2 * zk- I
FEEL? 7777/4×2"duel
i
DD DD -
- I
Merge Sort: Shortcomings
Merging A and B creates a new array C
No obvious way to efficiently merge in place
Extra storage can be costly
Inherently recursive
Recursive call and return are expensive
Alternative approach
Extra space is required to merge
Merging happens because elements in left half must move right and vice versa
Can we divide so that everything to the left is smaller than everything to the right?
No need to merge!
O24€27
Divide and conquer without merging
Suppose the median value in A is m
Move all values ≤ m to left half of A
Right half has values > m
This shifting can be done in place, in time O(n)
Recursively sort left and right halves
A is now sorted! No need to merge
T(n) = 2T(n/2) + n = O(n log n)
→7-⑤28 6 4 31 to 9 O
2,413 , 1,0M 7,816 , 10,9. Merge Sort-To - mid --④7/12
01 234 TEJ 678 920ms [e.mid) MSG'd .B- -f
Eye Oln)
New Sort median?
I0 Thanedom
Divide and conquer without merging
How do we find the median?
Sort and pick up middle element
But our aim is to sort!
Instead, pick up some value in A — pivot
Split A with respect to this pivot element
Pivot not in test
[0 , 1,337 53
Eo , if,D 51 5
he
co , lid ③ G
Quicksort
Choose a pivot element
Typically the first value in the array
Partition A into lower and upper parts with respect to pivot
Move pivot between lower and upper partition
Recursively sort the two partitions
-
C. A R Hoare
-
Quick sort: example
13 32 22 43 63 73 91 78
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 3 / 7
A
\ a-f32139863732
q
Instead4354>13%027139878637
Quick sort: example
13 32 22 43 63 73 91 78
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 3 / 7
43 32 98 63 73 13 78 22
22 32 13/43/7-39878 63
→ c-qurseksnt quicksort
def Quicksort(A,l,r): # Sort A[l:r] if r - l <= 1: # Base case return ()
# Partition with respect to pivot, a[l] yellow = l+1
for green in range(l+1,r): if A[green] <= A[l]: (A[yellow],A[green]) = (A[green],A[yellow]) yellow = yellow + 1
# Move pivot into place (A[l],A[yellow-1]) = (A[yellow-1],A[l])
Quicksort(A,l,yellow-1) # Recursive calls Quicksort(A,yellow,r)
Quicksort in Python A
" 7th→
e- Hf
←
Getting the code right
How can we check that ourcode is correct?
How do we design the codeto begin with?
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, a[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 4 / 7
Getting the code right
How can we check that ourcode is correct?
How do we design the codeto begin with?
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, a[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 4 / 7
Getting the code right
How can we check that ourcode is correct?
How do we design the codeto begin with?
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, a[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 4 / 7
"o
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
-
-
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
TED
o
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
11
"
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
Loop invariants
Our computation is iterative
What does start signify inouter loop?
l[0:start] hasstart-smallest values ofl in ascending order
What do i, minpos signifyin inner loop?
minpos is position ofminimum value inl[start:i]
Ensure that our loopsmaintain these invariants
def SelectionSort(l):
# Scan slices l[0:len(l)], l[1:len(l)], ...for start in range(len(l)):
# Find minimum value in slice . . .minpos = startfor i in range(start,len(l)):
if l[i] < l[minpos]:minpos = i
# . . . and move it to start of slice(l[start],l[minpos]) = (l[minpos],l[start])
Outer loop ends with start == len(l), by firstinvariant entire list is in ascending order
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 5 / 7
Loop invariants
Loop in insert
seq[0:pos] is inascending order
seq[pos:k] is inascending order
Not sure if seq[pos-1]< seq[pos]
When loop ends, pos is 0
By the second invariant,seq[0:k] is now inascending order
def InsertionSort(seq):isort(seq,len(seq))
def isort(seq,k): # Sort slice seq[0:k]if k > 1:
isort(seq,k-1)insert(seq,k-1)
def insert(seq,k): # Insert seq[k] into sorted seq[0:k-1]pos = kwhile pos > 0 and seq[pos] < seq[pos-1]:
(seq[pos],seq[pos-1]) = (seq[pos-1],seq[pos])pos = pos-1
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 6 / 7
It
? (
Loop invariants
Loop in insert
seq[0:pos] is inascending order
seq[pos:k] is inascending order
Not sure if seq[pos-1]< seq[pos]
When loop ends, pos is 0
By the second invariant,seq[0:k] is now inascending order
def InsertionSort(seq):isort(seq,len(seq))
def isort(seq,k): # Sort slice seq[0:k]if k > 1:
isort(seq,k-1)insert(seq,k-1)
def insert(seq,k): # Insert seq[k] into sorted seq[0:k-1]pos = kwhile pos > 0 and seq[pos] < seq[pos-1]:
(seq[pos],seq[pos-1]) = (seq[pos-1],seq[pos])pos = pos-1
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 6 / 7
Loop invariants
Loop in insert
seq[0:pos] is inascending order
seq[pos:k] is inascending order
Not sure if seq[pos-1]< seq[pos]
When loop ends, pos is 0
By the second invariant,seq[0:k] is now inascending order
def InsertionSort(seq):isort(seq,len(seq))
def isort(seq,k): # Sort slice seq[0:k]if k > 1:
isort(seq,k-1)insert(seq,k-1)
def insert(seq,k): # Insert seq[k] into sorted seq[0:k-1]pos = kwhile pos > 0 and seq[pos] < seq[pos-1]:
(seq[pos],seq[pos-1]) = (seq[pos-1],seq[pos])pos = pos-1
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 6 / 7
Loop invariants
Loop in insert
seq[0:pos] is inascending order
seq[pos:k] is inascending order
Not sure if seq[pos-1]< seq[pos]
When loop ends, pos is 0
By the second invariant,seq[0:k] is now inascending order
def InsertionSort(seq):isort(seq,len(seq))
def isort(seq,k): # Sort slice seq[0:k]if k > 1:
isort(seq,k-1)insert(seq,k-1)
def insert(seq,k): # Insert seq[k] into sorted seq[0:k-1]pos = kwhile pos > 0 and seq[pos] < seq[pos-1]:
(seq[pos],seq[pos-1]) = (seq[pos-1],seq[pos])pos = pos-1
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 6 / 7
# pones =L
te
II
Loop invariants
Loop in insert
seq[0:pos] is inascending order
seq[pos:k] is inascending order
Not sure if seq[pos-1]< seq[pos]
When loop ends, pos is 0
By the second invariant,seq[0:k] is now inascending order
def InsertionSort(seq):isort(seq,len(seq))
def isort(seq,k): # Sort slice seq[0:k]if k > 1:
isort(seq,k-1)insert(seq,k-1)
def insert(seq,k): # Insert seq[k] into sorted seq[0:k-1]pos = kwhile pos > 0 and seq[pos] < seq[pos-1]:
(seq[pos],seq[pos-1]) = (seq[pos-1],seq[pos])pos = pos-1
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 6 / 7
Oun
or seqfpos -D2 seedpod
Loop invariants
Loop in insert
seq[0:pos] is inascending order
seq[pos:k] is inascending order
Not sure if seq[pos-1]< seq[pos]
When loop ends, pos is 0
By the second invariant,seq[0:k] is now inascending order
def InsertionSort(seq):isort(seq,len(seq))
def isort(seq,k): # Sort slice seq[0:k]if k > 1:
isort(seq,k-1)insert(seq,k-1)
def insert(seq,k): # Insert seq[k] into sorted seq[0:k-1]pos = kwhile pos > 0 and seq[pos] < seq[pos-1]:
(seq[pos],seq[pos-1]) = (seq[pos-1],seq[pos])pos = pos-1
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 6 / 7
or seqEpos-B e seq(post- s
,
Loop invariants
A[l] is pivot
A[l+1:yellow] haselements smaller than pivot
A[yellow:green] haselements greater than pivot
A[green:r] yet to beexamined
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, A[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 7 / 7
Loop invariants
A[l] is pivot
A[l+1:yellow] haselements smaller than pivot
A[yellow:green] haselements greater than pivot
A[green:r] yet to beexamined
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, A[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 7 / 7
E
Loop invariants
A[l] is pivot
A[l+1:yellow] haselements smaller than pivot
A[yellow:green] haselements greater than pivot
A[green:r] yet to beexamined
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, A[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 7 / 7
>
Loop invariants
A[l] is pivot
A[l+1:yellow] haselements smaller than pivot
A[yellow:green] haselements greater than pivot
A[green:r] yet to beexamined
def Quicksort(A,l,r): # Sort A[l:r]if r - l <= 1:
return ()
# Base case# Partition with respect to pivot, A[l]yellow = l+1for green in range(l+1,r):
if A[green] <= A[l]:(A[yellow],A[green]) = (A[green],A[yellow])yellow = yellow + 1
# Move pivot into place(A[l],A[yellow-1]) = (A[yellow-1],A[l])
# Recursive callsQuicksort(A,l,yellow-1)Quicksort(A,yellow,r)
Madhavan Mukund Programming and Data Structures with Python PDSP, 25 Jan 2021 7 / 7
•-
]green
= r-
11 Skip Acy - D