lecture 1. introduction / insertion sort / merge...
TRANSCRIPT
![Page 1: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/1.jpg)
Copyright 2000-2018 Networking Laboratory
Lecture 1.
Introduction / Insertion Sort / Merge Sort
T. H. Cormen, C. E. Leiserson and R. L. Rivest
Introduction to Algorithms, 3nd Edition, MIT Press, 2009
Sungkyunkwan University
Hyunseung Choo
![Page 2: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/2.jpg)
Algorithms Networking Laboratory 2/28
Algorithm
A well-defined computational procedure that transforms the
input to the output
Describes a specific computational procedure for achieving
the desired input/output relationship
An instance of a problem is all the inputs needed to compute
a solution to the problem
A correct algorithm
halts with the correct output for every input instance.
is said to solve the problem
![Page 3: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/3.jpg)
Algorithms Networking Laboratory 3/28
Algorithm
Example: Sorting
Input: A sequence of n numbers <a1, a2, ..., an>
Output: A permutation (reordering) <b1, b2, ..., bn> of the input
sequence such that b1 b2 ... bn
Instance: <6, 4, 3, 7, 1, 4>
![Page 4: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/4.jpg)
Algorithms Networking Laboratory 4/28
Algorithm
Insertion Sort
In-place sorting: Uses only a fixed amount of storage
beyond that needed for the data
Example:
6 4 3 7 1 4 4 6 3 7 1 4
^ * ^ *
3 4 6 7 1 4 3 4 6 7 1 4
* ^ *
1 3 4 6 7 4 1 3 4 4 6 7
^ *
![Page 5: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/5.jpg)
Algorithms Networking Laboratory 5/28
Algorithm
Example: 6 4 3 7 1 4
6 4 3 7 1 4
![Page 6: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/6.jpg)
Algorithms Networking Laboratory 6/28
Algorithm
Pseudocode:
INSERTION-SORT(A) /* A is an array of numbers */
1 for j 2 to length[A]
2 do key A[j]
3 /* insert A[j] into the sorted sequence A[1 .. j-1] */
4 i j - 1
5 while i > 0 and A[i] > key
6 do A[i+1] A[i]
7 i i - 1
8 A[i+1] key
![Page 7: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/7.jpg)
Algorithms Networking Laboratory 7/28
Algorithm
![Page 8: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/8.jpg)
Algorithms Networking Laboratory 8/28
Algorithm
Example:
5 2 4 6 1 3
![Page 9: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/9.jpg)
Algorithms
Insertion Sort Algorithm
Video Content
An illustration of Insertion Sort with Romanian folk dance.
Networking Laboratory 9/28
![Page 10: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/10.jpg)
Algorithms
Insertion Sort Algorithm
Networking Laboratory 10/28
![Page 11: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/11.jpg)
Algorithms Networking Laboratory 11/28
Analyzing Algorithm
Predicting the resources, such as memory, bandwidth,
logic gates, or running time
Assumed implementation model
Random-access machine (RAM)
Running time: f ( input size )
Input size:
Sorting: number of items to be sorted.
Multiplication: number of bits.
Graphs: numbers of vertices and edges.
![Page 12: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/12.jpg)
Algorithms Networking Laboratory 12/28
Analyzing Algorithm
Running time for a particular input is the number of primitive
operations executed
Assumption: Constant time ci for the execution of the ith line (of
pseudocode)
Note: tj is the number of times the while loop test in line 5 is executed for the value of j.
![Page 13: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/13.jpg)
Algorithms Networking Laboratory 13/28
Analyzing Algorithm
![Page 14: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/14.jpg)
Algorithms Networking Laboratory 14/28
Analyzing Algorithm
Best case
Array is already sorted, so tj = 1 for j = 2, 3, ..., n.
T(n) = c1n + c2(n - 1) + c4(n - 1) + c5(n - 1) + c8(n - 1)
= (c1 + c2 + c4 + c5 + c8)n - (c2 + c4 + c5 + c8)
= an + b (linear in n)
Worst case
= an2 + bn + c (quadratic in n).
![Page 15: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/15.jpg)
Algorithms Networking Laboratory 15/28
Analyzing Algorithm
Average Case ?
Concentrate on worst-case running time
Provides the upper bound
Occurs often
Average case is often as bad as the worst case
Order of Growth
The order of a running-time function is the fastest growing term,
discarding constant factors
Insertion sort
Best case: an + b (n)
Worst case: an2 + bn + c (n2)
![Page 16: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/16.jpg)
Algorithms Networking Laboratory 16/28
Designing Algorithms
Incremental design
Iterative
Example: insertion sort
Divide-and-conquer algorithm
Recursive
Example: merge sort
Three steps in the divide-and-conquer paradigm
Divide the problem into smaller subproblems
Conquer subproblems by solving them recursively
Combine solutions of subproblems
![Page 17: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/17.jpg)
Algorithms Networking Laboratory 17/28
Designing Algorithms
Merge Sort
Divide the n-element sequence into two subsequence of n/2
elements each
Conquer (sort) the two subsequences recursively using merge
sort
Combine (merge) the two sorted subsequences to produce the
sorted answer
Note: Recursion bottoms out when only one element
to be sorted
![Page 18: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/18.jpg)
Algorithms Networking Laboratory 18/28
Divide ...
5 1 4 2 10 3 9 15
5 1 4 2 10 3 9 15
5 1 4 2 10 3 9 15
5 1 4 2 10 3 9 15
![Page 19: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/19.jpg)
Algorithms Networking Laboratory 19/28
And Conquer
1 2 3 4 5 9 10 15
1 2 4 5 3 9 10 15
5 1 4 2 10 3 9 15
1 5 2 4 3 10 9 15
![Page 20: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/20.jpg)
Algorithms Networking Laboratory 20/28
Merge Sort
For MERGE_SORT, an initial array is repeatedly divided
into halves (usually each is a separate array), until arrays
of just one element remain
At each level of recombination, two sorted arrays are
merged into one
This is done by copying the smaller of the two elements
from the sorted arrays into the new array, and then
moving along the arrays
1 13 24 26 2 15 27 38
Aptr Bptr Cptr
![Page 21: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/21.jpg)
Algorithms Networking Laboratory 21/28
Merging
1 13 24 26 2 15 27 38 1
Aptr Bptr Cptr
1 13 24 26 2 15 27 38 1 2
Aptr Bptr Cptr
1 13 24 26 2 15 27 38 1 2 13
Aptr Bptr Cptr
etc.
![Page 22: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/22.jpg)
Algorithms Networking Laboratory 22/28
Merge Sort Algorithm
MERGE_SORT(A, p, r)
1 if p < r
2 then q
3 MERGE_SORT(A, p, q)
4 MERGE_SORT(A, q+1, r)
5 MERGE(A, p, q, r)
( ) /p r 2
![Page 23: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/23.jpg)
Algorithms Networking Laboratory 23/28
Merge Sort Algorithm
![Page 24: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/24.jpg)
Algorithms
Merge Sort Algorithm
Video Content
An illustration of Merge Sort with Transylvanian-saxon (German)
folk dance.
Networking Laboratory 24/28
![Page 25: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/25.jpg)
Algorithms
Merge Sort Algorithm
Networking Laboratory 25/28
![Page 26: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/26.jpg)
Algorithms Networking Laboratory 26/28
Merge Sort Algorithm
Note:
The MERGE_SORT(A, p, r) sorts the elements in the subarray A[p ..
r]
If p >= r, the subarray has at most one element and is therefore
already sorted
The procedure MERGE(A, p, q, r), where p <= q < r, merges two
already sorted subarrays A[p ..q] and A[q+1 .. r]. It takes (n) time
To sort an array A[1 .. n], we call MERGE_SORT(A, 1, n)
![Page 27: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/27.jpg)
Algorithms Networking Laboratory 27/28
Analyzing Divide-And-Conquer
Algorithms
Running time is described by a recurrence equation or
recurrence
Assume:
A problem is divided into a subproblems, each of which is 1/b
the size of the original
Dividing the problem takes D(n) time
Combining the solutions to subproblems into the solution to
the original problem takes C(n) time
T(n) = (1) if n <= c,
= aT(n/b) + D(n) + C(n) otherwise.
![Page 28: Lecture 1. Introduction / Insertion Sort / Merge Sortmonet.skku.edu/wp-content/uploads/2018/03/Algorithm_01.pdf · 2018-03-03 · Example: insertion sort Divide-and-conquer algorithm](https://reader035.vdocuments.us/reader035/viewer/2022062918/5ede3b75ad6a402d66698c43/html5/thumbnails/28.jpg)
Algorithms Networking Laboratory 28/28
Analysis of Merge Sort
Divide: Computes the middle of the subarray D(n) = (1)
Conquer: We recursively solve two subproblems, each of size
n/2, contributing 2T(n/2)
Combine: The MERGE procedure takes (n),
so C(n) = (n)
The worst-case running time of merge sort is:
T(n) = (1) if n = 1,
= 2T(n/2) + (n) if n > 1
Analyzing Divide-And-Conquer
Algorithms