cs421 - course information
DESCRIPTION
CS421 - Course Information. Website Syllabus Schedule The Book:. What is an algorithm?. Informally, an algorithm is any well-defined computational procedure that takes some value(s) as input and produces some value(s) as output. Goals for an algorithm: Correct Terminates Efficient. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/1.jpg)
CS421 - Course Information
• Website
• Syllabus
• Schedule
• The Book:
![Page 2: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/2.jpg)
What is an algorithm?
Informally, an algorithm is any well-defined computational procedure that takes some value(s) as input and produces some value(s) as output.
Goals for an algorithm:1. Correct2. Terminates3. Efficient
![Page 3: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/3.jpg)
What is CS421 about?
We will engage in the theoretical study of the design and analysis of computer algorithms.
– Analysis: predict the cost of an algorithm in terms of performance and resources used
– Design: design algorithms which minimize such costs
![Page 4: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/4.jpg)
Machine Model Assumptions..
Random Access Machine (RAM) Model:1. Any memory cell can be accessed in 1 step.2. Memory is not limited (unbounded).3. Arbitrarily large integers can be stored in each memory
cell.4. Operations are executed sequentially.5. Operators include:
• primitive arithmetic (+, -, *, /, modulo, etc..) • logic (if..then, and, or, etc..) • comparators (<, >, =, etc..) and• function calls.
6. Each operation has a unit cost of 1.
![Page 5: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/5.jpg)
An Example.. Sorting
Input: A sequence of n numbers, a1, a2, …, an
Output: A permutation (reordering), a'1, a'2, …, a'n,
of the input sequence such that a'1 a'2 … a'n .
Example:
Input: <13, 7, 42, 3, 6, 41>Output: <3, 6, 7, 13, 41, 42>
![Page 6: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/6.jpg)
Insertion Sort
Pseudocode:
INSERTION-SORT (A)1 for j ← 2 to length[A]2 do key ← A[ j]3 i ← j – 14 while i > 0 and A[i] > key5 do A[i+1] ← A[i]6 i ← i – 17 A[i+1] = key
![Page 7: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/7.jpg)
Example of insertion sort
8 2 4 9 3 6
![Page 8: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/8.jpg)
Example of insertion sort
8 2 4 9 3 6
![Page 9: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/9.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
![Page 10: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/10.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
![Page 11: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/11.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
![Page 12: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/12.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
![Page 13: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/13.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
![Page 14: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/14.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
![Page 15: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/15.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
![Page 16: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/16.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
![Page 17: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/17.jpg)
Example of insertion sort
8 2 4 9 3 6
2 8 4 9 3 6
2 4 8 9 3 6
2 4 8 9 3 6
2 3 4 8 9 6
2 3 4 6 8 9 end
![Page 18: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/18.jpg)
How “Good” is Insertion Sort?
Recall our goals for an algorithm:1. Correct
2. Terminates
3. Efficient
![Page 19: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/19.jpg)
Correctness
• Informally: At each step the “current card”, j, is inserted into an already sorted subarray A[1.. j-1].
• More formally: The loop invariant (a condition that does not change if correct) is that at the start of each for-loop, the subarray A[1.. j-1] consists of the elements in A[1.. j-1] but in sorted order.
![Page 20: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/20.jpg)
Correctness
These properties must hold for a loop invariant:
• Initialization: It is true prior to the first iteration.• Maintenance: It is true before an iteration of the
loop and remains true before the next iteration.• Termination: When the loop terminates, the
invariant yields a useful property which helps show the algorithm is correct.
![Page 21: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/21.jpg)
Correctness
In the case of insertion sort:
• Initialization: If j=2 at initialization, A[1..j-1] consists of a single element which is by definition sorted.
• Maintenance: Informally, the for-loop works by moving A[j-1], A[j-2], etc. one position to the right until the correct position for A[j] is found. Since A[1..j] is now sorted, when j is incremented, A[1..j-1] is sorted.
• Termination: The loop terminates when j=n+1. From the Maintenance property we know A[1..j-1] is now sorted. That is A[1..n] is now sorted. Hence the algorithm is correct.
![Page 22: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/22.jpg)
Termination
• As we will see later in the semester determining if an arbitrary program terminates or halts is undecidable for Turing machines.
• In the specific instance of Insertion-Sort, it is easy to see that the two loops iterate at most n times each and thus the algorithm does not run indefinitely and does terminate.
![Page 23: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/23.jpg)
Efficiency
• Running time is a measure of how many steps or primitive operations were performed.
• We have already stated that in the RAM machine model each operator has cost 1, but lets assume instead each operator has cost ci, where i is a line in our algorithm.
![Page 24: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/24.jpg)
Kinds of Analysis
• Worst-case: T(n) = maximum time of algorithm on any input of size n.
• Average-case: T(n) = expected time of algorithm over all inputs of size n.– Requires assumption of statistical distribution of
inputs.
• Best-case: T(n) = minimum time of algorithm on any input of size n. – Problematic because a generally slow algorithm may
works fast on some input.
![Page 25: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/25.jpg)
Running Time
Lets analyze our algorithm once more..
INSERTION-SORT (A) cost times1 for j ← 2 to length[A] c1 n2 do key ← A[ j] c2 n-13 i ← j – 1 c3 n-14 while i > 0 and A[i] > key c4 ∑j=2..ntj
5 do A[i+1] ← A[i] c5 ∑j=2..n (tj-1)6 i ← i – 1 c6 ∑j=2..n(tj-1)7 A[i+1] = key c7 n-1
![Page 26: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/26.jpg)
Best Case
T(n) = (c1+c2+c3+c4+c7)n – (c2+c3+c4+c7)
INSERTION-SORT (A) cost times1 for j ← 2 to length[A] c1 n2 do key ← A[ j] c2 n-13 i ← j – 1 c3 n-14 while i > 0 and A[i] > key c4 ∑j=2..ntj
5 do A[i+1] ← A[i] c5 ∑j=2..n (tj-1)6 i ← i – 1 c6 ∑j=2..n(tj-1)7 A[i+1] = key c7 n-1
![Page 27: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/27.jpg)
Worst Case
T(n) = (c4+c5+c6)n2/2 + (c1+c2+c3+c4/2-c5/2-c6/2c7)n – (c2 + c3 + c4 + c7)
INSERTION-SORT (A) cost times1 for j ← 2 to length[A] c1 n2 do key ← A[ j] c2 n-13 i ← j – 1 c3 n-14 while i > 0 and A[i] > key c4 ∑j=2..ntj
5 do A[i+1] ← A[i] c5 ∑j=2..n (tj-1)6 i ← i – 1 c6 ∑j=2..n(tj-1)7 A[i+1] = key c7 n-1
![Page 28: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/28.jpg)
Average Case
tj = j/2 .. Only out of order half the time..
T(n) = (c4+c5+c6)n2/4 + (c1+c2+c3+c4/4-c5/4-c6/4c7)n – (c2 + c3 + c4 + c7)
INSERTION-SORT (A) cost times1 for j ← 2 to length[A] c1 n2 do key ← A[ j] c2 n-13 i ← j – 1 c3 n-14 while i > 0 and A[i] > key c4 ∑j=2..ntj
5 do A[i+1] ← A[i] c5 ∑j=2..n (tj-1)6 i ← i – 1 c6 ∑j=2..n(tj-1)7 A[i+1] = key c7 n-1
![Page 29: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/29.jpg)
Machine Independent Analysis
As the size of the input becomes large, the constants ci don’t matter as much as the exponents and log factors. The constants also make machine independent analysis impossible.– Ignore the constants.– Examine growth of T(n) as n → ∞.– Asymptotic Analysis
![Page 30: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/30.jpg)
Order of Growth
• The rate of growth is of primary interest, so we consider only the leading term and ignore all constants (e.g. n^2)
• Thus, the worst case running time of Insertion Sort is Θ(n2). Quadratic time.
• We will define this more precisely later.
![Page 31: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/31.jpg)
Design Approach: Divide and Conquer
• Divide the problem into a number of subproblems.
• Conquer the subproblems recursively.
• Combine the subproblem solutions into the solution for the original problem.
• Recursion: when an algorithm calls itself.
![Page 32: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/32.jpg)
Merge Sort
• Divide: Divide an n-element array into two subsequences of n/2 elements each.
• Conquer: Sort the two subsequences recursively with merge sort.
• Combine: Merge the two sorted arrays to produce the sorted sequence.
• Special Case: If the sequence has only one element the recursion “bottoms out” as the sequence is sorted by definition.
![Page 33: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/33.jpg)
Merge Sort
MERGE-SORT (A[1 . . n])1. If n = 1, return A.2. L = A[ 1 . . n/2 ] 3. R = A[ n/2+1 . . n ] 4. L = Merge-Sort(L)5. R = Merge-Sort(R)6. Return Merge(L, R)
![Page 34: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/34.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
![Page 35: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/35.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
![Page 36: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/36.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
![Page 37: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/37.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
![Page 38: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/38.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
![Page 39: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/39.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
![Page 40: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/40.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
![Page 41: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/41.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
![Page 42: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/42.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
![Page 43: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/43.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
![Page 44: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/44.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
20
13
12
![Page 45: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/45.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
20
13
12
12
![Page 46: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/46.jpg)
Merging two sorted arrays
20
13
7
2
12
11
9
1
1
20
13
7
2
12
11
9
2
20
13
7
12
11
9
7
20
13
12
11
9
9
20
13
12
11
11
20
13
12
12
Time = (n) to merge a total of n elements (linear time).
![Page 47: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/47.jpg)
Analyzing merge sort
MERGE-SORT A[1 . . n]1. If n = 1, done.2. Recursively sort A[ 1 . . n/2 ]
and A[ n/2+1 . . n ] .3. “Merge” the 2 sorted lists
T(n)(1)2T(n/2)
(n)
Sloppiness: Should be T( n/2 ) + T( n/2 ) , but it turns out not to matter asymptotically.
![Page 48: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/48.jpg)
Recurrence for merge sort
T(n) =(1) if n = 1;
2T(n/2) + (n) if n > 1.
• We shall usually omit stating the base case when T(n) = (1) for sufficiently small n, but only when it has no effect on the asymptotic solution to the recurrence.
![Page 49: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/49.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
![Page 50: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/50.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
T(n)
![Page 51: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/51.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
T(n/2) T(n/2)
cn
![Page 52: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/52.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
T(n/4) T(n/4) T(n/4) T(n/4)
cn/2 cn/2
![Page 53: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/53.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
![Page 54: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/54.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
h = lg n
![Page 55: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/55.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
h = lg n
cn
![Page 56: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/56.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
h = lg n
cn
cn
![Page 57: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/57.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
h = lg n
cn
cn
cn
…
![Page 58: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/58.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
h = lg n
cn
cn
cn
#leaves = n (n)
…
![Page 59: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/59.jpg)
Recursion tree
Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.
cn
cn/4 cn/4 cn/4 cn/4
cn/2 cn/2
(1)
…
h = lg n
cn
cn
cn
#leaves = n (n)
Total(n lg n)
…
![Page 60: CS421 - Course Information](https://reader035.vdocuments.us/reader035/viewer/2022062321/56812d71550346895d928707/html5/thumbnails/60.jpg)
Conclusions
• (n lg n) grows more slowly than (n2).
• Therefore, merge sort asymptotically beats insertion sort in the worst case.