assignmentsalgorithm design techniques •brute force: when stuck, start here and improve •divide...
TRANSCRIPT
![Page 1: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/1.jpg)
Assignments• Assignment 2: Done• Assignment 3: • Will be released on 2 Oct• Due: 23 Oct 23:59• Grace period ends: 24 Oct 13:00
![Page 2: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/2.jpg)
Final Project •Milestone 1:• Feedback sent on 26 Sep morning• We are resolving issues with feedback for 10
students because of inconsistencies in the name used in wattle and enrollment record• If you don’t want to have this type of delay, please
use the same name as in your enrollment record
![Page 3: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/3.jpg)
Final Project •Milestone 2:• Description released on 26 Sep night (please
read them)• Due: 8 Oct 23:59• Grace period ends: 9 Oct 13:00• Last chance to alter application and
assumptions• Can modify at most 2 functionalities• 2 things to submit• Design document• Program skeleton
![Page 4: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/4.jpg)
COMP3600/6466 – Algorithms Algorithm Design Techniques
[Lev Intro of ch. 2-6, 8-10]
Hanna Kurniawati
https://cs.anu.edu.au/courses/comp3600/[email protected]
![Page 5: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/5.jpg)
So far …üThe Problemü Algorithm definition + Model of Computation
üAnalysis of Algorithmsü Asymptotic Notationsü Recurrence Analysis (Divide-and-Conquer)ü Probabilistic Analysis (Avg. case + Randomized Algorithms) ü Correctnessü Empirical Analysis
üAbstract data structuresü Binary Search Treeü Heapsü AVL Treeü Red Black Treeü Hash Tables
![Page 6: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/6.jpg)
Algorithm Design Techniques• Brute Force• Divide & Conquer• Randomized Algorithm• Decrease & Conquer• Transform & Conquer• Dynamic Programming• Greedy• Iterative Improvement
![Page 7: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/7.jpg)
Algorithm Design Techniques• Brute Force: When stuck, start here and improve• Divide & Conquer: Discussed a bit in recurrence analysis• Randomized Algorithm: Discussed a bit in prob. analysis • Decrease & Conquer: Very similar to divide & conquer, but
reduce to 1 smaller sub-problem (e.g., Insertion Sort)• Transform & Conquer: Simplify the problem. For instance, by
pre-sorting and changing data representation (use suitable abstract data structures)
• Dynamic Programming• Greedy• Iterative Improvement
![Page 8: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/8.jpg)
COMP3600/6466 – Algorithms Dynamic Programming
[CLRS ch. 15 + 22.4]
Hanna Kurniawati
https://cs.anu.edu.au/courses/comp3600/[email protected]
![Page 9: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/9.jpg)
Topics• What is it?• Example: Fibonacci Sequence• How to develop DP algorithms?• Example: Shortest Path• Example: Chain matrix multiplication• Example: Longest Common Subsequence• Example: Decision-making under uncertainty
![Page 10: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/10.jpg)
What is Dynamic Programming (DP)• DP is an approach to solve problems by trying all
possibilities while avoiding recomputation• Key idea: Remember solutions that have been
computed and reuse as much as possible
![Page 11: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/11.jpg)
DP Requirements• Not all problems can be solved using DP. Can
only be applied to problems whose solutions can be constructed from solutions to its sub-problems• Specifically, two requirements:• Optimal substructure: Optimal solution to the
problem is formed by optimal solutions to sub-problems, which can be solved separately.• Overlapping sub-problems
![Page 12: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/12.jpg)
DP – Trivia • Invented by Richard Bellman in the ‘50s• ”Programming” here has somewhat different
meaning than computer programming• Many stories as to what the word “Programming”
actually means and why it is used• Regardless, it is a powerful algorithm design
technique• Often used for solving optimization problem
![Page 13: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/13.jpg)
Types of Dynamic Programming• Two types:• Top-down (memoization)• Start from the problem we want to solve• Solve sub-problems “reachable” (based on a
decomposition of the problem) from the problem and store the results (aka., made into a memo, memoization)• Reuse the stored solutions whenever possible• Bottom-up• Compute the sub-problems from small to large, starting from
the base, keep the results • Reuse whenever possible
![Page 14: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/14.jpg)
TopicsüWhat is it?• Example: Fibonacci Sequence• How to develop DP algorithms?• Example: Shortest Path• Example: Chain matrix multiplication• Example: Longest Common Subsequence• Example: Decision-making under uncertainty
![Page 15: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/15.jpg)
Example: Fibonacci Sequence• The nth Fibonacci number is
F(n) = F(n-1) + F(n-2)• Algorithm to output the nth Fibonacci number
NaiveFibonacci(n)If n ≤ 2
Return 1Else
Res = NaiveFibonacci(n-1) + NaiveFibonacci(n-2)Return Res
• Time complexity?• 𝑇 𝑛 = 𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + Θ 1 = Θ(2!)
![Page 16: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/16.jpg)
Upper bound:𝑇 𝑛 ≤ 2𝑇 𝑛 − 1 + 𝑐
= 2 2𝑇 𝑛 − 2 + 𝑐 + 𝑐= 2.2(2𝑇 𝑛 − 3 + 𝑐) + 2𝑐 + 𝑐= 2.2.2…2. 𝑇 2 + 𝑐 ∑!"#$%& 2!
= 2$%' + 𝑐 2$%' − 1= O(2$)
𝑇 𝑛 = 𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + Θ 1
𝑛 − 2
Lower bound:𝑇 𝑛 ≥ 2𝑇 𝑛 − 2 + 𝑐
= 2 2𝑇 𝑛 − 4 + 𝑐 + 𝑐= 2.2(2𝑇 𝑛 − 6 + 𝑐) + 2𝑐 + 𝑐
= 2.2.2…2. 1 + 𝑐 ∑!"#!" %( 2!
= 2!" + 𝑐 2
!" − 1
= Ω(2$)
𝑛2
𝑇 𝑛 = 𝑇 𝑛 − 1 + 𝑇 𝑛 − 2 + Θ 1 = Θ 2,
![Page 17: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/17.jpg)
Can we do better?• Yes, using DP• The problem satisfies the two requirements
for DP:• Optimal sub-structure: In this case, solution to
Fibonacci(n) is formed by solutions to Fibonacci(n-1) and Fibonacci(n-2), and the sub-problems can be solved separately • Overlapping sub-problems
![Page 18: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/18.jpg)
Overlapping Sub-problems
NaiveFibonacci performs too many repeated calculations
![Page 19: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/19.jpg)
Example: Memoization• Top-down DP (sometimes called memoization)
TopDownDP-Fibonacci(n)Fibo[1..n] = -1Return RecurseFibo(n, Fibo)
RecurseFibo(n, Fibo)If (Fibo[n] != -1)
Return Fibo[n]If n ≤ 2
F = 1Else
F = RecurseFibo(n-1, Fibo) + RecurseFibo(n-2, Fibo)Fibo[n] = FReturn F
Time complexity? Θ(𝑛)
![Page 20: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/20.jpg)
Example: Bottom-Up• Bottom-up DP
BottomUp-Fibonacci(n)Fibo[1] = 1 Fibo[2] = 1 For i = 3 to n
Fibo[i] = Fibo[i-1] + Fibo[i-2] Return Fibo[n]
• Time complexity? Θ(𝑛)
![Page 21: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/21.jpg)
Bottom-Up DP• No recursion• Although asymptotic time complexity is the same as top-down, in
practice bottom-up is faster because it reduces the number of function calls
• How to decide which sub-problem to compute first? • Reverse topological order of the sub-problem graph• Sub-problem graph encodes dependencies of the sub-problems• Vertices: Sub-problems• An edge from u to v: Solving the sub-problem represented by u
requires direct solving of the sub-problem represented by v• For DP to work this graph must be a directed acyclic graph (DAG)
![Page 22: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/22.jpg)
Topological Sort of DAG [CLRS 22.4]• Suppose G(V, E) is a DAG• Topological sort of G is a linear ordering of all its
vertices such that if G contains a directed edge (u, v) then u appears before v in the ordering • Use DFS that maintain time stamp• Discovery time of vertex v: The time v is first visited• Finishing time of vertex v: The time all vertices in E
adjacent to v have been visited
![Page 23: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/23.jpg)
[CLRS p.604]
![Page 24: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/24.jpg)
Example
• Start from u
![Page 25: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/25.jpg)
Topological Sort of DAG [CLRS 22.4]
• Example:
Time complexity: Θ( 𝑉 + 𝐸 )
![Page 26: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/26.jpg)
![Page 27: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/27.jpg)
Now, back to Fibonacci DP Example• Sub-problem graph:
![Page 28: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/28.jpg)
What if sub-problem graph has cycle?• Cause the DP to have infinite loop•What can be done?
![Page 29: AssignmentsAlgorithm Design Techniques •Brute Force: When stuck, start here and improve •Divide & Conquer: Discussed a bit in recurrence analysis •Randomized Algorithm: Discussed](https://reader035.vdocuments.us/reader035/viewer/2022081621/6129ffc17006e372a415ff0f/html5/thumbnails/29.jpg)
TopicsüWhat is it?üExample: Fibonacci Sequence• How to develop DP algorithms?• Example: Shortest Path• Example: Chain matrix multiplication• Example: Longest Common Subsequence• Example: Decision-making under uncertainty
Next: How to develop DP algorithms &
More examples