dynamic programming (continued)
DESCRIPTION
Dynamic Programming (continued). Programming Puzzles and Competitions CIS 4900 / 5920 Spring 2009. TCO 09 Algorithms (Feb.). Today. TCO 09 Algorithms (March). Lecture Outline. Order Notation (for real this time) Dynamic Programming w/ Iteration Fibonacci numbers (revisited) - PowerPoint PPT PresentationTRANSCRIPT
Dynamic Dynamic Programming Programming (continued)(continued)
Dynamic Dynamic Programming Programming (continued)(continued)
Programming Puzzles and CompetitionsProgramming Puzzles and CompetitionsCIS 4900 / 5920CIS 4900 / 5920
Spring 2009Spring 2009
TCO 09 Algorithms (Feb.) Today
TCO 09 Algorithms (March)
Lecture Outline• Order Notation (for real this time)• Dynamic Programming w/ Iteration• Fibonacci numbers (revisited)• Text Segmentation (revisited)• ACM ICPC
Algorithmic Complexity• We would like to be able to
describe how long a program takes to run
Algorithmic Complexity• We would like to be able to
describe how long a program takes to run
• We should express the runtime in terms of the input size
Algorithmic Complexity• Absolute time measurements are
not helpful, as computers get faster all the time
Algorithmic Complexity• Absolute time measurements are
not helpful, as computers get faster all the time
• Also, runtime is clearly dependent on the input (and input size)
Order Notation• We define O(*) as follows
Order Notation• Basically, we just want to count
the number of operations (without being too precise)
Examples• What’s the complexity of the
following code?
int c = a + b;
Examples• What’s the complexity of the
following code?
int c = a + b;
• Answer: O(1)
Examples• What’s the complexity of the
following code?
int c = a + b;
• Answer: O(1)– (it actually depends on how you count)
Example 2for(int i = 0; i < n; ++i) {
//do something}
Example 2for(int i = 0; i < n; ++i) {
//do something}
• Answer: O(n)
Example 2for(int i = 0; i < n; ++i) {
//do something}
• Answer: O(n)– O(c) operations for each i O(cn)
== O(n) total operations
Example 2for(int i = 0; i < n; ++i) {
for(int j = 0; j < n; ++j) {//do something
}}
Example 2for(int i = 0; i < n; ++i) {
for(int j = 0; j < n; ++j) {//do something
}}
• Answer: O(n2)
Example 2for(int i = 0; i < n; ++i) {
for(int j = 0; j < n; ++j) {//do something
}}
• Answer: O(n2)– O(n) operations for each i == n * n total
operations == O(n2) total operations
Example 3for(int i = 0; i < n; ++i) {
for(int j = i; j < n; ++j) {//do something
}}
Example 3for(int i = 0; i < n; ++i) {
for(int j = i; j < n; ++j) {//do something
}}
• The first iteration of the outer loop takes O(n)
Example 3for(int i = 0; i < n; ++i) {
for(int j = i; j < n; ++j) {//do something
}}
• The second iteration of the outer loop takes O(n-1)
Example 3for(int i = 0; i < n; ++i) {
for(int j = i; j < n; ++j) {//do something
}}
• The third iteration of the outer loop takes O(n-2)…
Example 3for(int i = 0; i < n; ++i) {
for(int j = i; j < n; ++j) {//do something
}}
• The last iteration of the outer loop takes O(1)
Example 3• n + (n–1) + … + 1 = ?
Example 3• n + (n–1) + … + 1 = ?• Call this sum N
Example 3• n + (n–1) + … + 1 = N• Call this sum N
Example 3• n + (n–1) + … + 1 = N• Call this sum N
N = n + (n-1) + … + 2 + 1
Example 3• n + (n–1) + … + 1 = N• Call this sum N
N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n
Example 3• n + (n–1) + … + 1 = N• Call this sum N
N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n
• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)
Example 3• n + (n–1) + … + 1 = N• Call this sum N
N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n
• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)
= n * (n + 1)
Example 3• n + (n–1) + … + 1 = N• Call this sum N
N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n
• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)
= n * (n + 1)N = n * (n + 1) / 2
Example 3• n + (n–1) + … + 1 = N• Call this sum N
N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n
• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)
= n * (n + 1)N = n * (n + 1) / 2 = O(n2)
Examples• This is also the number of ways 2 items
can be chosen from a set of (n + 1) items, disregarding order and without replacement (also called “(n + 1) choose 2”, binomial coefficient)
2
)1(
)!1(!2
)!1(
)!2)1((!2
)!1(2
1
nn
n
n
n
nn
Fibonacci Numbers• F0 = 0
• F1 = 1
• Fn = Fn-1 + Fn-2 for n > 1
• 0, 1, 1, 2, 3, 5, 8, 13, …
Fibonacci Numbersint fibonacci(int n){
if(n == 0 || n == 1)return n;
elsereturn fibonacci(n-1) + fibonacci(n-
2);}
Fibonacci Numbers: Computing F(5)
Fibonacci Numbers: Computing F(5)
F(5)
= active
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1) F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1) F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1) F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1) F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1) F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Fibonacci Numbersw/ Memoization
int fibonacci(int n){
static map<int, int> memo;
if(memo.find(n) != memo.end())return memo[n];
if(n == 0 || n == 1)memo[n] = n;
elsememo[n] = fibonacci(n-1) + fibonacci(n-
2);
return memo[n];}
Memoization
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Table Lookups
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
F(2)
F(0)F(1)
F(1)
F(0)F(1)
Table Lookups
Fibonacci Numbers:F(5) w/ memoization
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookups
Fibonacci Numbers: Computing F(5)
Fibonacci Numbers: Computing F(5)
F(5)
= active
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookup
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookup
Fibonacci Numbers: Computing F(5)
F(5)
F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookup
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookup
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookups
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookups
Fibonacci Numbers: Computing F(5)
F(5)
F(3)F(4)
F(2)F(3)
F(1)F(2)
F(0)F(1)
Table Lookups
Fibonacci Numbers: Computing F(5)
• Memoization eliminated 6 out of 15 recursive calls
Dynamic Programming• Can be done in one of two ways
– iteration (bottom-up)– recursion w/ memoization (top-down)
Fibonacci Numbers: Iteration
int fibonacci(int n){
int M[n + 1];
M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {
M[i] = M[i-1] + M[i-2];}
return M[n];}
Fibonacci Numbersw/ Memoization
int fibonacci(int n){
static map<int, int> memo;
if(memo.find(n) != memo.end())return memo[n];
if(n == 0 || n == 1)memo[n] = n;
elsememo[n] = fibonacci(n-1) + fibonacci(n-
2);
return memo[n];}
Fibonacci Numbers: Iteration
• What’s the runtime of the iterative algorithm?
Fibonacci Numbers: Iteration
int fibonacci(int n){
int M[n + 1];
M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {
M[i] = M[i-1] + M[i-2];}
return M[n];}
Fibonacci Numbers: Iteration
int fibonacci(int n){
int M[n + 1];
M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {
M[i] = M[i-1] + M[i-2];}
return M[n];}
Fibonacci Numbers: Iteration
• What’s the runtime of this code?for(int i = 2; i <= n; ++i) {
M[i] = M[i-1] + M[i-2];}
Fibonacci Numbers: Iteration
• What’s the runtime of this code?for(int i = 2; i <= n; ++i) {
M[i] = M[i-1] + M[i-2];}
• O(n)
Fibonacci Numbers: Iteration
int fibonacci(int n){
int M[n + 1];
M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {
M[i] = M[i-1] + M[i-2];}
return M[n];}
Text Segmentation• Given a string without spaces,
what is the best segmentation of the string?
Text Segmentation• Some languages are written
without spaces between the words– difficult for search engines to
decipher the meaning of a search– difficult to return relevant results
Text Segmentation:Recursive Solution
• Recursion:
Pmax(y) = maxi P0(y[0:i]) x Pmax(y[i:n]);
Pmax(“”) = 1;
where n = length(y)
Text Segmentation:Recursive Solution
Pmax(“theyouthevent”) =
max(P0(“t”) x Pmax(“heyouthevent”),
P0(“th”) x Pmax(“eyouthevent”),
…P0(“theyouthevent”) x Pmax(“”)
);
Text Segmentation:Recursive Solution
double get_Pmax(const string &s){
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) *
get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
return max;}
Text Segmentation:Recursive Solution
P0(“”) x Pmax(“”)
double get_Pmax(const string &s){
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) *
get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
return max;}
Text Segmentation:Recursive Solution
max(…)
P0(“”) x Pmax(“”)
double get_Pmax(const string &s){
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) *
get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
return max;}
Text Segmentation:Recursive Solution
max(…)
P0(“”) x Pmax(“”)
double get_Pmax(const string &s){
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) *
get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
return max;}
base case
double get_Pmax(const string &s){
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) *
get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
return max;}
Text Segmentation:Recursive Solution
max(…)
P0(“”) x Pmax(“”)
base case
Pmax(y) = maxi P0(y[0:i]) x Pmax(y[i:n])
Text Segmentation:Memoization
double get_Pmax(const string &s){
static map<string, double> probabilities;
if(probabilities.find(s) != probabilities.end())return probabilities[s];
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
probabilities[s] = max;return max;
}
Memoization
Text Segmentation:Memoization
double get_Pmax(const string &s){
static map<string, double> probabilities;
if(probabilities.find(s) != probabilities.end())return probabilities[s];
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
probabilities[s] = max;return max;
}
Memoization
P0(“”) x Pmax(“”)
Text Segmentation:Memoization
double get_Pmax(const string &s){
static map<string, double> probabilities;
if(probabilities.find(s) != probabilities.end())return probabilities[s];
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
probabilities[s] = max;return max;
}
Memoization
max(…)
P0(“”) x Pmax(“”)
Text Segmentation:Memoization
double get_Pmax(const string &s){
static map<string, double> probabilities;
if(probabilities.find(s) != probabilities.end())return probabilities[s];
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
probabilities[s] = max;return max;
}
Memoization
max(…)
P0(“”) x Pmax(“”)
base case
Text Segmentation:Memoization
double get_Pmax(const string &s){
static map<string, double> probabilities;
if(probabilities.find(s) != probabilities.end())return probabilities[s];
if(s.length() == 0)return get_P0(s);
double max = 0.0;for(int i = 1; i <= s.length(); ++i) {
double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));
if(temp > max)max = temp;
}
probabilities[s] = max;return max;
}
Memoization
Pmax(y) = maxi P0(y[0:i]) x Pmax(y[i:n])
max(…)
P0(“”) x Pmax(“”)
base case
Text Segmentation:Iteration
double get_Pmax(const string &s){
double probabilities[s.length() + 1];
probabilities[0] = 1.0;for(int i = 1; i <= s.length(); ++i) {
double max = 0.0;for(int j = 0; j < i; ++j) {
double temp =probabilities[j] * get_P0(s.substr(j,
s.length()-j));
if(temp > max)max = temp;
}probabilities[i] = max;
}
return probabilities[s.length() – 1];}
Text Segmentation:Iteration
• What’s the runtime of the iterative version of the algorithm?
Text Segmentation:Iteration
double get_Pmax(const string &s){
double probabilities[s.length() + 1];
probabilities[0] = 1.0;for(int i = 1; i <= s.length(); ++i) {
double max = 0.0;for(int j = 0; j < i; ++j) {
double temp =probabilities[j] * get_P0(s.substr(j,
s.length()-j));
if(temp > max)max = temp;
}probabilities[i] = max;
}
return probabilities[s.length() – 1];}
Text Segmentation:Iteration
• What’s the runtime of this code?
for(int i = 1; i <= s.length(); ++i) {double max = 0.0;for(int j = 0; j < i; ++j) {
double temp =probabilities[j] * get_P0(s.substr(j, s.length()-
j));
if(temp > max)max = temp;
}probabilities[i] = max;
}
Text Segmentation:Iteration
• What’s the runtime of this code?• O(n2)
for(int i = 1; i <= s.length(); ++i) {double max = 0.0;for(int j = 0; j < i; ++j) {
double temp =probabilities[j] * get_P0(s.substr(j,
s.length()-j));
if(temp > max)max = temp;
}probabilities[i] = max;
}
ACM ICPC• ACM - “Association for Computing
Machinery”
• ICPC - “International Collegiate Programming Contest”
ACM ICPC Results:World Finals 2008
ACM ICPC Results:World Finals 2008
ACM ICPC Results:Regionals 2008 (SE
U.S.A.)
ACM ICPC Results:Regionals 2008 (SE
U.S.A.)
ACM ICPC• Held once every year since 1977
– ACM Regionals are in the Fall (semester)
– ACM World Finals in the Spring
ACM ICPC• About 73 teams at this past year’s
Southeast USA Regional Contest (2008; feeds the 2009 World Finals)
• 100 teams at 2008 World Finals
ACM ICPC• Format
– Teams of 1-3 students (undergrads and 1st year grads)
– 1 computer per team– 5 hours– 10 problems
ACM ICPC• Format (cont.)
– Regionals – printed materials allowed (up to 12"x12"x2" in size)
– Worlds – no printed materials allowed
ACM ICPC• Eligibility decision tree:
http://cm2prod.baylor.edu/ICPCWiki/attach/StaticResources/eligibilitydecisiontree.pdf