dynamic programming (continued)

Dynamic Dynamic Programming Programming (continued)(continued)

Dynamic Dynamic Programming Programming (continued)(continued)

Programming Puzzles and CompetitionsProgramming Puzzles and CompetitionsCIS 4900 / 5920CIS 4900 / 5920

Spring 2009Spring 2009

TCO 09 Algorithms (Feb.) Today

TCO 09 Algorithms (March)

Lecture Outline• Order Notation (for real this time)• Dynamic Programming w/ Iteration• Fibonacci numbers (revisited)• Text Segmentation (revisited)• ACM ICPC

Algorithmic Complexity• We would like to be able to

describe how long a program takes to run

Algorithmic Complexity• We would like to be able to

describe how long a program takes to run

• We should express the runtime in terms of the input size

Algorithmic Complexity• Absolute time measurements are

not helpful, as computers get faster all the time

Algorithmic Complexity• Absolute time measurements are

not helpful, as computers get faster all the time

• Also, runtime is clearly dependent on the input (and input size)

Order Notation• We define O(*) as follows

Order Notation• Basically, we just want to count

the number of operations (without being too precise)

Examples• What’s the complexity of the

following code?

int c = a + b;

Examples• What’s the complexity of the

following code?

int c = a + b;

• Answer: O(1)

Examples• What’s the complexity of the

following code?

int c = a + b;

• Answer: O(1)– (it actually depends on how you count)

Example 2for(int i = 0; i < n; ++i) {

//do something}

Example 2for(int i = 0; i < n; ++i) {

//do something}

• Answer: O(n)

Example 2for(int i = 0; i < n; ++i) {

//do something}

• Answer: O(n)– O(c) operations for each i O(cn)

== O(n) total operations

Example 2for(int i = 0; i < n; ++i) {

for(int j = 0; j < n; ++j) {//do something

}}

Example 2for(int i = 0; i < n; ++i) {

for(int j = 0; j < n; ++j) {//do something

}}

• Answer: O(n2)

Example 2for(int i = 0; i < n; ++i) {

for(int j = 0; j < n; ++j) {//do something

}}

• Answer: O(n2)– O(n) operations for each i == n * n total

operations == O(n2) total operations

Example 3for(int i = 0; i < n; ++i) {

for(int j = i; j < n; ++j) {//do something

}}

Example 3for(int i = 0; i < n; ++i) {

for(int j = i; j < n; ++j) {//do something

}}

• The first iteration of the outer loop takes O(n)

Example 3for(int i = 0; i < n; ++i) {

for(int j = i; j < n; ++j) {//do something

}}

• The second iteration of the outer loop takes O(n-1)

Example 3for(int i = 0; i < n; ++i) {

for(int j = i; j < n; ++j) {//do something

}}

• The third iteration of the outer loop takes O(n-2)…

Example 3for(int i = 0; i < n; ++i) {

for(int j = i; j < n; ++j) {//do something

}}

• The last iteration of the outer loop takes O(1)

Example 3• n + (n–1) + … + 1 = ?

Example 3• n + (n–1) + … + 1 = ?• Call this sum N

Example 3• n + (n–1) + … + 1 = N• Call this sum N

Example 3• n + (n–1) + … + 1 = N• Call this sum N

N = n + (n-1) + … + 2 + 1

Example 3• n + (n–1) + … + 1 = N• Call this sum N

N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n

Example 3• n + (n–1) + … + 1 = N• Call this sum N

N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n

• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)

Example 3• n + (n–1) + … + 1 = N• Call this sum N

N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n

• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)

= n * (n + 1)

Example 3• n + (n–1) + … + 1 = N• Call this sum N

N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n

• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)

= n * (n + 1)N = n * (n + 1) / 2

Example 3• n + (n–1) + … + 1 = N• Call this sum N

N = n + (n-1) + … + 2 + 1N = 1 + 2 + … + (n-1) + n

• Now, add these two together:2N = (n + 1) + ((n-1) + 2) + … + (1 + n)

= n * (n + 1)N = n * (n + 1) / 2 = O(n2)

Examples• This is also the number of ways 2 items

can be chosen from a set of (n + 1) items, disregarding order and without replacement (also called “(n + 1) choose 2”, binomial coefficient)

2

)1(

)!1(!2

)!1(

)!2)1((!2

)!1(2

1

nn

n

n

n

nn

Fibonacci Numbers• F0 = 0

• F1 = 1

• Fn = Fn-1 + Fn-2 for n > 1

• 0, 1, 1, 2, 3, 5, 8, 13, …

Fibonacci Numbersint fibonacci(int n){

if(n == 0 || n == 1)return n;

elsereturn fibonacci(n-1) + fibonacci(n-

2);}

Fibonacci Numbers: Computing F(5)

Fibonacci Numbers: Computing F(5)

F(5)

= active

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1) F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1) F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1) F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1) F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1) F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Fibonacci Numbersw/ Memoization

int fibonacci(int n){

static map<int, int> memo;

if(memo.find(n) != memo.end())return memo[n];

if(n == 0 || n == 1)memo[n] = n;

elsememo[n] = fibonacci(n-1) + fibonacci(n-

2);

return memo[n];}

Memoization

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Table Lookups

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

F(2)

F(0)F(1)

F(1)

F(0)F(1)

Table Lookups

Fibonacci Numbers:F(5) w/ memoization

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookups

Fibonacci Numbers: Computing F(5)

Fibonacci Numbers: Computing F(5)

F(5)

= active

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookup

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookup

Fibonacci Numbers: Computing F(5)

F(5)

F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookup

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookup

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookups

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookups

Fibonacci Numbers: Computing F(5)

F(5)

F(3)F(4)

F(2)F(3)

F(1)F(2)

F(0)F(1)

Table Lookups

Fibonacci Numbers: Computing F(5)

• Memoization eliminated 6 out of 15 recursive calls

Dynamic Programming• Can be done in one of two ways

– iteration (bottom-up)– recursion w/ memoization (top-down)

Fibonacci Numbers: Iteration

int fibonacci(int n){

int M[n + 1];

M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {

M[i] = M[i-1] + M[i-2];}

return M[n];}

Fibonacci Numbersw/ Memoization

int fibonacci(int n){

static map<int, int> memo;

if(memo.find(n) != memo.end())return memo[n];

if(n == 0 || n == 1)memo[n] = n;

elsememo[n] = fibonacci(n-1) + fibonacci(n-

2);

return memo[n];}

Fibonacci Numbers: Iteration

• What’s the runtime of the iterative algorithm?

Fibonacci Numbers: Iteration

int fibonacci(int n){

int M[n + 1];

M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {

M[i] = M[i-1] + M[i-2];}

return M[n];}

Fibonacci Numbers: Iteration

int fibonacci(int n){

int M[n + 1];

M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {

M[i] = M[i-1] + M[i-2];}

return M[n];}

Fibonacci Numbers: Iteration

• What’s the runtime of this code?for(int i = 2; i <= n; ++i) {

M[i] = M[i-1] + M[i-2];}

Fibonacci Numbers: Iteration

• What’s the runtime of this code?for(int i = 2; i <= n; ++i) {

M[i] = M[i-1] + M[i-2];}

• O(n)

Fibonacci Numbers: Iteration

int fibonacci(int n){

int M[n + 1];

M[0] = 0;M[1] = 1;for(int i = 2; i <= n; ++i) {

M[i] = M[i-1] + M[i-2];}

return M[n];}

Text Segmentation• Given a string without spaces,

what is the best segmentation of the string?

Text Segmentation• Some languages are written

without spaces between the words– difficult for search engines to

decipher the meaning of a search– difficult to return relevant results

Text Segmentation:Recursive Solution

• Recursion:

Pmax(y) = maxi P0(y[0:i]) x Pmax(y[i:n]);

Pmax(“”) = 1;

where n = length(y)

Text Segmentation:Recursive Solution

Pmax(“theyouthevent”) =

max(P0(“t”) x Pmax(“heyouthevent”),

P0(“th”) x Pmax(“eyouthevent”),

…P0(“theyouthevent”) x Pmax(“”)

);

Text Segmentation:Recursive Solution

double get_Pmax(const string &s){

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) *

get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

return max;}

Text Segmentation:Recursive Solution

P0(“”) x Pmax(“”)

double get_Pmax(const string &s){

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) *

get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

return max;}

Text Segmentation:Recursive Solution

max(…)

P0(“”) x Pmax(“”)

double get_Pmax(const string &s){

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) *

get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

return max;}

Text Segmentation:Recursive Solution

max(…)

P0(“”) x Pmax(“”)

double get_Pmax(const string &s){

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) *

get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

return max;}

base case

double get_Pmax(const string &s){

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) *

get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

return max;}

Text Segmentation:Recursive Solution

max(…)

P0(“”) x Pmax(“”)

base case

Pmax(y) = maxi P0(y[0:i]) x Pmax(y[i:n])

Text Segmentation:Memoization

double get_Pmax(const string &s){

static map<string, double> probabilities;

if(probabilities.find(s) != probabilities.end())return probabilities[s];

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

probabilities[s] = max;return max;

}

Memoization

Text Segmentation:Memoization

double get_Pmax(const string &s){

static map<string, double> probabilities;

if(probabilities.find(s) != probabilities.end())return probabilities[s];

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

probabilities[s] = max;return max;

}

Memoization

P0(“”) x Pmax(“”)

Text Segmentation:Memoization

double get_Pmax(const string &s){

static map<string, double> probabilities;

if(probabilities.find(s) != probabilities.end())return probabilities[s];

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

probabilities[s] = max;return max;

}

Memoization

max(…)

P0(“”) x Pmax(“”)

Text Segmentation:Memoization

double get_Pmax(const string &s){

static map<string, double> probabilities;

if(probabilities.find(s) != probabilities.end())return probabilities[s];

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

probabilities[s] = max;return max;

}

Memoization

max(…)

P0(“”) x Pmax(“”)

base case

Text Segmentation:Memoization

double get_Pmax(const string &s){

static map<string, double> probabilities;

if(probabilities.find(s) != probabilities.end())return probabilities[s];

if(s.length() == 0)return get_P0(s);

double max = 0.0;for(int i = 1; i <= s.length(); ++i) {

double temp =get_P0(s.substr(0, i - 0)) * get_Pmax(s.substr(i));

if(temp > max)max = temp;

}

probabilities[s] = max;return max;

}

Memoization

Pmax(y) = maxi P0(y[0:i]) x Pmax(y[i:n])

max(…)

P0(“”) x Pmax(“”)

base case

Text Segmentation:Iteration

double get_Pmax(const string &s){

double probabilities[s.length() + 1];

probabilities[0] = 1.0;for(int i = 1; i <= s.length(); ++i) {

double max = 0.0;for(int j = 0; j < i; ++j) {

double temp =probabilities[j] * get_P0(s.substr(j,

s.length()-j));

if(temp > max)max = temp;

}probabilities[i] = max;

}

return probabilities[s.length() – 1];}

Text Segmentation:Iteration

• What’s the runtime of the iterative version of the algorithm?

Text Segmentation:Iteration

double get_Pmax(const string &s){

double probabilities[s.length() + 1];

probabilities[0] = 1.0;for(int i = 1; i <= s.length(); ++i) {

double max = 0.0;for(int j = 0; j < i; ++j) {

double temp =probabilities[j] * get_P0(s.substr(j,

s.length()-j));

if(temp > max)max = temp;

}probabilities[i] = max;

}

return probabilities[s.length() – 1];}

Text Segmentation:Iteration

• What’s the runtime of this code?

for(int i = 1; i <= s.length(); ++i) {double max = 0.0;for(int j = 0; j < i; ++j) {

double temp =probabilities[j] * get_P0(s.substr(j, s.length()-

j));

if(temp > max)max = temp;

}probabilities[i] = max;

}

Text Segmentation:Iteration

• What’s the runtime of this code?• O(n2)

for(int i = 1; i <= s.length(); ++i) {double max = 0.0;for(int j = 0; j < i; ++j) {

double temp =probabilities[j] * get_P0(s.substr(j,

s.length()-j));

if(temp > max)max = temp;

}probabilities[i] = max;

}

ACM ICPC• ACM - “Association for Computing

Machinery”

• ICPC - “International Collegiate Programming Contest”

ACM ICPC Results:World Finals 2008

ACM ICPC Results:World Finals 2008

ACM ICPC Results:Regionals 2008 (SE

U.S.A.)

ACM ICPC Results:Regionals 2008 (SE

U.S.A.)

ACM ICPC• Held once every year since 1977

– ACM Regionals are in the Fall (semester)

– ACM World Finals in the Spring

ACM ICPC• About 73 teams at this past year’s

Southeast USA Regional Contest (2008; feeds the 2009 World Finals)

• 100 teams at 2008 World Finals

ACM ICPC• Format

– Teams of 1-3 students (undergrads and 1st year grads)

– 1 computer per team– 5 hours– 10 problems

ACM ICPC• Format (cont.)

– Regionals – printed materials allowed (up to 12"x12"x2" in size)

– Worlds – no printed materials allowed

ACM ICPC• Eligibility decision tree:

http://cm2prod.baylor.edu/ICPCWiki/attach/StaticResources/eligibilitydecisiontree.pdf