umass lowell computer science 91.404 analysis of algorithms prof. karen daniels spring, 2001

UMass Lowell Computer Science 91.404

Analysis of Algorithms Prof. Karen Daniels

Spring, 2001

UMass Lowell Computer Science 91.404

Analysis of Algorithms Prof. Karen Daniels

Spring, 2001

Lectures 3/4Lectures 3/4Chapters 1 & 2Chapters 1 & 2

Wed. 2/7/01 and Thursday 2/8/01Wed. 2/7/01 and Thursday 2/8/01oror

““How to Make an Algorithm SandwichHow to Make an Algorithm Sandwich””adapted from the fall, 2000 talkadapted from the fall, 2000 talk

Finding the Largest Area Axis-Parallel Rectangle in a Polygon in Finding the Largest Area Axis-Parallel Rectangle in a Polygon in OO(n log(n log22 n) Time n) Timegiven at UMass Lowell MATHEMATICAL SCIENCES COLLOQUIUMgiven at UMass Lowell MATHEMATICAL SCIENCES COLLOQUIUM

Context: Taxonomy of ProblemsSupporting Apparel Manufacturing

Context: Taxonomy of ProblemsSupporting Apparel Manufacturing

Ordered Ordered ContainmentContainment

Geometric RestrictionGeometric Restriction

Distance-Based Distance-Based SubdivisionSubdivision

Maximum Maximum RectangleRectangle

Limited GapsLimited Gaps

Minimal Minimal EnclosureEnclosure

Column-Based LayoutColumn-Based LayoutTwo-Phase LayoutTwo-Phase Layout

Lattice Lattice PackingPacking

ContainmentContainment

Maximal CoverMaximal Cover

A Common (sub)Problem

Find a Good ApproximationFind a Good Approximation

Outer Outer InnerInner

Given a 2D polygon that:Given a 2D polygon that: does not intersect itselfdoes not intersect itself may have holesmay have holes has has nn vertices vertices

Find the Find the LLargest-Area Axis-Parallel argest-Area Axis-Parallel RRectangleectangle How “hard” is it?How “hard” is it? How “fast” can we find it?How “fast” can we find it?

What’s the Problem?

Explore the Explore the problem problem to gain intuition:to gain intuition: Describe it: What are the assumptions? Describe it: What are the assumptions? (model of computation, (model of computation,

etc...)etc...) Has it already been solved?Has it already been solved? Have similar problems been solved? (more on this later)Have similar problems been solved? (more on this later) What does best-case input look like?What does best-case input look like? What does worst-case input look like?What does worst-case input look like?

Establish worst-case Establish worst-case upper boundupper bound on the on the problemproblem using using an algorithman algorithm Design a (simple) algorithm and find an upper bound on its worst-case Design a (simple) algorithm and find an upper bound on its worst-case

asymptotic running time; this tells us problem asymptotic running time; this tells us problem can becan be solved in a certain solved in a certain amount of time. Algorithms taking more than this amount of time may amount of time. Algorithms taking more than this amount of time may exist, but won’t help us.exist, but won’t help us.

Establish worst-case Establish worst-case lower boundlower bound on the on the problemproblem Tighten each bound to form a worst-case “sandwich” Tighten each bound to form a worst-case “sandwich”

Approach

increasing worst-case asymptotic running time as a function of nincreasing worst-case asymptotic running time as a function of n

nn11 22nnnn22 nn33 nn44 nn55

Upper BoundUpper Bound

First AttemptFirst Attempt

Designing an Algorithmto Provide Upper BoundDesigning an Algorithmto Provide Upper Bound

Contacts reduce degrees of freedomContacts reduce degrees of freedom

First attempt uses a straightforward approach:First attempt uses a straightforward approach: ““brute-force”brute-force” ““naïve”naïve”

Will most likely produce a “loose” upper boundWill most likely produce a “loose” upper bound Characterize rectangle based on how it can “grow”Characterize rectangle based on how it can “grow”

O(n5) LR AlgorithmO(n5) LR Algorithm

Find_LR(Polygon P)Find_LR(Polygon P) area0 Find_LR_0_RC(P)area0 Find_LR_0_RC(P) area1 Find_LR_1_RC(P)area1 Find_LR_1_RC(P) area2 Find_LR_2_RC(P)area2 Find_LR_2_RC(P) area3 Find_LR_3_RC(P)area3 Find_LR_3_RC(P) area4 Find_LR_4_RC(P)area4 Find_LR_4_RC(P) return maximum(area0, area1, area2, area3, area4)return maximum(area0, area1, area2, area3, area4)

Find_LR_0_RC(P)Find_LR_0_RC(P)for i 1 to n [for each edge of P]for i 1 to n [for each edge of P]

for j 1 to nfor j 1 to n for k 1 to nfor k 1 to n for l 1 to nfor l 1 to n area area of LR for 0-RC determining set for (i,j,k,l)area area of LR for 0-RC determining set for (i,j,k,l)

if LR is empty, then update maximum areaif LR is empty, then update maximum area return maximum areareturn maximum areaO(n)O(n)

O(nO(n55))

this test has a O(n) loop down inside itthis test has a O(n) loop down inside it

First Upper Bound: What can we really conclude?First Upper Bound: What can we really conclude?

AlgorithmAlgorithm’s worst-case running time is in O(n’s worst-case running time is in O(n55).). ProblemProblem cancan be solved in O(n be solved in O(n55) time, even for worst-case ) time, even for worst-case

inputs.inputs. Note: there might exist algorithms for this problem that take more than nNote: there might exist algorithms for this problem that take more than n55

time, but they aren’t useful to us!time, but they aren’t useful to us!

If a worst-case input exists that causes If a worst-case input exists that causes algorithmalgorithm to to actually use time proportional to nactually use time proportional to n55, then , then algorithmalgorithm’s ’s worst-case running time is also in worst-case running time is also in (n(n55).). In this case,In this case, we can saywe can say algorithm algorithm’s worst-case running time is ’s worst-case running time is

in in (n(n55).).

nn11 nn22 22nnnn55


An inefficient algorithm for An inefficient algorithm for the problem might exist the problem might exist that takes this much time, that takes this much time, but would not help us.but would not help us.

Lower Bound Lower Bound


nn11 nn22 22nnnn55




First attempt will most likely produce a “loose” lower First attempt will most likely produce a “loose” lower bound that can be improved later.bound that can be improved later.

(1) and (1) and (n) are not hard to achieve:(n) are not hard to achieve: Rationale for Rationale for (n) is that we must examine (n) is that we must examine everyevery vertex at vertex at

least once in order to solve the problem. This holds for least once in order to solve the problem. This holds for everyevery algorithm that solves the problem, so it gives us a lower bound algorithm that solves the problem, so it gives us a lower bound on the problem in an even stronger sense than our upper on the problem in an even stronger sense than our upper bound. bound. Remember that our upper bound Remember that our upper bound does not guaranteedoes not guarantee that that nono algorithm algorithm

exists for this problem that takes more than O(nexists for this problem that takes more than O(n55) time, but if one ) time, but if one exists it will not help us! exists it will not help us!

nn11 22nnnn55

worst-case boundsworst-case bounds on problemon problem

An inefficient algorithm for An inefficient algorithm for the problem might exist the problem might exist that takes this much time, that takes this much time, but that would not help us.but that would not help us.

No algorithm for the problem No algorithm for the problem exists that can solve it for exists that can solve it for worst-case inputs in less worst-case inputs in less than linear time .than linear time .

Know the Difference!Know the Difference!

nn11 22nnnn55



No algorithm for the problem No algorithm for the problem exists that can solve it for exists that can solve it for worst-case inputs in less worst-case inputs in less than linear time .than linear time .

Strong BoundStrong Bound: This : This worst-case worst-case lowerlower bound bound on the problem holds for on the problem holds for everyevery algorithm that algorithm that solves the problem and solves the problem and abides by our problem’s abides by our problem’s assumptions.assumptions.

Weak BoundWeak Bound: This worst-case : This worst-case upperupper bound on the problem bound on the problem comes from just considering comes from just considering oneone algorithm. Other, less algorithm. Other, less efficient algorithms that solve efficient algorithms that solve this problem might exist, but this problem might exist, but we don’t care about them!we don’t care about them!

Both the upper and lower bounds Both the upper and lower bounds are probably are probably looseloose (i.e. probably (i.e. probably can be can be tightenedtightened later on). later on).

Upper BoundUpper Bound

Tightening ItTightening It

Think harder…Think harder… Design an algorithm that takes only O(nDesign an algorithm that takes only O(n22) time for ) time for

worst-case inputsworst-case inputs

Approach

nn11 22nnnn55


nn22

Now we no longer care Now we no longer care about this one!about this one!

tighter upper tighter upper boundbound

Think even harder…Think even harder… Run into brick wall!Run into brick wall! Characterize the brick wall: this type of caseCharacterize the brick wall: this type of case Go around brick wall by looking at similar problems.Go around brick wall by looking at similar problems. This requires ability to compare functions!This requires ability to compare functions!

Approach (continued)

nn11 22nnnn55


nn22

Some Related ProblemsSome Related Problems

nn11 n log(n) n log(n) n logn log22(n) (n) 22nnnn55n n (n) log(n)(n) log(n)n n (n)(n)This slide and the next are included just to show a variety of functions. Don’t This slide and the next are included just to show a variety of functions. Don’t worry about the details!worry about the details!

(n) is the (n) is the veryvery slowly growingslowly growing inverse of Ackermann’s function inverse of Ackermann’s function

Some Related Problems

(continued)Some Related Problems

(continued)

nn11 n log(n) n log(n) n logn log22(n) (n) 22nnnn55n n (n) log(n)(n) log(n)n n (n)(n)

Reduce the O(nReduce the O(n22) bound to O(n log) bound to O(n log22 n): n): Adapt a technique that worked for a similar problem in Adapt a technique that worked for a similar problem in

order to develop a general framework for the order to develop a general framework for the problematic caseproblematic case

Approach (continued)

nn11 22nnnn55


nn22n logn log22 n n

Lower BoundLower Bound

Tightening ItTightening It

ApproachApproach

Go around lower bound brick wall by:Go around lower bound brick wall by: examining examining strongstrong lower bounds for some similar lower bounds for some similar

problemsproblems transforming a similar problem to our problemtransforming a similar problem to our problem

[this process is similar to what we do when we prove problems NP-complete][this process is similar to what we do when we prove problems NP-complete]

nn11 22nnnn55


nn22n logn log22 n n

Lower Bounds for Related Problems Lower Bounds for Related Problems

nn11 n log(n) n log(n) n logn log22(n) (n) 22nnnn55

SmallestOuterRectangle: (n)

SmallestOuterCircle: (n)

LargestInnerRectangle: (n)

LargestInnerCircle: (n log n)

point set, polygonpoint set, polygon

point set, polygonpoint set, polygon

point setpoint set

polygonpolygon

LargestInnerRectangle: (n log2(n)) polygonpolygon

worst-case bounds on our problemworst-case bounds on our problem

Lower Bound of (n log n) by Transforming a (seemingly unrelated) ProblemLower Bound of (n log n) by Transforming a (seemingly unrelated) Problem

MAX-GAP MAX-GAP instanceinstance: given : given nn real numbers { x real numbers { x11, x, x22, ... x, ... xn n } find the } find the maximum difference between 2 consecutive numbers in the maximum difference between 2 consecutive numbers in the sortedsorted list. list.

O(n) time transformationO(n) time transformationspecialized specialized polygonpolygon

xx22xx44 xx33 xx11

Rectangle Rectangle area is a area is a solution to the solution to the MAX-GAP MAX-GAP instanceinstance

Rectangle algorithm must take as least as much time as MAX-GAP.Rectangle algorithm must take as least as much time as MAX-GAP.MAX-GAP is known to be in MAX-GAP is known to be in (n log n).(n log n).

Rectangle algorithm must take Rectangle algorithm must take (n log n) time(n log n) timefor specialized polygons. for specialized polygons.

[[Transforming yet another different problem yields bound for unspecialized polygons.]Transforming yet another different problem yields bound for unspecialized polygons.]

First attempt:First attempt: Establish weak (and loose) worst-case Establish weak (and loose) worst-case upper boundupper bound on the on the

problemproblem using an algorithm using an algorithm Establish strong (and loose) worst-case Establish strong (and loose) worst-case lower boundlower bound on the on the

problemproblem

Tighten bounds to make an “algorithm sandwich”Tighten bounds to make an “algorithm sandwich” Establish weak (but tighter) worst-case Establish weak (but tighter) worst-case O(n logO(n log22 n) n) upper upper

boundbound on the on the problemproblem using an algorithm using an algorithm Establish strong (and tighter) worst-case Establish strong (and tighter) worst-case lower boundlower bound on the on the

problemproblem by transforming a problem with known lower bound by transforming a problem with known lower bound

Summary

nn11 n log(n)n log(n) n logn log22(n)(n) 22nnnn55

bounds on problembounds on problem

For More InformationFor More Information

Computational Geometry:Computational Geometry: Graduate CS course in Computational Geometry Graduate CS course in Computational Geometry to be offered at UMass Lowell in Spring ‘01to be offered at UMass Lowell in Spring ‘01 Introductory texts:Introductory texts:

Computational Geometry in CComputational Geometry in C (O’Rourke)(O’Rourke) Computational Geometry: An IntroductionComputational Geometry: An Introduction (Preparata & Shamos)(Preparata & Shamos)

Bibliography: ftp://ftp.cs.usask.ca/pub/geometry/Bibliography: ftp://ftp.cs.usask.ca/pub/geometry/ Software:http://www.geom.umn.edu/software/cglist/Software:http://www.geom.umn.edu/software/cglist/

My research:My research: http://www.cs.uml.edu/~http://www.cs.uml.edu/~kdanielskdaniels Journal paperJournal paper: “Finding the largest area axis-parallel rectangle in a polygon”: “Finding the largest area axis-parallel rectangle in a polygon”

((Computational Geometry: Theory and ApplicationsComputational Geometry: Theory and Applications)) Prof. Victor Milenkovic: Frequent co-author and former PhD advisorProf. Victor Milenkovic: Frequent co-author and former PhD advisor

http://www.cs.miami.edu/~vjmhttp://www.cs.miami.edu/~vjm

Remainder of 2/8 LectureRemainder of 2/8 Lecture

Homework 1 Homework 1

Clarifications & HintsClarifications & Hints

HW1 Clarifications & HintsHW1 Clarifications & Hints

Printed copy is missing some text from Printed copy is missing some text from problem 6(d). It should read:problem 6(d). It should read:(d)(d) Can you conclude from your answers to (b) and Can you conclude from your answers to (b) and

(c) that (c) that MysteryMystery’s worst-case running time is = ’s worst-case running time is = ((f(n))?f(n))? Why or why not? Why or why not?

Web copy: view it in Page Layout mode so you see Web copy: view it in Page Layout mode so you see arrows in problem 6 pseudocode and other special arrows in problem 6 pseudocode and other special (graphical) symbols (like square root!)(graphical) symbols (like square root!)

HW1 Clarifications & Hints (continued)HW1 Clarifications & Hints (continued)

General Hint: General Hint: Remember the definitions for O, Remember the definitions for O, and and

f(n) is in the set of functions O(g(n)) if (and only if) 0 <= f(n) <= cg(n) for all f(n) is in the set of functions O(g(n)) if (and only if) 0 <= f(n) <= cg(n) for all n >= nn >= n0 0 where c and nwhere c and n00 are positive constants are positive constants

f(n) is in the set of functions f(n) is in the set of functions (g(n)) if (and only if) f(n) >= cg(n) >= 0 for all n (g(n)) if (and only if) f(n) >= cg(n) >= 0 for all n >= n>= n0 0 where c and nwhere c and n00 are positive constants are positive constants

f(n) is in the set of functions f(n) is in the set of functions (g(n)) if (and only if) f(n) is in O(g(n)) and also in (g(n)) if (and only if) f(n) is in O(g(n)) and also in (g(n)). This means: (g(n)). This means:

0 <= f(n) <= c0 <= f(n) <= c11g(n) for all n >= ng(n) for all n >= n0 0 where where cc11 and n and n00 are positive constants are positive constants andand f(n) >= cf(n) >= c22g(n) >= 0 for all n >= ng(n) >= 0 for all n >= n0 0 where where cc22 and n and n00 are positive constants are positive constants

Important notes:Important notes: c is a constant means that c cannot depend on n!c is a constant means that c cannot depend on n! In the In the definition, the constants c definition, the constants c11 and c and c22 can be different from each other. But, the upper and can be different from each other. But, the upper and

lower bounds must work for the lower bounds must work for the samesame n n00!! For each constant, For each constant, you only need to find 1 value that worksyou only need to find 1 value that works!!


Problems 1 & 2 & 3: Problems 1 & 2 & 3: The definitions for O and The definitions for O and use inequalities. use inequalities. This means that:This means that:

If f(n) is in O(g(n)), then f(n) is also in O(any function If f(n) is in O(g(n)), then f(n) is also in O(any function largerlarger than g(n)) than g(n)) If f(n) is in If f(n) is in (g(n)), then f(n) is also in (g(n)), then f(n) is also in (any function (any function smallersmaller than g(n)) than g(n))


Problem 1: Problem 1: Be aware of the difference between a statement about a bound Be aware of the difference between a statement about a bound

on the on the running time of an running time of an algorithmalgorithm vs. a statement about a vs. a statement about a bound on the bound on the running time of a running time of a problemproblem. Make sure you . Make sure you understand slides 5,9,11,12 of this lecture, which talk about this. understand slides 5,9,11,12 of this lecture, which talk about this.

You can assume that in Problem 1 the statements You can assume that in Problem 1 the statements ““Let Let RR be a be a problem. Suppose that the worst-case asymptotic time complexity of problem. Suppose that the worst-case asymptotic time complexity of RR is in is in O(nO(n22)) and also in and also in (n lg n)(n lg n)” ” mean that:mean that: The upper bound is a The upper bound is a weakweak bound that means someone has found an algorithm bound that means someone has found an algorithm

whose running time is quadratic in n ; this shows the problem can be solved in whose running time is quadratic in n ; this shows the problem can be solved in quadratic time. It quadratic time. It does not meandoes not mean that there does not exist a very inefficient that there does not exist a very inefficient algorithm for this problem that takes more than quadratic time.algorithm for this problem that takes more than quadratic time.

The lower bound is a The lower bound is a strongstrong bound that means someone has proven that there bound that means someone has proven that there cannot existcannot exist any algorithm for this problem that can handle worst-case inputs in any algorithm for this problem that can handle worst-case inputs in less than time proportional to nlgn.less than time proportional to nlgn.


Problem 1 (continued): Problem 1 (continued): Also, be sure you understand what is necessary Also, be sure you understand what is necessary

to conclude that an algorithm has a matching to conclude that an algorithm has a matching upper and lower bound so that the upper and lower bound so that the notation notation can be used (see the discussion on slide 9 and can be used (see the discussion on slide 9 and think back to our discussion in class on loops think back to our discussion in class on loops that cannot exit early…)that cannot exit early…)


Problem 3:Problem 3: There are 5 functions, so there are 5! possible orders. To find the right one without There are 5 functions, so there are 5! possible orders. To find the right one without

examining all possible orders, let transitivity help you (see slide 28). examining all possible orders, let transitivity help you (see slide 28). Here’s an example of how to do this for 3 functions: 2Here’s an example of how to do this for 3 functions: 2nn n n n n33

First pick one pair of the functions First pick one pair of the functions (we’ll pick 2(we’ll pick 2nn and n and n33, but any pair will do), but any pair will do) You need to figure out whether nYou need to figure out whether n3 3 is in O(2is in O(2nn) or n) or n3 3 is in is in (2(2nn) )

That is, you need to solve: That is, you need to solve: nn3 3 ? c2? c2nn for all n >= nfor all n >= n0 0 where where ?? is either >= or <= is either >= or <= In some cases it is easy to isolate c and pick a value for c that satisfies the “n” part of the In some cases it is easy to isolate c and pick a value for c that satisfies the “n” part of the

expression whenever n is greater than some value. This current case is a challenge because expression whenever n is greater than some value. This current case is a challenge because of the power of n. One approach is to use logarithms. Taking lg of both sides yields:of the power of n. One approach is to use logarithms. Taking lg of both sides yields:

3lgn ? lgc + n3lgn ? lgc + n 3lgn – n ? lgc3lgn – n ? lgc Now, think about the function 3lgn – n and how it behaves for small vs. large nNow, think about the function 3lgn – n and how it behaves for small vs. large n

For large n, n dominates and makes the function negative. When the function is negative, any For large n, n dominates and makes the function negative. When the function is negative, any value of c >= 1 works. When does it become (value of c >= 1 works. When does it become (and remainand remain) negative?) negative?

Trying n=2,4,8,16 is interesting because we see that 3lgn – n increases from n=2 to n=4 and then Trying n=2,4,8,16 is interesting because we see that 3lgn – n increases from n=2 to n=4 and then decreases. By the time n=16, it is negative and stays negative.decreases. By the time n=16, it is negative and stays negative.

If we pick c=1 and nIf we pick c=1 and n00=16, then 3lgn <= lgc + n so =16, then 3lgn <= lgc + n so nn3 3 <= c2<= c2nn and nand n3 3 is in O(2is in O(2nn).). This means that in the ordering of our 3 functions, nThis means that in the ordering of our 3 functions, n3 3 must come before 2must come before 2nn..


Problem 3 Problem 3 (continued):(continued): Now consider the remaining function: n. We need to know whether n Now consider the remaining function: n. We need to know whether n

belongs before nbelongs before n33, between n, between n33 and 2 and 2nn or after 2 or after 2nn.. Let’s compare n with nLet’s compare n with n33. If we can show that n goes before n. If we can show that n goes before n33, then, by , then, by

transitivity, we’ll have the ordering: n ntransitivity, we’ll have the ordering: n n3 3 22nn So, let’s try to prove that n is in O(nSo, let’s try to prove that n is in O(n33). To do this, we must find a value of ). To do this, we must find a value of

c and a value of nc and a value of n00 that work so that: n <= c n that work so that: n <= c n33

To isolate c, divide both sides by nTo isolate c, divide both sides by n33 (n is positive, so this does not change (n is positive, so this does not change the sense of the inequality). n/ nthe sense of the inequality). n/ n33 <= c n <= c n3 3 1/ n1/ n22 <= c <= c

Since 1/ nSince 1/ n22 is <= 1 for all n >=1, there are many possibilities for c when n is <= 1 for all n >=1, there are many possibilities for c when n00 = 1. Let’s pick c=1. Having found c=1 and n= 1. Let’s pick c=1. Having found c=1 and n00 = 1 that make n <= c n = 1 that make n <= c n33 we we can conclude that n is in O(ncan conclude that n is in O(n33). ).

Because n is in O(nBecause n is in O(n33) and n) and n33 is in O(2 is in O(2nn), the ordering is: n n), the ordering is: n n3 3 22nn Thus, gThus, g11 = n = n gg22 = n = n33 and g and g33 = 2 = 2nn


Problem 4:Problem 4: Your answer should be a value of Your answer should be a value of nn that is that is greater than 0greater than 0.. You might want to write a short program that finds the value for you.You might want to write a short program that finds the value for you.

Problem 5:Problem 5: To explain why your pseudocode is correct, briefly say why you’re sure To explain why your pseudocode is correct, briefly say why you’re sure

that:that: Your pseudocode terminatesYour pseudocode terminates

If recursive, your base case must correctly stop the recursionIf recursive, your base case must correctly stop the recursion If iterative, your loops must stop (no infinite loops)!If iterative, your loops must stop (no infinite loops)!

When your pseudocode terminates, it returns the right answerWhen your pseudocode terminates, it returns the right answer Your pseudocode has the proper running time: Your pseudocode has the proper running time: (lg n) worst-case time in this case. (lg n) worst-case time in this case.

Since you’re justifying upper Since you’re justifying upper andand lower bounds on the worst-case running time, be lower bounds on the worst-case running time, be sure you say why:sure you say why:

a worst-case input for this problem won’t cause your algorithm to take more than time a worst-case input for this problem won’t cause your algorithm to take more than time proportional to lgn proportional to lgn

there exists a worst-case input for this problem that actually causes your algorithm to there exists a worst-case input for this problem that actually causes your algorithm to take time proportional to lgn take time proportional to lgn


Problem 6:Problem 6: You can assume that each You can assume that each printprint statement in the statement in the

MysteryMystery pseudocode takes one unit of time = one pseudocode takes one unit of time = one step.step.

umass lowell computer science 91.404 analysis of algorithms prof. karen daniels spring, 2001

Documents