Chapter 3 - Asymptotic Notations Search C455

Document last modified: 09/16/2013 08:51:5401/16/2013 23:54:10

Definition

Asymptotic

a line that continually approaches a given curve but does not meet it at any finite distance.

Example

x is asymptotic with x + 1

limit - f(x) = k

Roughly translated might read as:

x approaches ∞, f(x) approaches k

for x close to ∞, f(x) is close to k

Two limits often used in analysis are:

f(x) = 1/x = 0

f(x) = cx = ∞ for c>0

asymptotic - A way to describe the behavior of functions in the limit or without bounds.

Let f(x) and g(x) be functions from the set of real numbers to the set of real numbers.

We say that f and g are asymptotic and write f(x) ≈ g(x) if

f(x) / g(x) = c (constant)

Examples

x/ 2x = ½

½x / 2x = 1/4

x2/2x2 = ½

½x2 / 2x2 = 1/4

x3/2x3 = ½

½x3 / 2x3 = 1/4

Question 3.1 - What are:

(1/x + 3)

2x/10x

2x/10x2

lg x/x

lg 4294967296 = lg 232 = 32

O Big-O Notation (Omicron)

possibly asymptotically tight upper bound for f(n) - Cannot do worse, can do better

n is the problem size.

f(n) ∈ O(g(n)) where:

O(g(n)) = { h(n): ∃ positive constants c, n0 such that 0 ≤ h(n) ≤ cg(n), ∀ n ≥ n0}

Meaning for all values of n ≥ n0 h(n) is on or below g(n).

O(g(n)) is a set of all the functions h(n) that are less than or equal to cg(n), ∀ n ≥ n0.

If f(n) ≤ cg(n), c > 0, ∀ n ≥ n0

then f(n) ∈ O(g(n))

meaning f(n) is one of the h(n)'s in the set

and g(n) is an asymptotically tight upper bound for f(n).

We usually write: f(n) = O(g(n))

Expressed as limits:

O f(n) / g(n) = c for some 0 ≤ c < ∞

n3 / n2 = n = ∞ n3 ≠ O(n2)

n2+n / n2 = 1 + 1/n = 1 n2+n = O(n2)

Example - functions in O(n2)

n n/1000 n1.9 n2 n2+n 1000n2+50n

Example - functions not in O(n2)

n3 n2.1 2n

Question 3.2 - Show using limits.

a. 1000n2+50n = O(n2)

b. n3/1000 ≠ O(n2)

Question 3.3 - Which functions are in O(n2)?

n1/2 n2 n2+n2+n2 n3 n2.1

If f(n) ≤ cg(n), c > 0, ∀ n ≥ n0 then f(n) ∈ O(g(n))

n2 bounded above by n

n2 bounded below by n

Example Show 2n2 = O(n3)

0 ≤ h(n) ≤ cg(n) Definition of O(g(n))

0 ≤ 2n2 ≤ cn3 Substitute

0/n3 ≤ 2n2/n3 ≤ cn3/n3 Divide by n3

Determine c

0 ≤ 2/n ≤ c 2/n = 0

2/n maximum when n=1

0 ≤ 2/1 ≤ c = 2 Satisfied by c=2

Determine n0

0 ≤ 2/n0 ≤ 2

0 ≤ 2/2 ≤ n0

0 ≤ 1 ≤ n0 = 1 and n0=1

0 ≤ 2n2 ≤ 2n3 ∀ n ≥ n0=1

Example - 1000n2 + 50n = O(n2)

0 ≤ h(n) ≤ cg(n)

0 ≤ 1000n2 + 50n ≤ cn2

0/n2 ≤ 1000n2/n2 + 50n/n2 ≤ cn2/n2 Divide by n2

0 ≤ 1000 + 50/n ≤ c Note that 50/n → 0 as n → ∞

Greatest when n = 1

0 ≤ 1000 + 50/1 = 1050 ≤ c = 1050 With c=1050

0 ≤ 1000 + 50/n0 ≤ 1050 Find n0

-1000 ≤ 50/n0 ≤ 50

-20 ≤ 1/n0 ≤ 1

-20n0 ≤ 1 ≤ n0 = 1 n0=1

0 ≤ 1000n2 + 50n ≤ 1050n2 ∀ n ≥ n0=1, c=1050

If f(n) ≤ cg(n), c > 0, ∀ n ≥ n0 then f(n) ∈ O(g(n))

Question 3.4.1 - Show that 2n2+n is in O(n2) by finding c and n0

Example - n2 + 50n = O(n2)

0 ≤ h(n) ≤ cg(n)

0 ≤ n2 + 50n ≤ cn2

0/n2 ≤ n2/n2 + 50n/n2 ≤ cn2/n2 Divide by n2

0 ≤ 1 + 50/n ≤ c Note that 50/n → 0 as n → ∞

Pick n = 50

0 ≤ 1 + 50/50 = 2 ≤ c = 2 With c=2

0 ≤ 1 + 50/n0 ≤ 2 Find n0

-1 ≤ 50/n0 ≤ 1

-20n0 ≤ 50 ≤ n0 = 50 n0=50

1000n2 + 50n = O(n2)

with c=1050 and n0=1

Graph illustrates that:

0 ≤ 1000n2 + 50n ≤ 1050n

Note that the y-scale is much greater than the x-scale.

n2 + 50n = O(n2)

with c=2 and n0=50

0 ≤ n2 + 50n ≤ 2n2 ∀ n ≥ n0=50, c=2

Example - n = O(n lg n)

0 ≤ h(n) ≤ cg(n)

0 ≤ n ≤ cn lg n

0/n lg n ≤ n/n lg n ≤ cn lg n/n lg n Divide by n lg n

0 ≤ 1/lg n ≤ c Note that 1/lg n → 0 as n → ∞

Greatest when n = 2

0 ≤ 1/lg 21 = 1 ≤ c = 1 With c=1

0 ≤ 1/lg n0 ≤ c = 1 Find n0

0 ≤ 1/lg n0 ≤ 1

0 ≤ 1 ≤ lg n0 = 1 lg n0 = 1

when

n

0=21=2

0 ≤ n ≤ cn lg n ∀ n ≥ n0=2, c=1

Example - n2+n2+n2 = 3n2 = O(n3)

0 ≤ h(n) ≤ cg(n)

0 ≤ 3n2 ≤ cn3 Guessing c and n0

0 ≤ 3*22 = 12 ≤ 1*23 = 8 with c=1 and n0=2 fails to hold

0 ≤ 3*22 = 12 ≤ 2*23 = 16 with c=2 and n0=2

0 ≤ 3*32 = 27 ≤ 1*33 = 81 with c=1 and n0=3

Determining c and n0

0 ≤ h(n) ≤ cg(n)

0 ≤ 3n2 ≤ cn3

0/n3 ≤ 3n2/n3 ≤ cn3/n3

0 ≤ 3/n ≤ c Find c=3 when n=1

because 3/n → 0 as n → ∞, maximum when n=1

3/n only grows smaller as n → ∞

0 ≤ 3/n0 ≤ 3 Find n0 with c=3

0 ≤ 1/n0 ≤ 1

0 ≤ 1 ≤ n0 = 1 n0 = 1

0 ≤ 3n2 ≤ 3n3 ∀ n ≥ n0=1

If f(n) ≤ cg(n), c > 0, ∀ n ≥ n0 then f(n) ∈ O(g(n))

Question 3.4.2 - Show that lg n is in O(n) by finding c and n0

Question 3.4.3 - Verify that for a>0, b>0 any linear function an+b = O(n2) by finding c and n0.

Example - n3 ≠ O(n2)

0 ≤ h(n) ≤ cg(n)

0 ≤ n3 ≤ cn2

0/n2 ≤ n3/n2 ≤ cn2/n2

n = O(n lg n)

with c=1 and n0=2

3n2 = O(n3)

with c=1 and n0=2 fails

with c=1 and n0=3 holds

0 ≤ n ≤ c because n → ∞, no c exists where ∀ n ≥ n0 true

Example - 5n3+10n ≠ O(n2)

0 ≤ h(n) ≤ cg(n)

5n3+10n ≤ cn2

5n3/n2 + 10n/n2 ≤ cn2/n2

5n + 10/n ≤ c

(5n2 + 10)/n ≤ c because (5n2 + 10)/n → ∞, no c exists where ∀ n ≥ n0 true

Question 3.5 - Show 2n3+n ≠ O(n2)

Ω Omega Notation

possibly asymptotically tight lower bound for f(n) - Cannot do better, can do worse

f(n) ∈ Ω(g(n)) where:

Ω(g(n)) = {h(n): ∃ positive constants c > 0, n0 such that 0 ≤ cg(n) ≤ h(n), ∀ n ≥ n0}

Meaning for all values of n ≥ n0 h(n) is on or above g(n).

Ω(g(n)) is a set of all the functions h(n) that are greater than or equal cg(n), ∀ n ≥ n0.

If cg(n) ≤ f(n), c > 0 and ∀ n ≥ n0, then f(n) ∈ Ω(g(n))

meaning f(n) is one of the h(n)'s in the set

and g(n) is an asymptotically tight lower bound for f(n).

Expressed as limits:

Ω f(n) / g(n) = c for some 0 < c ≤ ∞

n3 / n2 = n = ∞ n3 = Ω(n2)

n / n2 = 1/n = 0 n ≠ Ω(n2)

Example - functions in Ω(n)

n n/1000 n1.999 n2 n2+n 1000n2+50n

Example - functions not in Ω(n2)

n n1.999 v n

Question 3.6.1 - Show using limits:

a. 1/n ≠ Ω(n2)

b. n2/1000 = Ω(n2)

Example - n3 = Ω(n2) with c=1 and n0=1

0 ≤ cg(n) ≤ h(n)

0 ≤ 1*12 = 1 ≤ 1 = 13

0 ≤ cg(n) ≤ h(n)

0 ≤ cn2 ≤ n3

0/n2 ≤ cn2/n2 ≤ n3/n2

0 ≤ c ≤ n

0 ≤ 1 ≤ 1 with c=1 and n0=1 since n increases.

n = ∞

Question 3.6.2 - Can we use n0=0?

Ω(g(n)) = {h(n): ∃ positive constants c > 0, n0 such that 0 ≤ cg(n) ≤ h(n), ∀ n ≥ n0}

Example - 3n2 + n = Ω(n2)

0 ≤ cg(n) ≤ h(n)

0 ≤ cn2 ≤ 3n2 + n

0/n2 ≤ cn2/n2 ≤ 3n2/n2 + n/n2

0 ≤ c ≤ 3 + 1/n 3+1/n = 3

0 ≤ c ≤ 3 c = 3

0 ≤ 3 ≤ 3 + 1/n0

-3 ≤ 3-3 ≤ 3-3 + 1/n0

-3 ≤ 0 ≤ 1/n0

n0=1 satisfies

1/n = 0

Question 3.6.3 - From the two graphs at right, is

c=3 and n0=1 or

c=1 and n0=1

the better choice for 3n2 + n = Ω(n2)?

Why?

Example - v n = Ω(lg n)

n3 = Ω(n2)

with c=1 and n0=1

3n2 + n = Ω(n2) with c=1 and n

3n2 + n = Ω(n2) with c=3 and n

0 ≤ cg(n) ≤ h(n)

0 ≤ c lg n ≤ v n

0 ≤ c lg 16 ≤ v 16 n0 = 16

0 ≤ c 4 ≤ 4 lg 24 = 4 = v 16

0 ≤ c ≤ 1 c = 1

Question 3.7 - How does the graph indicate that this is true for n ≥ n0 = 16?

v n / lg n = ? See graph at right.

If cg(n) ≤ f(n), c > 0, ∀ n ≥ n0, then f(n) ∈ Ω(g(n))

Ω f(n) / g(n) = c for some 0 < c ≤ ∞

Question 3.8 - Which functions are in Ω(n2)?

n1/2 n2 n2+n2+n2 n! n3 n2.1 n1.999

Question 3.9.1 - Show that 2n2+n is in Ω(n2)

What are c and n0?

Example - n ≠ Ω(n2)

0 ≤ cg(n) ≤ h(n)

0 ≤ cn2 ≤ n

0/n2 ≤ cn2/n2 ≤ n/n2

0 ≤ c ≤ 1/n

c depends on n and there is no n0 that satisfies as n increases.

Only c = 0 would satisfy but not allowed by definition.

1 / n = 0

Example - 3n + 5 ≠ Ω(n2)

0 ≤ cg(n) ≤ h(n)

0 ≤ cn2 ≤ 3n + 5

0/n2 ≤ cn2/n2 ≤ 3n/n2 + 5/n2

0 ≤ c ≤ 3/n + 5/n2

c depends on n and there is no n0 that satisfies as n increases.

Only c = 0 would satisfy but not allowed by definition.

3/n + 5/n2 = 0

Question 3.9.2 - Show that 2n2+n ≠ Ω(n3)

If cg(n) ≤ f(n), c > 0, ∀ n ≥ n0, then f(n) ∈ Ω(g(n))

v n = Ω(lg n) with c=1 and n

Θ Theta Notation

asymptotically tight bound for f(n)

f(n) ∈ Θ(g(n)) where

Θ(g(n)) =

{h(n): ∃ positive constants c1, c2, n0 such that

0 ≤ c1g(n) ≤ h(n) ≤ c2g(n), ∀ n ≥ n0}

Positive means greater than 0.

Θ(g(n)) is a set of all the functions h(n) that are between c1g(n) and c2g(n), ∀ n ≥ n0.

If f(n) is between c1g(n) and c2g(n), ∀ n ≥ n0, then f(n) ∈ Θ(g(n)),

meaning f(n) is one of the h(n)'s in the set,

and g(n) is an asymptotically tight bound for f(n).

Expressed as limits:

Θ f(n) / g(n) = c for some 0 < c < ∞

n2+n / n2 = 1 + 1/n = 1

Example - n2/2-2n = Θ(n2)

c1g(n) ≤ h(n) ≤ c2g(n)

c1n2 ≤ n2/2-2n ≤ c2n

2

c1n2/n2 ≤ n2/2n2-2n/n2 ≤ c2n

2/n2 Divide by n2

c1 ≤ 1/2-2/n ≤ c2

O: Determine c2 = ½

½-2/n ≤ c2 = ½ ½-2/n = ½

maximum of ½-2/n

Ω: Determine c1 = 1/10

0 < c1 ≤ 1/2-2/n 0 < c1 minimum when n=5

0 < c1 ≤ 1/2-2/5

0 < c1 ≤ 5/10-4/10 = 1/10

n0 : Determine n0 = 5

1/10 ≤ 1/2-2/n0 ≤ 1/2

1/10 ≤ 1/2-2/n0 ≤ 1/2

2/n0 ≤ 1/2-1/10 ≤ 1/2

2/n0 ≤ 4/10 ≤ 1/2

2/n0 ≤ 2/5 ≤ 1/2

n0 ≥ 5 n0 = 5 satisfies

Verify

c1n2 ≤ n2/2-2n ≤ c2n

2 with c1=1/10, c2=½ and n0=5

1/10*52 ≤ 52/2-2*5 ≤ ½*52

25/10 ≤ 25/2-20/2 ≤ 25/2

5/2 ≤ 5/2 ≤ 25/2 Holds

In general: 1/10n2 ≤ n2/2-2n ≤ ½n2 for n ≥ 5

Question 3.10 - Does the following hold for ∀n ≥ n0?

c1n2 ≤ n2/2-2n ≤ c2n

2 with c1=1/2, c2=1/2 and n0=5

Example - Show that ½n2 - 3n ∈ Θ(n2)

Determine ∃ c1, c2, n0 positive constants such that:

n2/2-2n = Θ(n2) c1=1/10, c2=1/2 and n

c1n2 ≤ ½n2 - 3n ≤ c2n

2

c1 ≤ ½ - 3/n ≤ c2 Divide by n2

O: Determine c2 = ½

½ - 3/n ≤ c2

as n → ∞, 3/n → 0 ½ - 3/n = ½

therefore ½ ≤ c2 or c2 = ½ ½ - 3/n maximum for as n → ∞

Ω: Determine c1 = 1/14

0 < c1 ≤ ½ - 3/n ½ - 3/n > 0 minimum for n0 = 7

0 < c1 = ½ - 3/7 = 7/14 - 6/14 = 1/14

Determine n0 = 7

c1 ≤ ½ - 3/n0

1/14 ≤ ½ - 3/n0

3/n0 ≤ ½ - 1/14 = 6/14

1/n0 ≤ 2/14

7 = 14/2 ≤ n0

n0 ≥ 7

½n2 - 3n ∈ Θ(n2) when c1 = 1/14, c2 = ½ and n0 = 7

Example - Recall we found a closed-end equation for the InsertionSort of T(n) = an2 + bn + c.

a, b and c are > 0.

How do we know that? These constants are based on instruction execution counts.

Show: T(n) = an2 + bn + c = Θ(n2)

c1g(n) ≤ h(n) ≤ c2g(n)

c1n2 ≤ an2 + bn + c ≤ c2n

2

c1n2/n2 ≤ an2/n2 + bn/n2 + c/n2 ≤ c2n

2/n2

c1 ≤ a + b/n + c/n2 ≤ c2

as n → ∞, b/n, c/n2 → 0

a + b/n + c/n2 greatest when n0 = 1

c1 ≤ a + b + c when c1 = a

a + b + c ≤ c2 when c2 = a + b + c

Question 3.12 - (Levitin page 56)

Show that ½(n2 - n) ∈ Θ(n2)

½n2-3n = Θ(n2) c1=1/14, c2=1/2 and n

Theorem 3.1 page 46

For any two functions, f(n) and g(n):

f(n) = Θ(g(n)) ↔ f(n) = O(g(n)) and f(n) = Ω(g(n))

You might recall that p ↔ q is read as "p if and only if q" and called an equivalence that is true when p=q.

p q p ↔ q

T T T

T F F

F T F

F F T

Question 3.13 - Does n2=Θ(n2) imply n2=O(n2) and n2=Ω(n2)?

Qualified Notation

Ω Omega Notation

best case Ω - describes a lower bound for all input (it can't get any better than this).

Example: the array is already correctly sorted.

worst case Ω - describes a lower bound for worst case input, possibly greater than best case.

Example: the array is sorted in reverse order.

just Ω - same as best case Ω

v n = Ω(lg n)

with c=1 and n0=16

O Big-O Notation

best case O - describes an upper bound for best case input, possibly lower than worst case.

Example: the array is already correctly sorted.

worst case O - describes an upper bound for all input (it can't get any worse than this).

Example: the array is sorted in reverse order.

just O - same as worst case O

n = O(n lg n)

with c=1 and n0=2

Θ Theta Notation

best case Θ - not used

worst case Θ - describes asymptotic bounds for worst case input

n2/2-3n = Θ(n2) c1=1/14, c2=1/2 and n0=7

just Θ - same as worst case Θ

Asymptotic - a line that continually approaches a given curve but does not meet it at any finite distance.

Little-o Notation (omicron)

non-asymptotically tight upper bound for f(n)

f(n) ∈ o(g(n)) where

o(g(n)) = {h(n): for any constant c > 0, ∃ a constant n0 > 0, such that: 0 ≤ h(n) < cg(n), ∀ n ≥ n0}

f(n) / g(n) = 0

for any 0 < c < ∞

O(g(n)) = {h(n): for some constant c > 0, ∃ a constant n0 > 0, such that: 0 ≤ h(n) ≤ cg(n), ∀ n ≥ n0}

f(n) / g(n) = c

for some 0 ≤ c < ∞

O is a possibly asymptotically tight upper bound for f(n)

2n2 = O(n2) is asymptotically tight 2n2 / n2 = 2

2n = o(n2) is non-asymptotically tight 2n / n2 = 0

Question 3.14 - What is implied by f(n) / g(n) = 0?

Examples o(n2)

n1.999

n2/lg n n2/lg n/n2 = 1/lg n = 0

2n 2n / n2 = 0

Examples not o(n2)

2n2 ≠ o(n2) 2n2 / n2 = 2

n2 ≠ o(n2) (just as 2 is not < 2)

n = o(n2)

c = 1/10 and c = 1/20

n2/1000 ≠ o(n2)

Since in 0 ≤ f(n) < cg(n), c is any constant > 0

0 ≤ n2/1000< cn2

0 ≤ 1/1000 < c contradicted by picking any c < 1/1000

Can also show by:

n2/1000 / n2 = 1/1000 = 1/1000 not 0

o(g(n)) = {h(n): for any constant c > 0, ∃ a constant n0 > 0, such that:

0 ≤ h(n) < cg(n), ∀ n ≥ n0}

Question 3.15 - Using limits show:

n2/lg n = o(n2) f(n) /

g(n) = 0

n2 ≠ o(n2)

Question 3.16 - Classify n2.001 and n1.999 as o(n2) or not,from the graph at right.

f(n) ∈ o(g(n)) where

o(g(n)) = {h(n): for any constant c > 0, ∃ a constant n0 > 0, such that: 0 ≤ h(n) < cg(n), ∀ n ≥ n0}

f(n) / g(n) = 0 for any 0 < c < ∞

Little-ω Notation (omega)

non-asymptotically tight lower bound for f(n)

f(n) ∈ ω(g(n)) where

ω(g(n)) = {h(n): for any constant c > 0, ∃ a constant n0 > 0, such that 0 ≤ cg(n) < h(n), ∀ n ≥ n0}

f(n) / g(n) = ∞

for any 0 < c ≤ ∞

Ω(g(n)) = {h(n): for some constant c > 0, ∃ a constant n0 > 0, such that 0 ≤ cg(n) ≤ h(n), ∀ n ≥ n0}

f(n) / g(n) = c

for some 0 < c ≤ ∞

Ω possibly asymptotically tight lower bound

ω non-asymptotically tight lower bound

Examples

n2 = ω(n) n2/n = ∞

n2.001 = ω(n2) n2.001/n2 = ∞

n2 lg n = ω(n2) n2 lg n/n2 = lg n = ∞

n2 = ω(n)

n2 ≠ ω(n2) (just as 2 is not > 2) n2/n2 = 1

n2/1000 ≠ ω(n2)

Since in 0 ≤ cg(n) < f(n) , c is any constant > 0

0 ≤ cn2 < n2/1000

0 ≤ c < 1/1000 contradicted by picking any c ≥ 1/1000

Can also show by:

n2/1000 / n2 = 1/1000 = 1/1000 not ∞

c = 20 and c = 30

Question 3.17

a. What is implied by f(n) / g(n) = ∞?

b. Show n2/lg n ≠ ω(n2)

c. What is the n1.9999/n2?

d. Is n1.999 = ω(n2)?

f(n) ∈ ω(g(n)) where

ω(g(n)) = {h(n): for any constant c > 0, ∃ a constant n0 > 0, such that 0 ≤ cg(n) < h(n), ∀ n ≥ n0}

f(n) / g(n) = ∞

Asymptotic notation in equations and inequalities

Have been using notation:

n = O(n2) → n ∈ O(n2)

3n + 1 = Θ(n) → 3n + 1 ∈ Θ(n)

where

2n2 + 3n + 1 = 2n2+ Θ(n) means 2n2 + f(n)

for f(n) ∈ Θ(n)

in this case: f(n) = 3n + 1 ∈ Θ(n)

In equations

2n2 + 3n + 1 = 2n2+ Θ(n) = Θ(n2)

2n2 + Θ(n) = Θ(n2)

finer detail coarser detail

Comparison of Functions

An imprecise analogy between the asymptotic comparison of two function f and g and the relation between their values:

f(n) = O(g(n)) ≈ f(n) ≤ g(n)

f(n) = Ω(g(n)) ≈ f(n) ≥ g(n)

f(n) = Θ(g(n)) ≈ f(n) = g(n)

f(n) = o(g(n)) ≈ f(n) < g(n)

f(n) = ω(g(n)) ≈ f(n) > g(n)