r asymptotic analysis11

8/16/2019 R Asymptotic Analysis11

http://slidepdf.com/reader/full/r-asymptotic-analysis11 1/65

Asymptotic Analysis



Asymptotic Analysis

Topics Asymptotic notation

Using Basic Methods

Using Limits

Analysis of summation



What is Complexity?

The level in difficulty in solving mathematically posed problems as measured by

• The time (time complexity)

• Number of steps or arithmetic operations (computational

complexity)

• Memory space required (space complexity)

Major Factors in Algorithms Design

1. Correctness

An algorithm is said to be correct if For every input, it halts ith correct output! An

incorrect algorithm might not halt at all "# $t might halt ith an anser other than

desired one! Correct algorithm solves a computational problem!

. Algorithm !fficiency

%easuring efficiency of an algorithm, do its analysis i!e! groth rate! Compare

efficiencies of different algorithms for the same problem!



Algorithms &roth #ate

Algorithm "ro#th $ates

$t measures algorithm efficiency

%hat means by efficient&$f running time is bounded by polynomial in the input

Notations for Asymptotic performance

'o running time increases ith input sie", "mega, Theta, etc! for asymptotic running time

These notations defined in terms of functions hose domains are natural numbers

con'enient for #orst case running time

Algorithms, asymptotically efficient best choice



Complexity Analysis

Algorithm analysis means predicting resources such as• computational time• memory• computer hardare etc

%orst case analysis

• rovides an upper bound on running time

A'erage case analysis• rovides the expected running time• *ery useful, but treat ith care+ hat is average-?

#andom (e.ually li/ely) inputs

#eal0life inputs



Worst0case Analysis

(et us suppose that

• 1n 2 set of inputs of sie n for the problem

• $ 2 an element of 1n!

• t($) 2 number of basic operations performed on $• 1efine a function W by

W(n) 2 max3t($) 4 $ ∈ 1n5

called the orst0case complexity of the algorithm

• W(n) is the maximum number of basic operations performed by the

algorithm on any input of sie n!

• lease note that the input, $, for hich an algorithm behaves orst dependson the particular algorithm!



Average Complexity

• 6et r ($) be the probability that input $ occurs!• Then the average behavior of the algorithm is defined as

A)n* + Σ,r )-* t)-* summation o'er all - ∈ Dn

• We determine t($) by analying the algorithm, but r($) cannot be computedanalytically!

• Average cost 2A)n* + ,r)succ* Asucc)n* / ,r)fail* Afail)n*

• An element $ in 1n may be thought as a set or e.uivalence class that affect

the behavior of the algorithm

%orst Analysis computing a'erage cost

• Ta/e all possible inputs, compute their cost, ta/e average



Asymptotic 7otations roperties

• Categorie algorithms based on asymptotic groth rate e!g! linear, .uadratic, polynomial, exponential

• $gnore small constant and small inputs

• 8stimate upper bound and loer bound on groth rate of time complexity

function

• 1escribe running time of algorithm as n gros to ∞!

• 1escribes behavior of function ithin the limit!

(imitations• not alays useful for analysis on fixed0sie inputs!

• All results are for sufficiently large inputs!



Asymptotic 7otations

Asymptotic Notations , O, Ω, o, ω

We use Θ to mean “order exactly”,

O to mean “order at most”,

Ω to mean “order at least”,

o to mean “tight upper bound”,

ω to mean “tight lower bound”,

Defne a set o unctions: which is in practice used tocompare two unction sizes



Asymptotic Analysis"b9ective

The purpose of asymptotic analysis is to examine the behavior of an algorithm for

large input sie! %ore specifically, if T(n) is the running time for an input of size

n , e ould ant to /no the behavior or growth rate of T(n) for very largevalues of n! An analysis of algorithm for large input is referred to as asymptotic

analysis!

The asymptotic behavior of an algorithm is often compared to some standard

mathematical function, such as n2, n lg n etc The relationship or similarity of

behavior is often expressed by a special notation hich is called asymptotic

notation!

The standard asymptotic notations commonly used in the analysis of algorithms

are /non as O (Big Oh), Ω (Big Omega), and θ(Theta).

:ometimes, additional notations o( small-oh) and ( small-omega) are also used!



Asymptotic 7otation



"07otation

c.g(n) for all n! n"

then f(n) # O(g(n)) (#ead f(n) is Big-Oh of g(n) )

1efinition

$f f(n) is running time of an algorithm, and g(n) is some standard groth functionsuch that for some positive constants c and n" ,

" $ f(n) %

The behavior of f(n) and g(n) is portrayed in the diagram! $t follos, that for n<n0 , f(n) may

be above or belo g(n), but for all n n0 , f(n) falls consistently belo g(n)! The function g(n)is said to be asymptotic upper boun& for f(n)



"07otation

;sing basic definition e sho that !n2 " #0n $ %(n2 )

Consider, #0 & n for n #0 ( obvious <)

#0n & n2 for n #0 ( %ultiplying

both sides ith n )

!n2"#0n & 'n2 for n #0 (Adding !n2 to

both sides )!n2 " #0 n & cn2 for n n0 ( c$' and n0$#0 )

Therefore, it follos from the basic definition that

!n2 "#0n

$

%(n2 )

8xample



"07otation8xample contd

The preceding result shos that

!n2

" #0 n = c n2,

for c#' and n ! "

'oever, the choice of c is not uniue! We can select some other c and correspon&ing n" so that the

relation still holds!

Consider,!n2 " #0n ≤ !n2 " #0 n2

2#! n>

for n ≥# ( #eplacing n ith n2 on the right

side )for n≥ #!n2 "#0n ≤ cn2 ,

for n≥

n0 , ( c#* and n"# ) The graph depicts the to solutions! "bserve that both *n+ and 'n+ eventually grow faster than

*n+," g(n)# cn+

c#* g(n)#cn+

c#'

f(n)#*n+ , " n

n"#"

n"#



"07otation:et uilder

There can be a several functions , say f#(n), f2(n), f!(n), !!for hich g(n) is the upperboun& ! All such functions are said belong to a class identified by O(g(n)) as shon belo!

f#(n) & c# g(n) for n*n#

f2(n) & c2 g(n) for n*n2

f!(n) & c! g(n) for n*n!O( g(n) )# f(n) f+(n)

f*(n)/0

;sing set-buil&er notation the relationship

is expressed as

O(g(n)) # f(n)1 there e2ist positiveconstants c an& n" " % f(n) % c.g(n) for all

n ! n" 0

∈ :ymbolically , f(n)

O(g(n))



@07otation1efinition

$f f(n) is running time of an algorithm, and g(n) is some standard groth function

such that for some positive constants c n" ,

" $ cg(n) % f(n) for all n!n"

then f(n) # Ω(g(n)) (#ead f(n) is Big-Omega of g(n) )

The behavior of f(n) and g(n) is portrayed in the graph! $t follos, that for n<n0 , f(n) may be

above or below g(n), but for all n n0, f(n) falls consistently above g(n) The function g(n) is

said to be asymptotic lower boun& for f(n)



We sho that n+ -"n # +(n2 )

For , n n 2 n - #0

for n 0 ( "bvious < )n (2 . #0) for n #0 ( 1ivide right side by B)

$ n 20n2 / #0n n2 20 for n #0 ( %ultiply both sides ith n to maintain ine.uality )n2 / #0n c n2 for n n0 , here c$# 20 and n0$#0

The behavior of functions n2-#0n and n2 20 is shon in the graph! "bserve that for

n#0, the function n2 20 falls belo the function n2-#0n

Therefore, n2 -#0n $ +(n2 )

@07otation8xample0

f(n)$n2 -#0n

n0$#0

g(n)$cn2 , c $ #20 Loer asymptotic 1ound



@07otation

n n 2 for n 0 n- 2! ! n (2 . 2)

$ ! n 0

( "bvious < )for n 3 ( 1ivide right side by >DE G!E)

for n 3!n2 / 2n 3n2 0 for n 3

( %ultiply both sides ith En to maintain ine.uality )!n2 / 2 n c n2 for n n0 , here c#3 4 5" and n"#3

The behavior of functions En2-2n and Hn2 0 is shon in the graph! "bserve that for

n3, the function Hn2 0 falls belo the function En2-2n

Therefore, En2 -2n$+(n2 )

f(n)$!n2 -2n

g(n)$cn2, c$ 30

Loer asymptotic 1ound

n

8xample0>

7ext e sho that En2 -2n $ +(n2 )

n0$3



@07otation:et uilder

There can be a several functions , say f#(n), f2(n), f!(n), for hich g(n) is the lowerboun& ! All such functions are said to belong to a classs i&entifie& by Ω(g(n))!

f#(n) c# g(n) for n*n#

f2(n) c2 g(n) for n*n2

f!(n) c! g(n) for n*n!

Ω( g(n) )# f(n) f+(n) f*(n)/0

∈

;sing set-buil&er notation the relationship is expressed as

Ω(g(n)) # f(n)1 there e2ist positive constants c an& n" f(n) ! c.g(n) for all n ! n" 0

:ymbolically , f(n)

+(g(n))



I07otation

n" ," $ c+ .g(n) % f(n) % c .g(n) for all n!n"

then f(n) # θ(g(n)) (#ead f(n) is theta of g(n) )

The behavior of f(n) and g(n) is portrayed in the graph! $t follos, that for n < n0 , f(n) may

be above or below g(n), but for all n ! n" , f(n) falls consistently between c .g(n) an& c+ .g(n).Function g(n) is sai& to be the asymptotic tight boun& for f(n)

There can be a several functions for hich the g(n) is asymptotic tight boun& ! All such

functions are said belong to the class identified by g(n)! :ymbolically, e denote the

relationship as f(n)∈ 4(g(n))

1efinition

$f f(n) is running time of an algorithm, and g(n) is some standard grothfunctionsuch that for some positive constant sc c+ and



Therefore, 0n+ -3n # θ(n+ )

I07otation We sho that

8xamplen2 -#3n $ 4(n2 )

7ext, consider the lower boun& ,

n n 2for n 0

( "bvious < )

n- #3 n (2 .#3)

for n ' ( 1ivide right side by HD2 E!G)

$ n !5 for n '

n2 / #3n 2n2 !5 for n ' ( %ultiply both sides ith n )

n2 / #3 n c2 n2 for n n2 , here c+#+5 4 *6 and n+#'

Consider the upper boun& ,n2 / #3n & n2 for n 0

n2 / #3 n & c# n2 for n n# , here c#5 and

n#"

$t follos, " $ c+ .n+ 0n+ - 3n % c .n+ for n ! n" here n"#' c#5 and c+#+54*6.



I07otation

The relation 0n+ -3nasymptotic upper and loer bounds of the function f(n)$ n2-#3 n

g(n)#c+ .n+

c+#+54*6

Loer 1ound

n

8xample0contd

# θ(n+

) is illustrated in the figure belo! The graph shos the

g(n)#cn+

c#5

f ( n )#5n+

-

3n

Upper 1ound

n"#'



Asymptotic 7otationConstant #unning Time

$f running time T(n)#c is a constant i!e independent of input sie, then , by convention,

asymptotic behavior is denoted by the notation

%(c) $ %(#) , 4 (c) $ 4(#), +(c) $ +(#)

The convention implies that the running time of an algorithm ,hich does not depend

on the sie of input, can be expressed in any of the above ays!

$f c is constant then using basic definition it can be shon that

%( cf(n) ) $ %(f(n))

4( cf(n) ) $ 4(f(n))

+( cf(n) ) $ +(f(n))

The above relations imply that in asymptotic notation the multiplier constants can beignore& For example, %(#000n)$%(n), 4(6lgn ) $ 4 (lg), +(#00n7)$ +(n7)



Asymptotic 7otationI, ", @ #elationship

then

7f f(n) # θ( g(n) )

f(n) # Ω( g(n) ) an& f(n)#

8onversely if f(n) # Ω( g(n) )

then f(n) # θ( g(n) )

O( g(n) )

an& f(n) #O( g(n) )

The above properties follo directly from the basic definitions

!2ample)1*3 :ince, n(n-#)2 # 4(n2 ), therefore it follos that

n(n-#)2 # +(n2 )

n(n-#)2 # %(n2

)!2ample)*3 $t can be shon that

n2"# # +(n2 )

and n2"# $ %(n2 )

Therefore, n2 " # $ 4(n2 )



Asymptotic Analysis/QAU2008/Dr.A.Sattar/18

Asymptotic 7otation#elationship



Asymptotic :et 7otation8xample

The relationship among the "0,@0,I07otation can be expressed using set notation

Consider, for example, the folloing sets of groth functions

3 f(n)8 9n, n", lg n"'n, n#"n, 9n"n2 , n2"n, lg n"'n2 , n#"!n2 5

f(n)83 9n"n2 , n2"n, lg n"'n2 , n#"!n2 5

# 3 f(n)8 9n"n2, n2"n, lg n"'n2 , n#"!n2 , n2"n! , n! "n2"n, lg n"'n' , nlg n"!n' 5

J 9 O(n+ )#

J 9 Ω(n+ )

J 9 θ(n+ )#

$t follos that 9 θ(n+ ) # 9 O(n+ ) ∩ 9 Ω (n+ )

here 9 O(n+ ) is a set of functions f(n) ∈ O(n+ )

here 9 Ω(n+ ) is a set of functions f(n)∈ Ω(n+ )

here 9 θ(n+ ) is a set of functions f(n) ∈θ(n+ )



Asymptotic 7otation"rder Theorem

Theorem1 $f f #(n) $ %( g #(n) ) and f 2(n) $ %( g 2(n) ) then

f #(n) " f 2(n)$

:roof1 y definition,

%( ma.( g #(n) , g 2(n) )

6et n0 2 ma.( n# , n2 ) f #(n) & c! g #(n)

c!$ma.(c# , c2 ) for n n0

f 2(n) & c! g 2(n) for n n0

c!g #(n) " c! g 2(n) ) for n n0 f #(n) " f 2(n) &

6et h(n) $ ma.( g #(n) , g 2(n) )

f #(n) " f 2(n) & 2c!h(n) $ c h(n) here c$2c!

f #(n) " f 2(n) & c h(n) $ c ma.( g #(n) , g 2(n) )

Therefore, f #(n) " f 2(n) $ %( ma.( g #(n) , g 2(n) )

The theorem also applies to θ and Ω notations

for n n0

for n n0

f #(n) & c# g #(n) for n n#

f 2(n) & c2 g 2(n) for n n2



Asymptotic 7otation;sing "rder Theorem

The relation

f #(n) " f 2(n)$ %( ma.( g #(n) , g 2(n) )

implies that in a summation, the lower or&er growth function can be &iscar&e& in favor

of the highest ran;ing function

$n general,

f #(n)"f 2(n)"f !(n) :"f ; (n)$ %(ma.(g #(n),g 2(n),g !(n):,g ; (n))

!2ample3 Consider the summation f(n) consisting of basic functions +

f(n) $ n "9n " n#" lg n " n lg n " (lg n)2 " n2

We have seen that lg n < (lgn)2 < √n < n <n lg n <n# < n2

The function n2 gros faster than all other functions in the expression

Thus , %( ma2 ( n " 9n " n#" lg n " n lg n " (lg n)2 " n2 ) ) $%( n2 )



;sing 6imits



Asymptotic Analysis;sing 6imits

;se of basic definition for determining the asymptotic behavior is often a/ard! $t

involves ad hoc approach or some /ind of manipulation to prove algebraic relations!

Calculus provides an alternative method for the analysis! $t depends on evaluating

the folloing limit!

f(n)lim 2 K

n= g(n)

here f(n) is a given groth function for an algorithm and g(n) is a standard function

1epending upon the limit K , the relation beteen f(n) and g(n) can be expressed in terms

of asymptotic notations

$t ill be seen that the condition n< = is e>ui?alent to the condition all n ! n" in the

basic definition of asymptotic notation! 8ither of these conditions implies large input

;se of limits simplifies the asymptotic analysis!



"07otation;sing 6imit

$f f(n) is running time of an algorithm and g(n) is some standard groth function such

that f(n)

!2ample)1*+ !n2 " n" 20 $

!n2 " n " 20

%(n2 )

limn =

2!" n "20 n2 $ ( !"0"0)$!

Therefore, !n2 " n" 20 $ %(n2 )

!2ample)*+ #0n2 " 2n" 6 $ %(n! )

#0n2 " 2n " 6 limn =

$#0 n" 2 n2 " 6 n! $ (0"0"0)$0

Therefore, #0n2 " 2n" 6 $ %(n! )

n < =

g(n)

then f(n) # O) g(n))

lim >> #c

where " % c $ = (infinity is e.cluded)

n2

n!



"07otation8xamples contd

!2ample)4*+ lg n # O(n)

$n order to compute differential of lg n e first convert binary logarithm to natural logarithm!Converting lg n (1inary ) to ln n ( natural), by the using formula lg n $ ln n ln 2

n = ln 2

n

lim

Therefore, lg n $ %(n)

!2ample)5*+ n+ # O(+n )

limn =

#n (ln 2)n

lg n (ln n)$$∞ / ∞$ 0 ( 1ifferentiating the numerator and denominator )

limn =

n = 2n

lim

2

$∞ / ∞ ( 1ifferentiating the numerator and denominator)

(ln2)2 2n

$ %(2n

)

n2

$ 0

Therefore, n2

n = ln 2 2nlim $ = = (1ifferentiating again the numerator and denominator)2n



@ 07otation;sing 6imit


that f(n)

n < =

g(n)

then f(n) $ 6) g(n))

lim >> #c

where " $ c % = (@ero is e.cluded)

!2ample)1*+ Ln2 " #2n"5$ +(n2 )

6n2 " #'n " 5limn =

2L" #' n "5 n2 $(6 " 0 " 0)$6

Therefore, Ln2 " #'n" 5

!2ample)*+ #0n! "

n" 2

$ +(n2 )

$ +(n2 )

#0n! " n " 2limn =

$#0 n" n " 2 n2 $ = " 0 " 0 $ =

Therefore, #0n! " n" 2 $ + (n2 )

n2

n2



I07otation;sing 6imit


that f(n)

n < = g(n)lim >> # c where " $ c $ = ( @ero and infinity is e.cluded )

then f(n) $ θ ) g(n))

!2ample)1*+ 'n! 0 !n2 - n" 20 $ 4(n! )

'n! -!n2 - n " 20n!

limn =

2 M0E n - n2 "20n! $('-0 -0)$'

Therefore, 'n! 0 !n2 - n" 20 $ 4(n! )

!2ample)*+ n lg n " n " n2 $ 4(n2 )

limn =

$ lg n n "#n "#$(0"0"#)$#n2

n lg n " n " n2

Thus, n lg n " n " n2 2 4(n2 )



o07otation1efinition

$f f(n) is running time and g(n) is some standard groth function such that

f(n)

n " 20limn =

2 n "20 n2 $ ( 0"0)$0

Therefore, n" 20 $ o(n2 )

!2ample)*+ #0n2 " 2n" 6 $ o(n! )

#0n2 " 2n " 6 limn =

$#0 n" 2 n2 " 6 n! $ (0"0"0)$0

n < = g(n)

then f(n) # o) g(n)) (ead f (n) is small-oh of g(n))

!2ample)1*+ n" 20 $ o(n2 ) For ,

lim >> #"

n2

n!

!2ample)4*+ lg n $ o(n), because

lg nlimn =

$ 0 ( Using Lopital ule to compute the limit)n



N07otation1efinition

$f f(n) is running time and g(n) is some standard groth function such that

f(n)

n < = g(n)

then f(n) # 7) g(n)) (ead f(n) is small-omega of g(n))

!2ample)1*+ n!" 20n2"n"#0 $ C(n2 ), For

lim >> #=

n! " 20n2"n"#0

n2

limn =

2n "20" # n"#0n2 $ (="20"0"0)$=

!2ample)*+ #0n2 " 2n" 6 $ C(n), because

#0n2 " 2n " 6 limn =

$ n" 2 " 6 n $ (="2"0)$=n

!2ample)4*+ n7 2N(2n ) , because

n7limn =

$ ∞ (Using Dtirlings formula for n7)

2n



Asymptotic 7otation:ummary

lim EEE $F

6et f(n) be time complexity and g(n) standard function, such that

n=

f(n) g(n)

Table belo summaries the asymptotic behavior of f(n) in terms of g(n)

?otation @sing Basic Aefinition @sing imits Csymptotic Boun&

f(n)$%( g(n) ) f(n) & cg(n) for some c*0, and n n0 0 & F < = tight upper

f(n)$o( g(n) ) f(n) <cg(n) for all c*0, and n n0 F $ 0 loose upper

f(n)$+( g(n) ) f(n) cg(n) for some c*0 and n n0 0 < F & = tight loer

f(n)$C( g(n) ) f(n) * cg(n) for all c*0 and n n0 F $ = loose loer

f(n)$4( g(n) ) c#g & f(n) & c2g(n) for some c#*0, c2*0

and n n0

0 < F < = tight



Analysis of :ummations



Arithmetic :ummationAsymptotic ehavior

The sum of first of n terms of arithmetic series is +

# " 2 " !:::"n $ n(n"#)2

6et f(n)# n(n,)4+

and g(n)# n+

lim

n=

f(n) 2

g(n)

n(n"#)2 n2 2 " n22 # 2 " #2n $#2 "0$#2

n2

2

n2

:ince the limit is non-zero an& finite it follos

f(n) # θ( g(n)) # θ(n+ )

"r , + , ///.., n # θ(n+ )



&eometric :ummationAsymptotic ehavior

The asymptotic behavior of geometric series

# "r " r 2"::"r ndepends on geometric ratio r ! Three cases need to be considered

Case r D 1 $t can be shon that sum f(n) of first n terms is as follos

f(n) # # "r " r 2

"::"r n

$ EEEEE r n"#

- #

r - #

Case r # 1 This is trivial

f (n)2 # " # " #" ::"# 2 n $G(n)

Case r $ 1 $t can be shon that

f(n) # # "r " r 2"::"r n $ EEEEE # - r

The asymptotic behavior in first case and third case is explored by computing

limits!

# - r n"#



&eometric :ummationCase r O

6et f(n) # ,r , r +,//..,r n # >>>>> r n, -

r -

4(r n ) for r*#

6et g(n)# r n

Consider, the limit

f(n)

g(n)lim

n=r n"# - #(r - #)r n

r - #r n

(r - #)22

:ince r*# , #r n 0 as n =

f(n)

g(n)

lim

n=

The

refore, f(n) $

2

(r - #)

r OB, since r * #

"r, ,r , r +,//..,r n # θ(r n ) for r D



&eometric :ummationCase r P

Consider ,r , r +,//..,r n # >>>>> r n,-

r - 6et g(n)#c where c is some positive constant

Ta/ing the limit

4(g(n))$4(c) $ 4(#) for r <#

f(n)

g(n)lim

n=r n"#

- #

(r - #)c (#- r )c22

:ince r<# , r n"# 0 as n =

f(n)

g(n) (# - r)c

lim

n=

Therefore, f(n)$

2#

OB, since r < #

#- r n"#

"r, ,r , r +,//..,r n #θ() for r $

To sum up ,,r,r +,........., r n $

E(r n ) when rD

E() when r$



6ogarithm :ummationAsymptotic ehavior

The logarithmic series has the summationlg(#)"lg(2)" lg(!)": "lg(n)

6et f(n) # lg(+), lg(* ),/.,.lg(n) $ lg(+.*//..n) #lg( nF)and g(n) # n lg n

lg ( 9(2Hn)(ne)n

)limn= f(n) g(n) n lg nlg n7

n lg n ==

7o, lg ( 9(2Hn)(ne)n ) $ (#"lg H " lg n)2 " n lg n - n lg e , therefore

lg ( 9(2Hn)(ne)n )lim

n= n lg n

$ ( #"lg H " lg n ) (2 nlg n) " # - lg e lg n $ (0" #- 0)$#

:ince limit is non-zero an& finite, it follos

f(n)$ 4(g(n))

%r, lg(),lg(+), lg(*),/.. ,lg(n) # θ(n lg n):

(using :tirlingQs approximation)



'armonic :ummation

:ince limit is finite an& non-zero it follos

f(n) $ 4( g(n))

"r, , 4+ , 4*,/..,4n # θ (lg n )

$t can be shon that #" #2" #!":"#n $ lg (n) " I " #2n / ##2n2" here I J 0662

lim

n=

f(n) g(n)

Asymptotic ehavior

The sum of first n terms of Garmonic series is#" #2 " #!":"#n

6et f(n) # , 4+ , 4*,/..,4nand g(n) # lg n

lg n " I " #2n / ##2n ":

lg n=

2

$ #"0"0-0"0":$#

lim f(n)n= g(n)

#"#2"#!" :"#n

lg n=



#eflexive #elation

Definition+

6et R be a non0empty set and # is a relation over R then # is said to be reflexive if

(a, a) ∈ #, ∀ a ∈ R,

!2ample 136et & be a graph! 6et us define a relation # over & as if node x is connected to y then (x,y) ∈ &! #eflexivity is satisfied over & if for every node there is a self loop!

!2ample 36et be a set of all persons, and : be a relation over such that if (x, y) ∈ : then x has

same birthday as y!

"f course this relation is reflexive because(x, x) ∈ :, ∀ a ∈ ,



#eflexivity #elations over Θ, Ω, "

!2ample 1

:ince, B ≤ f(n) ≤ cf(n) ∀ n ≥ nB 2 , if c 2

'ence f(n) 2 "(f(n))

!2ample

:ince, B ≤ cf(n) ≤ f(n) ∀ n ≥ nB 2 , if c 2

'ence f(n) 2 Ω(f(n))

!2ample 4

:ince, B ≤ cf(n) ≤ f(n) ≤ c>f(n) ∀ n ≥ nB 2 ,if c2 c> 2

'ence f(n) 2 Θ(f(n))

Note3 All the relations Ω 8 are refle2i'e



6ittle o and ω are not #eflexivity #elations

!2ample

As e can not prove that f(n) P f(n), for any n, and for all c O B

Therefore

! f(n) ≠ o(f(n)) and>! f(n) ≠ ω(f(n))

Note 3

'ence small o and small omega are not reflexive relations



:ymmetry

Definition+6et R be a non0empty set and # is a relation over R then # is said to be symmetric if

∀ a, b ∈ R, (a, b) ∈ # ⇒ (b, a) ∈ #

!2ample 136et be a set of persons, and : be a relation over such that if (x, y) ∈ : then x has the

same sign as y!

This relation is symmetric because

(x, y) ∈ : ⇒ (y, x) ∈ :!2ample 36et be a set of all persons, and be a relation over such that if (x, y) ∈ then x is

brother of y!

This relation is not symmetric because

(Aner, :adia) ∈ ⇒ (:aida, rother) ∉

:ymmetry over Θ



:ymmetry over ΘProperty : pro!e that

f(n) 2 Θ(g(n)) ⇔ g(n) 2 Θ(f(n))

Proof

:ince f(n) 2 Θ(g(n)) i!e! f(n) ∈ Θ(g(n)) ⇒

∃ constants c, c> O B and nB ∈ 7 such that

B ≤ cg(n) ≤ f(n) ≤ c>g(n)∀ n ≥ nB ()

() ⇒ B ≤ cg(n) ≤ f(n) ≤ c>g(n) ⇒ B ≤ f(n) ≤ c>g(n)

⇒ B ≤ (Dc>)f(n) ≤ g(n) (>)

() ⇒ B ≤ cg(n) ≤ f(n) ≤ c>g(n) ⇒ B ≤ cg(n) ≤ f(n)

⇒ B ≤ g(n) ≤ (Dc)f(n) (E)From (>),(E)+ B ≤ (Dc>)f(n) ≤ g(n) ∧ B ≤ g(n) ≤ (Dc)f(n)

⇒ B ≤ (Dc>)f(n) ≤ g(n) ≤ (Dc)f(n)

:uppose that Dc> 2 cE, and Dc 2 cM,

7o the above e.uation implies that

B ≤ cE

f(n) ≤ g(n) ≤ cM

f(n), ∀ n ≥ nB

⇒ g(n) 2 Θ(f(n)), ∀ n ≥ nB

9ence it pro'es that

f(n) 2 Θ(g(n)) ⇔ g(n) 2 Θ(f(n))

!2ercise3

prove that big ", big omega Ω, little ω, and little o, do not satisfy the symmetry

property!



Transitivity

Denition:"et # be a non$empty set and % is a relation o!er # then % is said to be

transiti!e i∀ a, b, c ∈ #, &a, b' ∈ % ∧ &b, c' ∈ % ⇒ &a, c' ∈ %

Example 1:"et ( be a set o all persons, and ) be a relation o!er ( such that i &x, y' ∈ )

then x is brother o y *his relation is transiti!e this is because

&x, y' ∈ ) ∧ &y, z' ∈ ) ⇒ &x, z' ∈ )Example 2:"et ( be a set o all persons, and + be a relation o!er ( such that i &x, y' ∈ +

then x is ather o yO course this relation is not a transiti!e because i &x, y' ∈ + ∧ &y, z' ∈ + ⇒

&x, z' ∉ +



Transitivity Relation over Θ, Ω, %,o and ω

Prove the following

! f(n) 2 Θ(g(n)) S g(n) 2 Θ(h(n)) ⇒ f(n) 2 Θ(h(n))

>! f(n) 2 "(g(n)) S g(n) 2 "(h(n)) ⇒ f(n) 2 "(h(n))

E! f(n) 2 Ω(g(n)) S g(n) 2 Ω(h(n)) ⇒ f(n) 2 Ω(h(n))

M! f(n) 2 o (g(n)) S g(n) 2 o (h(n)) ⇒ f(n) 2 o (h(n))! f(n) 2 ω(g(n)) S g(n) 2 ω(h(n)) ⇒ f(n) 2 ω(h(n))

Note

$t is to be noted that all these algorithms complexity measuring notations are in fact

relations hich satisfy the transitive property!



Transitivity Relation overΘ

Property 1

f(n) 2 Θ(g(n)) S g(n) 2 Θ(h(n)) ⇒ f(n) 2 Θ(h(n))

Proof

:ince f(n) 2 Θ(g(n)) i!e! f(n) ∈ Θ(g(n)) ⇒

∃ constants c, c> O B and nB ∈ 7 such thatB ≤ cg(n) ≤ f(n) ≤ c>g(n)∀ n ≥ nB ()

>! 7o since g(n) 2 Θ(h(n)) i!e! g(n) ∈ Θ(h(n)) ⇒

∃ constants cE, cM O B and nB> ∈ 7 such that

B ≤ cEh(n) ≤ g(n) ≤ cMh(n) ∀ n ≥ nB> (>)

E! 7o let us suppose that nB

2 max (nB

, nB>

)



Transitivity Relationover Θ

M! 7o e have to sho that f(n) 2 Θ(h(n)) i!e! e have to prove that

∃ constants c, c O B and nB ∈ 7 such that

B ≤ ch(n) ≤ f(n) ≤ ch(n)?

(>) ⇒ B ≤ cEh(n) ≤ g(n) ≤ cMh(n)

⇒ B ≤ cEh(n) ≤ g(n) (E)

(1) ⇒ B ≤ cg(n) ≤ f(n) ≤ c>g(n)

⇒ B ≤ cg(n) ≤ f(n)

⇒ B ≤ g(n) ≤ (Dc)f(n) (M)

From (E) and (M), B ≤ cEh(n) ≤ g(n) ≤ (Dc)f(n)

⇒ B ≤ ccEh(n) ≤ f(n) ()



Transitivity Relation over Θ

() ⇒ B ≤ cg(n) ≤ f(n) ≤ c>g(n)

⇒ B ≤ f(n) ≤ c>g(n) ⇒ B ≤ (Dc>)f(n) ≤ g(n) ()

(>) ⇒ B ≤ cEh(n) ≤ g(n) ≤ cMh(n)

⇒ B ≤ g(n) ≤ cMh(n) (L)

From () and (L), B ≤ (Dc>)f(n) ≤ g(n) ≤ (cM)h(n)

⇒ B ≤ (Dc>)f(n) ≤ (cM)h(n)

⇒ B ≤ f(n) ≤ c>cMh(n) (G)

From (), (G), B ≤ ccEh(n) ≤ f(n) ∧ B ≤ f(n) ≤ c>cMh(n)

B ≤ ccEh(n) ≤ f(n) ≤ c>cMh(n)

B ≤ ch(n) ≤ f(n) ≤ ch(n)

And hence f)n* + )h)n** n ≥ n:

Transitivity Relation over



Transitivity Relation overBig %Property 2

f(n) 2 "(g(n)) S g(n) 2 "(h(n)) ⇒ f(n) 2 "(h(n))

Proof

:ince f(n) 2 "(g(n)) i!e! f(n) ∈ "(g(n)) ⇒∃ constants c O B and nB ∈ 7 such that

B ≤ f(n) ≤ cg(n) ∀ n ≥ nB ()

>! 7o since g(n) 2 "(h(n)) i!e! g(n) ∈ "(h(n)) ⇒

∃ constants c> O B and nB> ∈ 7 such that

B ≤ g(n) ≤ c>h(n) ∀ n ≥ n

B> (>)

E! 7o let us suppose that nB 2 max (nB, nB>)

7o e have to to e.uations

B ≤ f(n) ≤ cg(n) ∀ n ≥ nB ()

B ≤ g(n) ≤ c>h(n) ∀ n ≥ nB> (>)

(>) ⇒ B ≤ cg(n) ≤ cc>h(n) ∀ n ≥ nB> (E)

From () and (E)

B ≤ f(n) ≤ cg(n) ≤ cc>h(n)

7o suppose that cE2 cc>

B ≤ f(n) ≤ cc>h(n)

And hence f)n* + 8)h)n** n≥

n:



Transitivity Relation overBig ΩProperty

f(n) 2 Ω(g(n)) S g(n) 2 Ω(h(n)) ⇒ f(n) 2 Ω(h(n))

Proof

:ince f(n) 2 Ω(g(n)) ⇒

∃ constants c O B and nB ∈ 7 such that

B ≤ cg(n) ≤ f(n) ∀ n ≥ nB ()

>! 7o since g(n) 2 Ω(h(n)) ⇒

∃ constants c> O B and n

B> ∈ 7 such that

B ≤ c>h(n) ≤ g(n) ∀ n ≥ nB> (>)

E! :uppose that nB 2 max (nB, nB>)

M! We have to sho that f(n) 2 Ω(h(n)) i!e! e have to prove that

∃ constants cE O B and nB ∈ 7 such that

B ≤ cEh(n) ≤ f(n) ∀ n ≥ nB ?

(>) ⇒ B ≤ c>h(n) ≤ g(n)

() ⇒ B ≤ cg(n) ≤ f(n)

⇒ B ≤ g(n) ≤ (Dc)f(n) (E)

From (>) and (E), B ≤ c>h(n) ≤ g(n) ≤ (Dc)f(n)

⇒ B ≤ cc>h(n) ≤ f(n)

hence f)n* + Ω)h)n** n ≥ n:

Transitivity Relation over little



a s t ty e at o o e little

o,roperty 5

f(n) 2 o(g(n)) S g(n) 2 o(h(n)) ⇒ f(n) 2 o(h(n))

,roof

:ince f(n) 2 o(g(n)) i!e! f(n) ∈ o(g(n)) ⇒∃ constants c O B and nB ∈ 7 such that

B ≤ f(n) P cg(n) ∀ n ≥ nB ()

>! 7o since g(n) 2 o(h(n)) i!e! g(n) ∈ o(h(n)) ⇒

∃ constants c> O B and nB> ∈ 7 such that

B ≤ g(n) P c>h(n) ∀ n ≥ nB> (>)

E! 7o let us suppose that nB 2 max (nB, nB>)

7o e have to to e.uations

B ≤ f(n) P cg(n) ∀ n ≥ nB ()

B ≤ g(n) P c>h(n) ∀ n ≥ nB (>)

(>) ⇒ B ≤ cg(n) P cc>h(n) ∀ n ≥ nB> (E)

From () and (E)B ≤ f(n) ≤ cg(n) P cc>h(n)

7o suppose that cE2 cc>

B ≤ f(n) P cc>h(n)

And hence f)n* + o)h)n** n ≥ n:1



Transitivity Relation overlittle ωProperty !

f(n) 2 ω(g(n)) S g(n) 2 ω(h(n)) ⇒ f(n) 2 ω(h(n))

Proof

:ince f(n) 2 ω(g(n)) ⇒

∃ constants c O B and nB ∈ 7 such that

B ≤ cg(n) P f(n) ∀ n ≥ nB ()

>! 7o since g(n) 2 ω(h(n)) ⇒

∃ constants c> O B and nB> ∈ 7 such thatB ≤ c>h(n) P g(n) ∀ n ≥ nB> (>)

E! :uppose that nB 2 max (nB, nB>)

M! We have to sho that f(n) 2 ω(h(n)) i!e! e have to prove that

∃ constants cE O B and nB ∈ 7 such that

B ≤ cEh(n) ≤ f(n) ∀ n ≥ nB ?

(>) ⇒ B ≤ c>h(n) P g(n)

() ⇒ B ≤ cg(n) P f(n)

⇒ B ≤ g(n) P (Dc)f(n) (E)

From (>) and (E), B ≤ c>h(n) ≤ g(n) P (Dc)f(n)

⇒ B ≤ cc>h(n) P f(n) hence f(n) 2 ω(h(n)), ∀ n ≥ nB

S



Transpose Symmetry

Property 1

rove that f(n) 2 "(g(n)) ⇔ g(n) 2 Ω(f(n))

Proof

:ince f(n) 2 "(g(n)) ⇒

∃ constants c O B and nB ∈ 7 such thatB ≤ f(n) ≤ cg(n) ∀ n ≥ nB

1ividing both side by c

B ≤ (Dc)f(n) ≤ g(n) ∀ n ≥ nB

ut Dc 2 cQ

B ≤ cQf(n) ≤ g(n) ∀ n ≥ nB

'ence, g(n) 2 Ω(f(n))



TransposeSymmetry

Property 2

rove that f(n) 2 o(g(n)) ⇔ g(n) 2 ωf(n))

Proof

:ince f(n) 2 o(g(n)) ⇒

∃ constants c O B and nB ∈ 7 such thatB ≤ f(n) P cg(n) ∀ n ≥ nB

1ividing both side by c

B ≤ (Dc)f(n) P g(n) ∀ n ≥ nB

ut Dc 2 cQ

B ≤ cQf(n) P g(n) ∀ n ≥ nB

9ence g)n* + ω)f)n**



#elation beteen Θ, Ω , "

Trichotomy property o'er real numbers For any to real numbers a and b, exactly one of the folloing must hold+ a P b, a 2 b,

or a O b!

The asymptotic comparison of to functions f and g and the comparison of to real

numbers a and b!

Trichotomy property o'er Ω and 8! f (n) 2 "(g(n)) ≈ a = b

>! f (n) 2 Ω (g(n)) ≈ a ≥ b

E! f (n) 2 Θ (g(n)) ≈ a 2 b

M! f (n) 2 o (g(n)) ≈ a P b

! f (n) 2 ω(g(n)) ≈ a O b



:ome "ther :tandard 7otations

Monotonicitymonotonically increasing if m ≤ n ⇒ f(m) ≤ f(n)!

monotonically decreasing if m ≤ n ⇒ f(m) ≥ f(n)!

strictly increasing if m P n ⇒ f(m) P f(n)!

strictly decreasing if m P n ⇒ f(m) O f(n)!

,olynomials&iven a positive integer d, a polynomial in n of degree d is a function of the form given

belo, ai are coefficient of polynomial!

( ) i

d

i

inan p ∑

=

=B



:tandard 6ogarithms 7otations

:ome 1efinitions

!2ponentx 2 log b a is the exponent for a 2 bx!

Natural log

ln a 2 log e a;inary loglg a 2 log>a

<quare of loglg>a 2 (lg a)>

(og of (oglg lg a 2 lg (lg a)



:tandard 6ogarithms 7otations

alogclog

a

b

b b

c

c b

bn

b

log

b b ca

blogAalog

alog(ADa)log

blogalogalog

anlogalog

loglog)(log

=

=

−=

=

=

+=

=

1aa11a

ccc

a1

r asymptotic analysis11

Documents