2. analysis of algorithms
TRANSCRIPT
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 1/26
ANALYSIS OFALGORITHMS
Introducing Some Limited Concepts about Algoritms
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 2/26
Outline
!""icienc# o" Algoritms
!$amples
r er o ro%t
&orst' A(erage and )est Case Anal#sis
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 3/26
!""icienc# o" Algoritms T%o "undamental *uestions+
Ho% "ast %ill m# program run, -Time Comple$it#.
Ho% muc memor# it %ill ta/e,
o" operations per"ormed b# te algoritm
&e %ill assume tat all operations ta/e te same
amount o" time0
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 4/26
!$amplesFahrenheit to Celsius Converter (code)
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 5/26
!$amplesFahrenheit to Celsius Converter (analysis)
Tree aritmetic operationsOne subtraction' once multiplication' and one di(ision
'
generali1e and sa# %e a(e tree operations
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 6/26
!$amplesAdding Array Elements (code)
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 7/26
!$amplesAdding Array Elements (analysis)
For an arra# o" si1e N' o% man# operations are rougl# e$ecuted,
Statement Frequency of Execution
int sum 2 3 4
int i2 3 4 4
Te total number o" operations clearl# depends on input si1e
&e %ill not obsess about te small details-constant "actors.
i 5 list0lengt N
i66 N
sum2sum6list-i. N
Total 27N68
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 8/26
!$amplesAdding Square Matrices(code)
For an N N matri$' o% man# operations arerougl# e$ecuted,
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 9/26
!$amplesAdding Square Matrices(analysis)
Statement Frequency of Execution
dbouble9:9: sum000 4
int i2 3 4 4
i 5 list0lengt N
i66 N
int ; 2 3 N
;5 list0lengt N8
;66 N8
sum9i:9;: 2 a9i:9;: 6 b9i:9;: N8
Total 27N86 7N 6 8
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 10/26
Order o" Gro%tEffect of Problem Sie
Onl# large problem si1es are %ort stud#ing<
Modern computers are (er# "ast0
usuall# complete in under a minute
Our anal#sis o" algoritm "ocuses on o% an
algoritm scales %it problem si1es0
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 11/26
Order o" Gro%t!ro"th #ate of Functions
Consider+ " -n. 2 n8 6 433n 6 log -n. 6 4333
Let us consider o% tis "unction bea(es "or small
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 12/26
Order o" Gro%t!ro"th #ate of Functions(cont$d)
For te pre(ious "unction' " -n. 2 n8 6 433n 6 log -n. 64333' %e can see For large n' " -n. n8
Tis enables us to use te appro$imation+ " -n. n8' "or asu c en # arge n
Tis simpli"ication %or/ed despite te large coe""icient in "ront o"n
Generall#' "or most "unctions' lo%er order terms become lessrele(ant as te (alue o" n gets larger
Tis =as#mptotic= simpli"ication is (er# use"ul "or anal#1ingalgoritms Remember tat onl# large problems are %ort stud#ing
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 13/26
Order o" Gro%t%ig&' otation
Instead o" sa#ing an algoritm ta/es 7n8 6 >n 6 8steps' its simpler to lea(e out lo%er order terms suc
as >n and 8 and e(en te coe""icient 7 and sa te
gro%t rate is O-n8. -pronounced as big?o o" n8.
)ig?O is te most %idel# used as#mptotic notation in
algoritm anal#sis
&ill be used trougout te course
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 14/26
Order o" Gro%t)ig?O Notation ? Common Sense Approac
)ig?O enables us to speci"# o% an algoritm@s per"ormancescales %it input si1e
&e %ill discard lo%er order terms since te# are dominated na dominates nb i" a b+ "or e$ample n7 dominates n8
an# e$ponential dominates an# pol#nomial+ 7n dominates Bn
an# pol#nomial dominates an# logaritm+ n dominates -log -n..7
Multiplicati(e constants can be omitted+ 4>n8 becomes n8
Constant terms can be important-especiall# i" te# are large.0
)ut+ &e cannot easil# stud# algoritms %itout te simplicit#a""orded b# big?O notation0
Constant terms depend ea(il# on te ard%are arcitecture ? soits ard to come up %it an accurate number
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 15/26
Order o" Gro%t%ig&' otation & Formal efinition
enition Let T-n. and " -n. be "unctions "rom positi(e
20
-%ic means T-N. gro%s no "aster tan " -n.. i"
tere are positi(e constants c and n3 suc tat T-N.
2 c"-N. %en N n30
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 16/26
Order o" Gro%t!ro"th #ate Com*arisons
I" %e assume a gi(en operation ta/es onenanosecond-43?D. on some computer' %e can
construct te "ollo%in table
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 17/26
Order o" Gro%t+hat do "e mean by a fast algorithm,
&e cannot simpl# sa# a "ast algoritm is one tat runs*uic/l# %en gi(en a real input instance
Tis de"inition depends on te speed o" te computer ? a gor ms run as on some mac nes an s o% # on
oter macines
i""erent algoritms bea(e di""erentl# on di""erentinput si1es
A some%at standard denition %e %ill "ollo%
efinition
An algorithm if fast if it is growth rate is O(n) or less.
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 18/26
&orst' A(erage and )est CaseSequential Search (code)
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 19/26
&orst' A(erage and )est CaseSequential Search (analysis)
Our anal#sis depends on te input
)est Case+ O-4. I" te item being searced is "ound at te rst position
Tis appens onl# once in a %ile
ors ase+ I" te item being searced is "ound at te end o" te list'or NOT "ound at
all
Te is te upper bound
A(erage Case
Hard to determine anal#ticall# unless %e /no% te t#pe o" input %e areprocessing
Re"lects te actual situation
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 20/26
&orst' A(erage and )est Case
%inary Search ("hat is it,)
A searc algoritm tat %or/s onl# on sorted lists0 Similar to o% #ou %ould searc "or a %ord in te
A (er# %idel# used searc algoritm
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 21/26
&orst' A(erage and )est Case
%inary Search (the big idea)
9Image ta/en "rom+ Sed%ic/ and &a#ne' Algoritms' Fourt ed0:
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 22/26
&orst' A(erage and )est Case
)inar# Searc -code.
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 23/26
&orst' A(erage and )est Case
%inary Search (analysis)
Our anal#sis depends on te input
)est Case+ O-4. I" te item being searced is "ound e$actl# at te middle
Tis appens onl# once in a %ile
ors ase+ og n ? %e % scuss s s or # I" te item being searced is "ound at te end o" te list' or NOT "ound
at all
Te is te upper bound
A(erage Case
Hard to determine anal#ticall# unless %e /no% te t#pe o" input %e areprocessing
Re"lects te actual situation
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 24/26
&orst' A(erage and )est Case
%inary Search ("orst case analysis)
Te meaning o" log8-n. 2 m+ i" #ou di(ide n b# 8o(er and o(er' m times' #ou get 40
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 25/26
&orst' A(erage and )est Case
%inary Search ("orst case analysis) & cont$d
In binar# searc' %e /eep on reducing te searc domain b# a "actor o"
t%o
I" te initial input as N elements+
&e a(e NE8 items to be searced in second iteration' NE88 iterations in te
tird iteration' NE87 in te "ourt iteration000
In te %orst case' te searc goes on until one element is le"t in te list on' sa#'
te Mt iteration
)ased on our pre(ious obser(ation o" te logaritm' "or an input si1e o" N
and a total o" M iterations-a"ter %ic onl# one item is le"t.' %e a(e te
relation sip
8/9/2019 2. Analysis of Algorithms
http://slidepdf.com/reader/full/2-analysis-of-algorithms 26/26
&orst' A(erage and )est Case
+orst- Average- and %est Case
)est Case Considers cases tat are unli/el#0 Not used muc0
Indicates te lo%er bound on te gro%t0
A(erage Case+ ard to e(aluate' but (er# use"ul &orst Case+
er# use"ul since it tells us te upper bound on te gro%trate
Muc easier to calculate tan a(erage case ? %ill be used(er# o"ten0