CS2210DiscreteStructuresAlgorithmsandComplexity

Fall2017SukumarGhosh

Whatisanalgorithm

Afiniteset(orsequence)ofpreciseinstructions

forperformingacomputation.

Example:Maximafindingproceduremax (a1,a2,…,an:integers)max :=a1fori :=2 ton

ifmax <ai thenmax :=aireturnmax {thelargestelement}

Flowchartformaximafinding

max:=a1

i:=2

max<ai

max=ai

i =n?

i:=i +1

start

yes

no

end

yes

no

Givenn elements,canyoucountthetotalnumberofoperations?

TimecomplexityofalgorithmsCountsthelargestnumberofbasic operations requiredtoexecuteanalgorithm.

Example:Maximafindingproceduremax (a1,a2,…,an:integers)max :=a1 1operationfori :=2 ton 1operationi:=2

ifmax <a1 thenmax :=ai {n-1times}{2ops+1optocheckifi >n +1optoincrementi}

returnmax {thelargestelement}

Thetotalnumberofoperationsis4(n-1)+2=4n-2

TimecomplexityofalgorithmsExampleoflinearsearch(Searchx inalist )

k :=1 {1op}whilek ≤n do {n opsk ≤n}

ifx =ak thenfoundelsek:=k+1} {2nops+1op}

Themaximum numberofoperationsis3n+2.Ifwearelucky,thensearchcanendeveninthefirstiteration.

Timecomplexityofalgorithms

{searchfailed}

Binarysearch(Searchx inasortedlist )

Howmanyoperations?Roughlylogn.Why?

BubbleSortprocedurebubblesort (A:listofitems)n =length(A)repeat

fori =1ton-1doifA[i-1]>A[i]thenswap(A[i-1],A[i])endif

endforn:=n - 1

until n=0endprocedure

BubbleSort

BubbleSort

Theworstcasetimecomplexityis(n-1)+(n-2)+(n-3)+…+2=n(n-1)/2-1

TheBig-OnotationItisameasureofthegrowthoffunctionsandoftenusedtomeasurethecomplexityofalgorithms.

DEF.Letf andg befunctionsfromthesetofintegers(orrealnumbers)tothesetofrealnumbers.Thenf isO(g(x))ifthereareconstantsC andk,suchthat

|f(x)|≤C|g(x)| forallx >k

Intuitively,f(x)grows“slowerthan”somemultipleofg(x)asx growswithoutbound.ThusO(g(x))definesanupperboundoff(x).

TheBig-Onotation

1 2

1

2

3

4y =x2

y=4x2

Definesanupperboundofthegrowthoffunctions

TheBig-Ω (omega)notation

DEF.Letf andg befunctionsfromthesetofintegers(orrealnumbers)tothesetofrealnumbers.Thenf isΩ(g(x))ifthereareconstantsCandk,suchthat

|f(x)|≥ C|g(x)| forallx >k

Example.7x2 +9x+4isΩ(x2),since7x2 +9x+4≥1.x2 forallxThus Ω definesthelowerboundofthegrowthofafunction

Question.Is7x2 +9x+4Ω(x)?

TheBig-Theta(Θ)notationDEF.Letf andg befunctionsfromthesetofintegers(orrealnumbers)tothesetofrealnumbers.Thenf isΘ(g(x))ifthereareconstantsC1 andC2 apositiverealnumberk,suchthat

C1.|g(x)|≤|f(x)|≤C2.|g(x)| forallx >k

Example. 7x2 +9x+4isΘ(x2),since1.x2 ≤7x2 +9x+4≤8.x2 forallx >10

AveragecaseperformanceEXAMPLE.Computetheaveragecasecomplexityofthelinearsearch algorithm.

Classificationofcomplexity

Complexity TerminologyΘ(1) ConstantcomplexityΘ(log n)Θ(log n)c

LogarithmiccomplexityPoly-logarithmiccomplexity

Θ(n) LinearcomplexityΘ(nc) PolynomialcomplexityΘ(bn)(b>1) ExponentialcomplexityΘ(n!) Factorialcomplexity

WealsousesuchtermswhenΘ isreplacedbyO(big-O)

ExerciseComplexityofn5 O(2n) Trueorfalse?Complexityof2n O(n5) Trueorfalse?Complexityoflog(n!) Θ(n logn) Trueorfalse?Complexityof12+22+32+…+n2 Ω(n3) Trueorfalse?”

LetS={0,1,2,…,n}.ThinkofanalgorithmthatgeneratesallthesubsetsofthreeelementsfromS,andcomputeitscomplexityinbig-Onotation.

GreedyAlgorithms

Inoptimizationproblems,manyalgorithmsthatusethebest

choiceateachsteparecalledgreedyalgorithms.

Example.Deviseanalgorithmformakingchangeforn cents

usingquarters,dimes,nickels,andpenniesusingtheleast

numberoftotalcoins?

GreedyChange-makingAlgorithm

Let bethedenominationofthecoins,

(and)

for i:=1tor

while n ≥ci

begin

addacoinofvalueci tothechange

n :=n- ci

end

Question.Isthisoptimal?Doesitusetheleastnumberofcoins?

Letthecoinsbe1,5,10,25cents.Formaking38cents,youwilluse

1quarter1dime3cents

Thetotalcountis5,anditisoptimum.

GreedyChange-makingAlgorithm

Butifyoudon’tuseanickel,andyoumakeachangefor

30centsusingthesamealgorithm,theyouwilluse1quarter

and5cents(total6coins).Buttheoptimumis3coins

(use3dimes!)

So,greedyalgorithmsproduceresults,buttheresults

maybesub-optimal.

GreedyRoutingAlgorithm

A B

IfyouneedtoreachpointBfrompointAinthefewestnumberofhops,

Thenwhichroutewillyoutake?Iftheknowledgeislocal,thenyouare

temptedtouseagreedyalgorithm,andreachBin5hops,although

itispossibletoreachBinonlytwohops.

C

Otherclassificationofproblems• Problemsthathavepolynomialworst-casecomplexityare

calledtractable.Otherwisetheyarecalledintractable.

• Problemsforwhichnosolutionexistsareknownasunsolvable problems(likethehaltingproblem).Otherwisetheyarecalledsolvable.

• Manysolvableproblemsarebelievedtohavethepropertythatnopolynomialtimesolutionexistsforthem,butasolution,ifknown,canbecheckedinpolynomialtime.ThesebelongtotheclassNP(asopposedtotheclassoftractableproblemsthatbelongtoclassP)

Estimationofcomplexity

Source:D.Harel.Algorithmics:TheSpiritofComputing.Addison-Wesley,2ndedition,1992

TheHaltingProblem

TheHaltingproblem asksthequestion.

Givenaprogramandaninputtotheprogram,determineiftheprogram

willeventuallystopwhenitisgiventhatinput.

• Runtheprogramwiththatinput.Iftheprogramstops,thenweknowit

stops.

• Butiftheprogramdoesn'tstopinareasonableamountoftime,thenwe

cannotconcludethatitwon'tstop. Maybewedidn'twaitlongenough!

Thequestionisnotdecidableingeneral!

TheTravelingSalesmanProblem

Startingfromanode,youhavetovisiteveryothernodeandreturnToyoustartingpoint.Findtheshortestroute?NP-complete

3-SatisfiabilityProblem

Doesthereexistanassignmentofvaluesofx,y,zsothatthisformulaistrue?NP-Completeproblem!

Consideranexpressionlikethis: