introduction to computer sciencebytes.usc.edu/files/cs109/fa17/slides/csci109... · ucharacterize...
TRANSCRIPT
![Page 1: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/1.jpg)
Introduction to Computer ScienceCSCI109
AndrewGoodneyFall2017
China– Tianhe-2
ReadingsSt.Amant,Ch.4,Ch.8
Lecture5:DataStructures&Algorithms9/25,2017
“An algorithm (pronounced AL-go-rith-um) is a procedure or formula forsolving a problem. The word derivesfrom the name of the mathematician,Mohammed ibn-Musa al-Khwarizmi,who was part of the royal court inBaghdad and who lived from about780 to 850.”
![Page 2: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/2.jpg)
Reminders
u Quiz2today(coverslecturematerialfrom1/30and2/6)u Nolecturenextweek(Feb20)duetoPresidents’day
u Quiz3onFeb27(coverslecturematerialfromtoday(2/13))u HW2dueon2/27
u Midtermon3/20
1
![Page 3: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/3.jpg)
Where are we?
2
Date Topic Assigned Due Quizzes/Midterm/Final
21-Aug IntroductionWhatiscomputing,howdidcomputerscometobe?
28-Aug ComputerarchitectureHowisamoderncomputerbuilt?Basicarchitectureandassembly
HW1
4-Sep Laborday
11-Sep DatastructuresWhyorganizedata?Basicstructuresfororganizingdata
HW1
12-Sep
18-Sep DatastructuresTrees,GraphsandTraversals
HW2Quiz1onmaterialtaughtinclass8/21-8/28
25-SepMoreAlgorithms/DataStructures Recursionandrun-time
2-OctComplexityandcombinatorics
How"long"doesittaketorunanalgorithm.
HW2Quiz2onmaterialtaughtinclass9/11-9/25
6-Oct
9-Oct Algorithmsandprogramming(Somewhat)Morecomplicatedalgorithmsandsimpleprogrammingconstructs
Quiz3onmaterialtaughtinclass10/2
16-Oct Operatingsystems WhatisanOS?Whydoyouneedone? HW3Quiz4onmaterialtaughtinclass10/9
23-Oct Midterm MidtermMidtermonallmaterialtaughtsofar.
30-Oct ComputernetworksHowarenetworksorganized?HowistheInternetorganized?
HW3
6-Nov ArtificialintelligenceWhatisAI?Search,plannningandaquickintroductiontomachinelearning
Quiz5onmaterialtaughtinclass10/30
10-Nov
13-Nov Thelimitsofcomputation Whatcan(andcan't)becomputed? HW4Quiz6onmaterialtaughtinclass11/6
20-Nov RoboticsRobotics:backgroundandmodernsystems(e.g.,self-drivingcars)
Quiz7onmaterialtaughtinclass11/13
27-Nov Summary,recap,review Summary,recap,reviewforfinal HW4Quiz8onmaterialtaughtinclass11/20
8-DecFinalonallmaterialcoveredinthesemester
Finalexam11am-1pminSAL101
LastdaytodropaMonday-onlyclasswithoutamarkof“W”andreceivearefundorchangetoPass/NoPassorAuditforSession001
Lastdaytodropacoursewithoutamarkof“W”onthetranscript
Lastdaytodropaclasswithamarkof“W”forSession001
![Page 4: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/4.jpg)
Data Structures and Algorithms
uAproblem-solvingviewofcomputersandcomputing
uOrganizinginformation:sequencesandtreesuOrganizinginformation:graphsuAbstractdatatypes:recursion
3
Reading:St.Amant Ch.4Ch.8(partial)
![Page 5: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/5.jpg)
Overview
4
CPU,Memory,Disk,I/OProgram
Problems
Solution:Algorithms +DataStructures
Pseudocode
Low-levelinstructions
Executionsmanagedby
Compileto
Low-levelinstructionsLow-levelinstructions
ProgramProgramProgram
OperatingSystem
![Page 6: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/6.jpg)
Sequences, Trees and Graphs
5
u Sequence:alistv Itemsarecalledelementsv Itemnumberiscalledtheindex
uGraph
u TreeEric
Emily Jane
Terry Bob
Jim
Mike Chris
Bob
![Page 7: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/7.jpg)
Recursion: abstract data types
uDefiningabstractdatatypesintermsofthemselves(e.g.,treescontaintrees)
u SoatreeisEitherasinglevertex,ora vertexthatistheparent
ofoneormoretrees
6
Eric
Emily Jane
Terry Bob
Drew Pam
Kim
![Page 8: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/8.jpg)
Recursion: algorithms
u Definingalgorithmsintermsofthemselves(e.g.,quicksort)Checkwhetherthesequencehasjustoneelement.Ifitdoes,stopCheckwhetherthesequencehastwoelements.Ifitdoes,andtheyareintherightorder,stop.Iftheyareinthewrongorder,swapthem,stop.Chooseapivotelementandrearrangethesequencetoputlower-valuedelementsononesideofthepivot,higher-valuedelementsontheothersideQuicksortthelowerelementsQuicksortthehigherelements
7
![Page 9: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/9.jpg)
Recursion: algorithms
uHowdoyouwriteaselectionsortrecursively?uHowdoyouwriteabreadth-firstsearchofatreerecursively?Whataboutadepth-firstsearch?
8
![Page 10: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/10.jpg)
Analysis of algorithms
uHowlongdoesanalgorithmtaketorun?timecomplexity
uHowmuchmemorydoesitneed?spacecomplexity
9
![Page 11: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/11.jpg)
Estimating running time
uHowtoestimatealgorithmrunningtime?vWriteaprogramthatimplementsthealgorithm,runit,andmeasurethetimeittakes
vAnalyzethealgorithm(independentofprogramminglanguageandtypeofcomputer)andcalculateinageneralwayhowmuchworkitdoestosolveaproblemofagivensize
uWhichisbetter?10
![Page 12: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/12.jpg)
Analysis of binary search
u n=8,thealgorithmtakes3stepsu n=32,thealgorithmtakes5stepsu Forageneraln,thealgorithmtakeslog2nsteps 11
![Page 13: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/13.jpg)
Growth rates of functions
uLinearuQuadraticuExponential
12
![Page 14: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/14.jpg)
Big O notation
u Characterizefunctionsaccordingtohowfasttheygrowu Thegrowthrateofafunctioniscalledtheorderofthefunction.(hencetheO)
u BigOnotationusuallyonlyprovidesanupperbound onthegrowthrateofthefunction
u Asymptoticgrowth
f(x)=O(g(x))asx->∞ifandonlyifthereexistsapositivenumberM suchthatf(x)≤M*g(x)forallx>x0
13
![Page 15: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/15.jpg)
Examples
u f(n)=3n2 +70v Wecanwritef(n)=O(n2)v WhatisavalueforM?
u f(n)=100n2 +70v Wecanwritef(n)=O(n2)v Why?
u f(n)=5n+3n5
u Wecanwritef(n)=O(n5)
u Why?
14
u f(n)=nlognv Wecanwritef(n)=O(nlogn)v Why?
u f(n)=πnnv Wecanwritef(n)=O(nn)v Why?
u f(n)=(logn)5 +n5
u Wecanwritef(n)=O(n5)
u Why?
![Page 16: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/16.jpg)
Examples
u f(n)=logan andg(n)=logbn arebothasymptoticallyO(logn)v Thebasedoesn’tmatterbecauselogan =logbn/logba
u f(n)=logan andg(n)=loga(nc)arebothasymptoticallyO(logn)v Why?
u f(n)=logan andg(n)=logb(nc)arebothasymptoticallyO(logn)v Why?
u Whataboutf(n)=2n and g(n)=3n ?v Aretheybothofthesameorder?
15
![Page 17: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/17.jpg)
Conventions
uO(1)denotesafunctionthatisaconstantv f(n)=3,g(n)=100000,h(n)=4.7areallsaidtobeO(1)
u Forafunctionf(n)=n2 itwouldbeperfectlycorrecttocallitO(n2)or O(n3)(orforthatmatterO(n100))
uHoweverbyconventionwecallitbythesmallestordernamely O(n2)
16
![Page 18: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/18.jpg)
Complexity
u (Binary)searchofasortedlist:O(log2n)u Selectionsort
u Quicksortu Breadthfirsttraversalofatree
u Depthfirsttraversalofatree
u Prim’salgorithmtofindtheMSTofagraphu Kruskal’s algorithmtofindtheMSTofagraph
u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes
17
![Page 19: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/19.jpg)
Selection sort
u Puttingthesmallestelementinplacerequiresscanningallnelementsinthelist(andn-1comparisons)
u Puttingthesecondsmallestelementinplacerequiresscanningn-1elementsinthelist(andn-2comparisons)
u …u Totalnumberofcomparisonsis
v (n-1)+(n-2)+(n-3)+…+1v n(n-1)/2v O(n2)
u Thereisnodifferencebetweenthebestcase,worstcaseandaveragecase
18
![Page 20: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/20.jpg)
Quicksort
u Bestcase:v Assumeanidealpivotv TheaveragedepthisO(logn)v Eachlevelofprocessesatmostn elementsv Thetotalamountofworkdoneonaverageistheproduct,O(n logn)
u Worstcase:v Eachtimethepivotsplitsthelistintooneelementandtherestv So,(n-1)+(n-2)+(n-3)+…(1)v O(n2)
u Averagecase:v O(n logn)[butprovingitisabitbeyondCS109]
19
![Page 21: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/21.jpg)
BF and DF traversals of a tree
u Abreadthfirsttraversalvisitstheverticesofatreelevelbylevel
u Adepthfirsttraversalvisittheverticesofatreebygoingdeepdownonebranchandexhaustingitbeforepoppinguptovisitanotherbranch
uWhatdotheyhaveincommon?
20
![Page 22: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/22.jpg)
BF and DF traversals of a tree
u Abreadthfirsttraversalvisitstheverticesofatreelevelbylevel
u Adepthfirsttraversalvisittheverticesofatreebygoingdeepdownonebranchandexhaustingitbeforepoppinguptovisitanotherbranch
uWhatdotheyhaveincommon?u Bothvisitall theverticesofatreeu IfatreehasV vertices,thenbothBFandDFareO(V)
21
![Page 23: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/23.jpg)
Prim’s algorithm
u Initializeatreewithasinglevertex,chosenarbitrarilyfromthegraph
u Growthetreebyaddingonevertex.Dothisbyaddingtheminimum-weightedgechosenfromtheedgesthatconnectthetreetoverticesnotyetinthetree
u Repeatuntilallverticesareinthetree
u Howfastitgoesdependsonhowyoustoretheverticesofthegraph
u Ifyoudon’tkeeptheverticesofthegraphinsomereadilysortedorderthenthecomplexityisO(V2)wherethegraphhasVvertices
22
![Page 24: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/24.jpg)
Kruskal’s algorithm
u Initializeatreewithasingleedgeoflowestweightu Addedgesinincreasingorderofweight
u Ifanedgecausesacycle,skipitandmoveontothenexthighestweightedge
u Repeatuntilalledgeshavebeenconsideredu Evenwithoutmuchthoughtonhowtheedgesarestored(aslongaswesortthemonceinthebeginning),thecomplexityisO(ElogE)wherethegraphhasEedges
23
![Page 25: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/25.jpg)
Dijkstra’s algorithm
u Ateachiterationwerefinethedistanceestimatethroughanewvertexwe’recurrentlyconsidering
u InagraphwithV vertices,alooseboundisO(V2)
24
![Page 26: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/26.jpg)
Reminders
u Homework#2duetodayu Quiz#2laterinlecture
25
![Page 27: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/27.jpg)
Where are we?
26
Date Topic Assigned Due Quizzes/Midterm/Final
21-Aug IntroductionWhatiscomputing,howdidcomputerscometobe?
28-Aug ComputerarchitectureHowisamoderncomputerbuilt?Basicarchitectureandassembly
HW1
4-Sep Laborday
11-Sep DatastructuresWhyorganizedata?Basicstructuresfororganizingdata
HW1
12-Sep
18-Sep DatastructuresTrees,GraphsandTraversals
HW2Quiz1onmaterialtaughtinclass8/21-8/28
25-SepMoreAlgorithms/DataStructures Recursionandrun-time
2-OctComplexityandcombinatorics
How"long"doesittaketorunanalgorithm.
HW2Quiz2onmaterialtaughtinclass9/11-9/25
6-Oct
9-Oct Algorithmsandprogramming(Somewhat)Morecomplicatedalgorithmsandsimpleprogrammingconstructs
Quiz3onmaterialtaughtinclass10/2
16-Oct Operatingsystems WhatisanOS?Whydoyouneedone? HW3Quiz4onmaterialtaughtinclass10/9
23-Oct Midterm MidtermMidtermonallmaterialtaughtsofar.
30-Oct ComputernetworksHowarenetworksorganized?HowistheInternetorganized?
HW3
6-Nov ArtificialintelligenceWhatisAI?Search,plannningandaquickintroductiontomachinelearning
Quiz5onmaterialtaughtinclass10/30
10-Nov
13-Nov Thelimitsofcomputation Whatcan(andcan't)becomputed? HW4Quiz6onmaterialtaughtinclass11/6
20-Nov RoboticsRobotics:backgroundandmodernsystems(e.g.,self-drivingcars)
Quiz7onmaterialtaughtinclass11/13
27-Nov Summary,recap,review Summary,recap,reviewforfinal HW4Quiz8onmaterialtaughtinclass11/20
8-DecFinalonallmaterialcoveredinthesemester
Finalexam11am-1pminSAL101
LastdaytodropaMonday-onlyclasswithoutamarkof“W”andreceivearefundorchangetoPass/NoPassorAuditforSession001
Lastdaytodropacoursewithoutamarkof“W”onthetranscript
Lastdaytodropaclasswithamarkof“W”forSession001
![Page 28: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/28.jpg)
Recap
u (Binary)searchofasortedlist:O(log2n)u Selectionsort:O(n2)
u Quicksort:O(nlogn)u Breadthfirsttraversalofatree:O(V)
u Depthfirsttraversalofatree:O(V)
u Prim’salgorithmtofindtheMSTofagraph:O(V2)u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)
u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)
27
![Page 29: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/29.jpg)
What do they have in common?
u (Binary)searchofasortedlist:O(log2n)u Selectionsort:O(n2)
u Quicksort:O(nlogn)u Breadthfirsttraversalofatree:O(V)
u Depthfirsttraversalofatree:O(V)
u Prim’salgorithmtofindtheMSTofagraph:O(V2)u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)
u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)
28
![Page 30: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/30.jpg)
A knapsack problem
u Youhaveaknapsackthatcancarry20lbs
u Youhavebooksofvariousweights
u Isthereacollectionofbookswhoseweightaddsuptoexactly20lbs?
u Canyouenumerateallcollectionsofbooksthatare20lbs
29
Book Weight
Book 1 2
Book 2 3
Book 3 13
Book 4 7
Book 5 10
Book 6 6
![Page 31: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/31.jpg)
A knapsack problem
u Youhaveaknapsackthatcancarry20lbs
u Youhavebooksofvariousweights
u Isthereacollectionofbookswhoseweightaddsuptoexactly20lbs?
u Canyouenumerateallcollectionsofbooksthatare20lbs
30
Book Weight
Book 1 2
Book 2 3
Book 3 13
Book 4 7
Book 5 10
Book 6 6
![Page 32: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/32.jpg)
A knapsack problem
u Youhaveaknapsackthatcancarry20lbs
u Youhavebooksofvariousweights
u Isthereacollectionofbookswhoseweightaddsuptoexactly20lbs?
u Canyouenumerateallcollectionsofbooksthatare20lbs
31
Book Weight
Book 1 2
Book 2 3
Book 3 13
Book 4 7
Book 5 10
Book 6 6
![Page 33: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/33.jpg)
How many combinations are there?
32
#ofbooks
Combinations Combinations
0 {} 1
1 {2}{3}{13}{7}{10}{6} 6
2 {2,3} {2,13}{2,7}{2,10}{2,6}{3,13}{3,7}{3,10}{3,6}{13,7}{13,10}{13,6}{7,10}{7,6}{10,6}
15
3 {2,3,13} {2,13,7}{2,7,10}{2,10,6}{2,3,7}{2,3,10}{2,3,6}{2,13,10}{2,13,6}{2,7,6}{3,13,7}{3,13,10}{3,13,6}{3,7,10}{3,7,6}{3,10,6}{13,7,10}{13,10,6}{13,7,6}{7,10,6}
20
4 {2,3,13,7} {2,3,13,10}{2,3,13,6}{2,3,7,10}{2,3,7,6}{2,3,10,6}{2,13,7,10}{2,13,10,6}{2,13,7,6}{2,7,10,6}{3,13,7,10}{3,13,10,6}{3,13,7,6}{3,7,10,6}{13,7,10,6}
15
5 {2,3,13,7,10} {3,13,7,10,6}{13,7,10,6,2}{7,10,6,2,3}{10,6,2,3,13}{6,2,3,13,7} 6
6 {2,3,13,7,10,6} 1
TOTAL 64
![Page 34: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/34.jpg)
How many combinations are there?
33
#ofbooks
Combinations Combinations
0 {} 1
1 {2}{3}{13}{7}{10}{6} 6
2 {2,3} {2,13}{2,7}{2,10}{2,6}{3,13}{3,7}{3,10}{3,6}{13,7}{13,10}{13,6}{7,10}{7,6}{10,6}
15
3 {2,3,13} {2,13,7}{2,7,10}{2,10,6}{2,3,7}{2,3,10}{2,3,6}{2,13,10}{2,13,6}{2,7,6}{3,13,7}{3,13,10}{3,13,6}{3,7,10}{3,7,6}{3,10,6}{13,7,10}{13,10,6}{13,7,6}{7,10,6}
20
4 {2,3,13,7} {2,3,13,10}{2,3,13,6}{2,3,7,10}{2,3,7,6}{2,3,10,6}{2,13,7,10}{2,13,10,6}{2,13,7,6}{2,7,10,6}{3,13,7,10}{3,13,10,6}{3,13,7,6}{3,7,10,6}{13,7,10,6}
15
5 {2,3,13,7,10} {3,13,7,10,6}{13,7,10,6,2}{7,10,6,2,3}{10,6,2,3,13}{6,2,3,13,7} 6
6 {2,3,13,7,10,6} 1
TOTAL 64
![Page 35: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/35.jpg)
Subset sum problem
u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?
u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}
u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations
u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential– i.e.O(2NN)
34
![Page 36: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/36.jpg)
Big O notation
u Characterizefunctionsaccordingtohowfasttheygrowu Thegrowthrateofafunctioniscalledtheorderofthefunction.(hencetheO)
u BigOnotationusuallyonlyprovidesanupperbound onthegrowthrateofthefunction
u Asymptoticgrowth
f(x)=O(g(x))asx->∞ifandonlyifthereexistsapositivenumberM suchthatf(x)≤M*g(x)forallx>x0
35
![Page 37: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/37.jpg)
What do they have in common?
u (Binary)searchofasortedlist:O(log2n)u Selectionsort:O(n2)
u Quicksort:O(nlogn)u Breadthfirsttraversalofatree:O(V)
u Depthfirsttraversalofatree:O(V)
u Prim’salgorithmtofindtheMSTofagraph:O(V2)u Kruskal’s algorithmtofindtheMSTofagraph:O(ElogE)
u Dijkstra’s algorithmtofindtheshortestpathfromanodeinagraphtoallothernodes:O(V2)
36
![Page 38: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/38.jpg)
Subset sum problem
u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?
u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}
u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations
u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential– i.e.O(2NN)
37
![Page 39: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/39.jpg)
Travelling salesperson problem
u Givenalistofcitiesandthedistancesbetweeneachpairofcities,whatistheshortestpossibleroutethatvisitseachcityexactlyonceandreturnstotheorigincity?
u Givenagraphwhereedgesarelabeledwithdistancesbetweenvertices.Startataspecifiedvertex,visitallotherverticesexactlyonceandreturntothestartvertexinsuchawaythatsumoftheedgeweightsisminimized
u Therearen! routes(anumberontheorderofnn - muchbiggerthan2n)
u O(n!)
38
![Page 40: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/40.jpg)
Enumerating permutations
u Listallpermutations(i.e.allpossibleorderings)ofnnumbers
uWhatistheorderofanalgorithmthatcandothis?
39
![Page 41: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/41.jpg)
Enumerating permutations
u Listallpermutations(i.e.allpossibleorderings)ofnnumbers
uWhatistheorderofanalgorithmthatcandothis?uO(n!)
40
![Page 42: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/42.jpg)
Analysis of problems
u Studyofalgorithmsilluminatesthestudyofclasses ofproblems
u Ifapolynomialtimealgorithmexiststosolveaproblemthentheproblemiscalledtractable
u Ifaproblemcannotbesolvedbyapolynomialtimealgorithmthenitiscalledintractable
u Thisdividesproblemsinto#?ree groups:known polynomialtimealgorithmbutnotyetproventobeintractable
41
![Page 43: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/43.jpg)
Analysis of problems
u Studyofalgorithmsilluminatesthestudyofclasses ofproblems
u Ifapolynomialtimealgorithmexiststosolveaproblemthentheproblemiscalledtractable
u Ifaproblemcannotbesolvedbyapolynomialtimealgorithmthenitiscalledintractable
u Thisdividesproblemsintothreegroups:v Problemswithknownpolynomialtimealgorithmsv Problemsthatareproventohavenopolynomial-timealgorithmv Problemswithnoknownpolynomialtimealgorithmbutnotyetproventobeintractable
42
![Page 44: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/44.jpg)
Tractable and Intractable
u Tractableproblems(P)v Sortingalistv Searchinganunorderedlistv Findingaminimumspanningtree
inagraph
43
u Intractablev Listingallpermutations(all
possibleorderings)ofnnumbers
u Mightbe(in)tractablev Subsetsum:givenasetof
numbers,isthereasubsetthataddsuptoagivennumber?
v Travellingsalesperson:ncities,n!routes,findtheshortestroute
Theseproblemshavenoknownpolynomialtimesolution
Howevernoonehasbeenabletoprovethatsuchasolutiondoesnotexist
![Page 45: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/45.jpg)
Tractability and Intractability
u ‘Propertiesofproblems’(NOT ‘propertiesofalgorithms’)u Tractable:problemcanbesolvedbyapolynomialtimealgorithm(orsomethingmoreefficient)
u Intractable:problemcannotbesolvedbyapolynomialtimealgorithm(allsolutionsareproventobemoreinefficientthanpolynomialtime)
u Unknown:notknowniftheproblemistractableorintractable(noknownpolynomialtimesolution,noproofthatapolynomialtimesolutiondoesnotexist)
44
![Page 46: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/46.jpg)
Tractability and Intractability
u ‘Propertiesofproblems’(NOT ‘propertiesofalgorithms’)u Tractable:problemcanbesolvedbyapolynomialtimealgorithm(orsomethingmoreefficient)
u Intractable:problemcannotbesolvedbyapolynomialtimealgorithm(allsolutionsareproventobemoreinefficientthanpolynomialtime)
u Unknown:notknowniftheproblemistractableorintractable(noknownpolynomialtimesolution,noproofthatapolynomialtimesolutiondoesnotexist)
45
![Page 47: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/47.jpg)
Subset sum problem
u Givenasetofintegersandanintegers,doesanynon-emptysubsetsumtos?
u {1,4,67,-1,42,5,17}ands=24 Nou {4,3,17,12,10,20}ands =19 Yes {4,3,12}
u IfasethasN elements,ithas2N subsets.u CheckingthesumofeachsubsettakesamaximumofNoperations
u Tocheckallthesubsetstakes2NNoperationsu Someclevernesscanreducethisbyabit(2Nbecomes 2N/2,butallknownalgorithmsareexponential)
46
![Page 48: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/48.jpg)
Take away
u Somesimpleproblemsseemtobeveryhardtosolvebecauseofexponentialorfactorialrun-time
u Notsosimpleinpractice:
47
Problem NaïveSolution(s) Best?
Knapsack 2N 2N,pseudopolynomial
Subset-sum N2N 2N/2,pseudopolynomial
TravellingSalesman N! N22N
![Page 49: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/49.jpg)
P and NP
u P:setofproblemsthatcanbesolvedinpolynomialtime
u Considersubsetsumv Noknownpolynomialtimealgorithmv However,ifyougivemeasolutiontotheproblem,itiseasyformetocheckifthesolutioniscorrect– i.e.Icanwriteapolynomialtimealgorithmtocheckifagivensolutioniscorrect
uNP:setofproblemsforwhichasolutioncanbecheckedinpolynomialtime
48
Easytosolve
Easytocheck
![Page 50: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/50.jpg)
P=NP?
49
uAllproblemsinP arealsoinNPuArethereanyproblemsinNP thatarenotalsoinP?
uInotherwords,is
P =NP ?uCentralopenquestioninComputerScience
![Page 51: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/51.jpg)
P=NP?
u Whydowecare?u “Asidefrombeinganimportantproblemincomputationaltheory,aproofeitherwaywouldhaveprofoundimplicationsformathematics,cryptography,algorithmresearch, artificialintelligence, gametheory,multimediaprocessing, philosophy, economics andmanyotherfields.”
50
![Page 52: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/52.jpg)
Data Structures and Algorithms
uAproblem-solvingviewofcomputersandcomputing
uOrganizinginformation:sequencesandtreesuOrganizinginformation:graphsuAbstractdatatypes:recursion
51
Reading:St.Amant Ch.4Ch.8(partially)
![Page 53: Introduction to Computer Sciencebytes.usc.edu/files/cs109/fa17/slides/CSCI109... · uCharacterize functions according to how fast they grow uThe growth rate of a function is called](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e84448f4410b14d4a2c7860/html5/thumbnails/53.jpg)
Overview
52
CPU,Memory,Disk,I/OProgram
Problems
Solution:Algorithms +DataStructures
Pseudocode
Low-levelinstructions
Executionsmanagedby
Compileto
Low-levelinstructionsLow-levelinstructions
ProgramProgramProgram
OperatingSystem