umass lowell computer science 91.404 analysis of algorithms prof. karen daniels spring, 2001
DESCRIPTION
UMass Lowell Computer Science 91.404 Analysis of Algorithms Prof. Karen Daniels Spring, 2001. Final Review Mon. 5/14-Wed. 5/16. Overview of Next 2 Lectures. Review of some key course material Final Exam: Course Grade Logistics, Coverage, Format Handout for basis of 40% of test - PowerPoint PPT PresentationTRANSCRIPT
UMass Lowell Computer Science 91.404
Analysis of Algorithms Prof. Karen Daniels
Spring, 2001
UMass Lowell Computer Science 91.404
Analysis of Algorithms Prof. Karen Daniels
Spring, 2001
FinalFinal ReviewReview
Mon. 5/14-Wed. 5/16Mon. 5/14-Wed. 5/16
Overview of Next 2 LecturesOverview of Next 2 Lectures
Review of some key course materialReview of some key course material Final Exam: Final Exam:
Course GradeCourse Grade Logistics, Coverage, FormatLogistics, Coverage, Format Handout for basis of 40% of testHandout for basis of 40% of test
Course EvaluationsCourse Evaluations
Review of Key Course MaterialReview of Key Course Material
What’s It All About?What’s It All About?
AlgorithmAlgorithm:: steps for the computer to follow to steps for the computer to follow to
solve a problemsolve a problem Problem Solving GoalsProblem Solving Goals::
recognize structure of some common recognize structure of some common problemsproblems
understand important characteristics of understand important characteristics of algorithms to solve common problemsalgorithms to solve common problems
select appropriate algorithm & data select appropriate algorithm & data structures to solve a problemstructures to solve a problem
tailor existing algorithmstailor existing algorithms create new algorithmscreate new algorithms
Some Algorithm Application AreasSome Algorithm Application Areas
Computer GraphicsComputer Graphics
Geographic Geographic Information SystemsInformation Systems
RoboticsRobotics
BioinformaticsBioinformatics
AstrophysicsAstrophysicsMedical ImagingMedical Imaging
TelecommunicationsTelecommunications
Design
Apply
Analyze
Tools of the TradeTools of the Trade
Algorithm Design Patterns such as:Algorithm Design Patterns such as: binary searchbinary search divide-and-conquerdivide-and-conquer
Data Structures such as:Data Structures such as: trees, linked lists, hash tables, graphs trees, linked lists, hash tables, graphs
Theoretical Computer Science Theoretical Computer Science principles such as:principles such as: NP-completeness, hardnessNP-completeness, hardness
Growth of FunctionsGrowth of Functions
SummationsSummations
RecurrencesRecurrences
SetsSets
ProbabilityProbability
MATHMATHProofsProofs
Discrete Math Review Chapters 1-6
Discrete Math Review Chapters 1-6
Growth of Functions, Summations, Growth of Functions, Summations, Recurrences, Sets, Counting, ProbabilityRecurrences, Sets, Counting, Probability
TopicsTopics
Discrete Math Review Discrete Math Review : : Chapters 1-6Chapters 1-6 Solving Summations & RecurrencesSolving Summations & Recurrences Sets, Basic Tree & Graph conceptsSets, Basic Tree & Graph concepts Counting: Permutations/CombinationsCounting: Permutations/Combinations Probability: Basics, including Expectation of a Random VariableProbability: Basics, including Expectation of a Random Variable Proof Techniques: InductionProof Techniques: Induction
Basic Algorithm Analysis TechniquesBasic Algorithm Analysis Techniques: : Chapters 1-6Chapters 1-6 Asymptotic Growth of FunctionsAsymptotic Growth of Functions Types of Input: Best/Average/WorstTypes of Input: Best/Average/Worst Bounds on Algorithm vs. Bounds on ProblemBounds on Algorithm vs. Bounds on Problem Algorithmic Paradigms/Design Patterns: Divide-and-ConquerAlgorithmic Paradigms/Design Patterns: Divide-and-Conquer Analyze pseudocode running time to form summations &/or Analyze pseudocode running time to form summations &/or
recurrencesrecurrences
What are we measuring?What are we measuring?
Some Analysis Criteria:Some Analysis Criteria: ScopeScope
The problem itself? The problem itself? A particular algorithm that solves the problem?A particular algorithm that solves the problem?
““Dimension”Dimension” Time Complexity? Space Complexity?Time Complexity? Space Complexity?
Type of BoundType of Bound Upper? Lower? Both?Upper? Lower? Both?
Type of InputType of Input Best-Case? Average-Case? Worst-Case?Best-Case? Average-Case? Worst-Case?
Type of ImplementationType of Implementation Choice of Data StructureChoice of Data Structure
Function Order of GrowthFunction Order of Growth
O( ) upper boundO( ) upper bound
( ) lower bound( ) lower bound
( ) upper & lower bound( ) upper & lower bound
nn11 n lg(n) n lg(n) n lgn lg22(n) (n) 22nnnn55 lg(n)lg(n)lglg(n) lglg(n) nn22
know how to use asymptotic complexity notationknow how to use asymptotic complexity notation
to describe time or space complexityto describe time or space complexity
know how to order functions know how to order functions asymptoticallyasymptotically
(behavior as n becomes large)(behavior as n becomes large)
Types of Algorithmic InputTypes of Algorithmic Input
Best-Case InputBest-Case Input: of all possible algorithm inputs of size n, it : of all possible algorithm inputs of size n, it generates the “best” resultgenerates the “best” result
for Time Complexity: “best” is smallest running timefor Time Complexity: “best” is smallest running time
Best-Case Input Produces Best-Case Running TimeBest-Case Input Produces Best-Case Running Time provides a provides a lower bound on the algorithm’s asymptotic running timelower bound on the algorithm’s asymptotic running time
(subject to any implementation assumptions)(subject to any implementation assumptions)
for Space Complexity: “best” is smallest storagefor Space Complexity: “best” is smallest storage
Average-Case InputAverage-Case Input
Worst-Case InputWorst-Case Input
these are defined similarlythese are defined similarly
Best-Case Time <= Average-Case Time <= Worst-Case TimeBest-Case Time <= Average-Case Time <= Worst-Case Time
Bounding Algorithmic Time(using cases)
Bounding Algorithmic Time(using cases)
nn11 n lg(n) n lg(n) n lgn lg22(n) (n) 22nnnn55 lg(n)lg(n)lglg(n) lglg(n) nn22
T(n) = T(n) = (1)(1) T(n) = T(n) = (2(2nn))very loose bounds are not very useful!very loose bounds are not very useful!
Worst-CaseWorst-Case time of T(n) = time of T(n) = (2(2nn) tells us that worst-case inputs cause the algorithm to ) tells us that worst-case inputs cause the algorithm to take take at mostat most exponential time (i.e. exponential time is exponential time (i.e. exponential time is sufficientsufficient). ).
But, can the algorithm every really take exponential time? (i.e. is exponential timeBut, can the algorithm every really take exponential time? (i.e. is exponential time necessarynecessary?)?)
If, If, for arbitrary nfor arbitrary n, we find a worst-case input that forces the algorithm to use exponential , we find a worst-case input that forces the algorithm to use exponential time, then this time, then this tightenstightens the the lower boundlower bound on the on the worst-caseworst-case running time. If we can force running time. If we can force the lower and upper bounds on the worst-case time to match, then we can say that, for the lower and upper bounds on the worst-case time to match, then we can say that, for the worst-case running time, T(n) = the worst-case running time, T(n) = (2(2n n ) (i.e. we’ve found the ) (i.e. we’ve found the minimum upper boundminimum upper bound, , so the bound is so the bound is tight.) tight.)
Using “case” we can discuss lower and/or upper bounds on:Using “case” we can discuss lower and/or upper bounds on:best-case running timebest-case running time or or average-case running timeaverage-case running time or or worst-case running timeworst-case running time
Bounding Algorithmic Time(tightening bounds)
Bounding Algorithmic Time(tightening bounds)
nn11 n lg(n) n lg(n) n lgn lg22(n) (n) 22nnnn55 lg(n)lg(n)lglg(n) lglg(n) nn22
TTWW (n) = (n) = (2(2nn))
for example...for example...
Here we denote best-case time by THere we denote best-case time by TBB(n); worst-case time by T(n); worst-case time by TWW(n) (n)
TTBB(n) = (n) = (1)(1)1st attempt1st attempt 1st attempt1st attempt
TTBB (n) = (n) = (n)(n)
1st attempt1st attempt
2nd attempt2nd attempt
TTWW (n) = (n) = (n(n22))
TTBB(n) = (n) = (n)(n)2nd attempt2nd attempt
1st attempt1st attempt
TTWW(n) = (n) = (n(n22))
Algorithm BoundsAlgorithm Bounds
Explore the Explore the problem problem to gain intuition:to gain intuition: Describe it: What are the assumptions? Describe it: What are the assumptions? (model of computation, (model of computation,
etc...)etc...) Has it already been solved?Has it already been solved? Have similar problems been solved? (more on this later)Have similar problems been solved? (more on this later) What does best-case input look like?What does best-case input look like? What does worst-case input look like?What does worst-case input look like?
Establish worst-case Establish worst-case upper boundupper bound on the on the problemproblem using using an algorithman algorithm Design a (simple) algorithm and find an upper bound on its worst-case Design a (simple) algorithm and find an upper bound on its worst-case
asymptotic running time; this tells us problem asymptotic running time; this tells us problem can becan be solved in a certain solved in a certain amount of time. Algorithms taking more than this amount of time may amount of time. Algorithms taking more than this amount of time may exist, but won’t help us.exist, but won’t help us.
Establish worst-case Establish worst-case lower boundlower bound on the on the problemproblem Tighten each bound to form a worst-case “sandwich” Tighten each bound to form a worst-case “sandwich”
Approach
increasing worst-case asymptotic running time as a function of nincreasing worst-case asymptotic running time as a function of n
nn11 22nnnn22 nn33 nn44 nn55
Know the Difference!Know the Difference!
nn11 22nnnn55
worst-case boundsworst-case bounds on problemon problem
An inefficient algorithm for An inefficient algorithm for the problem might exist the problem might exist that takes this much time, that takes this much time, but would not help us.but would not help us.
No algorithm for the problem No algorithm for the problem exists that can solve it for exists that can solve it for worst-case inputs in less worst-case inputs in less than linear time .than linear time .
Strong BoundStrong Bound: This : This worst-case worst-case lowerlower bound bound on the problem holds for on the problem holds for everyevery algorithm that algorithm that solves the problem and solves the problem and abides by our problem’s abides by our problem’s assumptions.assumptions.
Weak BoundWeak Bound: This worst-case : This worst-case upperupper bound on the problem bound on the problem comes from just considering comes from just considering oneone algorithm. Other, less algorithm. Other, less efficient algorithms that solve efficient algorithms that solve this problem might exist, but this problem might exist, but we don’t care about them!we don’t care about them!
Both the upper and lower bounds Both the upper and lower bounds are probably are probably looseloose (i.e. probably (i.e. probably can be can be tightenedtightened later on). later on).
Master TheoremMaster Theorem
Master Theorem : Let with a > 1 and b > 1 .
Then : Case 1: If f(n) = O ( n (log
b a) - ) for some > o
then T ( n ) = ( n log
b a )
Case 2: If f (n) = (n log
b a )
then T ( n ) = (n log
b a * log n )
Case 3: If f ( n ) = (n (log
b a) + ) for some > o and if
a f( n/b) < c f ( n ) for some c < 1 , n > N0 then T ( n ) = ( f ( n ) )
)()()( nfb
naTnT
)()()( nfb
naTnT
Use ratio test to Use ratio test to distinguish distinguish between cases:between cases:
f(n)/ f(n)/ n log b
a
Look for Look for “polynomially “polynomially larger” dominance.larger” dominance.
SortingChapters 7-10
SortingChapters 7-10
Heapsort, Quicksort, LinearTime-Sorting, Heapsort, Quicksort, LinearTime-Sorting, MediansMedians
TopicsTopics
SortingSorting: : Chapters 7-10Chapters 7-10 Sorting Algorithms:Sorting Algorithms:
[Insertion & MergeSort from Chapters 1-6)], Heapsort, Quicksort, [Insertion & MergeSort from Chapters 1-6)], Heapsort, Quicksort, LinearTime-Sorting, MediansLinearTime-Sorting, Medians
Comparison-Based Sorting and its lower boundComparison-Based Sorting and its lower bound Breaking the lower bound using special assumptionsBreaking the lower bound using special assumptions Tradeoffs: Selecting an appropriate sort for a given situationTradeoffs: Selecting an appropriate sort for a given situation
Time vs. Space RequirementsTime vs. Space Requirements Comparison-Based vs. Non-Comparison-BasedComparison-Based vs. Non-Comparison-Based
Comparison-Based SortingComparison-Based Sorting
In algebraic decision tree model, comparison-based In algebraic decision tree model, comparison-based sorting of n items requiressorting of n items requires (n lg n) (n lg n) time.time.
HeapSortHeapSort
To breaking the lower bound and obtain linear To breaking the lower bound and obtain linear time, forego direct value comparisons and/or time, forego direct value comparisons and/or
make stronger assumptions about input. make stronger assumptions about input.
InsertionSortInsertionSort
MergeSortMergeSort
QuickSortQuickSort
(n lg n)(n lg n) (n(n22) )
BestCaseBestCase AverageCaseAverageCase WorstCaseWorstCaseTimeTime::AlgorithmAlgorithm::
(n lg n) (n lg n) (n (n lg n) lg n)
(n lg n) (n lg n) (n lg (n lg n) n)
(n lg n)(n lg n) (n lg n) (n lg n) (n(n22) )
Data StructuresChapters 11-14Data StructuresChapters 11-14
Stacks, Queues, LinkedLists, Trees, Stacks, Queues, LinkedLists, Trees, HashTables, Binary Search Trees, Balanced HashTables, Binary Search Trees, Balanced
TreesTrees
TopicsTopics
Data StructuresData Structures: : Chapters 11-14Chapters 11-14 Abstract Data Types: their properties/invariantsAbstract Data Types: their properties/invariants
Stacks, Queues, LinkedLists, (Heaps from Chapter 7), Trees, Stacks, Queues, LinkedLists, (Heaps from Chapter 7), Trees, HashTables, Binary Search Trees, Balanced (Red/Black) TreesHashTables, Binary Search Trees, Balanced (Red/Black) Trees
Implementation/Representation choices -> data structureImplementation/Representation choices -> data structure Dynamic Set Operations:Dynamic Set Operations:
Query [does not change the data structure]Query [does not change the data structure] Search, Minimum, Maximum, Predecessor, SuccessorSearch, Minimum, Maximum, Predecessor, Successor
Manipulate: [can change data structure]Manipulate: [can change data structure] Insert, DeleteInsert, Delete
Running Time & Space Requirements for Dynamic Set Running Time & Space Requirements for Dynamic Set Operations for each Data Structure Operations for each Data Structure
Tradeoffs: Selecting an appropriate data structure for a situationTradeoffs: Selecting an appropriate data structure for a situation Time vs. Space RequirementsTime vs. Space Requirements Representation choicesRepresentation choices Which operations are crucial?Which operations are crucial?
Advanced TechniquesChapters 16-17
Advanced TechniquesChapters 16-17
Dynamic Programming, Greedy Dynamic Programming, Greedy AlgorithmsAlgorithms
TopicsTopics
Advanced TechniquesAdvanced Techniques: : Chapters 16-17Chapters 16-17 Algorithmic Paradigms/Design Patterns: Algorithmic Paradigms/Design Patterns:
Divide-and-ConquerDivide-and-Conquer Dynamic ProgrammingDynamic Programming Greedy AlgorithmsGreedy Algorithms Brute-Force/NaiveBrute-Force/Naive
Using Dynamic Programming &/or Greedy Algorithms to solve Using Dynamic Programming &/or Greedy Algorithms to solve Optimization ProblemsOptimization Problems
Optimal SubstructureOptimal Substructure Greedy Choice Property: Locally optimal -> Globally optimalGreedy Choice Property: Locally optimal -> Globally optimal
Tradeoffs: Tradeoffs: Selecting an appropriate paradigm to solve a problemSelecting an appropriate paradigm to solve a problem Tackling a problem using a sequence of paradigms:Tackling a problem using a sequence of paradigms:
Brute-Force (high running time) then improve...Brute-Force (high running time) then improve...
Problem CharacteristicsProblem Characteristics
ModularModular
Independent piecesIndependent pieces
Divide-and-ConquerDivide-and-Conquer Dynamic ProgrammingDynamic Programming Greedy AlgorithmsGreedy Algorithms
ModularModular
OptimizationOptimization
Optimal substructure: Optimal substructure: optimal solution contains optimal solution contains optimal solutions to optimal solutions to subproblemssubproblems
Overlapping subproblemsOverlapping subproblems
ModularModular
OptimizationOptimization
Optimal substructure: Optimal substructure: optimal solution optimal solution contains optimal contains optimal solutions to solutions to subproblemssubproblems
Greedy choice property: Greedy choice property: locally optimal choices locally optimal choices
lead to globallead to global optimumoptimum
Graph AlgorithmsChapters 23-25
Graph AlgorithmsChapters 23-25
DFS/BFSTraversals, Topological Sort, DFS/BFSTraversals, Topological Sort,
MinimumSpanningTrees, Shortest PathsMinimumSpanningTrees, Shortest Paths
TopicsTopics
Graph AlgorithmsGraph Algorithms: : Chapters 23-25Chapters 23-25 Undirected, Directed GraphsUndirected, Directed Graphs Connected Components of an Undirected GraphConnected Components of an Undirected Graph Representations: Adjacency Matrix, Adjacency ListRepresentations: Adjacency Matrix, Adjacency List Traversals: DFS and BFSTraversals: DFS and BFS
Differences in approach: DFS: LIFO/stack vs. BFS:FIFO/queueDifferences in approach: DFS: LIFO/stack vs. BFS:FIFO/queue Forest of spanning treesForest of spanning trees Vertex coloring, Edge classification: tree, back, forward, crossVertex coloring, Edge classification: tree, back, forward, cross Shortest paths (BFS)Shortest paths (BFS)
Topological SortTopological Sort Weighted GraphsWeighted Graphs MinimumSpanningTrees: 2 different approachesMinimumSpanningTrees: 2 different approaches Shortest Paths: Single source: Dijkstra’s algorithmShortest Paths: Single source: Dijkstra’s algorithm Tradeoffs:Tradeoffs:
Representation Choice: Adjacency Matrix vs. Adjacency ListRepresentation Choice: Adjacency Matrix vs. Adjacency List Traversal Choice: DFS or BFSTraversal Choice: DFS or BFS
Traversals: DFS, BFSTraversals: DFS, BFS
DFS backtracks DFS backtracks visit most recently visit most recently discovered vertex discovered vertex LIFO structure LIFO structure stack data structurestack data structure
BFS BFS vertices close to v are visited before vertices close to v are visited before those further away those further away FIFO structure FIFO structure queue data structurequeue data structure
FINAL EXAMFINAL EXAM
Logistics, Coverage, FormatLogistics, Coverage, Format
Handout for basis of 40% of testHandout for basis of 40% of test
Course GradingCourse Grading
Homework 40%Homework 40% Exam 1 15% (closed book)Exam 1 15% (closed book) Midterm 20% (open book)Midterm 20% (open book) Final Exam 25% (open book)Final Exam 25% (open book)
Results are scaled if necessary.
Check grade status with us before final!
Final Exam: LogisticsFinal Exam: Logistics
Friday, 12/18Friday, 12/18 Olsen 311: 8:00-11:00 a.m.Olsen 311: 8:00-11:00 a.m. Open book, open notesOpen book, open notes Closed computers, neighborsClosed computers, neighbors CumulativeCumulative Worth 25% of gradeWorth 25% of grade
Note change Note change from registrar’s from registrar’s room numberroom number
Text/Chapter/Topic CoverageText/Chapter/Topic Coverage
Discrete Math Review & Basic Algorithm Analysis Discrete Math Review & Basic Algorithm Analysis Techniques Techniques : : Chapters 1-6Chapters 1-6 Summations, Recurrences, Sets, Trees, Graph, Counting, ProbabilitySummations, Recurrences, Sets, Trees, Graph, Counting, Probability, ,
Growth of Functions, Divide-and-ConquerGrowth of Functions, Divide-and-Conquer SortingSorting: : Chapters 7-10Chapters 7-10
Heapsort, Quicksort, LinearTime-Sorting, MediansHeapsort, Quicksort, LinearTime-Sorting, Medians Data StructuresData Structures: : Chapters 11-14Chapters 11-14
Stacks, Queues, LinkedLists, Trees, HashTables, Binary Search Trees, Stacks, Queues, LinkedLists, Trees, HashTables, Binary Search Trees, Balanced (Red/Black) TreesBalanced (Red/Black) Trees
Advanced TechniquesAdvanced Techniques: : Chapters 16-17Chapters 16-17 Dynamic Programming, Greedy AlgorithmsDynamic Programming, Greedy Algorithms
Graph AlgorithmsGraph Algorithms: : Chapters 23-25Chapters 23-25 Traversal, MinimumSpanningTrees, Shortest PathsTraversal, MinimumSpanningTrees, Shortest Paths
FormatFormat
Mixture of questions of the following types:Mixture of questions of the following types:1) Multiple Choice 1) Multiple Choice
2) True/False2) True/False
3) Short Answer3) Short Answer
4) Analyze Pseudo-Code and/or Data Structure 4) Analyze Pseudo-Code and/or Data Structure
5) Solve a Problem by Designing an Algorithm5) Solve a Problem by Designing an Algorithm Select an appropriate paradigm/ design patternSelect an appropriate paradigm/ design pattern Select appropriate data structuresSelect appropriate data structures Write pseudo-codeWrite pseudo-code Justify correctnessJustify correctness Analyze asymptotic complexityAnalyze asymptotic complexity
60%60%
40%40%