chapter 2 - mcgill universitycrypto.cs.mcgill.ca/~crepeau/comp610/02motivation.pdftypically takes 2n...
TRANSCRIPT
![Page 1: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/1.jpg)
1
Chapter 2 Basics of Algorithm Analysis
Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.
![Page 2: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/2.jpg)
2
As soon as an Analytic Engine exists, it will necessarily guide the future course of the science. Whenever any result is sought by its aid, the question will arise - By what course of calculation can these results be arrived at by the machine in the shortest time? - Charles Babbage (1791—1871)
2.1 Computational Tractability
![Page 3: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/3.jpg)
1991
![Page 4: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/4.jpg)
Computer Science Approach to problem solving
• If my boss / supervisor / teacher formulates a problem to be solved urgently, can I write a program to efficiently solve this problem ???
![Page 5: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/5.jpg)
Computer Science Approach to problem solving
• Are there some problems that cannot be solved at all ? and, are there problems that cannot be solved efficiently ??
![Page 6: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/6.jpg)
ALL PROBLEMS
COMPUTABLE
A few Computability Classes
![Page 7: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/7.jpg)
Alan Turing
In 1934, he formalized the notion of decidability of a language by a computer.
![Page 8: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/8.jpg)
Alonzo Church
In 1936, he proved that certain languages cannot be decided by any algorithm whatsoever...
![Page 9: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/9.jpg)
Emil Post
In 1946, he gave a very natural example of an undecidable language...
![Page 10: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/10.jpg)
(PCP) Post Correspondence Problem
An instance of PCP with 6 tiles.
A solution to PCP
aaabb
abb
bbba
aaa
bb
b
aaa
bbba
b
bb
bb
![Page 11: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/11.jpg)
Post Correspondence Problem
Given n tiles, u1/v1 ... un/vn where each ui or vi is a sequence of letters.
Is there a k and a sequence <i1,i2,i3,…,ik>( with each 1≤ij≤n ) such that
ui1 | ui2 | ui3 | ... | uik = vi1 | vi2 | vi3 | ... | vik ?
u1v1
u2v2
u3v3
unvn
. . .
![Page 12: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/12.jpg)
A Solution to Post Correspondence Problem
A solution is of this form:
u1v1
u2v2
u3v3
unvn
. . .
ui3vi3
ui4vi4
ui5vi5
uikvik
. . .ui1vi1
ui2vi2
with the top and bottom strings identical when we concatenate all the substrings.
![Page 13: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/13.jpg)
Post Correspondence ProblemTheorem:The Post Correspondence Problem cannot be decided by any algorithm (or computer program). In particular, no algorithm can identify in a finite amount of time some instances that have a No outcome. However, if a solution exists, we can always find it.
![Page 14: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/14.jpg)
Post Correspondence Problem
Proof:Reduction technique - if PCP was decidable then another undecidable problem would be decidable.
![Page 15: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/15.jpg)
The Halting ProblemNotice that an algorithm is a piece of text.
An algorithm can receive text as input.
An algorithm can receive the text description of an algorithm as input.
The Halting Problem: Given two texts A,B, consider A as an algorithm and B as an input. Will algorithm A halt (as opposed to loop forever) on input B?
![Page 16: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/16.jpg)
The Halting Problem and PCP
Any algorithm to decide PCP can be converted to an algorithm to decide the Halting Problem.
Conclusion: PCP cannot be decided either.
![Page 17: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/17.jpg)
ALL PROBLEMS
COMPUTABLE
A few Computability/Complexity Classes
P-Space
NPP
![Page 18: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/18.jpg)
Chapter 1 Introduction:Some Representative Problems
Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.
![Page 19: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/19.jpg)
1.2 Five Representative Problems
![Page 20: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/20.jpg)
20
Interval Scheduling
Input. Set of jobs with start times and finish times. Goal. Find maximum cardinality subset of mutually compatible jobs.
Time0 1 2 3 4 5 6 7 8 9 10 11
f
g
h
e
a
b
c
d
h
e
b
jobs don't overlap
![Page 21: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/21.jpg)
21
Weighted Interval Scheduling
Input. Set of jobs with start times, finish times, and weights. Goal. Find maximum weight subset of mutually compatible jobs.
Time0 1 2 3 4 5 6 7 8 9 10 11
20
11
16
13
23
12
20
26
![Page 22: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/22.jpg)
22
Bipartite Matching
Input. Bipartite graph. Goal. Find maximum cardinality matching.
C
1
5
2
A
E
3
B
D 4
![Page 23: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/23.jpg)
23
Independent Set
Input. Graph. Goal. Find maximum cardinality independent set.
6
2
5
1
7
34
6
5
1
4
subset of nodes such that no two joined by an edge
![Page 24: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/24.jpg)
24
Competitive Facility Location
Input. Graph with weight on each node. Game. Two competing players alternate in selecting nodes. Not allowed to select a node if any of its neighbours have been selected.
Goal. Select a maximum weight subset of nodes.
10 1 5 15 5 1 5 1 15 10
Second player can guarantee 20, but not 25.
X X X XX X
![Page 25: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/25.jpg)
25
Five Representative Problems
Variations on a theme: independent set.
Interval scheduling: n log n greedy algorithm.
Weighted interval scheduling: n log n dynamic programming algorithm.
Bipartite matching: nk max-flow based algorithm.
Independent set: NP-complete.
Competitive facility location: PSPACE-complete.
![Page 26: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/26.jpg)
complete
P-Space
NP
P
A few low order Complexity Classes
complete
Interval scheduling
Weighted interval scheduling
Bipartite matching
Competitive facility location
Independent set
= DeterministicPolynomial-Time
= non-DeterministicPolynomial-Time
= DeterministicPolynomial-Space
P = NP ? NP = P-Space ?
![Page 27: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/27.jpg)
Chapter 1 Introduction:Some Representative Problems
Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.
![Page 28: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/28.jpg)
Computer Science Approach to problem solving
• If my boss / supervisor / teacher formulates a problem to be solved urgently, can I write a program to efficiently solve this problem ???
![Page 29: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/29.jpg)
29
Polynomial-Time
Brute force. For many non-trivial problems, there is a natural brute force search algorithm that checks every possible solution. ■ Typically takes 2N time or worse for inputs of size N. ■ Unacceptable in practice.
Desirable scaling property. When the input size doubles, the algorithm should only slow down by some constant factor C.
Def. An algorithm is poly-time if the above scaling property holds.
Property: poly-time is invariant over all (non-quantum) computer models.
There exists constants a > 0 and d > 0 such that on every input of size N, its running time is bounded by a•Nd steps.
choose C = 2d
N ! for stable matchingwith N men and N women
![Page 30: Chapter 2 - McGill Universitycrypto.cs.mcgill.ca/~crepeau/COMP610/02Motivation.pdfTypically takes 2N time or worse for inputs of size N. Unacceptable in practice. Desirable scaling](https://reader035.vdocuments.us/reader035/viewer/2022081621/611d2dd419134d3dc73d99fb/html5/thumbnails/30.jpg)
51
Chapter 2 Basics of Algorithm Analysis
Slides by Kevin Wayne. Copyright © 2005 Pearson-Addison Wesley. All rights reserved.