models of computation prepared by john reif, ph.d. distinguished professor of computer science duke...
TRANSCRIPT
![Page 1: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/1.jpg)
Models of Computation
Prepared by
John Reif, Ph.D.Distinguished Professor of Computer Science
Duke University
Analysis of AlgorithmsWeek 1, Lecture 2
![Page 2: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/2.jpg)
Models of Computation (RAM)
a) Random Access Machinesb) Straight Line Programs and Circuitsc) Vector Machinesd) Turing Machinese) Pointer Machinesf) Decision Treesg) Machines That Make Random Choices
![Page 3: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/3.jpg)
Readings
• Main Reading Selection:• CLR, Chapter 1 and 2
![Page 4: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/4.jpg)
Random Access Machine (RAM)
![Page 5: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/5.jpg)
RAM Assumptions
1) Each register holds an integer2) Program can’t modify itself3) Memory instructions involve simple
arithmetica) Addition, subtractionb) Multiplication, division and control
states (got, if-then, etc.)
![Page 6: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/6.jpg)
Simplified Program Style
![Page 7: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/7.jpg)
Complexity Measures of Algorithms
• TimeA (X) = time cost of Algorithm A, input X
• SpaceA (X) = space cost of Algorithm A, input X
• Note: “time” and “space” depend on machine
Input X size n=|X| => =>Algorithm A output
![Page 8: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/8.jpg)
Complexity Measures of Algorithms (cont’d)
• Worst casetime complexity
• Average case complexityfor random inputs
• Worst casespace complexity
• Average case complexityfor random inputs
A AT (n) = max (Time (X))
{x:|x|=n}
A AE(T (n))= Time (X)Prob(X)
{x:|x|=n}
A AS (n) = max (Space (X))
{x:|x|=n}
A AE(S (n))= Space (X)Prob(X)
{x:|x|=n}
![Page 9: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/9.jpg)
Cost Criteria
• Uniform Cost Criteria Time = # RAM instructionsspace = # RAM memory
registers
• Logarithmic Cost Criteria– Time= L(i) units per RAM instruction
on integer size i– Space = L(i) units per RAM register
on integer size i
![Page 10: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/10.jpg)
Cost Criteria (cont’d)
• Example
Uniform time cost = nLogarithmic time cost > 2n
log |i| i 0where L(i) =
1 i 0
n2
Z 2
k=1 n Z Z Z
ouput Z = 2
for to do
![Page 11: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/11.jpg)
Varieties of Computing Machine Models
RAMsStraight line programsCircuitsBit vectorsLisp machines …Turning Machines
![Page 12: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/12.jpg)
Straight Line Programs
• Idea• fix n = input size• unroll each iteration loop until result
is loop-free program Pn
• For each n > 0, get a distinct program Pn
Note: this is only possible if we can eliminate all branching and all indirect addressing
![Page 13: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/13.jpg)
Example
• Given polynomial
with constant coefficients a0, a1, …, an
• Horner’s Rule for Polynomial Evaluation
n n-1n n-1 1 0p(x) = a x + a x + ... + a x + a
n
i
X
Y a
i = n-1 -1 0 Y (Y X) + a
Y
input
for by to do
ouput
RAM program in 2n steps
![Page 14: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/14.jpg)
Example (cont’d)
![Page 15: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/15.jpg)
Straight Line
Programs
Circuits
(DAG) graph model for straight line programs
1-1 correspondence
![Page 16: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/16.jpg)
Boolean Circuits (for VLSI Design)
• Restrictions1) All memory registers have value 0 or 12) Use only logical operations
∧ ∨ ⊕ ¬“and” “or” “parity” “not”
![Page 17: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/17.jpg)
Vector Machines
• logical operations ∧, ∨, ⊕, ¬ applied to vector elements
• memory locations hold Boolean vectors
• may also allow vector shift operations
![Page 18: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/18.jpg)
Vector Machines (cont’d)
• Examplegraph G
![Page 19: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/19.jpg)
Decision Trees
![Page 20: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/20.jpg)
Decision Trees (cont’d)
• To sort n keys• Any decision tree must have
n! output leaves• (n! = # permutations of n keys)
hence height of tree is
2log (n!) c n log n
![Page 21: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/21.jpg)
Pointer Machines Based on LISP programming
language
![Page 22: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/22.jpg)
The Turing Machine (TM):The VW of Machines
• Invented by Turing (a Cambridge logician)−Built by British for WWII cryptography !
![Page 23: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/23.jpg)
The Turing Machine (cont’d)
T(n) = time cost = max steps of TM
S(n) = space cost = max cells written by TM on memory
tapes
![Page 24: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/24.jpg)
Reductions Between TM and RAM Models
1) Given TM time cost T(n) then ∃ equivalent RAM (obvious)
Time c T(n) if uniform costcost c T(n) (log n) if log cost
2) Given RAM time cost T(n) with logarithmic cost then
∃ equivalent TM with time cost c’ T(n)2
Time cost
![Page 25: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/25.jpg)
Reductions Between TM and RAM Models (cont’d)
• Do arithmetic by Grammar School Method
Read write
memory
Proof idearegisters
![Page 26: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/26.jpg)
Extensions of RAMS
• Reasonable:(0) Modifiable Program(1) Random Choices(2) Non-uniformity
• Not Reasonable:(3) Non-deterministic Choices
![Page 27: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/27.jpg)
RASP Machine
• Same as RAM but allow program to change itself
• Same power as RAM
• Proof idea (due to Von Neumann)• Let RAM use memory registers to
store modifiable program of RASP
![Page 28: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/28.jpg)
Randomized Machines
• Extend RAM instructions to include−r RANDOM(k) gives a random k
bit number−Let AR(x) denote randomized
algorithm with input x, random choices R
• Expected Time Complexity−T(n) = max Time (x)
{x|n = |x|}
input X
R
Expected TimeTime (X) Prob (R)
Time (X)
![Page 29: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/29.jpg)
PROBABILITY
TIME T(n)
![Page 30: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/30.jpg)
A Randomized Computation
![Page 31: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/31.jpg)
1) If machine outputs value v with prob > ½ then v is considered its output.
2) Machine accepts input X if outputs 1 with prob > ½
3) Has 1-sided error if when not accepting 1 outputs only 0.
![Page 32: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/32.jpg)
Non-Uniformity
• For each input size n, allow the program a distinct, finite, “advice tape” to read
• Note:• If advice tape length 2n can solve
any Boolean output problem with n Boolean inputs (obvious).
![Page 33: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/33.jpg)
Surprising Result (Adelman)
• Given any polynomial time randomized algorithm with 1 side error,
∃ a non-uniform deterministic algorithm with polynomial time and polynomial advice!
• Gives way of derandomizing a randomized algorithm
![Page 34: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/34.jpg)
Nondeterministic Machines
• Allow “nondeterministic choice” branches
• If any sequence of choices succeed to accept x, then computation accepts.
accept x reject x
![Page 35: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/35.jpg)
NP and P
• NP = languages accepted by polynomial time nondeterministic TM machines.• Includes many hard problems:
1) Traveling Salesman Problem2) Propositional Satisfiability3) Integer Programming
• P = languages accepted by polynomial time deterministic machines
Not known probably noP = NP
?
![Page 36: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/36.jpg)
Another Surprising Result (Levin)
• If P=NP but we don’t know the proof (i.e., the polynomial time algorithm for NP)−find an optimal algorithm to find the
solution of any solvable search problem, in polynomial time!
• Proof depends on assumption that there is a finite length program for NP search problems, running in poly time)
![Page 37: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/37.jpg)
Conclusion
1) There are many possible machine models
2) Most (but not Nondeterministic) are “constructable” – so might be used if we have efficient algorithms to execute on machines.
3) New machine models can help us invent new algorithms, and vice versa!
![Page 38: Models of Computation Prepared by John Reif, Ph.D. Distinguished Professor of Computer Science Duke University Analysis of Algorithms Week 1, Lecture 2](https://reader035.vdocuments.us/reader035/viewer/2022070306/5517fb9a550346c6568b4fd4/html5/thumbnails/38.jpg)
Models of Computation
Prepared by
John Reif, Ph.D.Distinguished Professor of Computer Science
Duke University
Analysis of AlgorithmsWeek 1, Lecture 2