theory of computation dr. adam p. anthony lectures 25 and 26
TRANSCRIPT
THEORY OF COMPUTATIONDr. Adam P. Anthony
Lectures 25 and 26
Overview
Computer Science: do we need computers?
Computation Theory Functions Turing Machines Universal Programming Languages The Halting problem
Computer Science and Computers
Computer science is no more about computers than astronomy is about telescopes.
Edgser W. Dijkstra
Computability
Insight: the computation is separate, in concept, from the computer
A computer, then, is just some object that can carry out the computation Humans
Brain, often supplemented by pencil, paper Charles Babbage
Difference engine, analytical engine Controlled using clockwork-type components
ENIAC Controlled using vacuum tubes
Intel 8080 Controlled using micro-transistors
Strength of Computers
Can a simple calculator help you find your way around cleveland?
How about a (dumb) phone? Aside from making calls
How about a smart phone? How about a laptop? How about a desktop? Which of these count as computers?
Specific-Purpose vs. General Computers
Some ‘computers’ are designed only to achieve a limited number of specific tasks, and to do that either at high speed or at a low cost: Digital Phones (Cell and otherwise)
Encryption chips too! Various scientific measuring devices
Others are considered General Purpose Computers Anything that can be computed, can be
done on one of these machines
Theory of Computation
The theory of computation aims to answer the following questions: 1. What is a general purpose computer?2. What problems can I solve with a general
purpose computer?3. Is this specific computer general purpose?4. Given a general purpose computer, how
difficult will it be to solve a specific problem?
Alan Turing: “The Father of Computer Science”
Successful mathematician Cryptographer Helped build some early
(classified) computation devices Many ideas predated the first
computers Turing Machine Computability
Helped define what is possible on a computer, and what is not
Computable Functions
A function is a mapping of inputs to outputs Sum(2+2) = 4 Feet-Centimeter(500) = 15,240 Sort([1,3,2,3,6,9,7,8,0]) = [0,1,2,3,3,6,7,8,9] Father(Bill Smith) = Edward Smith
Some functions are computable Given the input, an algorithmic process can
always be applied to get an exact answer for the output
A general purpose computer can compute any computable function, and no others
Turing Machines
Control Unit: The actual machine Tape: infinitely long memory Read/Write Head: Used to read information from the tape,
erase information, write new information Reads one character at a time Moves left/right one position at a time
State: Description of current situation, based on tape values
State = START
How a Turing Machine Works1. Each new Turing Machine has an alphabet
of characters that it understands, and a set of states that help it make decisions
2. Given the state the current character read by the the read/write head, and a program of execution, the Control Unit decides to:
a) Stop running (HALT state)b) Write over the current characterc) Move one space left/rightd) Change States
A More Complex Machine
Alphabet: {0,1,*} A single binary number is represented as
*101010* States:
ADD,RETURN,CARRY,OVERFLOW,HALT Program to increase a positive binary
integer by 1:
Useful Turing Machine Facts
Multiple Turing machines are no more powerful (though possibly faster) than a single Turing Machine
Any Turing Machine can ‘simulate’ another Turing Machine
Result: We can use unambiguous complex commands in the control unit’s program! Command: “Move 5 Spaces to the left”
Turing Machine reads: “Execute the Turing Machine routine that moves 5 spaces to the left”
Theoretically speaking, one should typically demonstrate the sub-program is computable first
Church-Turing Thesis
Any function that can be computed using a Turing Machine is also computable using any other general purpose computer (i.e., the function is computable)
SO WHAT???Who Cares?
Impact of Church-Turing Thesis1. If Power = ‘number of functions I can
compute,’ then a Turing machine is the most powerful computer imaginable Or, at least, it ties with any other computer It computes ALL of the computable functions!
2. If a Turing machine can’t solve a problem, then neither can a real computer, no matter how ‘powerful’ it is
Modern Computers As Turing Machines
Control Unit = Processor Alphabet = {0,1} States = Op Codes Read/Write Head = BUS Programs = Software Tape = RAM
Infinite????? No, but for most purposes it is long enough to solve the
problem Tape = External Storage
Only limited by the number of natural resources we can obtain from the entire universe (so, probably infinite!)
Bare-Bones Computer Language Programming languages usually market their
‘features’ Meant to make programming easier
Bare-Bones Language: Only includes features that are 100% necessary to be
equivalent to a Turing machine: Variable names: all variables are in binary clear statement: set a variable = 0 (clear X;) incr statement: increase a variable by 1 ( incr X; ) decr statement: decrease a variable by 1 (decr X;) While/end: continue execution until a variable = 0
while X not 0 do;….end;
Group Work!
Can you use Bare-Bones to: Set the variable Z = 4? Add X + Y = Z? (use one variable each for
X,Y,Z) Copy the value of X into Y?
About the Bare-Bones Language Computer scientists have proven that
any computer that can execute the Bare-Bones language is equivalent in power to a Turing Machine Heaven forbid!
Useful conclusion: Any programming language does at least
the same as Bare-Bones (hopefully more!) will also be Turing Equivalent
The extra features are just for convenience
Where We’ve Been
Computers are just tools for completing computations
Theory of computation: what is possible/impossible for all computers? What is computable?
Turing Machine: imaginary ‘all powerful’ computer Church-Turing thesis states no computer can do better
Modern computers are equivalent to Turing Machines
Any algorithm we implement on a computer is computable
Where We’re headed
It’d be nice to know, before we start if a problem is noncomputable Halting problem as an example
Even if a problem is computable, it would be nice to know in advance if it is easy or hard to solve
Even if we can solve a problem, it would be nice to know how long it will take to solve it Save effort in solving complex problems Take advantage of complexity
The Halting Problem
Some problems can’t be solved. Consider: Given the source code for any
computer program, can you analyze the code and decide if it will it ever stop running?
The Halting Problem
Does this program halt?
int X = 3while( X > 0)x = x -1
The Halting Problem
Does this program halt?
int X = 3repeatx = x +1;until x = 0
The Halting Problem
How about this program?virtual void estimate_sigmas(){
sigmas = std::vector< std::vector<Matrix> >(num_clusters);
for(int i = 0; i<num_clusters; i++){
sigmas[i] = std::vector<Matrix>(num_clusters);
}
for(int i = 0; i<num_clusters; i++){
for(int j = 0; j<num_clusters; j++){
sigmas[i][j] = zero_matrix<double>(proper_size,proper_size);
}
}
Vector temp_v(proper_size);
edge_iterator ebg,end;
int c_i,c_j;
for(tie(ebg,end) = edges(data); ebg!=end; ++ebg){
if(data[*ebg].type == edge_type && data[*ebg].exists){
c_i = data[source(*ebg,data)].clustering();
c_j = data[target(*ebg,data)].clustering();
temp_v = get_edge_vector(*ebg) - ic_means[c_i][c_j]
sigmas[c_i][c_j] = sigmas[c_i][c_j] + outer_prod(temp_v,trans(temp_v))/observed_edge_prob(c_i,c_j);
}
}
}
Computability Explained
Sometimes, we can work out answers for simple, example inputs of hard problems, but: What algorithm did you use to decide for the first
two programs? Can you generalize it to the third?
To prove something is not computable, we’ll use the following strategy: 1. Assume that there is an algorithm that can solve the
problem all the time2. Show that, regardless of how the algorithm works,
that there is at least one case where the algorithm will fail
1. Contradicts part 1, which claimed it ‘always works’
The Halting Problem Is Not Computable (PROOF!)
1. Assume, for the sake of contradiction, that there exists a computer program that, given any other computer program as input, can tell us if it stops:
STOPS(Program)
2. Make a new program: Opposite(X):
1. If STOPS(X), then run forever. 2. Otherwise, stop!
3. Opposite is a program, which itself accepts program code as input. What happens when we try to run
Opposite(Oppisite)?1. If STOPS(Opposite), then Opposite will run forever2. Otherwise, stop!
Halting Problem Implications Existence proof: Since there’s one program
that exists which we can’t compute if it halts, then there may be (probably are) others
If there’s one problem that seems computable, but is not, then there are others Look up Wang Tiles for an interesting example!
Program Analysis: “Does my program compute X?” Any place in the code where X is computed, add a
HALT command Changes to “Does my program ever halt?”
Algorithmic Complexity
Algorithmic Complexity refers to how many resources (time and memory) a computer will need to solve a problem How long will it take to process all the data? How much space (Memory) will we need? If we use more space, will it take less time? Are some problems harder to solve than
others? Can we figure that out before we try to solve
them? How can we take advantage of complexity?
Problems Vs. Solutions
It’s one thing to take a specific algorithm and say it’s complex (or not):
PROCEDURE Add(X,Y):sum = X + YRETURN sum
Because solving the problem, and doing so efficiently, are two different things:
PROCEDURE BadAdd(X,Y):Z = 1000000000sum = 0REPEAT Z = Z - 1UNTIL Z = 0sum = X + YRETURN sum
To say that a PROBLEM is difficult, you need to prove that there are no easy ways to solve it
Run-Time Complexity
Looked at briefly in chapter 5 Principal method for analyzing algorithm complexity:
How many steps does it take to complete the entire algorithm?
Steps are often based on the size of the input: PROCEDURE add-all(L):
sum = 0count = 0WHILE count < length(L):
sum = sum + L[count]RETURN sum
How many steps to add a list with 10 numbers? 1000 numbers?
Space Complexity
Many algorithms only need enough space to hold the input data Procedure add-all (L) Only needs enough
space to store L Others, because the problem is more
difficult, use supplementary data EX: Binary Search Trees
Still others use extra memory to be faster EX: Dynamic Programming Fibonacci
sequence
Complexity Classes
The most reoccurring algorithmic runtimes are (in order) constant, log(N), N, N*log(N), N2, N3, and an
A polynomial problem is any problem for which the best known algorithm for solving it has time complexity that is no worse than a polynomial function f(N) = Nd where d can be any number and N is the size of the input.
All problems that we can solve with an exact solution is a reasonable amount of time are in the polynomial class
Problems outside this class are referred to as intractible For short, we refer to the entire set of all
polynomial problems in the world a the set P
Complexity Classes, continued. A Nondeterministic machine: is a theoretical
machine that just knows how to solve a problem, no matter how hard it may be
A Nondeterministic Polynomial Problem is a problem for which the best known algorithm for solving has a polynomial runtime, but its execution would require a nondeterministic machine
Another intuition: these are problems for which finding the solution is hard, but checking the solution for correctness is easy
We refer to the set of problems in this domain as NP
NP DOES NOT MEAN “NOT POLYNOMIAL”
NP-Completeness
In General, the class of problems in NP consists of problems that are difficult, but useful Traveling Salesman example—best solution is exponential
Within a single class, some problems are harder than others In P, it is harder to sort a list than it is to add two numbers
In the class NP, we identify a set of problems that are the most difficult to solve in the entire set Called NP-Complete problems The speed at which we can solve these problems
determines how fast we can solve the lesser problems
P vs. NP
All problems that are in the set P must also be in the set NP Why?
Big, unknown question in Computer Science: DOES P = NP???
What does it mean if P = NP? One approach: Find a polynomial solution for
an NP-Complete problem Thousands have tried, all have failed
Most people believe, but can’t prove P NP