Undecidability

Church-Turing Thesis

A Turing machine that halts on all inputs is the precise formal notion corresponding to the intuitive notion of an algorithm

An “algorithm” means a precisely defined set of instructions

This thesis cannot be formally proven See this link.

Consequence of Church-Turing thesis: If a problem cannot be solved by a Turing machine then it cannot be solved by a human using a precisely defined sequence of instructions, either

Knowing a problem is undecidable helps avoid wasted work

Can extend Church-Turing thesis to semidecidability (e.g. theorem proving)

Universal Turing machine

A UTM is a general purpose computer Can simulate any other Turing machine How simple can it be? U: Given description “T” of machine T and

“x” of input to T, simulates T on x and halts if and only if T halts on x.

Need to represent arbitrary TM in fixed alphabet; see this link.

Details of U are uninteresting. But try to understand how one or more UTM work.

Examples:– Hopcroft and Ullman 40 state UTM– A 4 state 6 symbol UTM– Minsky 7 state 4 symbol UTM– Wolfram’s Universal Turing Machines– Details for Hopcroft and Ullman machine

Unsolvability of Halting Problem

A Turing machine can be represented by a sequence of 4-tuples (state, symbol, new state, direction or symbol) or 5-tuples of the form (state, symbol, new state, direction, new symbol)

Thus a TM can be represented by a string of symbols

This string can be encoded in binary

Thus a TM can be represented as a binary string

Not all binary strings represent TM; if a binary string does not represent a TM we can say it represents a TM that always halts right away

A binary string can be interpreted as a binary integer j

Thus each binary integer j corresponds to a TM (call it Tj )

The integer j can also be represented in unary

This permits j to be read by a TM having only one symbol other than blank in its alphabet

Every TM can be represented as Tj for some j Let be the problem whose base set is the

positive integers, and such that the right answer for an integer j is “yes” if Tj does not halt on input j, “no” otherwise

Claim: is not partially decidable. Proof: By contradiction.

– Suppose is partially decidable. Then there is a Turing machine M that always gives the right answer and halts on input j if Tj does not halt on input j, and M loops otherwise.

– But then M must be Ti for some i.– Then for any j, M halts on input j if and only if Tj

does not halt on input j.– In particular, for j = i, this is true.– Thus M halts on input i if and only if Ti does not

halt on input i.– But M is Ti .– Thus Ti halts on input i if and only if Ti does not

halt on input i.– This is a contradiction.– Thus M does not exist and is not partially

decidable.

Note that self-reference is essential for this proof.

The problem occurs when Tj reads input j. Thus Tj is reading a description of itself. Self awareness is part of consciousness. Does this mean that consciousness is in

some way releted to undecidability?

Corollary: is not decidable either. Thus the halting problem is not decidable. The halting problem H: The base set is the

set of pairs (T,x) where T is a Turing machine and x is the input to T.

The right answer is “yes” if T halts on input x and “no” otherwise.

If H were decidable then would be too. So H is undecidable.

H is partially decidable (use a universal Turing machine).

H is not partially decidable (if it were then H would be decidable).

H is not decidable on any computing device.

Don’t waste time trying to solve it. Illustration: In a certain village the barber

shaves everyone who does not shave himself, and only those people who do not shave themselves.

Who shaves the barber?

Unsolvability of H results from TM running on their own descriptions. A kind of self-awareness. Is this significant?

Other Unsolvable Problems

Show other problems unsolvable by showing that if you could solve them, you could solve the halting problem.

Many problems are known to be unsolvable. Examples:

– Blank tape halting problem– Hilbert’s tenth problem– Theorem proving in first-order logic

But many of these unsolvable problems are partially decidable.

A proof that the blank tape halting problem is undecidable:

Let M have start state s, blank B, and these 5-tuples:– (s,0,t,0,R) (s,1,t,1,R) (s,B,y,B,R)– (t,0,s,0,R) (t,1,s,1,R) (t,B,n,B,R)

M tests if a binary string has even length. Let M’ have start state a, blank symbol B, all 5 tuples

of M, and these 5-tuples:– (a,B,b,0,R) (b,B,c,1,R) (c,B,d,1,L) (d,1,s,1,L)

Then M’ started on blank tape writes 011, moves left to the 0, and transfers control to M.

Unsolvability of BTHP

Thus M halts on input 011 if and only if M’ halts on blank tape.

If we could solve the blank tape halting problem then we could use this to tell whether M halts on input 011.

The same idea works in general: If we could solve BTHP we could solve the general halting problem.

Because the halting problem is unsolvable, BTHP is unsolvable.

Busy Beaver Problem (step counting version) BB(n) is the largest number of steps taken

by any n state TM with a binary alphabet, starting on blank tape, that eventually halts

Suppose there were 5 TM having 3 states. Suppose M1 took 3 steps and halted, M2 took 50 steps and halted, M3 looped, M4 took 7 steps and halted, and M5 looped. Then BB(3) = 50.

Now BB(n) is not computable and no computable function is always larger.

Proof: If we could compute BB(n) we could solve the halting problem. (How?)

But with a few extra bits for each n we could compute BB(n).– Just say which TM runs the longest and halts.

Thus with a few extra bits we could solve the halting problem. These bits will never be known unless by revelation.

BB(n) grows very fast, faster than any computable function.

The problem is classifying Turing computations that do not obviously halt or loop.

This will forever be impossible to do unless some computing device is more powerful than a Turing machine. The behavior of Turing machines is too subtle for us to fully understand.

See this link There is no easy way to distinguish the

beginning of a halting computation from the beginning of a looping computation.

If there were an easy way to look at the beginning of a computation and tell that it was going to halt, then one would quickly know that the other computations were looping.

If there were an easy way to look at the beginning of a computation and tell that it was going to loop, then one would quickly know that the other computations would halt.

Neither is possible. Halting and looping computations are very hard to distinguish.

Known bounds on BB(n)

The function Sigma(n) denotes the maximal number of tape marks which a Turing Machine (TM) with n internal states and a two-way infinite tape can produce onto an initially empty tape and then halt. The function S(n) denotes the maximal number of steps (shifts) which such a TM can do (it needs not produce many tape marks). The following table gives some known values:

n Sigma(n) S(n) 1 1 1 2 4 6 3 6 21 4 13 107 5 >= 4098 >= 47,176,870 6 > 1.29*10865 > 3*101730

Some researchers believe that Sigma(5) will never be known exactly. There are too many Turing machines to consider and their behavior is too subtle.

Following are transition tables and values of S and Sigma for a few Turing machines.

Try simulating them for a few steps to understand how they work!

The difficulty of understanding such machines helps us to see “why” the halting problem is unsolvable.

No matter how advanced we become, there will be infinitely many machines that are beyond our understanding.

We will not know whether they eventually halt. Our math is not strong enough to say.

The following table lists for each input state and symbol the output state, symbol and tape movement. The initial state is A, the halting state is H. The blank symbol is 0 and the tape is 2-way infinite.

No A0 A1 B0 B1 C0 C1 D0 D1 E0 E1 ones steps 1 B1L C1R C1L B1L D1L E0R A1R D1R H1L A0R 4098 47176870 2 B1L A1L C1R B1R A1L D1R A1L E1R H1R C0R 4098 11798826 3 B1L D0R C1R D1L A1R C1R H1L E1L A1L B0L 4097 23554764 4 B1L A1L C1R D1L A1R C1R H1L E0L C1R B1L 4097 11798796 5 B1L A1L C1R D0L A1R C1R H1L E1L E0R B1L 4096 11804910 6 B1L A1L C1R D0L A1R C1R H1L E1L C1R B1L 4096 11804896 7 B1L D1L C1R E0R A0L B0R E1L H1L C1R C1L 1471 2358064

Record 5-state Turing Machines

Record 6-state Turing machines

This is a list of 6-state TMs producing many ones, found by J.Buntrock and H.Marxen by scanning around 10% of all 6-state TMs with a program from June 2000 to October 2000. The blank symbol is 0 and the tape is 2-way infinite.

Name = a ones = 17485734 steps = 95547257425490 ones > 1.7*10^7 steps > 9.5*10^13 A: B1R C0L B: A1L A0R C: D0L Z1R D: E1R D1L E: F0L E0L F: F1R B0L

Name = b ones = 36109969 steps = 1137975066814256 ones > 3.6*10^7 steps > 1.1*10^15 A: B1R C0L B: A1L E0R C: D1L F0L D: B0L C1L E: B1R A1R

F: B1R Z1R

Name = c ones = 36109970 steps = 758650111948072 ones > 3.6*10^7 steps > 7.5*10^14 A: B1R C0L B: A1L E0R C: D1L F1L D: B0L C1L E: B1R A1R

F: C0L Z1R

Name = q ones > 6.4*10^462 steps > 6.1*10^925 A: B1R B0L B: C0R B1L C: D1R A0L D: E1L F1L E: A1L D0L F: Z1R E1L

Name = r ones > 1.2*10^865 steps > 3.0*10^1730 A: B1R F0L B: C0R D0R C: D1L E1R D: E0L D0L E: A0R C1R F: A1L Z1R

Simulation Results

Check these links for the results of simulations of machines 1,2,3,a,r:

Busy Beaver Simulation 1 Busy Beaver Simulation 2 Busy Beaver Simulation 3 Busy Beaver Simulation a Busy Beaver Simulation r (the record holder)

Chaitin’s Omega

A real number between 0 and 1 If we could compute it we could solve the

halting problem. Omega is “random.” Writing n bits of

omega requires an n bit program. Omega has no simple patterns.

See this link and this link.

= pP2-|p|

where P is the set of all halting Turing machines and |p| is the length in bits of the encoding of p.

Let Pn be the set of TM that halt in n steps or less.

n = pPn2-|p|

Then n approaches as n approaches .

Omega

0

0.05

0.1

0.15

0.2

0.25

0.3

Steps

Steps

Probability Omega

Omegan

OmegaP

n can be computed to any accuracy. To solve the halting problem using :

– If M halts in n steps then answer “yes.”– If - n < 2-|M| then answer “no.”– Do this for n = 1, 2, 3, … until an answer is

given.– Eventually either a “yes” or “no” answer will

be given. Thus will never be known. It is

unknowable unless it is given by revelation.

Partial solutions to the halting problem A loop tester is a Turing machine M that on

input i, halts if Ti loops on input i, and M loops otherwise.

A partial loop tester is a Turing machine M such that if M halts on input i, then Ti loops on input i.

No loop tester exists. But partial loop testers exist.

Improving partial loop testers

Suppose M is a partial loop tester. Let M be Tj. What does M do on input j?

If M halts on input j then Tj loops on input j. But Tj is M. Contradiction.

Therefore M loops on input j. But M does not “know” that it loops on input j.

M cannot solve this instance of the loop testing problem. But we can solve it.

Thus we excel every partial loop tester.

Let b(M) (“better” M) be M modified so that it halts on input j.

Thus b(M) knows that M halts on input j. It is easy to construct Turing machine b(M)

from M. b(M) is also a partial loop tester. But b(M) does not “know” that b(M) halts

on input k where b(M) is Tk. Consider b(b(M)). This machine knows

that b(M) halts on input k. Et cetera.

In this way we construct b(M), b2(M), b3(M), et cetera. Each halts more often than the previous one.

Let b(M) be a Turing machine that halts on input i if any of M, b(M), b2(M), et cetera halt on input i. This improves all bn(M).

Such a machine can be constructed by simulating M, b(M), b2(M), … in parallel.

Then one can improve b(M) to get b(b(M)), b(b(b(M))) et cetera.

Notation: b+1(M), b+2(M) … .

One obtains in the same way b 2 (M), b3(M), b4(M), …, and eventually b(M).

This can be written b2(M). Similarly one obtains b3(M) b4(M) b5(M)

… . This goes on for a long time! Related to the ordinals.

A paradox of consciousness

Suppose H is a Turing machine encoding human thought about looping of Turing machines.

Suppose H halts on input i if a human with lots of time can eventually deduce mathematically that Ti loops on input i. H loops otherwise.

Then H is a partial loop tester. Therefore H loops on input j where H is Tj.

Because H loops on input j, this means that humans cannot deduce that Tj loops on input j.

Tj is H, so humans cannot deduce that H loops on input j.

But we just showed that H loops on input j. We are humans. Contradiction.

Possibilities:– H is not a partial loop tester; human logic is

faulty.– Humans can never know H.– Human thought cannot be represented by a

Turing machine.

All possibilities are disturbing.– The first means that we can’t think correctly even

when reasoning mathematically.– The second means that we can never know how we

think.– The third means that our thought in some way

exceeds the capacity of a Turing machine. Note that not much human logic is needed to

reason that H loops on input j. Some people think that our consciousness, or

self-awareness, permits us to excel Turing machines.

Another possibility: H is a partial loop tester but we can never know that H is a partial loop tester.

This implies that human mathematical thought employs reasoning that human thought cannot justify.

What would be an example of this?