THEORY & PRACTICE | ' I/,! A a JULifL _ .̂, ■ I I \ Mlir f - 'JUII - lT^ ifl l lfl IWhj l Qi■< - . . - .■ :■■■■ .■■■ £ , * # f , ■ a - W i - ■ ■ I " i J T 7 -

^■O B ^ H B I B w H l H E ^ f t f e t J t E J S S S

Of course, unless one has a theory, onecannot expect much help from a computer(unless it has a theory). . . .

—Marvin Minsky

insky wrotethese words inconnection with

a famous little stinker of a problem, onethat has resisted both theorizing and computation for some 65 years.

The problem goes like this. Take anystring of Os and Is at least three digitslong. Examine the first digit. If it is a 0,delete the first three digits and append 00to the end ofthe string; if it is a 1, deletethe first three digits and add 1101 to theend. Now repeat the procedure with theresulting string, and then with the resultof that transformation, and so on. Thequestion is: what is the ultimate fate ofthestring? Does it grow infinitely long ordwindle away to nothing, or does it justmeander forever between these extremes?

Consider the starting pattern 1000100.Since the first digit is a 1, we delete thethree leftmost digits and add 1101, yielding01001101. Now the leading digit is a 0, andso we delete three digits and append 00;the result is 0110100. In this case it is easyto see what the outcome will be: in just afew more steps the string shrinks to 000,and then it falls below the minimumlength of three digits and disappearsentirely.

The starting string 1001 provides anexample that comes to a different end. Insix applications ofthe rules it yields thefollowing sequence of strings:

11101011101

10100001101

10100001101

The last two patterns will now berepeated in an unending cycle and so thestring never grows any longer than sixdigits or any shorter than five. A thirdexample, which runs for 16 steps before it

Tag—you're itBy Brian Hayesenters a repetitive cycle, is worked out inFigure 1.

Still more patterns could be traced inthe same way. No amount of such case-by-case analysis, however, can settle the general question of what happens to all possible strings. Indeed, the case-by-casemethod cannot even be guaranteed todetermine the fate of a single string. If thestring eventually dwindles away or entersa repeating cycle, then of course theanswer will be found. If the string growswithout limit, however, we will neverknow it because the calculation will neverend. What we need to solve the problem isa decision algorithm: a procedure that willdecide the fate of any string and that iscertain to terminate after a finite numberof steps when given any valid input.

This little problem in string manipulation was first explored by Emil L. Postin 1921, when he was a doctoral candidatein mathematics at Princeton University. Acolleague suggested naming the problem"tag" because the two ends ofthe stringseemed to chase each other in a way thatreminded him ofthe children's game. Anumber of other investigators (includingMinsky) have worked on the problemsince then, and in recent years they havebrought the power ofthe computer to bearon it.

Nevertheless, we live in a state ofalmost perfect ignorance about the natureof Post's tag system. No one has solvedthe problem by devising a decision algorithm, but no one has proved that it cannotbe solved. No one has found a string thatdoes not either dwindle away or enter acycle, but no one has shown that suchstrings do not exist.

I do not have a theory about tag systems, and neither does my computer. Allthe same, some interesting empiricalobservations can be made about the evolution ofthe systems. Maybe those observations will lead keener minds andmachines toward the development of atheory.

The fates of stringsA tag system can have just three possiblefates. First, the string can shrink to fewerthan three digits and vanish. This willhappen if at any point in the calculation allthe digits becomes 0s. Actually, it is noteven necessary for all the digits to be 0s; it

is sufficient for 0s to appear at positions 1,4, 7,10, and so on. It is only these positions (which I shall call the key positions)that are examined in applying the tagrules, and so they alone determine the fateof a string.

The second possibility is that the system will enter a repetitive cycle. This fateis inevitable if any one string appearsmore than once in the evolution ofthe system. It is easy to see why. Each stringuniquely determines its own successor;there can be no branching in the development of a tag system. Suppose string Agives rise to string B, which in turn produces C, D, and a series of furtherdescendants; if any one of these succes-

ring100101011011011101110111011110111010111011101101110100110100110110011011101110111011101111011101110101110111011101 I1011101110100110111010011011110100110111010100110111011101011011101110100 1.01110111010000 1.1 0 111 0 1 0 0 0 0 0 0 l ;11010000001101 1.100000011011101 1.0000110111011101 li011011101110100011101110100001 0 111 0 1 0 0 0 0 0 0 i11010000001101 1.100000011011101 1.0000110111011101 1<

Application of the tag rules to the stastring 10010 leads ultimately to a cycle Vr,a period of six. The 15-digit string producedat step 16 appears again at step 22, andhence the entire cycle of six strings will be

■*--Si "*:

sors generates string A again, then B, C,and all the rest must follow in an endlesscycle.

The third possibility, of course, is thatthe system will go merrily on for aninfinite number of steps. It is important torecognize that for this to happen, thestring must grow longer without limit.The proof is straightforward. Supposethere were some initial string that evolvedendlessly without dwindling away orcycling but also without ever growinglonger than 100 digits. There are only 2I(X)binary strings of 100 or fewer digits, soafter no more than 2100 steps at least onestring would have to be repeated. The system then inevitably enters a cycle, whichcontradicts the original assumption; thusno such initial pattern exists.

In analyzing a tag system it is temptingto try a probabilistic approach. Each timea 1 comes to the head ofthe string, thelength increases by one digit, and eachtime a 0 appears the length decreases byone. Thus if the digits at the key positionshave a uniform, random distribution, onemight expect the length to remain constanton the average.

This argument could be taken a stepfurther. The variations in the length ofthestring should describe a one-dimensionalrandom walk centered on the averagelength. Any one-dimensional randomwalk, if it is continued long enough, canbe expected to visit all the sites availableto it; in this case the string should at somepoint becomes arbitrarily short, dropbelow the three-digit threshold, and dwin

dle away. (The blind man, stumblingabout at random on top ofthe cliff, eventually falls over the precipice.) Evenbefore that happens, the system may lightupon a pattern that leads into a cycle.

The trouble with this analysis is that thepattern of 0s and Is is not random; on thecontrary, it is generated by a fully deterministic process. Moreover, even if mostofthe strings have the statistical properties of random patterns, there may well beexceptional strings that behave very differently. At best a probabilistic argumentcan predict the typical outcome, but whatis of greatest interest is the singular case.

The tag problem can be expressed as aTuring machine program. It is most conveniently formulated for a two-headedTuring machine, as shown in Figure 2.One head reads digits at the front ofthestring and instructs the other head to writenew digits at the rear. The question is thenwhether the two heads ever come togetheror whether they grow infinitely far apart.This formulation gives a clue to why theproblem is so hard. Predicting the outcome of all possible tag systems may beequivalent to the Turing machine haltingproblem, the granddaddy of all intractableproblems in computer science.

Another way to view tag systems is interms of formal languages. A languagecan be defined as a set of strings, whereeach string is made up of symbols drawnfrom a finite alphabet. Here the alphabetconsists ofthe two symbols 0 and 1, andthe tag transformation rules form a grammar defining which strings (out of all possible binary strings) are members ofthelanguage. The two grammar rules can beexpressed as:

0XX$ -* $001XX$ -> $1101

^^^^^^^^^^^^^^^^^^^^^^^^^H■*— Tape motion

IDDI1DDQDDH!/ R e a d h e a d W r i t e h e a d ^ / \

O n r e a d i n g 0 w r i t e 0 0 jOn reading 1 write 1101

S3

Finite-state control

Although Emil Post investigated the tag problem 15 years before Alan Turing inventuniversal computer, a tag system can be regarded as a Turing machine program. A rehead examines the first digit written on an infinite tape and instructs a write head to aceither 00 or 1101 after the last digit. The read head then advances by three digits. Thustag problem would be solved if one could solve the Turing machine halting problem (thgeneral question of whether a Turing machine halts when given an arbitrary input strinThe halting problem is known to be insoluble, but it has not been proved that the speci

3d his

the

i

In these rules X stands for any single symbol (that is, either 0 or I), and $ representsany string of symbols of arbitrary length,including the empty string. Hence the firstrule states that if a string consists of aOfollowed by any two symbols and the arbitrary substring $, then it can be transformed into $ followed by 00.

These rules differ in an important wayfrom the rules commonly used to specifyprogramming languages. The grammarsof virtually all programming languagesare said to be context-free, which means(among other things) that each grammarrule has only one symbol on its left side.In the tag-system grammar an arbitrarilylong string of symbols appears on the leftside of each rule. The grammar is a member ofthe most powerful and most complex class of grammars, the unrestrictedrecursive grammars.

How to play tagAlthough tracing individual tag systemscannot answer the deepest questions aboutthe nature ofthe systems, it is a usefulway to gather information about them. Acomputer, even one without theories, canbe of much assistance in this information-gathering. Many initial patterns go on forthousands of steps before settling into acycle or dwindling away, and it is not hardto find patterns that continue for millionsof steps. Following their progress by handwould be tedious at best.

The first step in writing a tag programis to choose a data structure for the stringof 0s and Is. The data structure must allowdigits to be deleted at the front ofthestring and added at the rear. The structureI chose is a circular queue, or ring. One ofits advantages is that when digits areadded or deleted, the remaining digitsnever have to be moved. Some Turbo Pascal procedures for manipulating the queueare shown in Listing 1.

The usual way of implementing a queueis to declare an array of memory cells andthen maintain pointers to the two cells thatrepresent the head and the tail ofthequeue. This works well enough, but mypersonal preference is for a slight variation: instead of a tail pointer, I prefer torecord the current length ofthe queue.When using head and tail pointers, specialcare must be taken to distinguish a fullqueue from an empty one, since both conditions might be signaled by having thehead and tail pointers point to the samecell. With a head pointer and a lengthvariable, the ambiguity cannot arise.

The routines EnQ and DeQ add andremove digits from the queue. One smallpeculiarity in their code is worth mentioning. Because a circular queue is beingimplemented by a linear array of memorycells, all calculations of position withinthe queue must be made modulo the queuesize. In other words, if the head pointer isat the last cell in the array and it is

Figure 2.

22 COMPUTER LANGUAGE ■ AUGUST 1986/' _

advanced by one cell, it should point tothe first cell. An expression such as (Head+ I) mod Qsize would accomplish this,but the mod operator is comparativelyslow in most languages. (It entails adivision, which is almost invariably theslowest arithmetic operation.) By makingQsize a power of 2, the modulo calculation can be avoided. (Head + 1) andQmask, where Qmask is a constant equalto Qsize - 7, has the same effect and runsmuch faster.

I decided to implement the queue as anarray of characters, which is an extravagant waste of space. Each digit takes up afull byte, when it could be represented bya single bit. The underlying reason forthis decision was laziness—I didn't wantto complicate the routines with code forextracting individual bits—but I rationalized that I was trading space for time. Ithought that on a byte-oriented machineroutines for manipulating a characterarray would be faster than those for a bitarray. On thinking about it further I'm notso sure. It's true that extracting bits wouldrequire masking operations and eventualconversion to character form; on the otherhand, the more compact bitmap woulddrastically reduce the number of memoryaccesses. Perhaps some reader will settlethe issue by trying both methods.

The function TagStep is at the heart ofthe program; it is called once for each stepin the evolution ofthe tag system. Itexamines the first digit ofthe string,deletes three characters, and then addseither 00 or 1101 to the end. An initial version, based on calls to EnQ and DeQ, isshown in Listing 2. A more efficient version that eliminates the overhead of theseprocedure calls is given in Listing 3.

Detecting repetitionMuch else is needed in a complete program for exploring tag systems: routinesfor input and output, for stepping througha sequence of initial patterns, for determining the fate of a pattern, for collectingstatistics, and so on. Most of this code isstraightforward, but one other area has anelement of subtlety to it. How can a program detect when a tag system has entereda cycle? A brute-force solution is guaranteed to work: store each string as it is generated and then compare every new stringwith all the preceding ones. As soon as amatch is found, the tag system is forevercommitted to a cycle.

The trouble with the brute-forcemethod is that it requires entirely toomuch brute force. For a pattern that runsthrough a million iterations before entering a cycle, with an average string lengthof 1,000 digits, the storage requirementwould be at least a gigabyte. Furthermore, roughly 500 million string com

parisons would be needed. There are better ways, based on sampling rather thanexhaustive record-keeping.

Consider a program that stores everyhundredth string and compares each newstring with these selected stored ones.Suppose a particular tag system enters acycle at step 917 and that the period ofthecycle—the number of steps betweenrepeated values—is six. Thus for everyvalue of/V equal to or greater than 917, thestring at step N will be identical to thestring at step N + 6.

The program will not detect this cycle

at the earliest possible moment (step 923),but soon after the next checkpoint (step1,000), the repetition will become apparent. The string at step 1,006 will be identical to the one stored at step 1,000. Furthermore, the interval between thecheckpoint and the step number where thecycle is detected gives the period ofthecycle. The point at which the cycle actually begins can be found by returning tothe previous checkpoint (step 900) and

lobal declarations }s ti z e = 8 1 9 2 ; { m u s t b e p o w e r o f 2 }

Q m a s k = 8 1 9 1 ; { m u s t b e Q s i z e - 1 }typeStepCode = (OK, Over, Under); {result returned by TagStep}" YP& ~ record

i : a r r a y [ 0 . . Q s i z e ] o f c h a r ;ead : i n t ege r ;e n : i n t e g e r ;tepNo : real;»d;

{ Add 'Digit' to the tail of 'Q'. }procedure EnQ(var Q : Qtype; Digit : char);

g in.D[(Q.Head + Q.Len) and Qmask] := Digit;.Len := Succ(Q.Len);d;eturn a character removed from the head of 'Q'. }ction DeQ(var Q : Qtype) : char;g in'.Len := Pred(Q.Len);

Q.Head := Succ(Q.Head) and Qmask;

Listing 1

function TagStep (var Q : Qtype) : StepCoda; { prototype versi

Digit, Dump : char;v^egin

if Q.Len < 3 then TagStep := Under=lse if Q.Len = Qsize then TagStep := Over2lsebeginTagStep := OK;Digit := DeQ; Dump := DeQ; Dump := DeQ;if Digit = '1' thenbeginEnQ(' l ' ) ; EnQ(' l ' ) ; EnQ('0 ') ; EnQ(' l ' ) ;

endelsebeginEnQ('0'); EnQ('0');

end;Q.StepNo := Q.StepNo + 1.0;

end;

{ queue empty }{ queue full }

{ read first and{ dump next two

{ append 1101

{ append 00

j f i Listing 2.

comparing successive pairs of strings separated by an interval of six steps.

A number of refinements can be addedto this cycle-detection method, and thestorage requirement can be reduced to asfew as two strings. (In my implementationI kept track of five.) The optimum algorithm was worked out by R. William Gosper of Symbolics Inc. It is describedbriefly by Donald E. Knuth of StanfordUniversity in volume 2 of The Art of Computer Programming. (The context there,

incidentally, is the problem of detectingcycles in the output of a pseudorandom-number generator.)

Whatever method is used for detectingcycles, the program is sure to spend a substantial amount of its time comparingstrings of digits. It is worthwhile makingthe comparison as efficient as possible.The function EqQ in Listing 4 is one solution. It exploits the fact that if two stringsare identical, their lengths must be identical. A comparison ofthe lengths can settle

rtion TagStep(var Q : Qtype) : StepCoda; { optimized versii n

f Q.Len < 3 then TagStep := Under { underflow }lse if Q.Len = Qsize then TagStep := Over { overflow }lse

TagStep := OK;if Q.D[Q.Head] = '1' thenbeginQ.Head := (Q.Head + 3) and Qmask;Q.Len := Q.Len - 3;Q.D[(Q.Head + Q.Len) and Qmask] := '1';

13.Len := Succ(Q.Len);

}.D[(Q.Head + Q.Len) and Qmask] := '1';2.Len := Succ(Q.Len);}.D[(Q.Head + Q.Len) and Qmask] := '0';2.Len := Succ(Q.Len);2.D[(Q.Head + Q.Len) and Qmask] := '1';3.Len := Succ(Q.Len);

beginQ.Head := (Q.Head + 3) and Qmask;Q.Len := Q.Len - 3;

.D[(Q.Head + Q.Len) and Qmask] := '0';

.Len := Succ(Q.Len);Q.D[ (Q.Head + Q.Len) and Qmask] := '0';Q.Len := Succ(Q.Len);nd;StepNo := Q.StepNo +1.0;

turn TRUE if two tag strings are identical. }tion EqQ(var Ql, Q2 : Qtype) : boolean;

i : integer;inQ := False;

f Ql.Len = Q2.Len then { check for equal lengths first }begin

ft N := 0 to (Ql.Len-1) dof Ql.D[(Ql.Head+N) and Qmask] <> Q2.D[ (Q2.Head+N) and Qm;then exit;

EqQ := True;

Listing 4.

the question for many pairs of stringswithout a digit-by-digit comparison ofthestring contents.

Observations and questionsFigure 3 shows the fates of 50 consecutivestarting patterns beginning with an arbitrarily chosen value 24 digits long. Notethat I have not traced all possible binarystrings in this range but only those thatdiffer at key positions. The reason is thatthe digits between the key positions aredeleted by the tag transformations withoutever being examined. For example, theinitial patterns 100,101,110, and 111 allyield the same tag system because onlythe first digit in each pattern is acted on bythe tag rules.

Along with the outcome of each pattern, Figure 3 gives the run length: thenumber of steps taken before the stringeither vanishes or enters a cycle. Forthose patterns that do cycle two additionalitems of information are listed: the periodofthe cycle and the number of digits in thefirst repeated string. About two-thirds ofthe 50 systems become cyclic, and the restdwindle away after no more than about400 steps. Most of those that enter a cycledo so after fewer than 100 steps, but thereare exceptions. One system goes on for2,125 steps before it gets stuck in a cycle;another lasts for 1,543 steps. More thanhalf of the cycles have a period of six, andanother 30% have a period of 10.

Looking at a larger sample suggeststhat these observations are fairly representative. A statistical summary of resultsfor the first 10,000 consecutive tag systems is given in Figure 4. Three-fourthsofthe systems cycle and the rest dwindle.Periods of 6 and 10 continue to dominate;indeed, together they account formore than 90 percent of all the cycles.The longest run before a cycle begins is24,563 steps, and the longest run before asystem dwindles to extinction is 1,398steps.

Based on these findings, there are a fewobservations I would make and a fewquestions I would ask.

The period of every cycle is an evennumber. The reason is not hard to find.(Hint: Consider why there can be no cyclewith a period of one, that is, a single pattern that repeats continuously.)

The distribution of periods in any largesample of tag systems is more difficult tounderstand. In the first 10,000 systems theonly periods that occur are 2,4, 6, 8,10,12,16, 28, and 40. Do the interveningeven numbers ever turn up? Why are thereno long periods of, say, 1,000 or 10,000steps? What can explain the curiouspredominance of 6 and 10? Is there something about the tag transformation itselfthat favors cycles with these periods?

One clue to the distribution of periods

r t ing str ing"100100000100000000

100100000100000100-1001000001001000001100100000100100100'100100100000000000100100100000000100100100100000100000100100100000100100100100100100000000100100100100000100100100100100100000100100100100100100

/0000000000000000000000000000000000000000001'00000000000000000001000iOOOOOOOOOOOOOOOOOOO1001•0000000000000000100000000000000000000001000001000000000000000010010000000000000000000100100100000000000001000000000000000000000010000000010000000000000100000100000000000000001000001001000000000000010010000000000000000000100100000100000000000001001001000000000000000010010010010000000000100000000000000000000001000000000001'00000000001000000001000iOOOOOOOOOO1000000001001

0000000000001000001000000000000000000100000100000100000000000010000010010000000000000001000001001001IOOOOOOOOOO100100000000000000000001001000000001000000000010010000010000000000000100100000100100000000001001001000000

JOOOOOOOOOO1001001000001000000000000100100100100000000000000010010010010010000000001000000000000000000000000100000000000000100000000010000000000010000000000001000000000001001

10000000100000000100000000000001000000001000001

a l s y s t e m s t e s t e d : 5 0s t e m s t h a t c y c l e : 3 1t e r n s t h a t d w i n d l e : 1 9

Fa te Run P e r i o d D i g i t scycle 59 \ ■■dwindle 396cycle 143 6 3 3cycle 53 6 3 7cycle 45 1 0 3 1cycle 47 6 3 3cycle 917 WZfmcycle 21 6 3 3cycle 927 K Z flcycle 2125 K Mcycle 1543 WMcycle 853dwindle 23dwindle 27cycle 18 6 1 3dwindle 33dwindledwindlecycle 4 1 3cycle 18cycle 20dwindle 31cycle 46 ' i f /

cycle 26 jv-cycle 16 jE-cycle 88 D ® :cycle 24 j icycle 80 | ® - i l ldwindle 31dwindle 37dwindle 43dwindle 415dwindle 421cycle 56 1 0 3 1cycle 86 1 0 3 1cycle 118 1 0 3 1dwindle 37dwindle 419cycledwindle

- wcycle l j @ ;■ ' ;cycle 1 0 3 1cycle 8 2 3cycledwindle

1 0 3 1

dwindlecycle 4 1 3cycle 18 ^ ndwindle 33dwindle 423

62.00%38.00%

t r i b u t i o n o f p e r i o d s a m o n g s y s t e m s t h a t e n t e r a c y c l er i o d N u m b e r P e r c e n t

3 9 . 6 8 %1 7 5 4 . 8 4 %

1 3 . 2 3 %' 2 9 . 0 3 %l 3 . 2 3 %

Fifty initial patterns are classified according to whether they eventually dwindle av>mter a cycle; the third possible fate—endless growth—has never been observed. The

labeled "Run" gives the number of steps required before the fate is determined,or strings that dwindle the run is assumed to end when the number of digits in the string

falls below three; in other words, the starting pattern 000 has a run length of 1. For cyclicpatterns the run ends when a string that will be repeated first appears; for example, thepattern traced in Figure 1 has a run length of 16. The period of a cycle is the number of

m one appearance of any repeated string to the next appearance. The "Digits"ords the length of the string at the step where a cycle begins.

is that many systems are in fact enteringthe same cycle. For example, a number ofsystems converge on the 19-digit string0000011011101110100, which cycles with aperiod of 6. The 37-digit string0000011011101110100000011011101110100 isessentially a doubling of this pattern, andit cycles with the same period. There areanalogous patterns of 55 digits, 73 digits,and so on. Given that all the cycles investigated so far have quite short periods,there may be only a small number of distinct cycles being observed. Questions ofthis kind were addressed in the 1960s byShigeru Watanabe ofthe University ofTokyo.

There is some evidence that as the starting patterns get longer, the proportion ofsystems that cycle increases (and hencethe proportion that dwindle decreases).Does this trend continue indefinitely, sothat for very long starting patterns theprobability of entering a cycle approachesunity?

One might guess that as the number ofIs at key positions increases the averagerun length would also increase. (The converse is unquestionably true: with Os in allkey positions, the system dwindles awayimmediately.) Figure 5 tabulates the fatesofthe first 50 strings that have Is in all keypositions. In the notation used in the figure, (100)2 means 100100, (100)3 means100100100, etc. There are certainly somelong runs among these systems, mostnotably (100)24, which continues for4,346,269 steps before degenerating intoa commonplace cycle with a period of 6.But if the curve is generally headedupward, there is a great deal of noise in it.Neither the size of the starting pattern northe number of Is in key positions wouldappear to be very reliable predictors ofrun length.

In both Figure 3 and Figure 5 the column headed "Digits," which gives thelength ofthe string at the step where thesystem first enters a cycle, has a mysterious regularity. All ofthe numbers inthis column are odd. When I firstobserved this fact after examining severalhundred tag systems, I made the obviousconjecture that it is always true and setabout finding out why.

I thought I was on the right track when Iwas able to show that if the number in the"Digits" column is always odd, then thecycle must always be entered on a 00 step,rather than a 1101 step. In other words, thelast string before the repetitive sequencebegins must have a 0 as its leading digit,so that the string length is reduced by onein the following step. This line of argument was looking very promising when Idiscovered that the conjecture itself is nottrue: a few starting patterns enter a cycleon a string with an even number of digits.In one sample of 1,000 tag systems, Ifound six such anomalous patterns. Thisis extremely vexing. In much of life, perhaps, the exception proves the rule, but in

mathematics a law valid in 99 and44/100% ofthe cases is an abomination.

The tag problem has a tantalizingresemblance to another famous littlestinker, generally known as the 3X +1problem. In this problem one begins withany positive integer X. IfXiseven,replace it with X/2; if it is odd, replace itwith 3X +1. Then apply the same rule tothe new X. The question is: does X invariably descend to a value of 1 or are theresome initial values of X for which theseries diverges toward infinity? Like thetag problem, the 3X +1 problem isunsolved. Is there any deep connectionbetween them?

The tag problem as stated here ismerely one example of an infinite class ofrelated problems. The transformationrules could call for deleting any numberof digits from the head ofthe string andappending any sequence of digits to theend. For example, one might delete twodigits and add either 0 or 101.

A further generalization would be touse an alphabet with more than two symbols and devise transformation rules to befollowed when each of these symbols isfound at the head ofthe string. Post wasable to solve all tag systems based on atwo-symbol alphabet where no more than

gest run before entering a cycle30010000000010010010010000010gest cycle period

10010010010010010000010Longest run before dwindling away

"000010000010010000010010000010

two symbols are deleted at each step.Minsky proved that when the alphabet hassix symbols and six symbols arc removedat each step, the tag problem is intrinsically insoluble. The gray area inbetween, including the (00,1101) problemitself, is the area of active interest.

P o s t m o r t e mWhen Post began work on the tag problemin the early 1920s, his motive was not idlecuriosity. He was engaged in an ambitiousattempt to secure the foundations of mathematics. At the turn ofthe century DavidHilbert had argued that all of mathematicsand logic could be reduced to a formalsystem: a set of symbols and a set of rulesfor manipulating the symbols withoutregard for their meaning. Hilbert envisioned a mechanical process—what wewould now call an algorithm—that inprinciple could complete mathematics.Given a set of axioms, or self-evidenttruths, the algorithm would generate alltrue statements derived from the axiomsand no falsehoods.

Post was trying to prove the existenceof such an algorithm when he stumbledonto the tag problem. He had alreadydemonstrated that one very simple set ofsymbols and rules is complete and consistent; applying the rules to any axiom written in these symbols would yield all statements logically consistent with the axiomand only those statements. Unfortunately,

(24,563 steps)

(40 steps)

(1,398 steps)

sms that cycle:ams that dwindle:

7 , 5 6 2 7 5 . 6 2 %2 , 4 3 8 2 4 . 3 8 %

ribution of periods among systems that enter a cycleN u m b e r Percent

0.29%1.44%

54.35%1.49%

37.52%0.19%1.24%2.99%

0.49%

e distribution of cycle periods is one of the most puzzling aspects of the Iids of 6 and 10 seem to be strongly favored over all other possibilities.

Figure 4.

1 9 ■ ■ ■r j i M /

WALTZLISP

The un ive rsa l , supere ffic ien tLISP for MS-DOS and CP/M.

Waltz Lisp is a very powerful and completeimplementation of Lisp. It is substantiallycompatible with established mainframe Lispssuch as Franz (the Lisp running under Unix),Common Lisp, and Maclisp.

- In independent tests, Waltz Lispwas up to twenty(!) times faster

than competing microcomputer Lisps.

I Built-in WS-compatible fullscreen file editor. Full debugging and error handling facilities are available at alltimes. No debuggers to link or load.

■ j,"T5|fW|| Random file access, binary filemMLiaiiSiiB support, and extensive string

operations make Waltz Lisp suitable for generalprogramming. Several utilities are included in thepackage.

CHH1BM Functions of type lambda (expr),■I linn m niambda (lexpr), lexpr, macro.

Splicing and non-splicing character macros. Fullsuite of mappers, iterators, etc. Long integers (upto 611 digits). Fast list sorting using user definedcomparison predicates. Built-in prettyprinting andformatting facilities. Nearly 300 functions in all.

■ dP-WMHi Transparent (yet programmable)mMoiiMsM handling of undefined functionreferences allows large programs to reside partiallyon disk at run time. Automatic loading of initialization file. Assembly language interface.

Each aspect ofthe interpreter is

described in detail. The 300+ page manualincludes an exhaustive index. Hundreds of illustrative examples.

Superbly documented.

16-bit versions require DOS 2.x or 3.x and 128KRAM (more recommended).

~**\ Z-80 version requires CP/M|\ 2.x or 3.x and 48K RAM'11 minimum. Waltz Lisp runs|1 on hundreds of different•il computer models and is

, ,.„ 'ill available in all disk formats.w a l t z l i s p I

i !" ' ' i l l "Manual only: S30 (refundable with order). Foreignorders: add S5 for surface

mail. S20 for airmail. COD add S3. Apple CP/M, hardsector, and 3" formats add S15. MC/Visa accepted.

For further information or to order call

1-800-522-LISPIn NJ and outside USA call 1-201-755-LISP

475 Watchung Ave.Watchung, NJ 07060

CIRCLE 110 ON READER SERVICE CARD

BWWWI'r1-1'' r'

Sequence Fate R u n P e r i o d(100' cycle 4 2(100 2 cycle 1 5 6(100 3 cycle 1 0 6(100 4 cycle 2 5 6(1005 dwindle 409(100 6 cycle 4 7 1 0(100 7 cycle 2 , 1 2 8 2 8(100 8 cycle 8 5 3 6(100 9 cycle 3 7 2 1 0(100 ,0 cycle 2 , 8 0 5 6(100 " cycle 3 6 6 6(100 u cycle 2 , 6 0 3 6(100 ,3 dwindle 701(100 M dwindle 37,910(100 ,5 cycle 6 1 2 6(100 '6 cycle 1 2 7 2 8(100 ,7 cycle 9 9 8 1 0(100 ,8 cycle 2 , 4 0 1 6100 ,9 cycle 1 , 0 0 1 0100 20 cycle 6 2 3 6100 21 cycle 5 , 2 8 0 6100 22 dwindle 1,776100 23 cycle 1 , 4 6 2 6100 24 cycle 4 , 3 4 6 , 2 6 9 6100 " dwindle 4,127100 26 cycle 3 , 2 4 1 (100 27 cycle 7 , 0 1 8 (100 28 cycle 3 , 8 8 5 (100 2"100 30

cycle 1 4 , 6 3 2 tcycle 7 , 0 1 9 (

100 31 cycle 4 , 5 6 4 I100 32 cycle 4 , 2 7 7 5 ' ,100 33 cycle 1 4 7 , 6 8 8 (10034 cycle 1 , 8 5 7 t100 * cycle 1 1 , 1 2 8 <100 36100 37

cyclecycle

8 1 , 1 4 1 (2 0 , 2 0 4 (

100 38 cycle 3 , 8 4 7 t100 39 cycle 1 1 6 , 0 1 4 (100 40 cycle 7 , 6 3 5 t100 4' cycle 6 , 4 8 8 I100 42 cycle 5 , 6 6 5 I100 43 cycle 6 , 1 4 2 t100 44 cycle 7 3 , 5 1 5 2 1100 45 cycle 5 , 8 2 6 6100 46 dwindle 6,060100 47 dwindle 3,779100 48 cycle 7 , 8 6 5 2 8100 49 cycle 2 8 , 6 3 0 6100 50 cycle 8 1 5 , 3 2 1 6

Digi ts

Statistics for run of 50 systemsS y s t e m s t h a t c y c l e : 4 3 8 6 %

„ . ; 7 1 4 %

stribution of periods among systems that enter a cycle

I i o d N u m b e r P e r c e n t

1 2 . 3 3 %3 3 7 6 . 7 4 %

4 9 . 3 0 %4 9 . 3 0 %1 2 . 3 3 %

^ reasonable hypothesis is that strings with 1 s at all the key positions might have longers than other strings. A tabulation of results for the initial strings 100,100100,100100100.includes some quite long runs, but there is no clear pattern. Note that only one period

- 4Liat was not seen in the first 10,000 tag systems appears in this sample: the stringjr the concatenation of 100 written 32 times, cycles with a period of 52 steps.

the set of symbols was too limited toexpress much of interest about logic ormathematics.

When Post tried to extend his results toricher formal systems, he ran into trouble. There were too many symbols thatcould be manipulated in too many ways,and so he retreated, in several steps, to thetag problem, which at first appeared to bemore tractable. As we have seen, he failedthere, too.

The reason became apparent a decadelater, when Kurt Godel, a young Austrianmathematician, showed that no formalsystem could possibly accomplish whatHilbert dreamed of. As soon as a formalsystem becomes powerful enough toencode its own rules, it also becomespowerful enough to expresscontradictions.

Gbdel's incompleteness theorem cameas a great surprise to mathematicians, andnot a pleasant one. It did not, however,end work on formal systems. Godel hadshown what cannot be done by pushingsymbols around on paper; AlonzoChurch, Alan Turing, and Post himselfwent on to show what can be done. Andthe power ofthe formal systems based ontheir work is not negligible; today we callthose formal systems languages, and wegive them names such as Pascal, LISP,and Ada.

ReferencesPost, Emil. "Absolutely Unsolvable Prob

lems and Relatively Undecidable Propositions: Account of an Anticipation," in TfieUndecidable: Basic Papers on UndecidablePropositions, Unsolvable Problems andComputable Functions, ed. by MartinDavis. Hewlett, New York: Raven Press,1965. [Post did not write a full account ofhis work on tag systems until 20 years afterthe fact, and that account was not publisheduntil another 20 years had passed. This collection is the only place it appears. The volume also includes fundamental papers byGodel, Church, Turing, and others.]

Watanabe, Shigeru. "Periodicity of Post'sNormal Process of Tag," in MathematicalTheory of Automata. Brooklyn, N.Y.: Polytechnic Press, 1963. [Watanabe analyzesthe kinds of patterns that can become periodic in a tag system.]

Minsky, Marvin L. Computation: Finite andInfinite Machines. Englewood Cliffs, N.J.:Prentice-Hall, 1967. [Includes a brief butlucid description ofthe tag problem in thecontext of computer science and languagetheory.]

Brian Hayes is a writer who works in bothnatural and formal languages. Until 1984he was an editor o/Scientific American,where he launched the Computer Recreations department.

COMPUTER LANGUAGE ■ AUGUST 1986