what turing did after he invented the universal turing machine · 2009-11-04 · what turing did...

What Turing Did after He Invented the Universal Turing MachineAuthor(s): B. Jack Copeland and Diane ProudfootSource: Journal of Logic, Language, and Information, Vol. 9, No. 4, Special Issue on AlanTuring and Artificial Intelligence (Oct., 2000), pp. 491-509Published by: SpringerStable URL: http://www.jstor.org/stable/40180239Accessed: 04/11/2009 17:35

Your use of the JSTOR archive indicates your acceptance of JSTOR's Terms and Conditions of Use, available athttp://www.jstor.org/page/info/about/policies/terms.jsp. JSTOR's Terms and Conditions of Use provides, in part, that unlessyou have obtained prior permission, you may not download an entire issue of a journal or multiple copies of articles, and youmay use content in the JSTOR archive only for your personal, non-commercial use.

Please contact the publisher regarding any further use of this work. Publisher contact information may be obtained athttp://www.jstor.org/action/showPublisher?publisherCode=springer.

Each copy of any part of a JSTOR transmission must contain the same copyright notice that appears on the screen or printedpage of such transmission.

JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range ofcontent in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new formsof scholarship. For more information about JSTOR, please contact [email protected].

Springer is collaborating with JSTOR to digitize, preserve and extend access to Journal of Logic, Language,and Information.

http://www.jstor.org

http://www.jstor.org/stable/40180239?origin=JSTOR-pdf

http://www.jstor.org/page/info/about/policies/terms.jsp

http://www.jstor.org/action/showPublisher?publisherCode=springer

^£ Journal of Logic, Language, and Information 9: 491-509,2000. ^gi ^rr © 2000 Kluwer Academic Publishers. Printed in the Netherlands.

What Turing Did after He Invented the Universal

Turing Machine

B. JACK COPELAND and DIANE PROUDFOOT The Turing Project, University of Canterbury, Private Bag 4800, Christchurch, New Zealand E-mail: bjcopeland@ canterbury. ac. nz, d.proudfoot@phil. canterbury. ac. nz; http://www.Alan Turing, net

(Received 1 June 1999; in final form 15 April 2000)

Abstract Alan Turing anticipated many areas of current research in computer and cognitive science. This article outlines his contributions to Artificial Intelligence, connectionism, hypercomputation, and Artificial Life, and also describes Turing's pioneering role in the development of electronic

stored-program digital computers. It locates the origins of Artificial Intelligence in postwar Britain. It examines the intellectual connections between the work of Turing and of Wittgenstein in respect of their views on cognition, on machine intelligence, and on the relation between provability and truth. We criticise widespread and influential misunderstandings of the Church-Turing thesis and of the halting theorem. We also explore the idea of hypercomputation, outlining a number of notional machines that "compute the uncomputable."

Key words: Artificial Intelligence, Artificial Life, Automatic Computing Engine (ACE), Church-

Turing thesis, Colossus, connectionism, Halting theorem, history of computing, hypercomputation, Turing, Wittgenstein

1. The Race to Build the First Computer

It is often said that, apart from specifying the universal Turing machine in 1935, Turing played little or no role in the development of computers. The reality is very different. In 1945 Turing produced a detailed design for his proposed Automatic Computing Engine or ACE (Turing, 1945; Copeland, 1999a; Copeland, 2000b). This design was the first relatively complete specification of an electronic stored- program general-purpose digital computer. The slightly earlier "First Draft of a Report on the EDVAC," produced by the Moore School group at the University of Pennsylvania (Von Neumann, 1945), contained little engineering detail, in particular concerning electronic hardware. Turing, on the other hand, supplied detailed circuit designs and specifications of hardware units, specimen programs in machine code, and even an estimate of the cost of building the machine (£1 1,200).

Turing saw that speed and memory were the keys to computing. His design had much in common with today's RISC architectures and called for a high-speed memory of roughly the same capacity as an early Macintosh computer (enormous by the standards of his day). Had Turing's ACE been built as planned it would have

492 B.J. COPELAND AND D. PROUDFOOT

been in a different league from the other early computers, but his colleagues at the National Physical Laboratory (NPL) thought the engineering work too difficult to attempt, and a considerably smaller machine was built. Known as the Pilot Model ACE, this machine ran its first program on May 10, 1950. With a clock speed of 1 MHz it was for some time the fastest computer in the world.

Sales of DEUCE, the production version of the Pilot Model ACE, exceeded 30 (confounding a prediction by a top adviser to the NPL that Britain's computing needs would be satisfied by a total of 3 digital computers). The fundamentals of Turing's ACE design were employed by Harry Huskey (at Wayne University, Detroit) in the Bendix G15 computer. The G15 was arguably the first personal computer and over 400 were sold worldwide. DEUCE and the G15 remained in use until about 1970. Another computer derived from Turing's ACE design, the MOSAIC, played a role in Britain's air defences during the Cold War period; other derivatives include the Packard-Bell PB250 (1961).

Unfortunately, delays beyond Turing's control resulted in the NPL's losing the race to build the world's first electronic stored-program digital computer - an hon- our that went to the University of Manchester, where, in the Royal Society Com- puting Machine Laboratory, the "Manchester Baby" ran its first program on 21 June 1948. As its name implies, the Baby was a very small computer, and the news that it had run what was only a tiny program - just 17 instructions long - for a mathematically trivial task was "greeted with hilarity" by the team building the

sophisticated Pilot Model ACE.* Turing designed the programming system for the Ferranti Mark I, the production version of the Baby's successor.** Completed in 1951, the Ferranti Mark I was the first commercially available electronic digital computer; about 10 were sold (in Britain, Canada, Holland and Italy).

Both the ACE and the Manchester computer came out of research that nobody would have guessed might have a practical application. In a lecture in Cambridge in 1935 the mathematician Max Newman - whose own role in the development of

computers has been insufficiently emphasised - introduced Turing to the concept that led directly to the Turing machine: Newman defined a constructive process as one that a machine can carry out (Copeland, 1999b).* Turing took Newman's words

literally and the result was a typescript, which Turing showed to Newman in April 1936, setting out the concept of the universal Turing machine.** According to New- man, Turing was interested right from the start in the possibility of actually making

* Michael Woodger (Turing's assistant at the NPL from May 1946) in personal communication with Copeland (1998).

** A digital facsimile of Turing's typewritten Programmers ' Handbook for Manchester Electronic

Computer is in the Turing Archive for the History of Computing and may be viewed on-screen at

http://www.AlanTuring.net * Newman in interview with Christopher Evans in 1976 ("The Pioneers of Computing: an Oral

History of Computing," Science Museum: London). ** The Turing machine concept was announced by Turing in an address to the London Math-

ematical Society on 12 November 1936, "On Computable Numbers, with an Application to the

Entscheidungsproblem" (Turing, 1936).

TURING AFTER THE UTM 493

a machine of the sort described in the paper.* In 1937-1938 Turing designed and wired up a simple binary multiplier, but it was not until the development during the war of high-speed electronic switching that the dream of building a miraculously fast general-purpose computing machine really took hold.

Following decades of secrecy, Turing's wartime work as leading codebreaker at the Government Code and Cypher School, Bletchley Park, is now well known. F.H. Hinsley, official historian of British Intelligence in the Second World War, has estimated that the Bletchley Park codebreakers shortened the war in Europe by as much as two years. Turing's main work concerned the 'Enigma' code (his "Treatise on the Enigma" has recently been declassified). Newman, in a different section, attacked the 'Fish' codes. Based on binary teleprinter code, Fish was used in preference to Morse-based Enigma for the encryption of high-level signals, for

example messages from Hitler and other members of the German High Command. Newman realised that the attack on Fish could be mechanised (Turing had already mechanised the attack on Enigma, with enormous success) and at Turing's sugges- tion sought the help of electronic engineer Tom Flowers.** The history of electronic data-processing begins with Flowers' pre-war work at the Post Office Research Sta- tion at Dollis Hill in London: during the period 1934-1939 Flowers experimented with high-speed electronic data-storage and designed electronic digital telephone equipment that used 3,000-4,000 vacuum tubes running continuously. Flowers has remarked that at the outbreak of war with Germany he was possibly the only person in Britain who realized that vacuum tubes could be reliably used on a large scale for high-speed digital computation. During 1943, in consultation with Newman, Flowers designed and built Colossus, the first large-scale fully-functioning electronic digital computer. Colossus first worked on 8 December 1943, some two years before the first comparable U.S. machine, the ENIAC. (ENIAC was constructed for the Army Ordnance Department by J. Presper Eckert and John Mauchly at the Moore School of Electrical Engineering, University of Pennsylvania.) Both Colossus and ENIAC were special-purpose computers (ENIAC's function was to calculate tables used by gunners when aiming artillery). Neither had internally stored programs - setting up these machines for a new task involved reconfigur- ing them by means of plugs and switches. During the construction of Colossus, Newman had tried to interest Flowers in Turing's 1936 paper - the origin of the stored-program concept - but Rowers "didn't really understand much of it." There can be little doubt that by 1943 Newman had firmly in mind the possibility of building a universal Turing machine using electronic technology. Colossus, involving approximately 2400 thermionic valves, established decisively and for the first time that large-scale electronic computing machinery was practicable.

In 1945, the war in Europe over, Turing and Newman followed different paths, Turing to the NPL to build the ACE and Newman to the Fielden Chair of Mathem-

* Newman, op. cit. ** All material herein relating to T.H. Flowers derives from personal communications from

Flowers to Copeland (1996-1998).


atics at the University of Manchester. Shortly after his arrival at Manchester, New- man wrote to von Neumann, a prominent figure in the development of computing in the U.S. (and strongly influenced by Turing):

I am ... hoping to embark on a computing machine section here, having got very interested in electronic devices of this kind during the last two or three

years. By about eighteen months ago I had decided to try my hand at starting up a machine unit when I got out. ... I am of course in close touch with

Turing.* Newman applied to the Royal Society for a grant to establish his Computing Ma- chine Laboratory, and the rest is history. Not that history has been particularly kind either to Newman or to Turing. Their logico-mathematical contributions to the triumph at Manchester have been neglected, and the Manchester machine is

nowadays remembered as the work of electronic engineers Williams and Kilburn.

Fortunately the words of the late Freddie Williams are preserved (Williams, 1975: 328):

Tom Kilburn and I knew nothing about computers, but a lot about circuits. Professor Newman and Mr. A.M. Turing . . . knew a lot about computers and substantially nothing about electronics. They took us by the hand and

explained how numbers could live in houses with addresses and how if they did they could be kept track of during a calculation.**

Turing's own phrase "the mechanic who . . . constructed the machine" (Turing, 1951a) perhaps reveals his attitude to Kilburn and Williams.

2. Artificial Intelligence

Turing was the first to carry out substantial research in the field now known as Artificial Intelligence. During the wartime years, Turing gave considerable thought to the issue of machine intelligence. Colleagues at Bletchley Park recall numerous

off-duty discussions with him on the topic, and at one point Turing circulated a

typewritten report (now lost) setting out some of his ideas. One of these colleagues, Donald Michie (who in the 1960s founded the Department of Machine Intelligence and Perception at the University of Edinburgh), remembers Turing often talking about the possibility of computing machines (1) learning from experience and

(2) solving problems by means of searching through the space of possible solutions, guided by (what are now called) heuristic principles.*

* Letter from Newman to von Neumann, 8 February 1946. **

Concerning Turing's knowledge of electronics, opinion among the engineers at Manchester seems to have been divided. G.C. Tootill - who bore much of the responsibility for liaising between the Computing Machine Laboratory and the engineers at Ferranti Ltd who built the Ferranti Mark I in collaboration with the University - spoke approvingly of Turing's "ability as a circuit designer" (letter from Tootill to Simon Lavington, 1 July 1975). (Tootill's "Informal Report on the Design of the Fer- ranti Mark I Computing Machine" (Royal Society Computing Machine Laboratory, November 1949) contains an interesting technical appendix by Turing entitled "Generation of Random Numbers")

* Donald Michie in personal communication with Copeland (1995).


At Bletchley Park Turing illustrated his ideas on machine intelligence by reference to chess. He experimented with two heuristics that later became common in AI: minimax (assume that your opponent will move in a way that maximises their gain and make your move in such a way as to minimise the loss that your opponent's expected move will cause you) and best first (rank the moves available to you according to a rule-of-thumb scoring system and examine their consequences, beginning with the highest-scoring move).*

In London in 1947 Turing gave what was, so far as is known, the earliest public lecture to mention computer intelligence, saying "What we want is a machine that can learn from experience" and "[t]he possibility of letting the machine alter its own instructions provides the mechanism for this" (Turing, 1947: 123). In 1948 he wrote (but did not publish) a report entitled "Intelligent Machinery." This was the first manifesto of Artificial Intelligence and in it Turing brilliantly introduced many of the concepts that were later to become central to the field, in some cases after reinvention by others. These included the concept of a genetic algorithm and of training an unorganised network of artificial neurons to perform specific tasks (see Section 3 below), the idea that "intellectual activity consists mainly of various kinds of search" (Turing, 1948: 23), and the theorem-proving approach to problem- solving. His 1950 paper, introducing the Turing Test, is of course well known. In 1951 Turing gave a lecture on machine intelligence on British radio and in 1953 he published a classic early article on chess programming (Turing, 1951b, 1953).

Both during and after the war Turing experimented with machine routines for playing chess: in the absence of a computer, the machine's behaviour was simulated by hand, using paper and pencil. The first true AI program had to await the arrival of a working general-purpose electronic digital computer.

The earliest AI programs** ran at Manchester University in the Royal Soci- ety Computing Machine Laboratory, of which Turing was appointed Deputy Dir- ector in 1948 (there was never a Director).* The first was written by Christopher Strachey, at the time a schoolmaster at Harrow and an amateur programmer (and later Director of the Programming Research Group at Oxford University). Strachey chose the board game of draughts (or checkers) as the domain for his experiment in machine intelligence. He wrote a draughts program for the Pilot Model ACE as early as February 1951, but was dissatisfied with the method employed in the program for evaluating board positions, and wrote an improved version for the Fer- ranti Mark I at Manchester (with Turing's encouragement and utilising the latter's recently completed Programmers9 Handbook for the machine, (Turing, 1950b)).**

* Ibid. ** See further Copeland (1993, 2000c). * In May of 1948 the vigour of the Manchester project and Newman's offer of a job had lured

a "very fed up" Turing away from his position at the NPL, where work on the ACE had drawn almost to a standstill (the quoted words are those of Turing's close friend Robin Gandy, personal communication with Copeland, 1995).

** It seems that the version for the Pilot Model ACE never ran successfully (personal communication from Michael Woodger to Copeland, 1999). An attempt in July 1951 foundered due to


By the summer of 1952 the Manchester version could play a complete game of

draughts at a reasonable speed. (The first AI program to run in the U.S. was also a draughts program, written in 1952 by Arthur Samuel of IBM for the IBM 701, IBM's first mass-produced electronic stored-program computer. Samuel took over the essentials of Strachey's program, which Strachey had described at a computing conference in Canada in 1952, and over a period of years considerably extended it.)

Prinz, who worked for the engineering firm of Ferranti Ltd., wrote the first chess program to be implemented. It ran in November 1951 on the Ferranti Mark I. Prinz's program was for solving simple problems of the mate-in-two variety. The program would examine every possible move until a solution was found. On

average several thousand moves had to be examined in the course of solving a

problem, and the program was considerably slower than a human player. Turing had started to program his own Turochamp chess-playing routine for the Ferranti Mark I but never completed the task. Unlike Prinz's program, the Turochamp could

play a complete game and operated not by exhaustive search but under the guidance of heuristics. Prinz also used the Ferranti Mark I to solve logical problems, and in 1949 and 1951 Ferranti built two small experimental special-purpose computers for

theorem-proving and other logical work. This was several years before the program by Newell, Simon and Shaw known as the "Logic Theorist" - often incorrectly described as being the first AI program - made its debut at the Dartmouth conference in 1956.

Oettinger's Shopper was the earliest program to incorporate rote-learning (de- tails of the program were published in 1952).* The program ran on Wilkes's EDSAC computer at the University of Cambridge. Oettinger was considerably influenced by Turing's views on machine learning, and suggested that a program of the Shopper type could pass a highly restricted version of the Turing Test. Shop- per's simulated world was a mall of eight shops. When sent out to purchase an item, Shopper would if necessary search for it, visiting shops at random until the item was found. While searching, Shopper would memorise a few of the items stocked in each shop visited (as a human shopper might). Next time Shopper was sent out for the same item, or for some other item that it had already located, it would go immediately to the correct shop. (Strachey had also investigated aspects of machine learning, taking the game of NIM as his focus, and in 1951 he reported a simple rote-learning scheme in a letter to Turing.)

3. Connectionism

Connectionists look back on Hebb and Rosenblatt as the originators of their approach, but in fact both were preceded by Turing, who anticipated much of modern

programming errors. A major hardware change later in 1951 meant that the program would never work without extensive recoding.

* "Shopper" is our name for what Oettinger terms "the shopping programme."


connectionism in his 1948 paper "Intelligent Machinery" (see Proudfoot and Cope- land, 1994; Copeland and Proudfoot, 1996, 1999a). Here Turing introduces what he calls "unorganised machines," giving as examples networks of neuron-like boolean elements connected together in a largely random manner (we call these "Turing Nets", Copeland and Proudfoot, 1999b). He describes a certain form of Turing Net as "the simplest model of a nervous system" and he hypothesises that "the cortex of the infant is an unorganised machine, which can be organised by suitable

interfering training" (Turing, 1948: 10, 16). The idea that an initially unorganised neural network can be organised by means of "interfering training" is undoubtedly the most significant aspect of Turing's discussion (and is not prefigured in the classic McCulloch-Pitts paper, 1943). In Turing's model, the training process renders certain neural pathways effective and others ineffective. He anticipated the modern procedure of simulating neural networks and the training process by means of an ordinary digital computer, saying (Turing, 1948: 21)

quite definite teaching policies' . . . could also be programmed into the [computer]. One would then allow the whole system to run for an appreciable period, and then break in as a kind of 'inspector of schools' and see what progress had been made.

Turing claimed a proof (now lost) of the proposition that an initially unorganised Turing Net with sufficient neurons can be organised to become a universal Turing machine with a given storage capacity (Turing, 1948: 15). This proof first opened up the possibility, noted by Turing (1948: 16), that the human cognitive system is a universal symbol-processor implemented in a neural network.

4. Hypercomputation

Turing's aim in his 1936 paper was to specify a machine, as simple as possible, that can perform any calculation that can be performed by a human mathematician with unlimited time, working with paper and pencil in accordance with some "rule-of- thumb" or rote method. (It is precisely because the universal Turing machine can carry out all such calculations that it is said to be "universal.") Modern digital computers, which are universal up to some resource limit, likewise possess the property of being abstract models of human beings engaged in rote calculation. As Turing wrote in his programming manual for the Ferranti Mark I, "Electronic computers are intended to carry out any definite rule-of-thumb process which could have been done by a human operator working in a disciplined but unintelligent manner" (Turing, 1950b: 1). An intriguing question arises: if we set aside the traditional idea that digital information-processing machines are to be modelled on human beings working in a rule-of-thumb manner, can we describe machinery that is capable of accomplishing a wider variety of tasks than the universal Turing machine? The answer to this question is that such machines can be specified on paper, but at the present time nobody knows whether it will be possible to build one. We call such machines hypercomputers (Copeland and Proudfoot, 1999a). A small but


growing international group of researchers is exploring the possibility of achieving hypercomputation (Copeland and Sylvan, 1999, surveys this emerging field). It is a matter for speculation whether the human brain itself is a naturally-occurring instantiation of a hypercomputer.

Turing was the first to investigate the idea of machines that are able to perform mathematical tasks too difficult for the universal Turing machine (this was in his Ph.D. thesis (Turing, 1938), supervised by Church, published as Turing, 1939). Turing described these as "a new kind of machine," calling them "O-machines" (Turing, 1939: 173). The primitive operations of an (ordinary) Turing machine are six in number: (i) move the tape left one square; (ii) move the tape right one square; (iii) read (i.e., identify) the symbol currently under the head; (iv) write a symbol on the square of tape currently under the head (after first deleting the

symbol already written there, if any); (v) change state; (vi) halt. These primitive operations are made available by unspecified subdevices of the machine - "black boxes." (The question of what mechanisms might appropriately occupy these black boxes is not relevant to the machine's logical specification.) An O-machine is a Turing machine augmented with a primitive operation that returns the values of some function (on the natural numbers) that is not Turing-machine-computable. This additional primitive operation is made available by a black box. Turing refers to this box as an "oracle" (Turing, 1939: 172). In other respects O-machines are similar to ordinary computing machinery. A digitally encoded program is fed in and the machine produces digital output from digital input by means of a step-by- step procedure. This step-by-step procedure consists of repeated applications of the machine's primitive operations, one of which is to pass data to the oracle and

register the oracle's response (see further Copeland, 1997, 1998a, 2000a). According to Turing's specification, each oracle returns the values of a two-

valued function. Let these values be written 0 and 1. Let p be the additional

primitive operation, p is called into action by means of a special state x» the call state. The machine inscribes the symbols that are to form the input to p on any convenient block of squares of its tape, using occurrences of a special symbol /z, the marker symbol, to indicate the beginning and the end of the input string. As soon as an instruction in the machine's program puts the machine into state x, the input is delivered to the subdevice that effects /?, which then returns the corresponding value of the function. On Turing's way of handling matters the value is not written on the tape. Rather a pair of states, the 1 -state and the 0-state, is employed to record values of the function. A call to p ends with a subdevice placing the machine in one or other of these two states, as the value of the function is 1 or 0. (When a function g is computable by an O-machine whose oracle serves to return the values of a function /, then g is sometimes said to be computable relative to /.)

One particular O-machine, the halting function machine, has as its "classical"

part the universal Turing machine specified by Turing in 1936 and as its "non- classical" part a primitive operation that returns the values of Turing's famous

halting function H(x, v). The halting function is defined thus: H(x, v) = 1 if


and only if the xth Turing machine eventually halts if set in motion with the integer y inscribed on its tape, say in binary code (think of y as being the machine's input); and H(x,y) = 0 otherwise.

The O-machine concept has been neglected in the philosophy of mind and cognitive science. Sadly, Turing's pioneering theoretical work has been forgotten even by those pursuing the goal of building hypermachines. Researchers talk of carrying out information-processing "beyond the Turing limit" and describe themselves as attempting to escape from "the Turing tar-pit" and to "break the Turing barrier."

Turing said nothing about what might be "inside" the logically specified black boxes, saying only that an oracle works by "unspecified means" and that "we shall not go any further into the nature of [an] oracle" (Turing, 1939: 172-173). However, notional machinery that fulfils the specifications of an oracle is not hard to concoct.

5. Notional Oracles and the Import of the Halting Theorem

Many textbooks on the fundamentals of computer science offer examples of information-processing tasks that are, it is claimed, absolutely uncomputable, in the sense that no machine can be specified to carry out these tasks. For example, it is said that no machine can respond in accordance with the following rules to inputs of arbitrarily selected finite strings of binary digits: (1) Answer "1" if the input string is a program that will cause a universal Turing machine to execute only a finite number of actions (such programs are called "terminating"). (2) Answer "0" if the string is not a terminating program - i.e., if the string is either a Turing machine program that does not terminate or is not a well-formed Turing machine program at all. (An example of a terminating program is one for finding the highest factor of a given integer, terminating on producing the answer. An example of a nonterminating program is one for calculating the digits of n .)

It is false that no conceivable machine can carry out the task just described (which we refer to as the terminating program or TP task). An AUTM, or Acceler- ating Universal Turing Machine, can carry out this task (Copeland, 1998b, 1998c; Stewart, 1991). An AUTM executes the program on its tape at an accelerating rate, performing each primitive operation that the program calls for in half the time that was taken for the immediately preceding primitive operation. So if the machine takes one unit of time to perform the first primitive operation, the second is performed in half a unit, the third in one quarter of a unit, and so on. Since

111 1

is less than 2, the AUTM requires less than two units of running time to do everything that the program on its tape instructs it to do. This is true even in the case of a program that does not terminate, for example a program that runs on forever calculating each successive digit of n: each of the infinite number of


operations that the nonterminating program instructs the machine to perform will be completed before the end of the second unit of running time.* (Copeland and Hamkins (forthcoming) discuss the issue of the physical plausibility of AUTMs with respect to Newtonian physics, relativistic physics, and quantum theory.)

Consider an AUTM that is set up to perform the TP task. The AUTM has a one-way infinite tape on which is inscribed the string that is to be tested. The initial square of the tape is used to display the result of the computation; at the start of the computation this square contains 0. The AUTM first determines whether or not the string being tested is a well-formed Turing-machine program. If the string is not a well-formed program then the AUTM halts. If the string is a well-formed program then the AUTM simulates it. If the string is a terminating program then, once the simulation is complete, the head of the AUTM moves to the initial square of the tape and replaces the 0 that was written there during the setting-up procedure by 1. The AUTM then halts. If the string is a nonterminating program, the head of the AUTM never returns to the start of the tape. Either way, at the end of the second unit of operating time the initial square contains the digit required by the above rules.

It is essential to distinguish between two senses in which a function may be said to be computable by a given machine, which we refer to respectively as the internal sense and the external sense (see Copeland, 1998b). A function is computable by a machine in the internal sense just in case the machine can produce values from

arguments (for all arguments in the domain), "halting" once any value has been produced, and where what counts as "halting" can be specified in terms of features internal to the machine and without reference to the behaviour of some device or

system - e.g., a clock - that is external to the machine. (This condition on the nature of halting behaviour will be referred to as the "internalist" condition.) Numerous behaviours on the part of a machine can satisfy this condition, for example com-

plete cessation of activity, or playing the British National Anthem,** or writing any sequence of digits in a certain location. A function is computable by a machine in the external sense just in case the machine can produce values from arguments (for all arguments in the domain), displaying each value at a designated location some

* This temporal patterning of operations seems to have been first described by Russell, in a lecture

given in Boston in 1914. In a discussion of Zeno's paradox, Russell said "If half the course takes half a minute, and the next quarter takes a quarter of a minute, and so on, the whole course will take a minute" (Russell, 1915: 172-173). Later he said that although it may be "medically impossible" for a person to run through "an infinite number of operations" by this means, it is nevertheless not

"logically impossible" to do so (Russell, 1936: 143-144). Boolos and Jeffrey (1980: 14) envisage Zeus being able to act so as to exhibit the Russell temporal patterning. By an extension of terminology (which Boolos and Jeffrey do not make) a Zeus machine is any machine exhibiting this temporal patterning. An AUTM is a Zeus machine, but not every Zeus machine is an AUTM.

** The first program of any significant size to run on the Ferranti Mark I - written by Strachey at

Turing's behest - brought its activity to a close by playing the National Anthem on the machine's "hooter."


prespecified number of time units after the corresponding argument is presented. The machine may or may not "halt" once a value has been displayed.

For example, it is in the external sense that a given function may be computable by a logic circuit. The value of the function is displayed at some designated node n time units after the argument is presented at the input nodes (mutatis mutandis for functions of more than one argument and functions whose values require more than one binary node for their expression). Before and after that critical moment, the activity of the output node may afford no clue as to the desired value. Even where the logic circuit never stabilises (in the sense of eventually producing an output signal that remains constant until such time as the input signal alters) the circuit nevertheless computes values of a function in the external sense if it displays them at the designated location at the prespecified times. The same is true of neural networks. A particular network may compute the values of a certain function in the external sense even though the network never stabilises (a network stabilises, or "halts," if and only if after some point there is no further change in the activity level of any of its units).

In some cases, a machine may compute the values of a function in both senses, there being an n such that whenever an argument is presented, the machine 'halts,' displaying the corresponding value, within n time units. A machine that computes a function in the external sense can readily be converted into one that computes the function in both senses by the addition of a bell triggered by an internal clock. The bell rings when the value (or, as appropriate, the last digit of the value) is produced, and the machine's ringing the bell constitutes its "halting." Of course, adding a clock to a machine may result in a machine not of the same type. A Turing machine plus a clock is not a Turing machine.

Turing's Halting Theorem of 1936 - in essence the proposition that the TP task is uncomputable - speaks only of computability by Turing machine in the internal sense, not of computability in the external sense: a Turing machine cannot carry out the TP task if it is required to produce the answer and then halt. But as the earlier example of the AUTM shows, a Turing machine can carry out the TP task if the internalist condition is lifted. While it may grate on one's ear to say so, it is nevertheless perfectly true that (even) a Turing machine can compute (in the external sense) that which is uncomputable (in the standard, internal sense).

The claim that no machine can carry out the TP task is evidently false, but is this weaker form true: no machine that delivers each of its answers after only a finite number of atomic operations (we call these finitely-operating machines) can carry out the TP task? The answer is that this is not true. It is possible to specify a machine that is finitely-operating in this sense and which carries out the TP task. Since each integer corresponds to a finite string of binary digits and vice versa (2 corresponding to 10, 3 to 11, 4 to 100, and so on), the TP task can be reformulated like this: given any integer, the machine is to output "1" if the binary string corresponding to the integer is a terminating program, and is to output "0" if the binary string corresponding to the integer is not a terminating program. Let us


write an for the correct output when integer n is the input (an is always 0 or 1). Now consider the following decimal specification of a number: 0.a\a2a3 .... We call this number r, for Turing (Copeland, 1998c; Copeland and Proudfoot, 1999a). Chaitin (1988) defines a number £2 that is analogous to, but not quite the same as, r. Like 7r, t is an irrational number. Perhaps the first few digits of r are 0.00000001 1 Let us imagine a device S that stores exactly r units of some physical quantity. A

measuring device M is able to measure the quantity stored in S to any specified number of significant figures. (Just as a hypothetical perfect frequency meter will measure the frequency in Hertz of a given signal to any desired number of significant figures.) M and S together form an oracle for the TP task (Copeland, 1997, 2000a). When any integer n is input into M, M determines an in a finite number of steps by measuring the quantity in S to an appropriate number of significant figures and outputting the nth digit of the result - which is an. (The first person to consider notional machines able to store arbitrary real numbers appears to have been Abramson. Abramson's "extended Turing machines" are able to store a real number on a single square of tape (Abramson, 1971).)

Of course, a TP-task oracle designed in this way would not work very well in

practice, since once n becomes very large, random noise would obstruct M's efforts to determine an accurately. No one yet knows whether it is possible to produce a realistic design for an oracle. But the search is on for a physically realisable archi- tecture capable in principle of computing more than a finitely-operating universal

Turing machine.

6. The Church-Turing Thesis

The statement that it may be possible to mechanise tasks that cannot be performed by the universal Turing machine is often met with bafflement and incredulity. For there is a myth that Turing and Church set out a principle concerning the limits of mechanisability, to the effect that the universal Turing machine can simulate the behaviour of any other machine. This (hypothesised) principle is commonly known as the Church-Turing thesis (see further Copeland, 2000a). The thesis that Turing and Church actually put forward is quite different: the universal Turing machine can perform any calculation that can be done by a human clerk working by rote with paper and pencil. This, the real Church-Turing thesis, does not entail that

hypermachines are impossible devices. This myth concerning the work of Turing and Church is widespread. For

example: Paul and Patricia Churchland, writing on Artificial Intelligence (Churchland and

Churchland, 1990: 26): Turing's results entail something remarkable, namely that a standard digital computer, given only the right program, a large enough memory and sufficient time, can

compute any rule-governed input-output function. That is, it can display any systematic pattern of responses to the environment whatsoever.


An O-machine carrying out the TP task responds to its input in a systematic, rule- governed manner. Turing had no result entailing that the systematic pattern of responses which this machine displays to its environment can be displayed by a standard digital computer - exactly the reverse, in fact.

Searle, writing on the question whether the operations of the brain can be simulated by the universal Turing machine (Searle, 1992: 200-201):

Given Church's thesis that anything that can be given a precise enough char- acterization as a set of steps can be simulated on a digital computer, it follows trivially that the question has an affirmative answer.

An O-machine carries out the step-by-step procedure dictated by its program. So if the brain is an O-machine, it is true that the brain's processing can be characterised as a set of steps but it need not be true that the brain can be simulated by the universal Turing machine. Church never advanced the thesis that Searle ascribes to him, and nor did Turing. The thesis is false.

Langton, writing on foundational issues in Artificial Life (Langton, 1989: 12):

There are certain behaviours that are incomputable' - behaviours for which no formal specification can be given for a machine that will exhibit that behaviour. The classic example of this sort of limitation is Turing's famous halting problem: can we give a formal specification for a machine which, when provided with the description of any other [sic] machine together with its initial state, will . . . determine whether or not that machine will reach its halt state? Turing proved that no such machine can be specified.

Even Turing's biographer, Hodges, says this about Turing's work of 1935-1936 (Hodges, 1992: 109):

Alan had . . . discovered something almost . . . miraculous, the idea of a universal machine that could take over the work of any machine.

The sooner philosophy and cognitive science are free of this myth the better.

7. Turing and Wittgenstein

Both Fellows of Cambridge colleges, Turing and Wittgenstein had a "wary respect" for each other.* Turing attended Wittgenstein's seminars from as early as 1935.** An apparently well-thumbed offprint of Turing's 1936 article "On Com- putable Numbers" was found among Wittgenstein's effects (Nedo and Ranchetti, 1983: 308). The published notes of Wittgenstein's lectures on the foundations of mathematics in 1939 include lengthy discussions with Turing (Wittgenstein, 1976). (This record and Turing's contributions to the now published radio broadcast "Can Automatic Calculating Machines Be Said To Think?" (Turing, 1952b) are the only

* The quoted words are Gandy's, in personal communication with Copeland (1995). ** Hodges' biography of Turing suggests that prior to 1937 Turing had seen Wittgenstein only at

the Moral Sciences Club (Hodges, 1992: 136). But see Nedo and Ranchetti (1983: 357-358).


examples we have of Turing in discussion.) Although it is now impossible to determine the precise nature of any influence between the two men, there is significant overlap in the work of Turing and Wittgenstein.

Concerning mathematics, it is interesting that during the early and mid 1930s, Wittgenstein and his students were considering the question "Is every mathematical problem solvable?" (Ambrose, 1935: 186, 188). Ambrose says of the following view that "[t]his, or a view similar to it, has been put forward by Dr. L. Wittgenstein" in lectures delivered at Cambridge in 1932-1935 (Ambrose, 1935: 333):

To say that a form has meaning is to say that in the symbolic system an answer to the question whether it is true, or false, is provided for ... And if the form . . . *acquires meaning' ... it has become the conclusion of a proof which is a new bit of mathematics. And this proof, on which the meaning of the form depends, provides the answer to the question whether the form is true or false.

Turing's views on these and related matters are nicely summarised in a letter to Newman:

You say "If all this whole formal outfit is not about finding proofs which can be checked on a machine it's difficult to know what it is about". When you say "on a machine" do you have in mind that there is (or should be or could be, but has not been actually described anywhere) some fixed machine on which proofs are to be checked, and that the formal outfit is as it were about this machine? If you take this attitude . . . there is little more to be said: we simply have to ... resign ourselves to the fact that there are some problems to which we can never get the answer. On these lines my ordinal logics would make no sense. However, I don't think you really hold quite this attitude because you admit that in the case of the Godel example we can decide that the formula is true i.e. you admit that there is a fairly definite idea of a true formula which is quite different from the idea of a provable one. Throughout my paper on ordinal logics [Turing, 1938, 1939] I have been assuming this too. ... If you think of various machines I don't see your difficulty. One ima-

gines different machines allowing different sets of proofs, and by choosing a suitable machine one can approximate 'truth' by 'provability' better than with a less suitable machine, and can in a sense approximate it as well as

you please. The choice of a proof checking machine involves intuition, which is interchangeable with the intuition required for finding an Q if one has an ordinal logic A , or as a third alternative one may go straight for the proof and this again requires intuition. Or one may go for a proof finding machine. I am rather puzzled why you drew this distinction between proof finders and proof checkers. It seems to me rather unimportant as one can always get a proof finder from a proof checker *

* Turing to Newman, undated (probably written in 1939 or 1940).


Concerning the nature of the brain, Wittgenstein, like Turing, anticipated aspects of connectionism. Wittgenstein's objections to the representational theory of mind and his emphasis upon samples, training and "family resemblance" in concept-formation (Wittgenstein, 1965: 130ff, 1953: §65ff, §§208-210) prefigure connectionist accounts of mind. As early as 1946-1948 he conjectured, as do many modern connectionists, that the brain does not process representations (Wittgen- stein, 1967b: §608ff). On cognition, however, Wittgenstein and Turing appear to have held very different views. In his later work (e.g., Wittgenstein, 1953: §152ff) Wittgenstein argued that understanding, thinking, intending, and other examples of cognition are not processes (because, for example, of the difficulty of speaking of the duration of understanding or intending in the way in which we speak of the duration of a process). It follows that the computational theory of mind is false: cognition cannot consist in computation, since computation is a process (for further discussion see Proudfoot, 1997).

On Turing machines, Wittgenstein argued for a distinction between merely behaving in accordance with a rule (of meaning, arithmetic, inference, etc.) and following a rule. Cognition, he claimed, requires genuine rule-following. Wittgenstein argued, further, that "reading-machines" (his term: these include Turing machines) as a matter of fact merely behave in accordance with a rule and so do not really read, calculate, and so on (Wittgenstein, 1967a: 1 19).* For Wittgenstein, to say that a Turing machine does read or calculate is to engage in make-believe. AI tradition- ally has been anthropocentric (for example, the Turing Test holds that a computer is a genuine thinker if it resembles a human being to the degree that someone in- terviewing both the computer and a human by teletype cannot tell which is which). An effect of this anthropocentrism is to encourage the make-believe that computers possess human qualities. For example, Turing said that the P-type Turing machine which he "trained" by hand (Turing, 1948; Copeland and Proudfoot, 1996) received "reward" and "punishment" and that machines should be taught "discipline" and "initiative" (Turing, 1948: 17-21). The P-type is a "child-machine" and a child- machine could not be sent to school "without the other children making excessive fun of it" (Turing, 1950a: 456-457). Its education, however, "should be entrusted to some highly competent schoolmaster" (Turing, 1951a). Ongoing make-believe of this kind (see Proudfoot, 1999) has entrenched in AI the aim of building machines that resemble human beings, even if the human qualities mimicked are irrelevant to the actual AI project.

Wittgenstein's arguments present a challenge to Turing's romantic aim of producing an artificial "res cogitans." Wittgenstein argued that psychological states and their representational contents are individuated in terms of a subject's behaviour, history and social environment, irrespective of internal states. He also argued that ordinary (belief-desire) psychological explanation is not causal (Wittgenstein, 1965: 15, 110), and that, using such explanation, we can give different accounts of the behaviour of individuals who are physical duplicates of each other but have

* Wittgenstein's remarks here anticipate Searle's Chinese room argument, see Proudfoot (2000).


different histories or environments. Computers do not as a matter of fact have the

history necessary for genuine psychological states. This leads to the question: if

giving a computer the requisite history is either impossible or too costly in time and money, and if the computer minus this requirement nevertheless does all that we want it to do, then why should we care if it does not really think?*

8. Morphogenesis

In his final years Turing worked on what would now be called Artificial Life or A-Life, using the Ferranti Mark I to model biological growth (Turing, 1952a). In

February 1951 he wrote in a letter to a colleague at the NPL:

Our new machine [the Ferranti Mark I] is to start arriving on Monday. I am

hoping as one of the first jobs to do something about 'chemical embryology.' In particular I think one can account for the appearance of Fibonacci numbers in connection with fir-cones.**

Turing used the computer to simulate a chemical mechanism by which the genes of a zygote may determine the anatomical structure of the resulting animal or plant. During this period Turing achieved the distinction of being the first to engage in the computer-assisted exploration of non-linear dynamical systems (his theory used non-linear differential equations to express the chemistry of growth).

Turing wrote concerning his work on neural computation and on morphogenesis in a letter to the biologist Young:

I am afraid I am very far from the stage where I feel inclined to start asking any anatomical questions [about the brain]. According to my notions of how to set about it that will not occur until quite a late stage when I have a fairly definite

theory about how things are done. At present I am not working on the problem at all, but on my mathematical theory of embryology . . . This is yielding to treatment, and it will so far as I can see, give satisfactory explanations of

i) Gastrulation ii) Polyogonally symmetrical structures, e.g., starfish, flowers

iii) Leaf arrangement, in particular the way the Fibonacci series . . . comes to be involved iv) Colour patterns on animals, e.g., stripes, spots and dappling v) Patterns on nearly spherical structures such as some Radiolaria, but this is more difficult and doubtful. I am really doing this now because it is yielding more easily to treatment. I think it is not altogether unconnected with the other problem. The brain structure has to be one which can be achieved by the

genetical embryological mechanism, and I hope that this theory that I am now

working on may make clearer what restrictions this really implies. What you tell me about growth of neurons under stimulation is very interesting in this

* See Proudfoot (forthcoming), "Why did Wittgenstein think that computers don't follow rules, and does it matter?"

** Turing to Michael Woodger, undated, received 12 February 1951.


connection. It suggests means by which the neurons might be made to grow so as to form a particular circuit, rather than to reach a particular place.*

In June 1954 Turing died, while in the midst of this groundbreaking work. He left a large pile of handwritten notes and some programs.** This material is still not fully understood.

Acknowledgements

Research on which this article draws was supported in part by University of Can- terbury Research Grant No. U6271 (Copeland) and Marsden Fund Research Grant No. UOC905 (Copeland and Proudfoot).

* Turing to Young, 8 February 1951. ** These are in the Modern Archive Centre, King's College, Cambridge.

References

Abramson, EG., 1971, "Effective computation over the real numbers," in Twelfth Annual Symposium on Switching and Automata Theory, Northridge, CA: Institute of Electrical and Electronics

Engineers. Ambrose, A., 1935, "Finitism in mathematics (I and II)," Mind 35, 186-203 and 317-340. Boolos, G.S. and Jeffrey, R.C., 1980, Computabihty and Logic, 2nd edn., Cambridge: Cambridge

University Press. Chaitin, G.J., 1988, "Randomness in arithmetic," Scientific American 259, 80-85. Churchland, P.M. and Churchland, P.S., 1990, "Could a machine think?" Scientific American 262,

26-31.

Copeland, B.J., 1993, Artificial Intelligence, Oxford: Blackwell.

Copeland, B.J., 1997, "The broad conception of computation," American Behavioral Scientist 40, 690-716.

Copeland, B.J., 1998a, 'Turing's O-machines, Penrose, Searie, and the brain," Analysis 58, 128-138. Copeland, B.J., 1998b, "Even Turing machines can compute uncomputable functions," pp. 150-164.

in Unconventional Models of Computation, C. Calude, J. Casti, and M. Dinneen, eds., London: Springer-Verlag.

Copeland, B.J., 1998c, "Super Turing-machines," Complexity 4, 30-32. Copeland, B.J., ed., 1999a, "The Turing-Wilkinson lectures on the Automatic Computing Engine,"

pp. 381-444 in Machine Intelligence 15, K. Furukawa, D. Michie, and S. Muggleton, eds., Oxford: Oxford University Press.

Copeland, B.J., ed., 1999b, A lecture and two radio broadcasts by Alan Turing, pp. 445-475 in Machine Intelligence 15, K. Furukawa, D. Michie, and S. Muggleton, eds., Oxford: Oxford

University Press.

Copeland, B.J., 2000a, "Narrow versus wide mechanism," Journal of Philosophy 97, 1-32. Copeland, B.J., ed., 2000b, Alan Turing's Automatic Computing Engine, London: Springer-Verlag. Copeland, B.J., 2000c, "Artificial intelligence," Encyclopedia Britannica (in press). Copeland, B.J. and Proudfoot, D., 19%, "On Alan Turing's anticipation of connectionism," Synthese

108, 361-377.

Copeland, b.J. and Proudtoot, D., l*wa, Alan luring s forgotten ideas in computer science, Scientific American 280 (April), 76-81.

Copeland, B.J. and Proudfoot, D., 1999b, "The Legacy of Alan Turing," Mind 108, 187-195.

508 B J. COPELAND AND D. PROUDFOOT

Copeland, B.J. and Sylvan, R., 1999, "Beyond the universal Turing machine," Australasian Journal of Philosophy 77, 46-66.

Hodges, A., 1992, Alan Turing: The Enigma, London: Vintage. Langton, C.R., ed., 1989, Artificial Life, Redwood City, CA: Addison-Wesley. McCulloch, W.S. and Pitts, W., 1943, "A logical calculus of the ideas immanent in nervous activity,"

Bulletin of Mathematical Biophysics 5, 1 15-133. Nedo, M. and Ranchetti, M., 1983, Wittgenstein: Sein Leben in Bildern und Texten, Frankfurt:

Suhrkamn Verlac. Proudfoot, D., 1997, "On Wittgenstein on cognitive science," Philosophy 72, 189-217. Proudfoot, D., 1999, "How human can they get?" Science 284, 745. Proudfoot, D., 2000, "Wittgenstein's anticipation of the Chinese Room," in Views into the Chinese

Room, J. Preston and M. Bishop, eds., Oxford: Oxford University Press (in press). Proudfoot, D. and Copeland, B.J., 1994, "Turing, Wittgenstein and the science of the mind,"

Australasian Journal of Philosophy 72, 497-519. Russell, B.A.W., 1915, Our Knowledge of the External World as a Field for Scientific Method in

Philosophy, Chicago, IL: Open Court. Russell, B.A.W., 1936, "The limits of empiricism," Proceedings of the Aristotelian Society 36, 131-

150. Searle, J., 1992, The Rediscovery of the Mind, Cambridge, MA: MIT Press. Stewart I.. 1991. "Decidine the undecidable." Nature 352. 664-665. Turing, A.M., 1936, "On computable numbers, with an application to the Entscheidungsproblem,"

Proceedings of the London Mathematical Society, Series 2 42 (1936-37), 230-265. Turing, A.M., 1938, "Systems of logic based on ordinals," Dissertation presented to the faculty of

Princeton University in candidacy for the degree of Doctor of Philosophy. Turing, A.M., 1939, "Systems of logic based on ordinals," Proceedings of the London Mathematical

Society, Series 2 45, 161-228. Turing, A.M., 1945, "Proposal for development in the mathematics division of an Automatic Com-

puting Engine (ACE)," National Physical Laboratory, London, pp. 2(M5 in AM. Turing's ACE Report of 1946 and Other Papers, B.E. Carpenter and R.W. Doran, eds., Cambridge, MA.: MIT Press.

Turing, A.M., 1947, "Lecture to the London Mathematical Society on 20 February 1947," pp. 106- 124 in A.M. Turing's ACE Report of 1946 and Other Papers, B.E. Carpenter and R.W. Doran, eds., Cambridge, MA: MIT Press.

Turing, A.M., 1948, "Intelligent machinery," National Physical Laboratory, London, pp. 3-23 in Machine Intelligence 5, B. Meltzer and D. Michie, eds., Edinburgh: Edinburgh University Press.

Turing, A.M., 1950a, "Computing machinery and intelligence," Mind 59, 433-460. Turing, A.M., 1950b, Programmers' Handbook for Manchester Electronic Computer, Royal Society

Computing Machine Laboratory, University of Manchester. Turing, A.M., 1951a, "Intelligent machinery, a heretical theory," in Copeland (1999b). Turing, A.M., 1951b, "Can digital computers think?," BBC Radio Broadcast, in Copeland (1999b). Turing, A.M., 1952a, "The chemical basis of morphogenesis, Philosophical Transactions of the

Royal Society of London, Series B 237, 37-72. Turing, A.M., 1952b, "Can automatic calculating machines be said to think?" BBC Radio Broadcast,

in Copeland (1999b). Turing, A.M., 1953, "Digital computers applied to games," typescript, reproduced as part of a longer

article of the same name by Turing et al. in Faster than Thought, B.V. Bowden, ed., 1953, London: Pitman.

Von Neumann, J., ed., 1945, "First draft of a report on the EDVAC," Moore School of Electrical Engineering, University of Pennsylvania.

Williams, F.C., 1975, "Early computers at Manchester University," The Radio and Electronic Engineer 45, 237-331.


Wittgenstein, L., 1953, Philosophical Investigations, G.E.M. Anscombe, trans., Oxford: Blackwell.

Wittgenstein, L., 1965, The Blue and Brown Books, New York: Harper. Wittgenstein, L., 1967a, Remarks on the Foundations of Mathematics, G.H. von Wright, R. Rhees,

and G.E.M. Anscombe, eds. (2nd edn.), Oxford: Blackwell.

Wittgenstein, L., 1967b, Zettel, G.E.M. Anscombe and G.H. von Wright, eds., G.E.M. Anscombe, trans., Oxford: Blackwell.

Wittgenstein, L., 1976, Lectures on the Foundations of Mathematics, C. Diamond, ed., Ithaca: Cornell

University Press.

Wittgenstein, L., 1980, Remarks on the Philosophy of Psychology, Vol. I, G.E.M. Anscombe and G.H. von Wright, eds., G.E.M. Anscombe, trans., Oxford: Blackwell.

what turing did after he invented the universal turing machine · 2009-11-04 · what turing did...

Documents