Ml

■«*uww»_-_.ni -■**a*,j*m»itmrnt!'*£>- .

Future Generation Computer Systems 9(1993) 105-117North-Holland

105

The Japanese National Fifth GenerationProject: Introduction, survey, and evaluationEdward Feigenbaum a and Howard Shrobe b

" Knowledge Systems Laboratory, Stanford University, 701 Welch Road, Building

C,

Palo Alto, CA 94304, USAbAI Lab, Massachusetts Institute of Technology, 545 Technology Square, Cambridge, MA 02139, USA

Abstract

Projecting a great vision of intelligent systems in the service of the economy and society, the Japanesegovernment in 1982launched the national Fifth Generation Computer Systems (FGCS) project. The project was carried out by a centralresearch institute,

ICOT,

with personnel from its member-owners, the Japanese computer manufacturers (JCMs) and otherelectronics industry firms. The project was planned for ten years, but continues through year eleven and beyond. ICOTchose to focus its efforts on language issues and programming methods for logic programming, supported by specialhardware. Sequential 'inference machines' (PSI) and parallel 'inference machines' (PIM) were built. Performances of thehardware-software hybrid was measured in the range planned (150 million logical inferences per second). An excellentsystem for logic programming on parallel machines was constructed (XLI). However, applicationswere done in demonstra-tion form only (not deployed). The lack of a stream of applications that computer customers found

effective,

and the soleuse of a language outside the mainstream, Prolog, led to disenchantmentamong the JCMs.

Keywords. FGCS; inference engine; logic programming; parallel architecture.

1. Introduction so by the MITI planners and ETL scientists whotook charge of the project. A revised planning

In 1981, the emergence of the government-in- document was issued in May 1982 that set moredustry project in Japan known as Fifth Genera- realistic objectives for the Fifth Generation Pro-tion Computer Systems was unexpected and dra- ject.matic. The Ministry of International Trade and Newton's third law, loosely interpreted to ap-Industry (MITI) and some of its scientists at ply to the beliefs of people (and journalists), saysElectrotechnical Laboratory (ETL) planned a that grand actions generate equal and oppositeproject of remarkable scope, projecting both reactions. Ten years later, the opposite reactionstechnical daring and major impact upon the econ- have grown into a loud chorus of negative assess-omy and society. ments of the achievements of the Fifth Genera-

This project captured the imagination of the tion Project that have appeared in scientific jour-Japanese people (e.g. a book in Japanese by nals, in trade magazines, and in the popular press.Junichiro Uemae rcounting its birth was titled These reactions are (symmetrically) unrealisti-The Japanese Dream). It also captured the atten- cally negative.tion of the governments and computer industries We believe that the Fifth Generation Projectof the USA and Europe, who were already wary will have significant long term implications forof Japanese takeovers of important industries. A the computer industry in Japan. Several of MITI'sbook by Feigenbaum and McCorduck, The Fifth major goals were achieved. The foundation tech-Generation, was a widely-read manifestation of nology called for in the 1982 plan was broughtthis concern [3]. The Japaneseplan was grand but into existence. By the end of the project, theit was unrealistic, and was immediatelyseen to be hardware and software performed more or less as

0376-5075

/93/$06.00

© 1993 -Elsevier Science Publishers B.V. All rights reserved

mn

■ i ■' ■' ' ' ■ '

p

iii—___Mi-Mi _——___■■ ■Hniiit.irr

E. Feigenbaum, H. Shrobe106

originally planned. As important, the original (buttacit) educational goal of MITI was achieved.Thousands of young Japanese engineers weretrained in the advanced concepts of computerscience and technology that they were not beingtaught in their universities;and they were trainedto think and invent in a creative open-endedway.At a conference in Tokyo in 1986, one of MITI'smain planners of the Fifth Generation Project,Mr. Konishi, told his audience that the educationgoal was MITI's main goal in funding the project!

terns project in 1982. It is now celebrating theend of its eleventh year. ICOT's life may beextended for a total of 3 more years with areduced staffing level.

The Fifth Generation Computer Systems pro-ject was motivated by the observation that " Cur-rent computers are extremely weak in basic func-tions for processing speech, text, graphics, pictureimages, and other non-numeric data, and forartificial intelligence type processing such as in-ference, association, and learning." To addressthese shortcomings, the Fifth Generation projectwas commissioned to build the prototypes for anew (the fifth) generation of hardware and soft-ware. Early ICOT planning documents [6] iden-tify the following requirements:

At the project's 10th anniversary conference in1992, Japan's foremost corporate executive in in-formation technology research told us, "Do notunderestimate the importance of the Fifth Gen-eration Project; it has trained the next generationof Japanese computer architects."

.

(1) To realize basic mechanisms for inference,.The goals for a broad economic and societal

application of intelligent systems (realizedby logicprogramming) were not achieved. Of course thisis the dimension that has led to the disenchant-ment. Where are the language, speech, vision,and problem-solving applications? They did notemerge in the first decade, but that does notmean that they will not. The fruits of the PIPSproject (Pattern Information Processing Project)of the 1970s did not appear until the mid-to-late1980s in the form of a superb national imageprocessing capability. Perhaps the Fifth Genera-tion Project was somewhat early with its ad-vanced goals. We don't yet know where the capa-bilities that it developed will go in the Japaneseindustry and computer science.

association, and learning in hardware andmake them the core functions of the fifthgeneration computers.

(2) To prepare basic artificial intelligence soft-ware to fully utilize the above functions.

(3) To take advantageof pattern recognition andartificial intelligence research achievements,and realize man-machine interfaces that are .natural to man.

(4) To realize support systems for resolving the t

'software crisis' and enhancing software pro-tduction.

A fifth generation computer system in thisearly ICOT vision is distinguished by the central-ity of

I

(1) Problem solving and inference,(2) Knowledge-base management,(3) Intelligent interfaces.

In 1992, we served on an American visitingcommittee (called a JTEC committee) that madean intensive six day visit to Japan to assessJapanese applications of and research in knowl-edge-based systems. As part of that trip, we vis-ited ICOT, and spoke with several of its leaders.We and they knew each other well as scientificcolleagues, so the discussions were direct, open,and frank. Much of what they had to say isincorporated in the survey we will now present;and much of the text is taken from the report ofthat committee [4].

.

.Such a system obviously requires enormouscomputing power. In ICOT's view, it also re-quired a new type of computing power, one moresymbolic and inferential in character than con-ventional systems. Also, the system was explicitlyassumed to rely on very large knowledge basesand to provide specialized capabilities for knowl-edge and data base management. Finally, fifthgeneration systems were assumed to interact withpeople in a more human like manner, using natu-ral language in both printed and spoken form.

The 1982 ICOT planning document [6] callsfor a three tier system: At the base is a tier forKnowledge-Base Systems which includes a paral-lel database management hardware and knowl-edge base managementsoftware. This system was

.

2. History and goals .

ICOT was founded as the central researchlaboratoryof the Fifth Generation Computer Sys-

m

**««»mi im

■

■.■■liv.!. >>< -^-mmsmpfmmmMtmammßSA'im', , .,.mmmmaemu* " ' _ »H, H—jim. . - "MitmummuL*!nw

i The Japanese National Fifth GenerationProject 107

.

envisioned as " . . . A database machine with 100to 1000 Gb capacity" and able" ... to retrieve theknowledge bases required for answering a ques-tion within a few seconds."

"The intention of software for the knowledgebase management function will be to establishknowledge information processing technologywhere the targets will be development of knowl-edge representation systems, knowledge base de-sign and maintenance support systems, large-scaleknowledge base systems, knowledge acquisitionexperimental systems, and distributed knowledgemanagement systems . . . One particularly impor-tant aim will be semi-automated knowledge ac-quisition, that is, systems will be equipped with acertain level of learning functions."

Built on top of this are a problem solving andinference tier, including hardware for parallelinference, abstract datatyping and dataflow ma-chines; this tier includes software for a FifthGeneration kernel language (see below), coopera-tive problem solving mechanisms and parallel in-ference mechanisms.

The final tier is the Intelligent Man-Machineinterface system. This was supposed to includededicated special purpose hardware for speechand other signal processing tasks and software fornatural language, speech graphics and image pro-cessing:

"The intelligent interface function will have tobe capable of handling communication with thecomputer in natural language, speech, graphics,and picture images so that information can beexchanged in a ways natural to man. Ultimatelythe system will cover a basic vocabulary (exclud-ing specialist terms) of up to 100,000 words andup to 2000 grammatical rules, with a 99% accu-racy in syntactic analysis.

"The object speech inputs will be continuousspeech in Japanese standard pronunciation bymultiple speakers, and the aims here will be avocabulary of 50,000 words, a 95% recognitionrate for individual words, and recognition of pro-cessing within 3 times the real time of speech."

"The system should be capable of storingroughly 10,000 pieces of graphic and image infor-mation and utilizing them for knowledge informa-tion processing."

These three tiers were then supposed to sup-port a very sophisticated program developmentenvironment to raise the level of software pro-

ductivity and to support experimentation in newprogramming models. Also the basic three tierswere supposed to support a variety of basic appli-cation systems: Those listed in the 1982 docu-ment include: Machine Translation, ConsultationSystems, and Intelligent programming systems(including automated program synthesis and veri-fication).

The application systems were assumed tobe ofsignificant size and sophistication with the follow-ing being typical features:

" Number of objects: many thousands

" Inference rules: 10,000 or more" Semi-automated knowledge acquisition

" Interfaces with system: Natural language andspeech

" Vocabularysize: 50,000 words or more.

3. The commitment to logic programming

Achieving such revolutionarygoals would seemto require revolutionary techniques. Conventionalprogramming languages, particularly those com-mon in the late 1970s and early 1980s offeredlittle leverage. The requirements clearly sug-gested the use of a rich, symbolic programminglanguage capable of supporting a broad spectrumof programming styles. Two candidates existed:LISP which was the mainstream language of theUS Artificial Intelligence community and Prologwhich had a dedicated following in Europe. LISPhad been used extensively as a systems program-ming language and had a tradition of carryingwith it a featureful programming environment; italso had already become a large and somewhatmessy system. Prolog, in contrast, was small andclean, but lacked any experience as an implemen-tation language for operating systems or pro-gramming environments.

It was decided early on that ICOT would baseits work on Logic Programming rather than Lisp;that it would build on but significantly extendProlog. Also it was decided that the Logic Pro-gramming language would be a 'Kernel Lan-guage' that would be used for a broad spectrumof software, ranging from the implementation ofthe system itself up through the application lay-ers.

E. Feigenbaum, H. Shrobe108

In practice, ICOT's central focus became thedevelopment of such a logic programming kernellanguage and the development of hardware tai-lored to the efficient execution of this language.The system's performance target was to be from100 Mega LIPS (logical inferences per second,i.e. simple Prolog procedure calls) to 1 GigaLIPS. (As a reference point, ICOT documentsestimate that 1 Logical Inference takes about 100instructions on a conventional machine; a 1MLIPS machine would therefore be roughlyequivalent to a 100 MIPS processor, although thiscomparison may confuse more than it reveals.)The rather reasonable assumption was made thatachieving such high performance would requireparallel ' processing: "... the essential researchand development will concentrate... on high-level parallel architectures to support the symbolprocessing that is the key to inference." Further-more, the assumption was made that achievingthe desired performance target would requireabout 1000 processing elements per system givenreasonable assumptions on the performance of asingle such processing element.

4. Accomplishments

4. 1. Accomplishments in hardware

During the first 3 year phase of the project,the Personal Sequential Inference machine (PSI1) was built and a reasonably rich programmingenvironment was developed for it (Fig. 1).

>

To put this effort in context, we compare it tothe US project which it most resembles: the MITLisp Machine. The MIT project had begun in thelate 1970s and had just reached commercializa-tion at the time of ICOT's inception. Like theMIT machine, PSI was a microprogrammed pro-cessor designed to support a symbolic processinglanguage. The symbolic processing languageplayed the role of a broad spectrum 'Kernellanguage' for the machine, spanning the rangefrom low level operating system details up toapplication software. The hardware and its mi-crocode were designed to execute the kernel lan-guage with high efficiency. The machine was areasonably high performance work station withgood graphics, networking and a sophisticatedprogramming environment.

\

>

>

.

i

i

ICOT's development plans were segmentedinto three phases. The goal for the initial phasewas to develop a 'Personal Sequential InferenceMachine' (PSI), i.e. a workstation tailored to effi-cient execution of a sequential logic programminglanguage. This phase was also supposed to de-velop the system software for high capability pro-gramming environments for Fifth Generationsoftware. The initial considerations of parallelsystems were also to begin during this stage.

rWhat made PSI different was the choice of

language family. Unlike more conventional ma-chines which are oriented toward numeric pro-cessing or the MIT machine which was oriented

.

.

.In the second phase, a refined Personal Se-

quential Inferencewould be developed, the modelfor parallel programming would be settled uponand initial exploratory parallel architectureswould be prototyped.

.

.

IThe third phase would build the Parallel Infer-ence Machines (PIM). This would include notonly the hardware effort, but a parallel operatingsystem and a second generation kernel languageappropriate for parallel processing.

I

}

I1 In the early 1980s parallelism was the 'coming thing' in

computer science. Since a goal of the fifth generation pro-ject was to 'jump headlong into the future of computing' itwas not only necessary to embrace parallelism, it was desir-able.

.

I

Fig. 1 . Major accomplishments.

>

v

>*»■»——mmmm^mtmm i

■

mmmmmmimtm^^gmgmn^^^^

...'■rlr.Aiaiw.-iA.a.wa,

I

The JapaneseNationalFifth Generation Project I109

towards LISP the language chosen for PSI wasProlog. The primary appeal of Prolog-like lan-guages to ICOT was the analogy between thebasic operations of Prolog and simple rule-likelogical inferencing. A procedure in such a lan-guage can be viewed as simply reducing a goal toits subgoals. Given the emphasis on inference asa key component of the FGCS vision, the choiceseemed quite natural. However, the choice of alogic programming framework for the kernel lan-guage was a radical one since there had beenessentially no experience anywhere with usinglogic programming as a framework for the imple-mentation of core system functions.

PSI-1 achieved a performance of about 35KLIPS, comparable to DEC- 10 Prolog, the Prologperformance of the Symbolics 3600 (one of thefollow-ons to the MITLisp Machine)or Quintus'sProlog implementation for Sun-3 class machines.This was fast enough to allow the development ofa rich operating system and programming envi-ronment, but still quite slow compared to thePhase 3 goals (1000 processors achieving 1 GLIPS,implies at least 1 MLIPS per processor). Twoextended Prolog-like languages (ESP and KLO)were developed for PSI-1. ESP (Extended SelfContained Prolog) included a variety of featuressuch as coroutining constructs, non-local cuts,etc. necessary to support system programmingtasks as well as more advanced Logic Program-ming. SIMPOS, the operating system for the PSImachines, was written in ESP.

Several hundred PSI machines were built andinstalled at ICOT and collaborating facilities;andthe machine was also sold commercially. How-ever, even compared to specialized Lisp hardwarein the US, the PSI machines were unpracticallyexpensive. The PSI (and other ICOT) machineshad many features whose purpose was to supportexperimentation and whose cost/benefit tradeoffhad notbeen evaluated as part of the design; themachines were inherently non-commercial.

During Phase 1, it was decided to explore an'And-Parallel' approach to parallel logic pro-gramming. To simplify, this means that the sub-goals of a clause are explored in parallel withshared variable bindings being the means of com-munication. The process solving one subgoal cancommunicate with a process solving a sibling sub-goal by binding a shared variable to a concretevalue. It was also observed that subgoals would

have to spread out across the network of proces-sors constituting the parallel machine and that itwould require careful control to avoid thebuildupof communication bottlenecks. By the end ofPhase 1, the form of the parallel kernel languagewas clarified: it was to be a 'Flat Guarded HornClause' (FGHC) language. A Flat Guarded HornClause consists of three parts:(1) The head,(2) the guard,(3) the body.

The head plays exactly the same role as thehead of a Prolog Clause: it identifies a set ofgoals for which the clause is suitable (i.e. thosegoals which unify with the head). The guard andbody collectively play the role of the body of aProlog clause, i.e. they are a set of subgoalswhose truth implies the truth of the head. How-ever, the body of the clause is not executed untilall variables in the guard are instantiated and allliterals in the guard are satisfied. In the casewhere two (or more) clauses have heads thatunify with the same goal, only the body of thatclause whose guard is first satisfied will execute(hence the name guarded horn clause. 'Flat'means that the guard can only contain built-inpredicates, rather than those which are evaluatedby further chaining. This greatly simplifies themechanisms, without significantly reducing theexpressive power).

The execution of a FGHC program is summa-rized in four rules:(1) Relevance: Those clauses whose head unifies

with a goal are potentially executable.(2) Synchronization: Until the caller has suffi-

ciently instantiated the variables to allow theguard part of the clause to execute,executionof the guard is suspended.

(3) Selection: If there are more than one poten-tially executableclauses for a goal, that clausewhose guard succeeds first will execute itsbody and the bodies of all other competingclauses will never execute.

(4) Parallelism: The subgoals in the body areexecuted in parallel.

Figure 2 shows a set of FGHC for a primesieve algorithm and how they begin to elaborate aparallelprocess structure. One should notice thatthis interpretation model does not lead to anautomatic search mechanism as in Prolog. InProlog all relevant clauses are explored, and the

110 E. Feigenbaum, H. Shrobe

-' *MM"'M|'*w*"*ll'^**^",̂*^*< -'

■ ■

mi ipi.ii -i mi hi ■ nmmmammmmmmmmmmmmmmmmmmmtmmmtKtttt^^

*

i

>

>

s

.

\

paradigm with a variety of useful features such asa macro language.

KLI-p includes the 'pragmas' for controllingthe implementation of the parallelism. There arethree main pragmas. The first of these is a meta-level execution control construct named 'shoen'which allows the programmer to treat a group ofprocesses (i.e. a goal and its subgoals) as a unit ofexecution control. (A shoen is created by calling aspecial routine with the code and its arguments;this creates a new shoen executing the code andall generated subgoals. These subgoals are, how-ever, running in parallel). A failure encounteredby any sub-process of a shoen is isolated to thatshoen. Each shoen has a message stream and areport stream by which it communicates with theoperating system; shoens may be nested but theOS treats the shoen as a single element. Suspend-ing a shoen results in the suspension of all itschildren, etc. Fine grain process management ishandled by the shoen, freeing the OS from thisresponsibility.

Fig. 2. Prime sieve algorithm using guarded horn clauses.

order of exploration is specified by the program-ming model. In FGHC only a single relevantclause is explored; ICOT has had to conductresearch on how to recapture search capabilitieswithin the FGHC framework.

The second pragma allows the programmer tospecify the priority of a goal (and the process itspawns). Each shoen has a minimum and maxi-mum priority for the goals belonging to it; thepriority of a goal is specified relative to these.The second 3 yearphase saw the development

of the PSI 2 machine which provided a significantspeedup over PSI 1. Towards the end of Phase 2a parallel machine (the Multi-PSI) was con-structed to allow experimentation with the FGHCparadigm. This consisted of an 8 X 8 mesh of PSI2 processors, running the ICOT Flat GuardedHorn Clause language XLI (not to be confusedwith the knowledge representation language KL-ONE developed at Bolt Beranek and Newman).Multi-PSI supported the development of theICOT parallel operating system (PIMOS) andsome initial small scale parallel application devel-opment. PIMOS is a parallel operating systemwritten in XLI; it provides parallel garbage col-lection algorithms, algorithms to control task dis-tribution and communication, a parallel file sys-tem, etc.

The third pragma allows the programmer tospecify the processor placement for a body goal.This may be a specific processor or a logicalgrouping of processors. All three of these prag-mas are 'meta execution control' mechanismswhich themselves execute at runtime; KLI-p thusallows dynamic determination of the appropriatepriority, grouping and placement of processes.

Much of the current software is written inhigher level languagesembedded in XLI, particu-larly languages which establish an object orienta-tion. Two such languages have been designed:A'UM and AYA. Objects are modeled as pro-cesses communicating with one another throughmessage streams. The local state of an object iscarried along in the cyclical call chain from dis-patching routine to service subroutine back todispatching routine. Synchronization betweenprocesses is achieved through the binding of vari-ables in the list structure modeling the messagestream. (See also Bal's contribution in this issue[1]).

Phase 3 has been centered around the refine-ment of the XLI model and the development ofmassively parallel hardware systems to execute it.XLI had been refined into a three level language.KLI-b is the machine level language underlyingthe other layers. KLI-c is the core language usedto write most software; it extends the basic FGHC

Five distinct Parallel Inference Machines(PIMs) have been developed to execute XLI,

prime(Max, PS) <- true I generator (2,

Max,

Ns),

sifter

(Ns.Ps).

generator (NO,

Max,

Ns)

<-NO-<

Max 1

NeO

-(NO INil],NI :-

NO

♦1,generator (NI,

Max,

Nsl).generator (NO,

Max,

NsO) <"-

NO

> Max I

NsO

- [].

sifter

((PlXsl], ZsO) <- true I

ZsO

-[P I Zsl],

nitertP, X2l,

Ysl),

sifter

(Ys,Zsl).

sifter

<[], ZsO) <~ true I

ZsO

- [].

niter (P, [XlXsl], YsO) <""(Xmod P)

■/

" 0 I

YsO

- [XIYsl],

filter

(P,

Xsl,

Ysl).

filter

(P, [XlXsl],YsO) <» (X mod P) -- 0 I

filter

(P,

Xsl,

YsO).niter (P, [],YsO) <- true I

YsO

- {].

|generator! — [2,3,4, _ 1 *> |

sifter

1| generator] -- [2,3,4,

m

) -♦- |prime2l»f

flltert

I*H

»'fter

1| generator| — [2,3,4, ... ] | Altera | -*" [34,7,9, ...I—i

L^. |prime3| |

filters

1 |

sifter

|

I IHWIIMIIMT———~— ■" ■ "■' "" -----■■■■Ifil^rirMMilM.lilia^^

f|

The Japanese NationalFifth Generation Project 111

each built by a commercial hardware vendor as-sociated with ICOT. They vary in processor de-sign and communication network. The abstractmodel of all PIMs consists of a loosely couplednetwork connecting clusters of tightly coupledprocessors. Each cluster is, in effect, a sharedmemory multiprocessor; the processors in thecluster share a memory bus and implement acache coherency protocol. Three of the PIMs aremassively parallel machines: PIM/p, PIM/m andPIM/c. PIM/k and PIM/i are research ma-chines designed to study specific intracluster is-sues such as caching and bus communication.Multi-Psi is a medium scale machine built byconnecting 64 Psi's in a mesh architecture.PIM/m and Multi-Psi do not use a cluster archi-tecture (but may be considered as degeneratecases having one processing element per cluster).

The main features of their communication sys-tems are shown below:

Topology #Cluster #PE Memory/cluster

64 512 256 Mb256 256 80 Mb

32 256 160 Mb

PIM/p hypercubex 2PIM/m meshPIM/c crossbarPIM/k 4 16 1 Gb

2 16 320 Mb64 64 80 Mb

PIM/iMultiPsi mesh

#PEs/ #Network Transfer RateCluster Interfaces/

Cluster

33 Mb/sec X 28 Mb/sec

PIM/p 8PIM/m 1PIM/c 8PIM/k 16PIM/i 8MultiPsi 1

81

40 Mb/sec1

11 10Mb/sec

Relevant features about the processing elements' implemen-tation technology are shownbelow:

Instruction Cycle Fabrication LineSet Time Technology Width

PIM/p RISC 60 nsec standardcell 0.96 micronPIM/m CISC (ucode) 65 nsec standard cell 0.8 micronPIM/c CISC (ucode) 50 nsec gate array 0.8 micronPIM/k RISC 100 nsec custom 1.2micronPIM/i RISC 100nsec standard cell 1.2micronMultiPsi CISC (ucode) 200 nsec gate array 2.0 micron

It should be noted that the cycle times for theprocessing elements are relatively modest. Com-mercial RISC chips have had cycle times lowerthan these for several years (the lower the cycletime, the faster the instruction rate). Newly

emerging processor chips (such as the DEC AL-PHA) have cycle times as low as 5 nsec. Evengranting that special architectural features of thePIM processor chips may lead to a significantspeedup (say a factor of 3 to be very generous),these chips are disappointing compared to thecommercial state of the art. The networks used tointerconnect the systems have respectablethroughput, comparable to that of commerciallyavailable systems such as the CM-5. In certain ofthe PIMs each processor (or processor cluster)can have a set of disk drives; this may allow morebalance between processing power and I/Obandwidth for database applications, but there isas yet no data to either confirm or refute this.(See also Tick's contribution in this issue [9].)

4.2. Accomplishments in software

ICOT's software vision has beenradical. In theconventional view the hardware supports an Op-erating System and language implementation(s).These in turn support a window system and in thebest case perhaps a text editor, graphics editorand a user interface management system. Theapplication developer must then span the entireremaining distance to the needs of the end users.This is tractable if the end user is expected toneed no more than a spreadsheet and word pro-cessor.

However, in ICOT's initial vision, the end-useris expected to communicate with the computerusing natural languageand images. The computeris required to know a fair amount about the realworld and to be capable of performing intelli-gently. To imagine building such applications,one must assume a much higher level startingpoint. Therefore, ICOT's software strategy isbased on providing a much deeper set of softwarelayers to the application developer(see Fig. 3). Inthis view, the application developerbuilds on topof automated deduction systems, constraint sys-tems, and natural language systems.

So far we have been discussing the bottommost layer, that concerned with the operatingsystem and languageruntime system for parallellogic programming. On this foundation, ICOThas pursued research into(1) Databases and knowledge base support,(2) Constraint logic programming,

-»■"*mmmt*KMKtKNKtKKNH*l**",",*m"'w''

112 E. Feigenbaum, H. Shrobe

i

t

i

i

»

All such languages share the idea of merging intoa logic programming context constraint solversfor specific non-logical theories (such as linearequations or linear inequalities). Two languagesof this type developed at ICOT are CAL (Con-straint Avec Logique) which is a sequential con-straint logic programming language which in-cludes algebraic, Boolean, set and linear con-straint solvers. A second language, GDCC(Guarded Definite Clauses with Constraints) is aparallel constraint logic programming languagewith algebraic, Boolean, linear and integer paral-lel constraint solvers.

ExperimentalApplication SystemsParallel

VLSI-CAD

LegalReasoning

Genetic

SequenceMatching

Others?

Knowledge ProcessingNatural Language Software

✓""Conitriint

LogicVv<^ProgTami___is

Parallel Theorem Proven

< Basic Software

Another area explored is that of AutomaticTheorem Proving. ICOT has developed a paralleltheorem prover called MGTP (Model GenerationTheorem Prover). This is written in XLI and runson the PIMs. MGTP has obtained a more than100 fold speedup on a 128 processing elementPIM/m for a class of problems called 'condenseddetachment problems'. MGTP is based on theModel Generation proving methods first devel-oped in the SATCHMO system; however, theICOT version uses the unification hardware ofthe PIMs to speed this up for certain very com-mon cases. MGTP has been used as a utility in ademonstration legal reasoning system. It has alsobeen used to explore non-monotonic and abduc-tive reasoning. Finally, MGTP has been em-ployed in some program synthesis explorations,including the synthesis of parallel programs. (Seealso Stickel's contribution in this issue [B].)

Fig. 3. ICOT's layered approachto softwaredevelopment.

(3) Parallel theorem proving, and(4) Natural language understanding.

In the area of databases, ICOT has developeda parallel database system called Kappa-P. This isa 'nested relational' database system based on anearlier ICOT system called Kappa. Kappa-P is aparallel version of Kappa, re-implemented inXLI. It also adopts a distributed database frame-work to take advantage of the abilityof the PIMmachines to attach disk drives to many of theprocessing elements. Quixote is a KnowledgeRepresentation languagebuilt on top ofKappa-P.It is a constraint logic programming languagewith object-orientation features such as object-identity, complex objects described by the decom-position into attributes and values, encapsulation,type hierarchy and methods. ICOT also describesQuixote as a Deductive Object Oriented Database(DOOD). Quixote and Kappa-P have been usedto build a Molecular Biological Database, and aLegal Reasoning System. (See also Nishio's con-tribution in this issue [7].)

Natural Language Processing has been a finalarea of higher level support software developedat ICOT. There have been several areas of work:(1) A Language Knowledge Base consisting of aJapanesesyntax and dictionary(2) A language tool box containing morphologicaland syntax analyzers, sentence generator,concor-dance system, etc.,(3) A discourse system which rides on top of thefirst two. These are combined in a parallel, coop-erative language understanding system using typeinference. The dictionary has about 150,000 en-tries of which 40,000 are proper names (to facili-tate analysis of newspaper articles). (See alsoBarnett and Yamada's contribution in this issue[2].)

On top of these tools a variety of demonstra-tion application systems (not deployed applica-tions) have been developed. These were shown

ICOT has been one of the world class centersfor research into Constraint Logic Programming.

ut

1-_nflM_-_-_flH---IBfi--M_MiP-Vtt_Maen«n«

113

I

77ie Japanese NationalFifth Generation Project 113

running on the PIM machines at the 10th an-niversary FGCS conferences. They are discussedby van de Riet in this issue [11].

Table 1Positive elementsof evaluation

" ICOT has shown the ability of Japan to innovate in com-puter architectures.

4.3. The legacy " The ICOT architectures'peak parallel performance is withinthe range of the original performance goals.

When asked what they regarded as their legacy,the core achievement of the 10 year ICOT pro-gram, both Director Dr. Fuchi and his DeputyDr. Chikayama said that it was XLI (as opposedto the PIM hardware, the higher level software orthe application demos).

" The PIMs represent an opportunity to study tradeoffs inparallel symbolic computing which does not exist else-where.

" XLI is an interesting modelfor parallel symbolic computa-tion, but one which is unlikely to capture the imaginationof U.S. researchers.

" PIMOS has interesting ideas on control of distribution andcommunication which US researchers should evaluateseriously.There are at least three aspects to what has

been achieved in XLI: " ICOT has been one of the few research centers pursuingparallel symbolic computations.First the language itself is an interesting paral-

lel programming language. XLI bridges the ab-straction gap between parallel hardware andknowledge based application programs. Also it isa language designed to support symbolic (as op-posed to strictly numeric) parallelprocessing. It isan extended logic programming language whichincludes features needed for realistic program-ming (such as arrays). However, it should also bepointed out that like many other logic program-ming languages, XLI will seem awkward to someand impoverishedto others.

" ICOT has been the only center with a sustained effort inthis area.

" ICOT hasshown significant (i.e. nearlylinear) accelerationof non regular computations (i.e. those not suitable fordata parallelism of vectorized pipelining).

" ICOT created a positive aura for Al, Knowledge BasedSystems, and innovativecomputer architectures. Some ofthe best young researchers have entered these fieldsbecause of the existence of ICOT.

Second is the development of a body of opti-mization technology for such languages. Efficientimplementation of a language such as XLI re-quired a whole new body of compiler optimiza-tion technology. Because there are several archi-tecturally distinct PIMs (and the Multi-PSI) ICOThas been forced to develop a flexible implementa-tion strategy for XLI. XLI is compiled into anintermediate language KLI-B which plays a rolesimilar to that which the WAM plays for sequen-tial Prolog compilers. KLI-B is the abstract lan-guage implemented by each hardware system; itis a hardware model of a loosely coupled multi-processor in which some processors are linked intightly coupled clusters. To build a XLI imple-mentation, the architect must transform the ab-stract KLI-B specification into a physical realiza-tion; this is done semi automatically.

guage. This asks for a small amount of additionalhardware which provides a high degree of lever-age for the language implementation without nec-essarily slowing down the processor or introduc-ing undue complexity to the implementation. Suchfeatures, which might also support LISP and othermore dynamic languages may eventuallyfind theirway into commercial processors.

At the time of the 10th anniversary Fifth Gen-eration Conference all of the PIM designs wereoperational and were demonstrated running theapplications listed above.

5. Evaluation

Positive elements of our evaluation of the FifthGeneration Computer Systems project are shownin Table 1. Negative elements are shown in Table2.

The third achievement is noticing where hard-ware can play a significant role in supporting thelanguage implementation. Part of the architec-ture of the PIM (and PSI) processors is a tagchecking component which provides support forthe dynamic type checking and garbage collectionneeded to support a symbolic computing lan-

The early ICOT documents suggested a pushtowards very advanced and very large scaleknowledge based systems. This was a call for aquantum jump in the quality of services whichcomputers provide.

■"f?

_-»MSBMMW_WWWIIiIiPW* r 1̂* "■ e»aw*. «mmiiin»««i«»ii.»iimr-r~I

114 £. Feigenbaum, H. Shrobe

I

!

I

ij

i

.

Table 2Negative elementsof evaluation

" ICOT has done little to advance the state of knowledgebased systems, or Artificial intelligence per se.

" ICOT's goals in the area of natural language were eitherdroppedor spun out to EDR.

" Other areas of advanced man machine interfacing weredropped.Research on Very Large Knowledge bases were substan-tially dropped.ICOT's efforts have had little to do with commercial appli-cation of AI technology. Choice of language was critical.

" ICOT's architectures have been commercial failures. Re-quired both a switch in programming model and the pur-chase of cost ineffective hardware.

" ICOT hardware has lagged behind U.S. hardware innova-tion (e.g. the MIT Lisp Machine and its descendants andthe MIT Connection Machine and its descendants).

" Application systems of the scale described in the originalgoals have not been developed(yet).

" Very little work on knowledge acquisition.

In retrospect, it is clear that this was notachieved. The early documents discuss the man-agement of very large knowledge bases, of largescale natural language understanding and imageunderstanding with a strong emphasis on knowl-edge acquisition and learning. Each of these di-rections seems to have been either dropped, rele-gated to secondary status, absorbed into the workon parallelism or transferred to other researchinitiatives. 2

ICOT set out to build a machine that couldprovide upwards of a 100 MLIPs of 'symbolcrunching' power. However, there was no appli-cation whose clear need for such horsepowerdrove the development and whose success (orfailure) would serve as the tangible evaluation oftheeffort. From our current perspective it is clearthat there could not have been such an applica-tion, since any such application would necessarilyhave had to contain very large stores of knowl-edge; but in 1982 when the project began, therewere no techniques available for capturing andmanaging such a large knowledge base. Eventoday, after more than a decade of research into

2 Importantly, ICOT spun off a new research center, theElectronic Dictionary Research center, located in an adja-cent building and run by a former ICOT manager.

knowledge representation, we have only a smallbase of experience and very few tested techniquesfor this task.

Thus the focus on a quantum jump in thequality of services provided by computers wasreplaced by a crisper but less ambitious one. Thecentral perspective of ICOT's efforts has been todevelopparallel symbolic programming (in partic-ular, parallel logic programming) by developinganew language and by developing experimentalhardware to support the language. Higher levelfacilities such as theorem provers, deductivedatabases and natural language support assumeda secondary role; applications assumed a tertiaryrole.

The set of demo applications for the PIMmachines seem relatively routine. Even thougheach of these programs demonstrates the powerof parallelism and even though each embeds someadvance in parallel programming, when viewed asknowledge based systems, these systems bringlittle new to bear; they fail to qualitatively ad-vance the kind of services which computers pro-vide. For example, the multi-sequence matchingprogram has a new approach to simulated anneal-ing which uniquely capitalizes on the availableparallelism; however, it knows essentially nothingabout genetics and proteins and it provides onlysyntactically oriented services. Of the programsdemonstrated, the legal reasoning system is theonly one which might be fairly termed a knowl-edge based system. Here parallelism was used toaccelerate both case retrieval and logical argu-mentation. Nevertheless, for all the computa-tional power being brought to bear, the systemdid not seem to establish a new plateau of capa-bility.

Were we to ask 'Has ICOT advanced the stateof the art in Logic Programming and relatedtechniques?' the answer would be 'clearly yes.However, if we were to ask whether ICOT hasdirectly accelerated the development of knowl-edge based technology in Japan so far, the an-swer would have to be no.

However, there are other questions which arealso relevant:(1) Has ICOT indirectly affected the state of

knowledge based technology in Japan?(2) Is the ICOT work likely to produce a plat-

form which will ultimately accelerate knowl-edgebased technology in Japan?

-r>-^<rs_«r_

<mtee.;

The Japanese NationalFifth Generation Project 115

(3) Has ICOT's work advanced the state of paral-lel processing technology in Japan (or else-where)?

The answer to (1) is almost certainly yes. Al-most all companies we interviewed said thatICOT's work had little direct relevance to them.The reasons most frequently cited were: The highcost of the ICOT hardware, the choice of Prologas a language and the concentration on paral-lelism. However, nearly as often our hosts citedthe indirect effect of ICOT: the establishment ofa national project with a focus on 'fifth genera-tion technology' had attracted a great deal ofattention for Artificial Intelligence and knowl-edge based technology. Several sites commentedon the fact that this had attracted better peopleinto the field and lent an aura of respectability towhat had been previously regarded as esoteric.One professor in particular told us that AI nowgets the best students and that this had not beentrue before the inception of ICOT and the FifthGeneration project.

Question (2) is considerably more difficult toanswer. ICOT's work has built an elegant frame-work for parallel symbolic computing. Most A.I.people agree that without parallelism there willultimately be a barrier to further progress due tothe lack of compute power. However, this barrierdoes not seem imminent. Workstations with morethan 100 MIPS of uniprocessor performance arescheduled for commercial introduction this year.With the exception of those sub-disciplines with aheavy signal processing component (e.g. vision,speech, robotics) we are more hampered by lackof large scale knowledge bases than we are bylack of parallelism. It is, however, quite possiblethat in the near future this will be reversed andwe will be in need of parallel processing technol-ogy to support very large scale knowledge basedsystems. We will then be in dire need of program-ming methodology and techniques to capitalizeon parallel hardware, and ICOT's work mightwell provide a solution.

Has the ICOT research significantly impactedparallel computing technology? There are argu-ments to be made on both sides of this question.On the positive side we can argue that XLI is aninteresting symbolic computing language. Fur-thermore, it is a parallel symbolic computing lan-guage and virtually no interesting work has beendone elsewhere for expressing parallel symbolic

computation. Another positive point is that ICOThas the test bed of the several PIM machines.This is an unusual opportunity; no other site hasaccess to several distinct implementations of thesame virtual parallel machine. It is not unreason-able to expect significant insights to emergefromthis experimentation. Finally, we can add thatICOT has confronted a set of interesting techni-cal questions about load distribution,communica-tion and garbage collection in a parallel environ-ment.

On the negative side we may cite several argu-ments as well. The ICOT work has tended tobe aworld closed in upon itself. In both the sequentialand parallel phases of their research, there hasbeen a new language developed which is onlyavailable on the ICOT hardware. Furthermore,the ICOT hardware has been experimental andnot cost effective. This has prevented the ICOTtechnology from having any impact on or enrich-ment from the practical work.

Earlier we pointed out the similarities betweenthe ICOT PSI systems and the MIT Lisp Ma-chine and its commercial successors. It's notewor-thy that only a few hundred PSI machines weresold commercially, while therewere several thou-sand Lisp machines sold, some of which continueto be used in important commercial applicationssuch American Express's Authorizers Assistant.The one commercial use we saw of the PSI ma-chines was at Japan Air Lines, where the PSI-IImachines were employed; ironically, they wereremicrocoded as Lisp Machines. Furthermore, theMIT Lisp Machine acted as a catalyst, providinga powerful Lisp engine until better implementa-tion techniques for Lisp were developed for stockhardware. As knowledge based technology hasbecome more routinized in both the US andJapan, commercial KBS tools have been recodedin C. In the US the A.I. research communitycontinues to use LISP as a vehicle for the rapiddevelopment of research insights; there seems tobe little such use of the ICOT technology inJapan.

The PIM hardware seems destined for thesame fate. The processing elements in the PIMshave cycle times no better than 60 ns; even as-suming that the features which provide directsupport for XLI offer a speedup factor of 3, thisleaves the uniprocessor performance lagging be-hind the best of today's conventional micropro-

' ar

E. Feigenbaum, H. Shrobe116

.

fii

!

I

t

I1

.

cessors. Both HP and DEC have announced theimminent introduction of uniprocessors of be-tween 100 and 200 MIPS. The interconnectionnetworks in the PIMs do not seem to constitutean advance over those explored in other parallelsystems. Finally, the PIMs are essentially 'integermachines'; they do not have floating point hard-ware. While the interconnection networks of thePIMs have reasonable performance, this perfor-mance is comparable to that of commercial paral-lel machines in the US such as the CM-5.

It is interesting to compare the PIMs to Think-ing Machines Inc.'s CM-5; this is a massivelyparallel machine which is a descendant of theMIT Connection Machine project. The CM-5 isthe third commercial machine in this line of de-velopment. It can support a thousand Spare chips(and presumably other faster microprocessors asthey arise) using an innovative interconnectionscheme called Fat Trees. Although the Connec-tion Machine project and ICOT started at aboutthe same time, the CM-5 is commercially avail-able and has found a market within which it iscost effective. One reason for this is it has quitegood floating point performance. The PIMs areinteger machines. It appears that the only estab-lished market for massive parallelism is in scien-tific computing, leaving the PIMs with a disad-vantage which will be very difficult to overcome.

The leaders of ICOT were not unaware ofthese problems. ICOT is considering, or has be-gun, a project to build a XLI system for commer-cially available processors. This would decouplethe language from the experimental hardwareand make it more generally available. This greateravailability could in turn allow a greater numberof researchers whose interests are in largeknowl-edge based systems to begin to explore the use ofthe XLI paradigm. Given their implementationstrategy (explained above) this should not be anoverwhelming task. One of the PIM hardwaredesigners has also designed another parallel sys-tem (the AP-1000 which is mesh connected sys-tem of about 1000 SPARC chips); this might be alikely target for such an effort.

In contrast to the Connection Machine efforts(and virtually all other parallel system efforts)which have focused on massively parallel scien-tific computation, the ICOT effort has continuedto focus on symbolic computing. In contrast tothe MIT LISP Machine efforts, which didn't

achieve great enough commercial viability to af-ford a push forward into parallelism, ICOT hashas sustained long term government fundingwhich allowed them to persevere.

6. Concluding observations

Japanesecomputer manufacturers (JCMs), themember-owners of ICOT, complain that ICOTefforts and funds were too much focused onspecialized hardware. They complain that theynow see that it would have been more pertinentand perspicacious of the MITI planners to haveplanned for a project involving client-server net-works and UNIX software. They cite FGCS pro-ject difficulties which began with goals that wereimpossible to achieve and ended with squabblesover intellectual property rights. And they notefinally that because of substantial project delaysin the middle of the ten (now eleven) year pro-ject, not enough time was devoted to doing sub-stantial engineering evaluations and real-worldeffective applications. ICOT's many excellent re-search contributions are drowned in a sea ofcomplaints by the JCMs, whose work and productlines were essentially unaffected by the FifthGeneration Project.

Essentially the same sentiments were heard atthe end of the Pattern Information ProcessingProject, Japan's first national project that incor-porated Al-like goals (project of the 19705).Where was the practical output, the companiesasked. The practical output was in the methods,techniques, software, and know-how related toimage processing that was deposited in the headsof a large number of Japanese engineers. Thiscollective knowledge was used in the 1980s toenormous economic advantage in the engineeringof inexpensive, reliable, and high performancedevices like the Japanese fax machines, scanners,etc.

We believe that it will take several more years,perhaps a decade, for an analogous realization ofthe gain from the Fifth Generation national pro-ject to occur in Japan. But it will happen; and inparallel computation it is already happening.

For final words, we choose to quote from arecent personal retrospective on the Fifth Gener-ation project [9]:

-<" **

The Japanese National Fifth Generation Project 117

I

"ICOT did not create a revolution because itdid not fundamentally change the manufactur-ers... Either another project, or a radical re-structuring of the diametric cultures of educationand industry, will be required to propagate theadvances made in the FGCS project."

References

[1] H. Bal; Evaluation of XLI and the inference machine,Future Generation Comput. Syst., this issue.

[2] J. Barnett and K. Yamada, Evaluation of ICOT's naturallaguage research, Future Generation Comput. Syst., thisissue.

[3] E. Feigenbaum and P. McCorduck, The Fifth Generation,Artificial Intelligence and Japan's Computer Challenge tothe World (Addison-Wesley, Reading, MA 1983).

[4] E. Feigenbaum and H. Shrobe, The Japanese NationalFifth Generation Project: Introduction, survey, and eval-uation, Future Generation Comput. Syst., this issue.

[5] E. Feigenbaum et al., JTEC panel on Knowledge BasedSystems in Japan, Loyola College, Maryland, JTEC pro-ject, 1983, 1993 (forthcoming).

[6] ICOT, Fifth Generation Computer Sytems, ICOT Plan-ning Report No. 1, May 1982 (note: this is a modificationof MITI's original 1981 plan).

[7] S. Nishio, An evaluation of the FGCS data & knowledgebase system - Expectations and Achievements, FutureGeneration Comput. Syst., this issue.

[8] M. Stickel, Automated theorem proving research in theFGCS project: Model generation theorem provers, Fu-ture Generation Comput. Syst., this issue.

[9] E. Tick, Appraisal of parallel processing research atICOT, Future Generation Comput. Syst., this issue.

[10] E. Tick, Launching the new era, Comm. ACM 36 (3)(1993) 90.

[11] R. van de Riet, An overview and appraisal of the FifthGeneration Computer System Project, Future GenerationComput. Syst., this issue.

K.I

,:ii

.1 I<"eigi-nhaiim is Professor of

H Computer Science at the ComputerScience Department, Stanford Uni-versity. He is Co-Scientific Directorof the Heuristic Programming Projectat

Stanford,

a leading laboratory forwork in Knowledge Engineering andExpert Systems. Dr. Feigenbaum isalso Co-Principal Investigator of thenationalcomputer facility for applica-tions of Artificial Intelligence toMedicine and Biology known as theSUMEX-AIM facility, established by

NIH at Stanford University.He has been Chairman of the Computer Science Department

and Director of the Computer Center at Stanford University.He is the Past President of the American Association for

Artificial Intelligence. He has served on the National ScienceFoundation Computer Science Advisory Board; is now servingon a DARPA advisory committee for Information Scienceand Technology; and has served on the National ResearchCouncil's Computer Science and Technology Board. He hasbeen a member of the Board of Regents of the NationalLibrary of Medicine.He is the co-editor of the encyclopedia, The Handbook ofArtificial Intelligence, and of the early book, Computers andThought, published by McGraw-Hill. He is co-author of theMcGraw-Hill book, Applications of Artificial Intelligence inOrganic Chemistry: The DENDRAL Program and was thefounding editorof the McGraw-HillComputer Science Series.He is co-author with Pamela McCorduck of the book TheFifth Generation: Artificial Intelligence and Japan. ComputerChallenge to the World, published by Addison-Wesley (1983)and by New American Library (1984). He is also co-authorwith Penny Nii and Pamela McCorduck of the book, The Riseof the Expert Company, on corporate successes in the use ofexpert systems, published by Times Books in New York andMacmillan in London (1988).He is a co-founder of three start-up firms in applied artificialintelligence. IntelliCorp, Teknowledge and Design Power Inc.and served as a member of the Board of Directors of Intel-liCorp and Design Power Inc. He also was a Director ofSperry Corporation prior to its merger with Burroughs.He was elected to the National Academy of Engineering in1986. In the same year, he was elected to the ProductivityHall of Fame of the Republic of Singapore. He is an electedFellow of the American Association for Artificial Intelligenceand of the honorary American College of Medical Informat-ics. He was elected to the American Academy of Arts andSciences in 1991. He is the first recipient of the FeigenbaumMedal, an award established in his honor by the WorldCongress of Expert Systems.He received his B.S. from Carnegie Mellon University in 1956and his Ph.D. from the same school in 1960. In 1989, he wasawarded an honorary Doctor of Science degree from AstonUniversity in England.

Ho. -in! Shiol _ is a Principal Research Scientist at the MITArtificial Intelligence Laboratory. Until January 1993, he wasalso Technical Director at Symbolics Inc. His work hasspanned the areas of VLSI design, computer architecture,and Articial Intelligence. He received his M.S. and Ph.D.from the Artificial Intelligence Laboratory at MIT where hewas a cofounder of the Programmer's Apprentice project. In1979 he joined the staff of the MIT AI Lab as a PrincipalResearch Scientist and in that role was one of the maindesigners ofthe Scheme-81 microprocessor(a Lisp interpreteron a chip) and of the DPL/Daedalus Integrated CircuitDesign system. He also helped found the Hardware Trou-bleshooting project at the MIT AI Lab and is currentlyconducting research on Designing and Understanding Mecha-nisms.At Symbolics Dr. Shrobe was one of the architects of theIvory microprocessor and of the NS CAD system used todesign it. Since that time he has led the effort to developJoshua, an AI programming language which introduced thenotion of a Protocol of Inference.Dr. Shrobe is coauthor of the book Interactive ProgrammingEnvironments together with David Barstow and Eric Sande-wall. He also was editor of the AAAIbook ExploringArtificialIntelligence: Surveysfrom the National Conferences on ArtificialIntelligence.