Bridging fhe Gap behueen Specificafion and Implemenfdon Thomas 6. Dietterich, Oregon State University

T H E FUNDAMENTAL GOAL O F artificial intelligence is to develop meth- odologies for constructing effective, intel- ligent systems. At the most abstract level, we must solve three basic subproblems: specifying the desired behavior of the tar- get system, constructing a (runtime) prob- lem-solving architecture that can support effective performance, and bridging the gap (assuming one exists) between the specifications and the runtime architecture.

Researchers in knowledge compilation claim that the gap between specifications and runtime architecturesis substantial. They seek automatic methods for transforming specifications into implementations.

Let’s examine this claim. The great ma- jority of expert systems make no separa- tion between specifications and runtime implementations. Indeed, one of the major selling points of early expert systems work was that the “specifications” could be gathered incrementally, expressed trans- parently as rules, and executed interpre- tively to yield effective problem solving and useful explanations of system reason- ing. Clearly, such systems contain no gap and require no knowledge compilation.

As we move to more ambitious AI appli- cations, however, the gap begins to appear. Three forces act to open the gap. First, in some domains it is easiest to specify de- sired system performance in a form that is too inefficient to execute interpretively. Second, there are domains where it is im- possible to interpretively execute the spec- ifications because they are expressed in terms of unobservable quantities. Third, specifications can contain contradictions or conflicting criteria that must be detected and resolved prior to execution.

As an example of inefficient specifica- tions, consider tasks in engineering design. In design domains, the knowledge taught in textbooks and easily incorporated into computer systems is analysis knowledge - knowledge that is sufficient to deter- mine whether a proposed design will satis- fy its requirements. Virtually all comput- er-aided engineering in mechanical design,

80

for example, involves finite-element anal- ysis of structures, heat flow, fluid flow, and so on. A common criticism of modern engineering education is that the only methods taught to students are analysis methods and, consequently, newly gradu- ated engineers have difficulty designing successful products.

It is possible, in principle, to apply

example, the Leap system for VLSI design incrementally compiles analysis knowledge (sufficient to verify design correctness) into synthesis rules.27

Because of the large gap between analy- sis and synthesis in design systems, I be- lieve knowledge compilation methods will have their greatest impact in engineering- design domains.

The second problem - that the specifi- cations might be unexecutable - was first

the goal of Hearts is to “avoid taking points” - but this specification cannot be satis- fied, because the only way to avoid taking points is to know what cards are in your opponents’ hands. Of course, after a game is finished. this information is available.

Domhvs W O L m G NEW T E C ~ O L O G I E S ,

KMMZEDGE COMPL?ATKN MIGHT PROVIDE A cRu,-uL

SPEEDUP: 5 TO 10 YEARS OF HUMAN EXPERZENCE COULD

BE COMPRESSED INTO 5 TO 10 MONTHS OF CPU TIME.

analysis knowledge directly to perform design tasks. All one needs is an exhaus- tive generator of possible designs. Then the analysis knowledge can be employed to test these designs until a satisfactory one is found. Of course, such an approach is far too inefficient to be practical.

Consequently, rather than employing generate-and-test methods, most design systems are built around a generate- analyze-patch architecture. The process begins by applying synthesis rules that map from the requirements directly to a proposed design. This design is then ana- lyzed to determine which requirements are violated. Then a set of patching rules is applied to map from violations to design changes.

In principle, we can derive synthesis and patching rules from analysis knowledge, but the gap is substantial. Knowledge com- pilation methods can bridge this gap. For

However, during the game, aprogram must find observable “proxy” variables that can predict what cards are in an opponent’s hand. For example, if an opponent cannot follow suit, this reveals that the opponent has no cards in that suit (that is, the oppo- nent is “void’). Mostow’s system there- fore adopts the strategy “Lead cards in suits where opponents are not void.”

As another example, consider the prob- lem of controlling a machine tool. Part of the specification might be “Do not allow the drill bit’s temperature to exceed 400 degrees.” However, typically it is impossi- ble to measure the bit’s temperature direct- ly. Instead, a strategy such as “Do not operate the machine continuously for more than 150 seconds” must be adopted.

The third problem - conflicting speci- fications - arises quite frequently, espe- cially in design and configuration domains. For example, in the Mycin therapy selec- tion algorithm, the specifications include “Minimize the number of drugs prescribed” and “Maximize the number of suspected organisms covered.” Mostow and Swartout describe a collection of strategies for re- solving these conflicts.29 Specifically, the minimize drugs/maximize coverage con- flict can be resolved by lexicographical ordering. First, we insist on covering the

~. ~

IEEE EXPERT-

most likely organisms. Then we consider all drug therapies that achieve this and choose one that minimizes the number of drugs. In choosing a strategy to apply. a system must consider both the appropri- ateness and the efficiency of the strategy. T o support this, each strategy also has an attached implementation.

Given that a gap can exist between spec- ifications and the runtime architecture. there are two general approaches to closing this gap. One approach, which has been the focus of knowledge compilation research, converts specifications into a form that the runtime architecture can interpret directly. The other approach. which has been the focus of work on model-directed reasoning and task-specific architectures, changes the runtime architecture so that i t can interpret the given specifications directly.

Figure I provides a schematic view of these approaches. In the figure. circles rep- resent agents - that is, people or intelli- gent programs - and squares represent specifications. In Figure la , agent A Y (the expert) provides specification So. but agent A,L (the inference engine) cannot interpret this specification directly. This illustrates the gap.

Figure Ib shows the knowledge compi- lation approach. Agent A, (the compiler) transforms So into S(-. the compiled speci- fication, which is then interpreted directly. One claim of knowledge compilation re- searchers is that. in current expert systems. agent A, is often the knowledge engineer. This might partially explain why knowl- edge engineering i s so time consuming. To my knowledge, no one has performed the studies of real systems that are needed to support or refute this claim.

Finally, Figure I C shows the interpreter- modification approach. AgentAcconstructs a specialized interpreter, A,,,,,,. that can i n - terpret the given specification s,) directly. Obviously. the last two cases could be combined so that A(. reformulates the specification and produces an interpreter for that specification.

When feasible . the interpreter- modification approach i s probably best. because interpreters can easily support ex- planation, incremental modification. and source-level debugging. However, I believe this approach will not work in many do- mains. and the examples given above - engineering design. unexecutable specifi- cations. and contradictory specifications

I I

~~~~A Figure 1. Strategies for bridging the gap: (a) illustrating the gap; (b) the knowledge compilation approach; (c) the interpreter-modification approach.

--demonstrate this. Interpreters are neces- sarily local in their focus. Hence. they have great difficulty working with the kinds of global information available from engi- neering analysis knowledge. Similarly, the process of formulating proxy variables for unobservable quantities typically involves lengthy. global analysis. Finally. interpret- ers cannot perform the kind of global rea- soning required to detect and resolve con- t r a d i c t or y spec i fi c a t i o n s. From the s e arguments, we mu5t conclude that i n many domains the knowledge compilation ap- proach is the only feasible strategy.

Given that we want to build a knowledge compilation system. how can we design and implement it? No single knowledge compilation method will solve our prob- lem. Researchers have explored a large collection of methods for aiding knowl- edge compilation. and we need to develop and study many more.

To make problem solving inore effi- cient, we can apply such methods as infer- ence caching. explanation-based learning. partial evaluation of the problem solver, (admissable) heuristic search. and intro- duction of abstractions.”’.’’ All these methods maintain the correctness of the target system. In addition. i t i s sometimes necessary to sacrifice correctness to obtain efficiency. For example, i t i s often conve- nient to specify design tasks in terms of rninirniLing cost while maximiLing perfor- mance parameters. In practice. however. ”satisficing“ methods are applied to find a design of sufficiently low cost and suffi- ciently high performance:”

To address the prohlem of unexecutable specifications. one approach i s to analyze

the domain to find observable proxy variables that can approximate unobserv- able quantities. Another approach is to adopt conservative strategies that are guar- anteed not to violate specifications.

Finally. to resolve the problems of con- flicting specifications. we can use strate- gies of the sort formalized by Mostow and Swartout.2q

Having reviewed why knowledge com- pilation is needed and how it i s used, let’s look at the implications of this review for some of the i ssues raised by Ashok Goel in his introduction.

One issue is the question of how knowl- edge compilation relates to the distinction between deep and shallow knowledge. If the expert finds i t easiest to provide spec- ifications in the form of “deep” knowl- edge, and iftheruntimearchitectureisonly effective with “shallow” rules. then the knowledge compilation process is indeed the process of converting from deep to shallow knowledge. However, the work of Davis and others on model-based reason- ing suggests it is possible in some domains to build a runtime architecture for diagnos- tic tasks that can interpretively evaluate deep knowledge with reasonable efficien- cy. In these domains, the knowledge com- pilation approach i s not needed.

In domains where experts have many years of experience. i t i s unlikely that they will expreTs their knowledge in deep form. As many knowledge engineers have dis- covered. experts are more likely to describe their expertise in the form of associational rules. Consequently, knowledge compila- tion will be of little use in such domains.

On the other hand, in domains involving

new technologies, the only knowledge The main benefit is that the final system initially available tends to be “deep.” This is more likely to be complete and correct reflects the fact that it takes five to 10 years than is a system based on associational 1 thank J , ~ Bennett for many discussions of to become an expert in a field.’5 For these rules obtaineddirectly from human experts. this topic and for providing comments on drafts domains, knowledge compilation might Another benefit is that deep-knowledge of this article. This rtxarch was supported by

provide a crucial speedup: five to I O years systems give us a better understanding of ~ NASAA1nesKesearchCenterunder WntNAG 2.630. of human experience could be compressed domain problems and trade-offs. For ex-

Thomas 6. Dietterich into five to I O months of CPU time. In , ample. our understanding of medical diag- these kinds of domains, i t is appropriate to nosis in general. and of Mycin in particu- is associate professor of view the knowledge compilation process lar, is much better because of Clancey’s computer science at as automatically deriving the kinds of ex- efforts to”decompi1e”expert-supplied rules Oregon State University.

His knowledge compi- lation research stresses pertise that experts acquire over many years. to make explicit the underlying knowledge

Lenat’s work on automatically discover- and strategies?’ applications i n engi- ing design heuristics for three-dimensional neering design and VLSI technology is anice example of this.3h scheduling, while hi5

Another issue raised by Goel is the ques- inductive-learning re- search focuses on large-

ale problems i n speech synthesis and recogni- tion of when i t is worthwhile to perform knowledge compilation. The arguments CoMPILATIoN Is tion. Dietterich received his AB from Oberlin above indicate that knowledge conipila- College in 1977. his MS from the University of tion should be performed only when it is Illinois i n 1979. and his PhD from Stanford essential for producing effective perfor- U n i v e r s i t y i n 1984. He was the OrganiL-

er of the 1986 Workshop on Knowledge Compi- mance. However, some would argue that lation and. more recently, the technica l cochair additional benefits could be gained from for AAAI-90. Recently elected to the AAA1 specifying an expert system using deep Council. he is also an editor of the journal knowledge and then compiling i t to pro- Machirw z-r’trrr*ir’,~ and a of the lEEE

duce a runtime system. even if the deep C o ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ; ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ :;:;,if knowledge is harder to acquire from do- Computer science, orego” State University, main experts. Corvallis. OK 9733 I ; e-mail, tgd@cs.orst.edu

Acknowledgments

needed in many application domains. At present. our understanding of different knowledge compilation methods is frag- mentary, and much research must still be carried out. But even at this early stage, the knowledge compilation approach shows promise of expanding the range of systems that can be constructed and deployed, par- ticularly in areas such as engineeringdesign, process planning. and scheduling.

Applying Knowledge Compilation Techniques to ModelmBased Reasoning Richard M. Keller, Sterling Federal Systems, NASA Ames Research Center

O V E R THE PAST FIVE YEARS. A small community of artificial intelligence researchers has regrouped and emerged under the banner of “knowledge compila- tion.” This formative community is highly interdisciplinary and includes researchers from software engineering. automatic pro- gram mi n g , ex pert sy s tem s , machine learning, and problem solving. What this diverse collection of researchers shares is a set of general programming techniques for improving the efficiency of software sys- tems. Understandably, these AI research- ers have focused on improving the perfor- mance of “knowledge-based” (rather than conventional) software systems, but the techniques being developed are potentially

applicable across the wide spectrum of software systems.

I n this article, 1 focus on the application of knowledge compilation techniques to a specialized class of knowledge-based sys- tems known as model-based reasoning systems.’* MBR systems use a comprehen- sive model of some system or device to perform “first-principles” reasoning about its structure. behavior, and causality. (In contrast. traditional “shallow” expert sys- tems use purely pattern-directed associa- tional reasoning, without regard to under- lying causal or behavioral mechanisms.) MBR systems have been used principally in the context of various diagnosis, design, and simulation tasks.

What is knowledge compilation?

The term “knowledge compilation” was coined by Neves and Anderson in 1981 to refer to specific cognitive phenomena in their work on human skill acquisition. The term later took on broader meaning at the 1986 Workshop on Knowledge Compila- tion.” Although the term has no single, universally accepted definition. the litera- ture provides at least three alternatives:

( I ) Knowledge compilation is the pro- cess of shifting from a declarative to a pro- cedural form of knowledge representation.