mixed computation of prolog programs

New Generation Computing, 6 (1988) 119-141 OHMSHA, LTD. and Springer-Verlag

�9 OHMSHA, LTD. 1988

Mixed Computation of Prolog Programs

David A. F U L L E R * and Samson A B R A M S K Y Department of Computing, Imperial College of Science and Technology, London SW7 2BZ, England.

Received 17 March 1988

Abstract This paper describes theoretical as well as implementation issues involved in the design of a mix partial evaluator, starting from an interpretive specification using Prolog as the source language. First, the general theory of mixed computation is shown, together with the description of some of the problems involved in it. After this, the design of a m/x partial evaluator for a subset of Prolog is explained. Here, three algorithms with increasing complexity are shown. Finally, a correctness criterion for the m/x process is presented, together with the proof that the previously shown algorithms generate correct residual programs.

Keywords: Partial Evaluation, Self-Application, Compiler Generation, Correct- ness, Prolog.

w Introduction In Logic Programming, the concept of partial evaluation corresponds to

the evaluation at compile time of a goal with (possibly) some of its parameters bounded to grounded terms. In other words, the partial evaluation of a goal Q with respect to a program II will produce a new program II ' which is program II specialised to the goal Q. The goal Q should compute the same answers with respect to II and II' , but it should run more efficiently for II ' than for II.

Partial evaluators have been used as a tool for the opt imisat ion of programs, as described in Refs. 3), 21) and 19). Then also appeared in the literature as a possible but not yet practical way for generat ing compilers , compiler generators and compiler-compiler generators during the seven- ties 1~176 and later Refs. 11) and 7). There, apa r t i a l evaluator to generate such programs is called a or m/x partial evaluator. Only during the last two years have successful implementat ions of compiler generators been repor tedJ 2'17~

, On leave from the Computer Science Department, Pontificia Universidad Catolica de Chile, P. O. Box 6177, Santiago 22, Chile.

120 D. A, Fuller and S. Abramsky

Recently, partial evaluat ion is also being connected to meta-programming. 1s'19)

The next section discusses the general theory of m/x partial evaluators, and some problems involved in the implementat ion of such programs. Section 3 explains the design of a new partial evaluator for a subset of Prolog, and defines some notat ion to be used throughout the paper. Sections 4, 5, 6, 7 and 8 explain theoretical aspects, as well as implementat ion details of the new m/x system. Section 9 draws conclusions, along with some possibilities for further research in this area.

w The Mix Self-Applicable Partial Evaluation Approach In order to explain some principles involved in a m/x partial evaluator,

conside a set of programs and a set of data, where both programs and data are members of a domain D of symbols. Then, a programming language P may be represented by a "semantic function", defined as P: D - ~ D*- ~ D, where D* repersents the domain with zero or more elements of D. The above definition of P is in curried form. The s y m b o l - ~ means that P is defined as a partial function, showing that the execution of a program may non terminate. Thus, if p rogram p is a member of D, then P(p): D*- --> D and P ( p ) ( < dl . . . . . dn > ) is the result of running P on program p with data < d l , ..., dn > ~ D*. Also, define the sets P and S as the sets o f all syntactically valid programs in languages P and S, respectively.

Definition 2.1 Consider a programming language specified by the semantic function P, and a program p in the set P of all syntactically valid programs in P. Then, a program r in the set P is called a "residual program" for p, with respect to a known input < x l ..... X m > ~ D * , if and only i f P(p)( < x l ..... Xm, yl ..... yn> ) = P(r) ( < yl .... , y n > ) for all sequences of remaining input < y l ..... y n > ~ D * . Thus, we are specialising program p on partially known input < x~, ..., xm > , creating a new program, r. [ ]

Definition 2.2 A m/x partial evaluator is a program in P such that for any p ~ P, and part ial ly known input < x t ..... x m > ~ D * , P ( m i x ) ( < p , x~ ..... Xm>) is a residual program for p with respect to < xt, ..., xm > . []

In other words, this definition says that:

P ( P ( m i x ) ( < p , xt, ..., Xm>)) (<y t ... . . y ~ > ) : P ( p ) ( < x , . . . . . x~, y~, ..., y ~ > ) (1)

for all sequences of remaining input < y t . . . . , y ~ > ~ D * . Here, p is called the "subject program" and P the "subject language" of the partial evaluator. The data <Xl ..... Xm> is called the "known input", and <y~ ..... yn> the "unknown or residual input". Equat ion (1) says that given a program p and a partially

Mixed Computation of Prolog Programs 121

known input < xl . . . . . Xm >, it is possible to generate a semantically equivalent program r, which will take the remaining unknown input. The way of constructing this program r is by means of the partial evaluator m/x with input program p and the known input to p.

Remark 2.3 From the last definition, it is clear then that a m/x partial evaluator accepts as input programs in P together with partially known input, and produces a residual program, also in P. Since m/x is itself written in language P, it can be expected that, at least theoretically, it can accept a copy of itself. The term "m/x self-applicable partial evaluator" will be used in order to refer to this ability. []

Definition 2.4 An interpreter for language S is a program int in P, such that :

P ( i n t ) ( < s , dl . . . . . d , > ) = S ( s ) ( < d l . . . . . d , > ) (2)

for all programs s in S and input tuples < dl . . . . . d , > ~ D * . This definition is operational in the sense that defines int in terms of an equivalent machine S. []

Definition 2.5 A compiler from language S to language P is a program comp in P, such that for any program s in S:

P ( P ( c o m p ) ( < s > ) ) ( < d l . . . . , d n > ) = S ( s ) ( < d L . . . . , d n >) (3)

for all input tuples < d~, ..., dn> ~ D*. [ ]

Thus, program s ~ S is transformed to a semantically equivalent program r ~ P . The way of constructing r is by applying the operator comp(wr i t t en in P) to the input program s, i. e. r = P ( c o m p ) ( < s > ) .

Definition 2.6 A compiler generator (or compiler-compiler), accepting interpreters written in P and generating compilers in language P, is a program cocom in P such that:

P ( P ( P ( c o c o m ) ( < int > ))( < s > ))( < d~, ..., dn > ) : S ( s ) ( < d~ . . . . . d~ > )

(4)

for all interpreters int in P for language S, and all input tuples < d~ ..... d, > D*. [ ]

Lets see how to apply the above ideas to the consruction of compilers. Consider P to be an implementation language as explained before, m/x be a self-applicable partial evaluator as stated in Equation (1) and Remark 2.3, and let S be some programming language. Then, it is possible to prove the following three basic theorems. 1~

122 D.A. Fuller and S. Abramsky

Theorem 2.7 Using a mix evaluator, it is possible to generate an equivalent program target in P from a program int in P for programs in S, and a program s in S, such that target = P( mix )( < int, s>) . This equation says that using a mix partial evaluator and an interpretive version of a language S, it is possible to compile a program in language S into a program in language P. This means that we may use m/x as a universal compiler. Note that mix self-application is not needed here. []

Theorem 2.8 It is possibte to generate a compiler comp (in P) from language S to language P, using a m/x self-applicable partial evaluator and a program int in P for programs in S, such that comp=P(mix)(<mix, int>). Therefore, comp is a compiler written in language P, from language S to language P. []

Theorem 2.9 It is possible to generate a compiler generator (or compiler-compiler) cocom (in P) using a m/x self-applicable partial evaluator, capable of transforming interpreters into compilers, such that cocom=P(mix)(<mix, mix>). Therefore, cocom is a general compiler generator, transforming interpreters into compilers.

[]

In order to study implementation problems behind a m/x system, it will be helpful first to show an example. Consider the following C-Prolog TM program, which computes into variable Z the length of two lists L1 and L2.

lengthof2(L1, L2, Z) :- append(L1, L2, Zl), length(Zl, Z).

append([ ], Y, Y). append(IX [ Xs], Y, IX I Zs]) :- append(Xs, Y, Zs).

length([ ], [ ]). length([XlXs], [ ' l ' [Zs] ) : - length(Xs, Zs).

where variables are shown in capital letters, and '1' represents a constant. Suppose now that we want to partially evaluate the goal lengthof2(L 1, L2, Z) with respect to the previous program, for L1 to be the list of constants [a, b], and L2 and Z to be unknown. Then,

lengthof2([a, b], L2, Z):- append([a, b], L2, Zl), length(Zl, Z).

Thus, the task is now reduced to the partial evaluation of two subgoals, namely append([a, hi, L2, Z1) and length(Z1, Z), with respect to the original program. The expansion of the first subgoal produces the following three clauses:

append([a, b], L2, [a J Zs]) :- append([b], L2, Zs). append([b], L2, [b [ Zs]):- append([ ], L2, Zs).


append([ ], L2, L2).

These clauses can be contracted of append( [ a, b ], L2, [ a, b l L2~ ) and therefore, the original clause can be reduced to,

lengthot2([a, b], L2, Z):- length([a, bl L2~, Z).

Now, expanding length([a, blL2~, Z) with respect to the original program, the following four clauses are obtained.

length([a, b I L2], ['1' I Zs~):- length([b I L2], Zs). length([blL2], [ ' l ' lZs3):- length(L2, Zs). length([ 1, [ ]). length(IX I Xs], ['1' I Zs~):- length(Xs, Zs).

Note that the "educated" decision of inhibiting the expansion of predicate length(Xs, Zs) was taken. Finally, contracting the above clauses lead to the following residual program.

lengthof2([a, b], L2, ['1', ' l ' lZ s ] ) : - length(L2, Zs).

length(I 1, [ ]). length([XlXs], [ ' l ' ]Zs] ) : - length(Xs, Zs).

In summary, the residual program should be constructed by expanding predicate definitions, suspending predicate calls by deciding not to expand, and by contracting clauses. The implementation of our m/x system is based on these principles.

The following points can be drawn from the current state of partial evaluation8,91:

( i ) The theory of partial evaluation is still not well understood. This fact can be deduced from the many attempts of building partial evaluators.

( i i ) No m/x self-applicable partial evaluator has been constructed for Prolog applications so far, and the ones existing for functional languages are still very experimental.

(iii) Correctness issues should be investigated together with the construction of mix self-applicable partial evaluators, since it is essential to have "correctly" generated compilers and compiler-compilers.

This paper is intended to contribute to these main points.

w Mix Partial Evaluator in Prolog The attempt described.in Refs. 12) and 17) of building a mix partial

evaluator was done for a deterministic first-order functional language. In this paper, the problems involved in a mix partial evaluator for the language Prolog will be studied.

The choice of language has to be done very carefully. On one hand, one


may be tempted to make the implementation language as complicated as one would like, because a m/x system has to be written in such language. On the other hand, one may be tempted to simplify the language as much as possible, because the m/x system is also accepting an input program in this language. An excessively complicated language would lead to an excessively complicated mix system.

In order to study the problems involved in a m/x partial evaluator in Prolog, it was decided to build a prototype, taking a subset of C-Prolog as the implementation language as well as the input language. A description of a program II in the defined Prolog subset is shown below.

Definition 3.1 A program II in the defined Prolog subset is specified as the following system of clauses:

pl( Tll ..... Tin1):- BI

pk(Tki, ..., Tknk) :-- Bk

where k E N , and T , represent terms. A term can be a constant, a variable, a list or a function. Each pi together with its terms is called the head of predicate pi, and B~(i--1 . . . . . k) is called its body, with the form A p~(T~, ..., T~nj)(j~N~

J

where N O represents the set of natural numbers including zero, and A is the conjunction symbol. If the body of a clause is empty, then it is called a fact. Note that the use of the cut operator has been avoided. []

Definition 3.2 A goal in the defined Prolog subset has the form /~ p3(Tjl, ..., T~j)(IC_N),

j ~ I

where I is a finite index set with q elements, i. e. q is the number of subgoals p~- ( V j ~ I ) in the goal. [ ]

Remark 3.3 The standard procedural semantics TM for programs written in this language will be used. [ ]

The design of a mix partial evaluator for the defined Prolog subset is divided into three phases, as shown in Fig. 1. The Lexer/Parser phase takes the input program int in the defined Prolog subset for programs in language S, together with program s in S, and produces an abstract syntax tree ~) of that program. The Expansion/Contract ion phase is the most complex module in this system. It actually performs the partial evaluation by means of two processes, expansion and contraction. The last phase, the optimisation phase, does last- minute optimisations including specialisation of functions, as well as elimination of unnecessary information.


w Phase 1: Lexer/Parser This phase of the system corresponds to the task of lexing and parsing.

The mix system is confronted by two possible situations. The first one, as described by Theorem 2.7, is when it is being used as a universal compiler. In this case, the two inputs to the system are a program int, written in the reduced Prolog set and a program s, written in an "unknown" language S. The program int in the defined Prolog subset has to be lexed and parsed, in order to be converted to an abstract syntax tree. For lexing and parsing, the method described in Ref. 22) was used. The structure of the abstract syntax tree can be found in Ref. 8). Since program s is written in a different language, the simplification that the user will provide such a program as a list in Prolog will be done here.

The second possible situation which confronts the m/x system is when both input programs are in the reduced Prolog set, as described by Theorems 2.8 (compiler generation) and 2.9 (compiler-compiler generation). In this case, both programs will be lexed and converted into syntactic trees. Note that the problem of handling a program in a different language still remains here, because the generated compiler or compiler-compiler will have to deal with such a program.

In this phase, the construction of an implicit symbol table is also done. It is implicit because the information is actually attached to the abstract syntax tree. Here, each constant and variable in the input program is marked with a special symbol to speed up their recognition later on. This information will help to gain efficiency in the next phase.

w Phase 2: Expansion/Contraction This phase is the most critical and most expensive of all the phases. It is

here where partial evaluation is performed. In this section, a study of mix partial evaluation algorithms is shown. Three algorithms with increasing complexity are presented, together with examples. For the purpose of readability of the algorithms, no effort has been made to present more efficient code.

Algorithm 5.1 The partial evaluation process expand of a non-empty goal Goal with respect to a source program Prog, which produces a new program NewProg, is defined by predicate expand(Prog, Goal, NewProg) in terms of a reduced Prolog set program/zl as,

expand(Prog, [ ], Prog). expand(Prog, Goal, NewProg) : -

expand_goal(Prog, [ 1, Goal, NewProg, [ 1, LOut).

expand_goal(Prog, ProgIn, [ 1, ProgIn, L, L). expand_goal(Prog, ProgIn, [SubGoa l lGoa l ] , ProgOut, LIn, LOut ) : -


expand_subgoal(Prog, Progln, SubGoal, Progl, LIn, LI), expand_goal(Prog, Progl, Goal, ProgOut, Ll, LOut).

expand_subgoal(Prog, Progln, SubGoal, Progln, L, L):- detect loop(Prog, Progln, SubGoal, Progln, L).

expand_subgoal(Prog, Progln, SubGoal, ProgOut, LIn, LOut):- store(SubGoal, LIn, LI), unify_heads(Prog, SubGoal, UnifiedClauses), expand_clauses(Prog, Progln, UnifiedClauses, ProgOut, L1,

LOut).

expand_clauses(Prog, Progln, [ ], Progln, L, L). expand_clauses(Prog, Progln, [Clause lClauses], ProgOut, LIn, LOut):-

expand_clause(Prog, Progln, Clause, Progl, LIn, LI), expand_clauses(Prog, Progl, Clauses, ProgOut, LI, LOut).

expand_clause(Prog, Progln, Clause, ProgOut, LIn, LOut):- append(Progln, Clause, Progl), get body(Clause, Body), expand~oal(Prog, Progl, Body, ProgOut, LIn, LOut).

where unify_heads(Prog, SubGoal, UnifiedClauses) finds all clauses in Prog whose heads are unifiable with the contents of SubGoal, unifies them (including their bodies but no occur checking is performed), and puts them in UnifiedClauses. If no clauses are found, the empty list is returned in

int s

l Lexer/Parser Phase

Expansion/Contraction Phase

Optimisation Phase [

Residual Program

Fig. 1 The m/x self-applicable partial evaluator in Prolog.


UnifiedClauses. The predicate detect_loop( Prog, Progln, SubGoal, Progln, L) is true if no loop is detected in the partial evaluation process of a recursive program, as defined later. Predicate store(SubGoal, LIn, LOut) puts in LOut the list LIn of already expanded subgoals together with the subgoal being expanded. The remaining predicates, used but not defined, have obvious meanings. []

Example 5.2 Consider the following input program to predicate expand, as shown in the previous definition,

p(X, Y, Z, W) :- q(X, Y), r(Y, Z), s(Z, W). q(X, Y) :- t(x, v). t(a, b). r(a, c). r(b, c). r(X, Y) :- u(X). s(c, d). s(c, e).

together with the goal p(X, Y, Z, d). Then, the output program will be,

p(X, Y, Z, d) :- q(X, Y), r(Y, Z), s(Z, d). q(X, Y) :- t(X, Y). t(a, b). r(a, c). r(b, c). r(X, Y) :- u(X). s(c, d).

Here, only forward unification has been performed. Also notice that unrelated predicates have been eliminated. D

Definition 5.3 Consider two reduced Prolog set queries Q and Q' as specified by Definition 3.2. It will be said that Q' is less general than Q and written as Q '< Q if and only if for some substitution a, Q '= Qa. If o'~ ~b, i. e. the empty set, then Q' will be strictly less general than Q, and written as Q '< Q. []

Definition 5.4 Predicate detect_loop is true if a loop at partial evaluation time is detected, i. e. predicate detect_loop(Prog, Progln, SubGoal, Progln, L) will be true if and only if there is a previously expanded subgoal SubGoal" in structure L, such that SubGoal <_ SubGoal'. []

A suggestion for implementing predicate store(SubGoal, LIn, LOut) is to copy to LOut only the most general version of the already expanded subgoals in LIn, and thus, remove any less general version of SubGoal. Note that this


criterion may thereby inhibit further expansions of the same subgoal, but reduces considerably the number of generated clauses as well as partial evalua- t ion time. More specialised residual programs can be obtained by storing in L O u t every instance of the expanded clauses, as shown in Ref. 19), and inhibit unfolding only if further occurrences of exactly the same instance of an already expanded clause are ditected. The latter criterion is not suitable for real m/x applications, because it may lead to very long partial evaluation times and the need for a large memory space. No further implementat ion issues of predicates detect loop and store will be discussed here.

The above algori thm has several deficiencies. First, unification is done only forwards, never backwards. This results in a less specialised program and therefore in a slower residual program at run-time. Also, if no predicate is found for unification in the expansion of a subgoal, i. e. the search fails, the subgoal will be left untouched, and the predicate will fail at run-time, not at partial evaluation time. On the other hand, partial evaluation time is reduced, and probably, the number of clauses in the resulting program will be less. Later, more suitable algorithms for mix applications will be shown, leading to more specialised residual programs, but first some concepts have to be presented.

Definition 5.5 Consider a subgoal Q as specified by Definition 3.2 and a program II in the reduced Prolog set, both not empty. I f Q is satisfied by n caluses in II (n_~2), then we say that Q is the root of an OR-tree and the n clauses are their n children. [ ]

Definition 5.6 Consider a goal Q1A Q2 A ... A Qm as specified by Definition 3.2 with m subgoals, m = q , m ~ _ 1, and a non-empty program II in the reduced Prolog set. I f each subgoal Q i ( l ~ _ i ~ m ) is satisfied by at least one clause in II, we say that the goal Q1A Q~ A ... A Qm is the root of an AND-tree and each clause is called a child of the tree. I f there exists only one clause for each subgoal Qi we call it a unary AND-tree. []

Definition 5.7 A substitution 0 is a finite set of the form {v~\h . . . . , vn\&}, where each vi is a variable, each ti is a term distinct from vi, and the variables v~ ..... vn are distinct. Each element v~\t~ is called a binding for vi. The substitution 0 is called a ground substitution if the tl are all ground terms. []

Definition 5.8 Let O = {Ul\S1 . . . . . Um\Sm} and cr= {vl\ t l . . . . , V n \ t n } be substitutions. Then, the composi t ion 0o" of 0 and ~r is the substitution obtained from the set { u1\s10, ..., Um\SmCr, vl\tl . . . . . Vn\&} by deleting any binding u~\sar for which u~=s~a and deleting any binding v~\t~ for which % ~ { u l . . . . . urn}. []


Definition 5.9 Let E be a set of simple expressions. A substitution 0 is called a unifier for E if E0 is a singleton. A unifier 0 for E is called a most general unifier (mgu) for E, if for each unifier tr of E, there exists a substitution y such that o'= 0y. For more explanations and examples on this and the last two definitions, see Ref. 16). []

Algorithm 5.10 A partial evaluation algorithm suitable for rn/x applications in the reduced Prolog set is described by program g2, as follows.

expand(Prog, [ ], Prog). expand(Prog, Goal, NewProg):-

expand~goal(Prog, [ ], Goal, NewProg, Mgu, [ ], LOut).

expand~oal(Prog, Progln, [ ], ProgIn, [ ], L, L). expand_goal(Prog, ProgIn, [SubGoal[Goal], ProgOut, Mgu, LIn, LOut):-

expand_subgoal(Prog, Progln, SubGoal, Progl, MI, LIn, LI), unify_goal(Goal, M1, NewGoal), expand_goal(Prog, Progl, NewGoal, ProgOut, M2, L1, LOut), compose(M1, M2, Mgu).

expand_subgoal(Prog, ProgIn, SubGoal, ProgIn, [ ], L, L):- detecmloop(Prog, ProgIn, SubGoal, ProgIn, L).

expand_subgoal(Prog, ProgIn, SubGoal, ProgOut, Mgu, LIn, LOut):- store(SubGoal, LIn, L1), unify heads(Prog, SubGoal, UnifiedClauses, MI), expand_clauses(Prog, Progln, UnifiedClauses, ProgOut, M2, L1,

LOut), compose(M1, M2, Mgu).

expand_clauses(Prog, ProgIn, [ ], ProgIn, [ ], L, L). expand_clauses(Prog, ProgIn, [Clause[Clauses], ProgOut, Mgu, LIn, LOut):-

expand_clause(Prog, ProgIn, Clause, Progl, M1, LIn, L1), expand clauses(Prog, Progl, Clauses, ProgOut, M2, L1, LOut), set_mgu(Clauses, M1, Mgu).

expand_clause(Prog, ProgIn, Clause, ProgOut, Mgu, LIn, LOut):- getbody(Clause, Body), expand_goal(Prog, ProgIn, Body, Progl, Mgu, LIn, LOut), unify clause(Clause, Mgu, NewClause), append(NewClause, Progl, ProgOut).

set_mgu([ ], Mgu, Mgu). set_mgu(Clauses, Mgu, [ ]).

where unify_clause(Clause, Mgu, NewClause) creates a new clause in New-

130 D. A, Fuller and S. Abramsky

Clause by unifying the contents of Clause with the most general unifier Mgu already computed, and represented as a data structure. Here, the predicate unify_heads(Prog, SubGoal, UnifiedClauses, Mgu) unifies the contents of SubGoal with head clauses in Prog and puts them in UnifiedClauses. If UnifiedClauses has only one clause, the n Mgu brings back the computed set of most general unifiers. Otherwise, the value of Mgu is the empty list. The predicate unify_goal(Goal, Mgu, NewGoal) applies to the contents of Goal the set of most general unifiers Mgu computed before, and puts the result in NewGoal. The predicate eompose(M 1, M2, Mgu) takes two sets M 1 and M2 of most general unifiers, and compose them into Mgu, as specified by Definition 5.8. Finally, the predicate set_mgu sets the empty value to Mgu if more than one clause were expanded for the same subgoal. Therefore, no backwards unification is performed. []

Example 5.11 Consider the input program shown in Example 5.2, together with the same goal, but now as input to predicate expand as defined by Algorithm 5.10. In this case, the output program is as shown.

p(a, b, c, d) :- q(a, b), r(b, c), s(c, d). q(a, b) :- t(a, b). t(a, b). r(b, c). r(b, Y) :- u(b). s(c, d).

Notice that this program is considerably more specialised than the one generated in Example 5.2. []

In addition to doing forward unification, this algorithm does backward unification. Here, if the goal being unified forms a unary AND-tree, the computed set of most general unifiers is passed back to its predecessor for further unification. If an OR-tree or an AND-tree with OR-trees in its branches is found, the empty set is passed back, and therefore, no backward unification is performed. Note that we could have passed back a set of sets of most general unifiers and create more specialised equations by doing several instances of backward unification. This would have finished in an unmanageable number of specialised clauses, particularly in the case of self-application. The algorithm described in Ref. 21) generates these multiple number of specialised clauses. In the case that the expansion of one of the subgoals in the body of a clause fails, the clause being expanded is not removed from the final program. Therefore, at run-time that specific branch of the program will fail. Further optimisation can be done here by removing not only the clause, but the whole subtree until an OR-tree expansion is encountered.

Algorithm 5.10 is far more expensive than Algorithm 5.1 in terms of


partial evaluation time and possibly in terms of memory space, i. e. number of specialised clauses. In order to keep the number of equations small (or as small as possible), another process is introduced to the m/x algorithm, called here "contraction". Informally, the contraction processes are defined by two processes. The first one eliminates redundant clauses in a set of generated clauses, and its purpose is to avoid expansion of duplicate clauses, as explained later in Section 7. The second contraction process eliminates a subgoal in the body of a clause if the subgoal is unifiable with only one clause of the input program, and such a clause is a fact. Note that if the expansion of the clause's body forms a unary AND-tree, all its children are facts, and the subgoals in the body are unifiable with the corresponding facts, then the clause is converted to a fact. A third contraction process could also be performed over a clause, if the expansion of at least one of the subgoals in the expanded body fails. In that case, the clause can be safely eliminated from the generated program. This process will not be used when it comes to self-application and therefore, will not be formally defined here. The reason for this is that this process will cut all the failing branches, and therefore, no built-in predicates or external modules are allowed.

Algorithm 5.12 Here, a partial evaluation algorithm for doing m/x self-applications will be presented, together with the first two contraction processes. Because of the similarities with Algorithm 5.10, only the clauses which have been changed are shown.

expand(Prog, Goal, NewProg):- expand~goal(Prog, [ 1, Goal, Progl, Mgu, [ ], LOut), contract_program(Prog 1, NewProg).

expand_subgoal(Prog, ProgIn, SubGoal, ProgOut, Mgu, LIn, LOut):- store(SubGoal, LIn, L1), unify_heads(Prog, SubGoal, UnifiedClauses, M1), contract program(UnifiedClauses, NewClauses), expand_clauses(Prog, ProgIn, NewClauses, ProgOut, M2, L 1,

LOut), compose(M1, M2, Mgu).

expand_clause(Prog, ProgIn, Clause, ProgOut, Mgu, LIn, LOut):- get_body(Clause, Body), expand~goal(Prog, ProgIn, Body, Progl, Mgu, LIn, LOut), unify clause(Mgu, Clause, Clausel), contract clause(Progl, Clausel, NewClause), append(NewClause, Progl, ProgOut).

contract_program( [Clause [ Cls~, [Clause [ NCls]) :- eliminate_clause(Clause, Cls, TCls),


contract_program(TCls, NCls). contract_program([ ], [ ]).

contract clause(Prog, [Head I Body], [Head I NewBody]) :- contract_body(Prog, Body, NewBody).

contract_body(Prog, [ ], [ ]). contract body(Prog, [SubGoal[ Goal], NewGoal) :-

inprog(Prog, SubGoal), contract_body(Prog, Goal, NewGoal).

contract_body(Prog, [SubGoal I Goal], [SubGoal I NewGoal]):- contract_body(Prog, Goal, NewGoal).

The predicate eliminate_clause(Clause, Cls, TCls) eliminates repetitions of Clause in the list of clauses Cls and puts the result in TCls, and the predicate inprog(Prog, SubGoal) is true if there is a unique fact in the list of clauses Prog unifiable with the subgoal SubGoal. This algorithm will be called program g3.

[]

Example 5.13 Applying program/~3 to the input program and goal shown in Example 5.2, the following output program is obtained,

p(a, b, c, d). s(c, d). r(b, Y) :- u(b). r(b, c). q(a, b). t(a, b).

and applying/z3 two times, the program p(a, b, c, d) is obtained, which is a very efficient residual program. []

w P h a s e 3: O p t i m i s a t i o n Optimisation techniques are applied to the program generated in the

previous phase in order to simplify it. In particular, predicate specialisation and redundant clause elimination are performed here. Also, some standard compiler optimisation techniques, like tail recursion elimination, can be applied in this phase. This will be discussed with more detail in the next section.

Notice that the contraction processes as defined in the previous section can also be performed in this phase, as a separate pass. It was decided to do these together with the expansion process because this way the program being generated is kept smaller and the expansion process time is reduced.

w Achiev ing M i x Se l f -Appl i ca t ion Achieving m/x self-application was not as easy as it may seem at first.


Once the problem of the number of generated clauses was solved, the problem of large memory usage still remained. Several rewritings of the main algorithms, and some meetings with the "Prolog gurus" finally did the trick. In this section, the experience of achieving self-application, together with some of the most important tricks, will be explained.

First, the interpreter for the language NORMA2, presented in Ref. 17), was translated into 14 Prolog clauses. NORMA2 is an imperative two-register language with assignments, "goto's" and " i f ' statements. Where an interpreter was required, this program was used.

As a Prolog engine we used the Edinburgh Prolog on a very powerful computer with a large memory. The main data areas of this Prolog interpreter are the code area, containing data representing the program itself, and three stack areas: the global stack, the local stack and the trail. Each time a procedure is invoked, a new frame is created in each stack if needed. The global frame contains information representing the structures of the new terms created by the procedure call, but the memory to store those structures is taken from a heap. The local frame contains bookkeeping information and the value cells for local variables, i. e. required information for the execution of the procedure. This information includes the backtracking points. The trail frame contains references to all the variables that need to be reset when backtracking occurs.

As an experiment, we successfully compiled the NORMA2 interpreter, with the NORMA2 program "twice", also shown in the previous reference, into a residual program with 5 clauses.

The problems begun when we tried to partially evaluate the NORMA2 interpreter with respect to the m/x algorithm. After consuming more than 30 megabytes of main memory by having incredibly large global and local stacks, we realised that something was wrong. A list of the most important items found follows.

(1) When using reasonably large interpreters as input, the generated program contained repetitions of the same clauses several times. This fact was producing output programs three times the expected size, and therefore, three times the expected memory and generation time. Due to insufficiently instantiated subgoals, the same clauses were produced during the expansion process (redundancy). This problem was solved by eliminating the repeated clauses by using the predicate contract_program in Algorithm 5.12.

(2) Prolog interpreters usually keep a bounded size of the global stack by doing garbage collection on its heap. But the Edinburgh Prolog interpreter does not do garbage collection, and therefore, the memory used by the global stack grows considerably. To limit this space, the following predicate was used.

do_save(A, B):- not not (call(A), asserta(B)), clause(B, true).

134 D . A . Fuller and S. Abramsky

where not is the standard Prolog negation operator, call(A) executes the procedure call A, asserta(B) asserts B in the internal database as the first clause, and clause(B, C) checks in the internal database for a clause with head B and body C. To call this predicate we use, for example, do_save(parse( Progln, ProgOut), p( ProgOut ) ), where parse is the name of the procedure that does the parsing, Progln is instantiated to the lexed program, ProgOut will hold the result of the parsed program, and p is just a unique predicate name. In this case, predicate dosave works by executing procedure parse, asserting its result into the internal database, releases all the stack space used by the execution of parse as a conse- quence of the negation, and finally obtaining the result back from the internal database. Note that if procedure parse would have failed, so would do_save. In this way we reduced the size of the global stack from megabytes to kilobytes.

(3) Most languages, including Lisp, discard stack frames at the time the procedure returns its result. With Prolog, however, this is not in general possible since procedures may be non-determinate, i. e. they can return several results. Here, a stack frame is generally reclaimed only after backtracking has occurred and the procedure has generated all the possible solutions. This causes a Prolog interpreter to use large amounts of memory in its local stack when non-trivial programs are executed. Note that in our case the global stack will not be specially affected, due to the lack of garbage collection. Some Prolog systems detect that the last result of a procedure has been generated, reclaiming the corresponding stack frames immediately on return from the procedure, as in conven- tional languages. This feature is called "tail recursion optimisation" or "last call optimisation", z3) A Prolog system detects determinate situations either because the procedure is properly indexed and always yields only one possible matching clause or because it is properly imposed by the programmer through Prolog's cut operator. Therefore, it is possible to reclaim the stack frames of all the determinate procedure calls in a clause before reaching the last call. By including cut operators in our mix system, we were able to reduce the local stack size from the megabyte range to kilobytes. It is important to say that the inclusion of cuts in our m/x programs is only to make it memory efficient, and it is not part of our chosen language (Eventually, an automatic tool to detect determinacy and include this operator could be used).

(4) As a last issue in the extensive use o f memory, we noticed that some parameters were instantiated to large lists, sometimes representing entire programs. Apart from specialising predicates--with known values-- to new ones--wi thout them--during the last optimisation phase, no other solution has been implemented. The incorporation of "binding time


analysis" will be considered in the future to deal more efficiently with this problem

After having solved the problem of extensive memory usage, we successfully compiled the rn/x Algorithm 5.10 with the NORMA2 interpreter, to produce a 600 line NORMA2 compiler. Here, the definition of predicate unify_ heads was not included, mainly because it is too long. Also, a 1000 line compiler generator was obtained by using/sa, as specified by Theorem 2.9.

w Partial Correctness Proofs In this section, termination problems at partial evaluation time are

discussed first. In is the belief of the authors that any m/x self-applicable partial evaluator algorithm should include a proof of correctness, since it is essential to argue that the automatically generated compilers are equivalent to the interpretive version of the defined language. Therefore, a correctness criterion for proving Prolog generated programs correct is presented here, a formal method for the proof using this criterion is developed, and the proof is done with one of the previously shown algorithms. The method shows that a certain property remains invariant when the execution of the m/x program terminates.

Remark &l Termination in the Expansion/Contraction phase of a recursive program 11 is semidecidable, i. e. there is an algorithm that, for each instance of the problem, says yes if the algorithm terminates, and may never give an answer otherwise. Due to this fact, heuristics to guarantee termination have to be developed. Notice that the termination condition discussed in the previous section, i. e. predicate detect_loop is not sufficient to guarantee termination of the partial evaluation process. []

Example 8.2 Consider the following program, which has an infinite solution space.

ancestor(father(X), X). ancestor(Y, X):- ancestor(Y, father(X)).

together with the goal ancestor(Z, john), i. e. who are the ancestors of John? If this example runs in a Prolog interpreter, the solution Z =father(john) will be obtained. By successive resatisfactions of the goal, solutions Z = father(father(john)), etc., are obtained. Here, the generated instances of predicate ancestor are infinitely many, and each one is different from the other. Therefore, a loop cannot be detected by the previous loop detection scheme. []

The fact just described means that a run-time terminating program does not necessarily terminate at partial evaluation time. One approach to handle this problem, as in Refs. 21) and 12), has been by putting the responsibility of partial

136 D . A . Fuller and S. Abramsky

evaluation termination in the hands of the user by doing program call annotations. In this process, the user usually has to specify the functions or predicates to be expanded and specialised by the algorithm. The inconvenience of this approach is that usually a proper annotation scheme is achieved only by "expert annotators", and only for relatively small programs. Even in more recent partial evaluation algorithms, like the one in Ref. 19), no solution to this problem other than annotations has been provided.

At this time, three approaches to ensure termination for the process o f partial evaluation can be visualised. The first approach consists in detecting loops at partial evaluation time by using a collection of standard algorithms? ) Since these algorithms detect only certain types of loops, a more general approach may be needed. The second approach considers the partial evaluation process as a rewrite system. It consists in using the grammar constructions defined in Ref. 14) in order to guide the partial evaluation process. These approaches will be investigated in the future, but it is important to note that the incorporation of smarter loop detectors will inevitably make the process of partial evaluation longer. The third approach is, as done in Refs. 12) and 13), to incorporate binding time analysis, a pre-precessing phase which computes information to automatically classify the operators as foldable or safely unfolda- ble. This process is the less expensive of the three approaches since it is performed before the partial evaluation phase, but it requires further optimisa- t ion phases because it usually does not produce the "best" residual program.

As explained in Remark 8.1, termination is not guaranteed in the process o f partial evaluation. Thus, it can only be proved that if the process of partial evaluation terminates, it will generate correct programs. This is called partial correctness. In the rest of this section, a method for proving partial correctness in mixed computation of Prolog programs will be shown and used with Algorithm 5.1. The method consists in proving that a certain relation remains invariant during the execution of the program, 4) provided that the process terminates. The invariance relation is called here goodness, and it is based on the principle that a query Q to a program H and the same query Q with the residual program II'(generated from II and Q by the mix system) will compute the same set of most general unifiers 0.

Definition &3 For every program II and for every query Q in the reduced Prolog set, II ? Q - -0 , where 0 is the set of most general unifiers obtained by the computat ion of program II with query Q. Note that the case of failure is represented here by the empty set, and termination is being assumed. [ ]

Definition 8.4 The goodness property on the atomic propositions of the language of program /11 is defined further on. Essentially, an atomic proposit ion is good if it succeeds as a query from the program. The notation for the variables has been changed


to use more compact names. Also, bracketing has been used only where ambigu- ities may arise. The usual operator precedence conventions apply here.

(1) Vl]VQVl] ' [good(expand(n, Q, I ] ' ) ) ~ V Q ' [ Q ' < _ Q >l] ? Q ' - I I ' ? Q']]

(2) V l] VII,,, V Q V IIo,,, V ,b,, V a.o,,, [good(expand_goa l ( l ] , Ibm, Q, l]o~t , ~.~,,, ,t.o~t ) ) ,: : ,[VQ'[Q'<_Q >1I? Q"--IIo~, ? Q']]AIIo~,-~II, .]

(3) 'q'IIVii,nV SubGoal Vl]ou, V~.,.V~.oue [good( expand_subgoal(II, IIi., SubGoal, IIoue, 7ti., ~to~t ) ) ,'. ; ,VQ'[Q'<_SubGoal ",II? Q'- l]o~, ? Q']AIIo~t@II, .]

(4) Vl]Vl]InV ClausesVl]o.tVh.lnVh.out [good(expand_Clauses(U, IIin, Clauses, l]o~t, ~.1., ~.om ) ) ,'. " , V C i V Q ' [ C i ~ C l a u s e s A Q ' < h e a d ( C i ) ;.II? Q'-IIo~t ? Q']

A IIout ~ IIIn.]

(5) ~i iVl]InX~ ClauseVl]oueV ]tlnV ]tout [good(expand_clause(l], IIl., Clause, IIo~t, ~,., 2o~t)) ,'. ; ,VQ'[Q'<_head(Clause) ', II? Q'-IIo~t ? Q']

A IIout ~-- III.]

(6) Vl]Vl] inVl Io~tV SubGoalV )t [good(detect_loop(if, IIz., SubGoal, IIom, Yt )) ,'. ", SubGoal ~ 7t

A V Q ' [ Q ' < S u b G o a l "l]? Q ' - I I I . ? Q'] Al]o~t-~II1.]

(7) V SubGoal V Ytl~V Yto~t [good ( store( SubGoal, Yti., Xo~l ) ) ~ Ytz~ < > SubGoal =- Ytout ]

(8) V l ] V SubGoa lV UnifiedClauses [good(unify_heads(if , SubGoal, Unified Clauses ) ) ,'. ",V C~[ C ~ UnifiedClauses ;,SubGoal=--head(Cg)]]

(9) V Clause VBody[good(get_body(Clause, Body)) .~-obody(Clause) = - Body]

where the operator < > represents append of two lists, = is syntactic equality, and the functions head and body of a clause have obvious meanings. []

Definition 8.5 The goodness property is preserved in a clause P( 9-):- AR;(~-I)(i_>0) if and only if V g [ A good(R,(gi(g)))---~.good(P(~-(f f )))] , where ~- is a tuple of ground substitutions for the distinct variable symbols in the clause, 9- and /7; are tuples of terms, and 9-(d) is 9- under the substitution ft. []

Theorem 8.6


Goodness is preserved in every clause of program tzl.

Proof The proof is done by proving that each clause of program/zl preserves goodness. Because of the excessive size of this proof, it will not be shown here. The interested reader can reger to Ref. 8). []

Lemma 8.7 Goodness is preserved during the execution of program tz~. In other words, if /-tt I - P ( d ) then good(P(g)) is true.

Proof It will be shown that goodness is preserved by a computation of any length, by induction on the length of the compution of/xl I- P(d) .

( i ) Base Case (n=0) : In this case, P(tT) is an instance of a fact in /xl. Here, goodness is preserved, as proved in Theorem 8.6.

( i i ) Induction Hypothesis ( n=m) : Assume goodness is preserved in all R(b-) which take at most m computations.

(iii) Induction Step ( n = m + l ) : Now suppose P ( d ) takes m + 1 computations. Then there exists a clause in tzl such that P( d ) : - Ra(~-1) . . . . . Rk(~-k) and all /z~ I- Ri(~-i) take at most m computations. Therefore, by induction hypothesis, good(Ri(~)) is true. But in Theorem 8.6 it has been shown that if all subgoals in the body of a clause in /zl are good, then the head is good. []

Corollary 8.8 If I.tl I-expand(II, Q, II'), then II" is partially correct, i. e. ~' Q' [Q'~< Q ;,II ? Q ' - I I ' ? Q'~.

Proof If /zl 1-expand(II, Q, II') then by Lemma 8.7 expand (l-I, Q, II') is good. Therefore, by definition of the goodness of predicate expand, II" is partially correct. []

The above result shows that an invariance relation is preserved during the execution of program #1, assuming that such execution terminates. The preserva- tion of this relation assures the correction of the algorithm. Similar results for algorithms /zz and #a are now being investigated.

w Conclusions and Further Research A rn/x self-applicable partial evaluator can be used for doing program

optimisation, as a universal compiler, and to generate compilers and compiler- compilers. This system is different to a standard partial evaluator in that, due to


self-application, the former has to take special care to keep the number of generated clauses bounded. Otherwise, large pieces of code can be obtained, and a large memory space may be required. Also, when used as a universal compiler, a m/x system has to handle a program in an "unknown" language syntax.

The design of a m/x self-applicable partial evaluator for a subset of Prolog was divided into three phases. The first one does the task of lexing and parsing of the two input programs. This phase confronts a difficult situation when the mix system is used as a universal compiler. Since one of the input programs is written in a different language, the simplification that the user will provide it as a Prolog list will be done. This has been the standard approach in the construction of mix systems. In this phase, the construction of an implicit symbol table is also built. This table helps to gain time efficiency in the next phases.

In the second phase, the partial evaluation process is performed. Three algorithms with increasing partial evaluation time and memory use complexity are shown, and the concepts of expansion and contraction are defined. A loop detection scheme for the partial evaluation of recursive programs in mixed computations is also presented. The problem of non-termination at partial evaluation time is discussed, and lines for further research are given.

It is essential that the automatically generated compilers are equivalent to the interpretive version of the defined language, and it is claimed that every mix

algorithm should include a proof of correctness. Therefore, a correctness criterion for proving Prolog generated programs correct is presented, a formal method for the proof using this criterion is developed, and the proof is done with one of the previously shown algorithms.

Finally, in the third phase of the mix system, optimisation techniques are applied to the program generated in the previous phase in order to simplify it. Specifically, predicate specialisation and redundant clause elimination are performed here. Also, some standard compiler optimisation techniques, like tail recursion elimination, can be applied in this phase.

Mix self-application was achieved by producing a 600 line compiler from partially evaluating a copy of mix wrt. the NORMA2 interpreter. Also, a 1000 line compiler generator was obtained by partially evaluating a copy of mix wrt. another copy of mix. However, it is clear from this paper that many of the problems in designing a mix self-applicable partial evaluator are not yet solved. Specifically, the termination problem explained in Section 8 has to be further investigated in order to have a more automated universal compiler, compiler generator, and compiler-compiler generator. Also, efficiency issues have to be considered.

Acknowledgemen ts Thanks are due to Chris Moss, Mark Reynolds, and Neff Jones for


suggest ing many ideas and useful improveme n t s to this paper . W e are also

greteful to the organ ise rs o f the W o r k s h o p on Par t i a l E v a l u a t i o n a n d Mixed

C o m p u t a t i o n in G1. Avernaes , D e n m a r k , and to the ed i to rs o f this j o u r n a l .

References 1) Aho, A. V. and Ullrnan, D. J., Principles of Compiler Design, Addison-Wesley, 1979. 2) Bancilhon, F. and Ramakrishnan, R., " An Amateur's Introduction to Recursive Query

Processing Strategies," Proceedings of SIGMOD, 1986. 3) Beckman, L., Haraldson, A., Oskarsson, O. and Sandewall, E., " A Partial Evaluator,

and Its Use as a Programming Tool," Artificial Intelligence, 7, North-Holland, pp. 319-357, 1976.

4) Clark, K. L+, "Predicate Logic as a Computational Formalism," Research Report, DOC79/59, Dept. of Computing, Imperial College of Science and Technology, London, 1979.

5) Ershov, A. P. and ltkin, V. E., "Correctness of Mixed Computation in Algol-Like Programs," Lecture Notes in Computer Science, Vol. 53 Springer-Verlag, pp. 59-77, 1977.

6) Ershov, A. P., "On the Essence of Compilation," in Formal Descriptions of Program- ming Concepts (E. J. Neuhold, ed.), North-Holland, 1978.

7) Ershov, A. P., "Mixed Computation: Potential Applications and Problems for Study," Theoretical Computer Science, 18, North-Holland, pp. 41-67, 1982.

8) Fuller, D. A. and Abramsky, S., "Mixed Computation of Prolog Programs: An Extended Paper," Research Report, DoC 87/12, Dept. of Computing, Imperial College of Science and Technology, London, June, 1987.

9) Fuller, D. A., "Towards Efficient Self-Applicable Partial Evaluation in Prolog," Research Report, Dept. of Computing, Imperial College of Science and Technology, London, February, 1988.

10) Futamura, Y., "Partial Evaluation of Computation Process--An Approach to a Compiler-Compiler," Systems, Computers, Controls, Vol. 2, No. 5, pp. 45-50, 1971.

11) Futamura, Y., "Partial Computation of Programs," Lecture Notes in Computer Science, Vol. 147, Springer-Verlag, pp. 1-35, 1982.

12) Jones, N. D., Sestoft, P. and Sondergaard, H., "An Experiment in Partial Evaluation: The Generation of a Compiler Generator," Lecture Notes in Computer Science, Vol. 202, Dijon, France, pp. 124-140, 1985.

13) Jones, N. D., "Automatic program specialization: a re-examination from first principles," DIKU Research Report, University of Copenhagen, October, 1987.

14) Jones, N. D., "Flow Analysis of Lazy Higher Order Functional Programs," in Abstract Interpretation of Declarative Languages (S. Abramsky and C. Hankin, eds.), Wiley, 1987.

15) Kowalski, R., "Predicate Logic as a Programming Language," in Information Process- ing (IEIP74), (H. J. Kugler, ed.) Elsevier Science, North Holland, 1974.

16) Lloyd, J. W., in Foundations of Logic Programming, Springer-Verlag, 1984. 17) Sestoft, P., "The Structure of a Self-Applicable Partial Evaluator," Research Report,

85/11, DIKU, University of Copenhagen, 1985. 18) Takeuchi, A., "Affinity between Meta Interpreters and Partial Evaluation," in Informa-

tion Processing (IFIP 86), (H. J. Kugler, ed.), Elsevier Science, North Holland, pp. 279-282, 1986.

19) Takeuchi, A. and Furukawa, K., "Partial Evaluation of Prolog Programs and Its Application of Meta Programming," in Information Processing (IFIP 86) (H. J.


Kugler, ed.), Elsevier Science, North Holland, pp. 415-420, 1986. 20) Turchin, V. F., "A Supercompiler System based on the Language REFAL," S1GPLAN

Notices, Vol. 14, No. 2 pp. 46-54, February, 1979. 21) Venken, R., "A Prolog Meta-Interpreter for Partial Evaluation and Its Implication to

Source to Source Transformation and Query-Optimisation," in Advances in Artificial Intelligence (T. O'Shea, ed.), Pisa, Italy, pp. 347-356. September, 1984.

22) Warren, D. H. D., "Logic Programming and Compiler Writing," D. A. 1. Research Report, No. 44, Edinburgh University, September, 1977.

23) Warren, D. H. D. "An improved Prolog implementation which optimises tail recursion," Proc. o f the Logic Programming Workshop, Debrecen, Hungary, 1980.

24) Warren, D., Bowen, D. and Pereira, L., in C-Prolog User's Manual (F. Pereira, ed.), January, 1985.

mixed computation of prolog programs

Documents