cs 598 scripting languages design and implementation 10. interpreters part i: lisp

CS 598 Scripting Languages Design and Implementation

10. InterpretersPart I: Lisp

2

Interpreters• Typically, the simplest way to implement a dynamic

language.• The resulting implementation could be slow, but it is often

adequate as testified by the numerous interpreted laguages that are widely used.

• An interpreter is a practical tool, but also its specification can be used to formally specify the semantics of a language.

• However, formal definitions are seldom used by practitioners.

3

Lisp• Again from the wikipedia: Lisp (historically, LISP) is a family of computer

programming languages with a long history and a distinctive, fully parenthesized Polish prefix notation.[1] Originally specified in 1958, Lisp is the second-oldest high-level programming language in widespread use today; only Fortran is older (by one year). Like Fortran, Lisp has changed a great deal since its early days, and a number of dialects have existed over its history. Today, the most widely known general-purpose Lisp dialects are Common Lisp and Scheme.

• Lisp was invented by John McCarthy in 1958 while he was at the Massachusetts Institute of Technology (MIT). McCarthy published its design in a paper in Communications of the ACM in 1960, entitled Recursive Functions of Symbolic Expressions and Their Computation by Machine, Part I (Part II was never published). He showed that with a few simple operators and a notation for functions, one can build a Turing-complete language for algorithms.

4

Scheme• Scheme and Common Lisp are the two principal dialects of the

computer programming language Lisp. Unlike Common Lisp, however, Scheme follows a minimalist design philosophy that specifies a small standard core accompanied by powerful tools for language extension.

• Scheme was created during the 1970s at the MIT AI Lab and released by its developers, Guy L. Steele and Gerald Jay Sussman, via a series of memos now known as the Lambda Papers. It was the first dialect of Lisp to choose lexical scope and the first to require implementations to perform tail-call optimization, giving stronger support for functional programming and associated techniques such as recursive algorithms.

5

Simple scheme tutorial

• https://classes.soe.ucsc.edu/cmps112/Spring03/languages/scheme/SchemeTutorialB.html

6

Lisp metacircular evaluator• You can read Lisp by reading a reference manual or by

studying eval• The fact that the definition of eval is available in Lisp

means that the programming environment is part of the language.

• Even today, the fact that eval can be defined in Lisp means that it’s easy to experiment with new models of implementation and debugging.

• … it must give up the possiblity of many optimizations. In other words, using eval is not without consequences

7

Types of procedures

• Predicates and• Selectors

• Approach introduced by McCarthy• Both types found in VDL

8

Metacircular evaluatorBasic procedures

• (apply-primmitive-procedure < procedure > < args > )• (primitive-procedure? < procedure > )

– returns whether the procedure is a primitive procedure.

• (lookup-variable-value < variable > <env>) – returns the value of < variable > in the environment < env >.

• (extend-environment < variables > < values > < base-env >) – adds a frame to < base-env > with < variables > bound to < values >.

• (define-variable! < variable > < value > < env >)– adds < variable > with value < value > to the environment’s top frame.

• (set-variable-value! < variable > < value > < env >)

9

eval

(define (eval exp env) (cond ((self-evaluating? exp) exp)

((quoted? exp) (text-of -quotation exp))((variable? exp) (lookup-variable-value exp env)) ((definition? exp) (eval-definition exp env))((assignment? exp) (eval-assignment exp env)) ((lambda? exp)(make-procedure exp env) )((conditional? exp) (eval-cond (clauses exp) env) ) ((application? exp) (apply (eval (operator exp) env) (list-of-values (operands exp) env))

(else(error "Unknown expression type/EVAL” exp))))

10

eval




11

apply

(define (apply procedure arguments) (cond ((primitive-procedure? procedure)

(apply-primitive-procedure procedure arguments)) ((compound-procedure? procedure) (eval-sequence

(procedure-body procedure)(extend-environment (parameters procedure) arguments (procedure-environment procedure)))) (else (error "Unknown procedure type\APPLY"

procedure))))

12

apply




procedure))))

13

eval-sequence

(define (eval-sequence exps env) (cond ((last-exp? exps) (eval (first-exp exps)env)) (else (eval (first-exp exps) env) (eval-sequence (rest-exps exps)env))))

14

eval




15

list-of-values

(define (list-of-values exps env) (cond ((no-operands? exps) '()) (else (cons (eval (first-operand exps) env) (list-of-values (rest-operands exps) env)))))

16

eval




17

eval-cond

(define (eval-cond clist env) (cond ((no-clauses? clist) nil) ((else-clause? (first-clause clist)) (eval-sequence (actions (first-clause clist)) env)

((true? (eval (predicate (first-clause clist))env)) (eval-sequence (actions (first-clause clist)) env))

(else (eval-cond (rest-clauses clist) env))))

18

eval




19

eval-assignment

(define (eval-assignment exp env) (let ((new-value (eval (assignment-value exp) env))) (set-variable-value! (assignment-variable exp) new-value env) new-value))

20

eval




21

eval-definition

(define(eval-definition exp env) (define-variable! (definition-variable exp) (eval (definition-value exp) env) env) (definition-variable exp))

22

eval




23

self-evaluating?quoted?

variable? • The only self-evaluating items are numbers:

(define (self-evaluating? exp) (number? exp))

• Quotations are expressions of the form (quote ( text-of-quotation ) ) :

(define (quoted? exp) (if (atom? exp) nil (eq? (car exp) ‘quote) ) )

(define (text-of-quotation exp) (cadr exp) )

• Variables are represented by symbols:

(define (variable? exp) (symbol? exp) )

Primitive predicates

24

eval




25

assignment? • Assignments are expressions of the form:

(set! < variable> < value >)

(define (assignment? exp) (if (atom? exp) nil (eq? (car exp) ‘set!)))

26

eval




27

definition? • Definitions are expressions of the form (define < variable> < value > )• or of the form

(define (<variable> < parameter1 > . . . <parametern > ) < body>)

• The latter form (standard procedure definition) is syntactic sugar for

(define <variable> (lambda (< parameter1 > . . . <parametern > ) ) < body>))

28

definition? • The corresponding syntax procedures are the following:

(define (definition? exp) (if (atom? exp) nil (eq? (car exp) 'define)))

(define (definition-variable exp) (if (variable? (cadr exp) ) (cadr exp) (caadr exp) ) )

(define (definition-value exp) (if (variable? (cadr exp) ) (caddr exp) (cons ‘lambda (cons (cdadr exp) ;formal parameters (cddr exp ))))))) ;body

29

eval




30

lambda?

(define (lambda? exp) (if (atom? exp) nil (eq? (car exp) 'lambda) ) )

31

eval




32

conditional?clauses

(define (conditional? exp) (if (atom? exp) nil (eq? (car exp) 'cond) ) ) (define (clauses exp) (cdr exp) )

33

eval




34

conditional? clauses

(define (conditional? exp) (if (atom? exp) nil (eq? (car exp) 'cond) ) ) (define (clauses exp) (cdr exp) )

35

eval




36

eval-cond

(define (eval-cond clist env) (cond ((no-clauses? clist) nil) ((else-clause? (first-clause clist)) (eval-sequence (actions (first-clause clist)) env)

((true? (eval (predicate (first-clause clist))env)) (eval-sequence (actions (first-clause clist)) env))

(else (eval-cond (rest-clauses clist) env))))

37

no-clauses?/first-clause/ rest-clauses/predicate/actions/ true?/else-

clause? (define (no-clauses? clauses)

(null? clauses) )

(define (first-clause clauses) (car clauses)) (define (rest-clauses clauses) (cdr clauses) ) (define (predicate clause) (car clause) )

(define (actions clause) (cdr clause) ) (define (true? x) (not (null? x) ) )

(define (else-clause? clause) (eq? (predicate clause) 'else) )

38

apply




procedure))))

39

eval-sequence

(define (eval-sequence exps env) (cond ((last-exp? exps) (eval (first-exp exps)env)) (else (eval (first-exp exps) env) (eval-sequence (rest-exps exps)env))))

40

last-exp?/first-exp/rest-exps

(define (last-exp? seq) (null? (cdr seq) ) ) (define (first-exp seq) (car seq) )(define (rest-exps seq) (cdr seq) )

41

eval




42

application?operator operands

;application if it is non atom and is none of the above (define (application? exp) (not (atom? exp) ) )

(define (operator app) (car app) )

(define (operands app) (cdr app) )

43

eval




44

list-of-values

(define (list-of-values exps env) (cond ((no-operands? exps) '()) (else (cons (eval (first-operand exps)env) (list-of-values (rest-operands exps) env)))))

45

no-operands? first-operand

rest-operands

(define (no-operands? args) (null? args) ) (define (first-operand args) (car args) ) (define (rest-operands args) (cdr args) )

46

eval




47

make-procedure (define (make-procedure lambda-exp env) (list 'procedure lambda-exp env)) Since a lambda expression has the syntax (lambda <parameters> <exp1> . . . <expn>) the result of make-procedure is(procedure (lambda <parameters> <exp1> . . . <expn> ) < env>)

48

eval




49

apply




procedure))))

50

compound-procedure?parameters

procedure-bodyprocedure-environment

(procedure (lambda <parameters> <exp1> . . . <expn> ) < env>)

(define (compound-procedure? proc) (if (atom? proc) nil (eq? (car proc) 'procedure) ) )

(define (parameters proc) (cadr (cadr proc) ) )(define (procedure-body proc) (cddr (cadr proc) ) )(define (procedure-environment proc) (caddr proc) )

51








52

lookup-variable-value (define (lookup-variable-value var env) (let ((b (binding-in-env var env))) (if (found-binding? b) (binding-value b) (error "Unbound variable" var))))

(define (binding-in-env var env) (if (no-more-frames? env) no-binding (let ((b (binding-in-frame var (first-frame env)))) (if (found-binding? b) b (binding-in-env var (rest-frames env))))))

53








54

extend-environmentset-variable-value!define-variable!

define (extend-environment variables values base-env) (adjoin-frame (make-frame variables values) base-env ))

(define (set-variable-value! var val env) (let ((b (binding-in-env var env))) (if (found-binding? b) (set-binding-value! b val) (error "Unbound variable" var))))

(define (define-variable! var val env) (let ((b (binding-in-frame var (first-frame env)))) (if (found-binding? b) (set-binding-value! b val) (set-first-frame! env (adjoin-binding (make-binding var val) (first-frame env))))))

55

Representing environments(define (first-frame env) (car env))

(define (rest-frames env) (cdr env))

(define (no-more-frames? env) (null? env))

(define (adjoin-frame frame env) (cons frame env))

(define (set-first-frame! env new-frame) (set-car! env new-frame))

56

Frames(define (make-frame variables values) (cond ((and (null? variables) (null? values) ) ‘() ) ((null? variables) (error "Too many values supplied" values) ) ((null? values) (error "Too few values supplied" variables) ) (else (cons (make-binding (car variables) (car values) ) (make-frame (cdr variables) (cdr values))))))

(define (adjoin-binding binding frame) (cons binding frame) )

57

(define (assq key bindings)(cond ((null? bindings) no-binding) ((eq? key (binding-variable (car bindings))(car bindings)) (else (assq key (cdr bindings))))) (define (binding-in-frame var frame)(assq var frame)) (define (found-binding? b)(not (eq? b no-binding) ) ) (define no-binding nil)

58

Representing bindings(define (make-binding variable value) (cons variable value) ) (define (binding-variable binding) (car binding) ) (define (binding-value binding) (cdr binding) ) (define (set-binding-value! binding value) (set-cdr! binding value) )

59

(define primitive-procedure-names‘(car cdr cons ( <names of more primitives> ) ) )

(define primitive-procedure-objects(<list of objects representing primitives> ) )

60

(define (setup-environment) (let ((initial-env (extend-environment primitive-procedure-name primitive-procedure-objects ‘() ) ) )

(define-variable! 'nil nil initial-env)

(define-variable! 't (not nil) initial-env) initial-env) ) (define the-global-environment (setup-environment) )

61

(define (primitive-procedure? proc) (if (atom? proc) nil (eq? (car proc) 'primitive) ) )

(define (primitive-id proc) (cadr proc) ) (define primitive-procedure-objects ‘( (primitive car)(primitive cdr) (primitive cons) (<more primitives> ) ) )

62

(define (apply-primitive-procedure proc args) (let ( (p (primitive-id proc) ) ) (cond ( (eq? p ‘car) (car (car args) ) ) ( (eq? p ‘cdr) (cdr (car args) ) ) ( (eq? p ‘cons) (cons (car args)(cadr args))) (<more primitives> ) (else (error "Unknown primitive procedure” proc))))

63

A Lisp interpreter of micro algol

• From “A FORMAL DESCRIPTION OF A SUBSET OF ALGOL” By John McCarhty, 1964.

• “Microalgol is a language for programming about, not for programming in”.

• Two statement:– Assignments– if <p> then goto <label>

64

Example of microalgol program

root :=1;a: root := 0.5 * (root + x/root);

error := root * root – x;perror := if error > 0.0 then error else 0.0 –

error;If perror > 0.0001 then goto a;

65

Representation

• It will be an abstract syntax tree, in the form of a Lisp object as follows:

• A program is (<stmt> <stmt> …<stmt>)

• A statement is either– (‘assignment <variable> <expression>) or– (‘ifstatement <expression> <label>)

66

Selecting a statement by position

(define (statement prog pc) (cond ((null? prog) nil) ((= pc 1) (car prog)) (else (statement (cdr prog) (- pc 1)))))

67

Representation

• An expression is just represented in prefix notation.

• For example:z*(x+2.0)*(2.0+y) → (‘prod (‘prod ‘z (‘sum ‘x 2.0)) (‘sum 2.0 ‘y))

• andx := t + if error > 0.0 then e else 0.0 - e →(‘assignment ‘x (‘add ‘t (‘ifexpression (‘less 0.0 ‘e) ‘e (‘diff 0.0 ‘e)))

68

Predicates and their selectorsvariable?number?sum? addend augenddiff? subtraend minuendprod? multiplier multiplicandquotient? numerator denominatorifexpression? proposition antecedent(then) consequent(else)equal? lefteq righteqless? leftlt rightltassignment? left rightifstatement? proposition destination

The form of these predicates is the obvious one. variable? and number? are as above. The others have the following form:

(define (prod? exp) (if (atom? exp) nil (eq? (car exp) ’prod) ) )

69

Selectors

(define (addend exp ) (cadr exp))(define (subtraend exp ) (cadr exp))(define (multiplier exp ) (cadr exp))(define (numerator exp ) (cadr exp))(define (proposition exp ) (cadr exp))(define (lefteq exp ) (cadr exp))(define (leftlt exp ) (cadr exp))(define (left exp ) (cadr exp))(define (proposition exp ) (cadr exp))

70

Predicates and their selectorsvariable?number?sum? addend augenddiff? subtraend minuendprod? multiplier multiplicandquotient? numerator denominatorifexpression? proposition antecedent(then) consequent(else)equal? lefteq righteqless? leftlt rightltassignment? left rightifstmt? proposition destination

The form of these predicates is the obvious one. variable? and number? are as above. The others have the following form:

(define (prod? exp) (if (atom? exp) nil (eq? (car exp) ’prod) ) )

71

Selectors

(define (augend exp ) (caddr exp))(define (minuend exp ) (caddr exp))(define (multiplicand exp ) (caddr exp))(define (denominator exp ) (caddr exp))(define (antecedent exp ) (caddr exp))(define (consequent exp ) (cadddr exp))(define (righteq exp ) (caddr exp))(define (rightlt exp ) (caddr exp))(define (right exp ) (caddr exp))(define (destination exp ) (caddr exp))

72

Interpreter(define (eval prog pc env)(let ((s (statement prog pc))) (cond ((null? s) env) ((assignment? s) (eval prog (+ pc 1) (define-variable! (left s) (op (right s) env) env) env)) ((ifstatement? s) (eval prog (cond ((true? (op (proposition s) env) ) (lookup-variable-value (destination s) env) ((else (+ pc 1))) env)) (else error “Unknown statement type – EVAL”))))

73

op(define (opexpr env) (cond ((variable? expr) (lookup-variable-value expr eval)) ((number? expr) expr) ((sum? expr) (+ (op (addend expr) env) (op (augend expr) env) ((diff? expr) (- (op (subtraend expr) env) (op (minuend expr) env)) ((prod? expr) (* (op (multiplier expr) env) (op (multiplicand expr) env)) ((quotient? expr) (/ (op (numerator expr) env) (op (denomintor expr) env)) ((ifexpression? expr) (cond (true? (op (proposition expr) env) (op (antecedent expr) env)) (else (op (consequent expr) env))) ((equal? expr) (= (op (lefteq expr) env) (op (righteq expr) env)) ((less? expr) (< (op (leftlt expr) env) (op (rightlt expr) env)) (else error “Unknown operator – EVAL-EXPR”)))

74

Processing the program as a sequence of characters

(define (statement pc prog) ; Get statement number pc (if (= pc 1) (delim “;” prog) (statement (- pc 1) (strip “;” prog)))

(define (first prog) ; Get first character of program (if (equal? prog “”) “” (substring prog 0 1) )

(define (rest prog) ; Get the whole program but the first character (if (equal? prog “”) “” (substring prog 1 (string-length prog) )

75

(define (delim char prog) ; Get the sequence of characters (if (equal? (first prog) char); before the first “;” “” (string-append (first prog) (delim char (rest prog)))))

(define (strip char prog) ; Remove all occurences of char (if (equal? (first prog) char) ; from prog (rest prog) (strip char (rest prog))))

76

(define (sum? exp) (operator exp 0 “+”)) ; Sum operation

(define (operator exp lev op) (cond ((equal? exp “”) nil) ((equal? (first exp “(“)) (operator (rest exp) (+ lev 1) op)) ((equal? (first exp “)“)) (operator (rest exp) (- lev 1) op)) ((> lev 0) (operator (rest exp) lev op)) ((equal? (first exp op)) #t) (else (operator (rest exp) pos op))))

77

(define (augend exp) (delimexp “+” 0 exp)

(define (delimexp op lev exp) (cond ((equal? (first exp “(“)) (string-append (first exp) (delimexp op (+ lev 1) (rest exp))) ((equal? (first exp “)“)) (string-append (first exp) (delimexp op (- lev 1) (rest exp))) ((> lev 0)) (string-append (first exp) (delimexp op lev (rest exp))) ((equal? (first exp) op) “”) (else (string-append (first exp) (delimexp op lev (rest exp))))

78

The program as a sequence of tokens

• Has the advantage that we don’t need to do the lexical scanning as part of the intepretation process.

• Each token has a symbol assosciated with it, except that variables and numbers are pairs.

79

root :=1;a: root := 0.5 * (root + x/root);

error := root * root – x;perror := if error > 0.0 then error else 0.0 –

error;If perror > 0.0001 then goto a;

(list (‘var . ‘root) ‘assign (‘number . 1) ‘semicolon (‘label ‘a) ‘colon (‘variable . root) ‘assign (‘number 0.5) ‘mpy ‘leftpar (‘variable .‘root) ‘sum (‘variable . ‘x) ‘div (‘variable . ‘root) ‘rightpar ‘semicolon … (‘variable ‘perror) ‘assign ‘if (‘variable . ‘error) ‘gt (‘number . 0.0) ‘then (‘variabl ‘error) ‘else ...

80

Example evaluator(number, ‘sum, ‘mpy)

(define (eval-exp exp) (cond ((= (length exp) 1) (cdar exp)) ( (operator exp 0 ’sum ) (+ (eval-exp (left-operand exp 0 ’sum)) (eval-exp (right-operand exp 0 ’sum )))) ((operator exp 0 'mpy ) (* (eval-exp (left-operand exp 0 'mpy)) (eval-exp (right-operand exp 0 'mpy)))) (else (eval-exp (depar exp)))))

(define (operator exp lev op) (cond ((equal? exp '()) #f) ((equal? (car exp) 'leftpar) (operator (cdr exp) (+ lev 1) op)) ((equal? (car exp) 'rightpar) (operator (cdr exp) (- lev 1) op)) ((> lev 0) (operator (cdr exp) lev op)) ((eq? (car exp) op) #t) (else (operator (cdr exp) lev op))))

81

(define (left-operand exp lev op) (cond ((equal? exp '()) '()) ((equal? (car exp) 'lp) (cons (car exp) (left-operand (cdr exp) (+ lev 1) op))) ((equal? (car exp) 'rp) (cons (car exp) (left-operand (cdr exp) (- lev 1) op))) ((> lev 0) (cons (car exp) (left-operand (cdr exp) lev op))) ((equal? (car exp) op) '()) (else (cons (car exp ) (left-operand (cdr exp) lev op)))))

(define (right-operand exp lev op) (cond ((equal? exp '()) '()) ((equal? (car exp) 'lp) (right-operand (cdr exp) (+ lev 1) op)) ((equal? (car exp) 'rp) (right-operand (cdr exp) (- lev 1) op)) ((> lev 0) (right-operand (cdr exp) lev op)) ((eq? (car exp) op) (cdr exp) ) (else (right-operand (cdr exp) lev op))))

82

(define (depar exp) (if (equal? (car exp) 'leftpar ) (dep1 (cdr exp)) exp))

(define (dep1 exp) (if (equal? (cdr exp) '()) '() (cons (car exp) (dep1 (cdr exp)))))

cs 598 scripting languages design and implementation 10. interpreters part i: lisp

Documents