a theory of hygienic macros phd thesis proposal david herman
TRANSCRIPT
A Theory ofHygienic Macros
PhD Thesis ProposalDavid Herman
2
The power of macros
Derived (user-defined) syntactic constructs Defined by rewriting to existing constructs Translated at compile-time
(define-macro (increment! a) (set! a (+ a 1)))
3
The power of macros
Abstractions for scope, control, program structure Powerful meta-programming idioms Allow for a minimal language core
functions, recursion, iteration, primitive datatypes
custom loop forms, derived datatypes, first-class module systems, object-orientation, test harnesses, data
constructors, laziness, short-circuit boolean operators, iteration protocols,
static analyses, parser generators, partial evaluation, debuggers,
coroutines, exceptions, threads, type checkers,
introspection, etc…
local bindings, software contracts,
languagedesigner
language user
4
The need for hygiene
(define-macro (or e1 e2) (with ([t e1]) (if t t e2)))
(with ([x 1]) (or #f x))=>(with ([x 1]) (with ([t #f]) (if t t x)))
(with ([t 1]) (or #f t))=>(with ([t 1]) (with ([t #f]) (if t t t)))
5
The need for hygiene
(define-macro (or e1 e2) (with ([t e1]) (if t t e2)))
(with ([x 1]) (or #f x))=>(with ([x 1]) (with ([t #f]) (if t t x)))
(with ([t 1]) (or #f t))=>(with ([t 1]) (with ([t′ #f]) (if t′ t′ t)))
6
What are hygienic macros?
7
What are hygienic macros?
(define-macro (or e1 e2) (with ([t e1]) (if t t e2)))
(with ([x 1]) (or #f x))=>(with ([x 1]) (with ([t #f]) (if t t x)))
(with ([t 1]) (or #f t))=>(with ([t 1]) (with ([t #f]) (if t t t)))
8
What are hygienic macros?
9
What are hygienic macros?
KFFD ’86:
Generated identifiers that become binding instances in the completely expanded program must only bind variables that are generated at the same transcription step.
i.e., not provided as the macro’s input
i.e., a single rewriting of a macro application
10
What are hygienic macros?
Clinger and Rees ’91:
1. It is impossible to write a macro that inserts a binding that can capture references other than those inserted by the macro.
2. It is impossible to write a macro that inserts a reference that can be captured by bindings other than those inserted by the macro.
again: not provided as the macro’s input
11
Why isn’t this enough?
12
Pathological macros in Scheme
Impossible with hygienic macros, right?
(loop (begin (when (prime? i) (break i)) (increment! i)))
13
Pathological macros in Scheme
Contrast with the explicit version:
(loop/explicit break (begin (when (prime? i) (break i)) (increment! i)))
provided as input
14
Pathological macros in Scheme
Petrofsky extraction:
(loop (begin (when (prime? i) (break i)) (increment! i)))
15
Pathological macros in Scheme
Dumpster diving:
(loop (begin (when (prime? i) (break i)) (increment! i)))
16
Pathological macros in Scheme
Dumpster diving:
(loop/explicit break (begin (when (prime? i) (break i)) (increment! i)))
17
Pathological macros in Scheme
(define-macro (murky a e) (begin (set! a e) (lambda (a) e)))
(let ([foo 10]) (murky foo (+ foo 1)))
18
Pathological macros in Scheme
(define-macro (indecisive ([a e]) body) (if-zero …complicated computation… ((lambda (a) body) e) ((lambda (a) e) body)))
19
What do we know about hygiene?
20
Hygiene and lexical scope
(define-macro (or e1 e2) (with ([t e1]) (if t t e2)))
(with ([x 1]) (or #f x))=>(with ([x 1]) (with ([t #f]) (if t t x)))
(with ([t 1]) (or #f t))=>(with ([t 1]) (with ([t′ #f]) (if t′ t′ t)))
=
21
Hygienic macro expansion isinsensitive to -conversion.
The essence of hygienic macros
In other words,hygienic = respects
!
22
What is the scope of with?
(define-macro (with ([a e1]) e2) …)
First try: look at results of expansion
(with ([x 1]) (or #f x))=>((lambda (x) ((lambda (t) (if t t x)) #f) 1)
(with ([t 1]) (or #f t))=>((lambda (t) ((lambda (t′) (if t′ t′ t)) #f) 1)
23
Using expansion to define scope
Goal: define hygienic macro expansion in terms of -equivalence.
Strategy: define -equivalence in terms of hygienic macro expansion.
Oops.
24
What is the scope of with?
(define-macro (with ([a e1]) e2) …)
Better idea: provide a binding specification
with consumes an argument of shape(with ([a e1]) e2) and produces an expression, where:
a is an identifier e1 and e2 are expressions a is bound in e2
25
What is the scope of with?
(define-macro (with ([a e1]) e2) …)
Better idea: provide a binding specification
with : (with ([<a> expr]) expra) → expr
26
Annotated with macro
(define-macro (with ([a e1]) e2) : expr [(a : binder) (e1 : expr) (e2 : expr a)] …)
States macro writer’s intention about scope Admits rigorous -equivalence definition Provides correctness criterion for hygienic
macro expansion
27
Thesis
The key to specifying and verifying hygienic macro expansion is specifying the binding structure of macros.
28
A Theory of Hygienic Macros
m, a model of hygienic macros
1. Macro binding specifications (types)2. Theory of -equivalence3. High-level expansion semantics4. Definition and proof of hygiene
29
A Theory of Hygienic Macros
m, a model of hygienic macros
1. Macro binding specifications (types)2. Theory of -equivalence3. High-level expansion semantics4. Definition and proof of hygiene
30
m: a model of hygienic macros
(with ([x 1]) x)
(x.x) 1
S-expressions
-calculus
31
m: a model of hygienic macros
→ (with ([x 1]) x)
→ ((lambda (x) x) 1)
→ ((x.x) 1)
→ (x.x) 1
32
m: a model of hygienic macros
let syntax fun = macro ((a) e) : ((<a>) expra) => a.e with = macro (([a e]) body) : (([<a> expr])
expra) => ((a.body) e)in (with ([f (fun (x) 42)]) (f y.y))end
33
A Theory of Hygienic Macros
m, a model of hygienic macros
1. Macro binding specifications (types)2. Theory of -equivalence3. High-level expansion semantics4. Definition and proof of hygiene
34
1. Macro binding specifications
Shape types based on Culpepper and Felleisen ’03:
with : (with ([id expr]) expr) → expr
Types for reasoning about scope inspired by nominal datatypes and nominal logic of Gabbay and Pitts ’01
35
1. Type checking
let syntax m = macro p : => …in x.eend
[m : → expr] ⊢ x.e : expr
36
1. Type checking
let syntax m = macro p : => …in x.eend
[m : → expr, x : expr] ⊢ e : expr
37
1. Meta-level scope
let syntax m = macro (e1 e2) : (expr expr) => t.(e1 e2)in …end
[t : expr], [e1 : expr, e2 : expr] ⊢ e : expr
38
1. Two dimensions of binding
: program bindings (object-level) : macro pattern bindings (meta-level)
, ⊢ s :
41
A Theory of Hygienic Macros
m, a model of hygienic macros
1. Macro binding specifications (types)2. Theory of -equivalence3. High-level expansion semantics4. Definition and proof of hygiene
42
2. Theory of -equivalence Adapted to macro applications by exploiting
shape type annotations Relies on novel shape-directed -conversion Based on Gabbay/Pitts “swapping” formulation of
-equivalence
43
2. -equivalence via swapping
let val x = x in x (fn x => x) end=
let val y = x in y (fn w => w) end
binding occurrence
44
2. -equivalence via swapping
let val z = x in x (fn x => x) end=
let val z = x in y (fn w => w) end
free
45
2. -equivalence via swapping
let val z = x in x (fn x => x) end=
let val z = x in y (fn w => w) end
46
2. -equivalence via swapping
let val z = x in z (fn z => z) end=
let val z = x in z (fn w => w) end
47
2. -equivalence via swapping
let val z = x in z (fn z′ => z′) end=
let val z = x in z (fn z′ => z′) end
48
2. -equivalence via swapping For each binder x, pick fresh name z Rename the binding occurrence of x In the body, swap all occurrences of x with z Recur on subterms Compare results syntactically
49
2. -equivalence for m
(with ([x x]) (x (fun (x) x)))
=
(with ([y x]) (y (fun (w) w)))
?
50
2. Shape-directed -conversion
(with ([x x]) (x (fun (x) x)))
(with ([<a> expr]) expra)
(with ([y x]) (y (fun (w) w)))
binding occurrence
51
2. Shape-directed -conversion
(with ([z x]) (x (fun (x) x)))
(with ([<a> expr]) expra)
(with ([z x]) (y (fun (w) w)))
not in scope of a;not converted
52
2. Shape-directed -conversion
(with ([z x]) (x (fun (x) x)))
(with ([<a> expr]) expra)
(with ([z x]) (y (fun (w) w)))
entering scope of a;swap all occurrences of x / y
with z
53
2. Shape-directed -conversion
(with ([z x]) (z (fun (z) z)))
(with ([<a> expr]) expra)
(with ([z x]) (z (fun (w) w)))
54
2. Shape-directed -conversion
(with ([z x]) (z (fun (z) z)))
(fun (<b>) exprb)
(with ([z x]) (z (fun (w) w)))
55
2. Shape-directed -conversion
(with ([z x]) (z (fun (z′) z′)))
(fun (<b>) exprb)
(with ([z x]) (z (fun (z′) z′)))
56
2. -equivalence for m
(with ([z x]) (z (fun (z′) z′)))
=
(with ([z x]) (z (fun (z′) z′)))
57
But what is hygienic expansion?
58
A Theory of Hygienic Macros
m, a model of hygienic macros
1. Macro binding specifications (types)2. Theory of -equivalence3. High-level expansion semantics4. Definition and proof of hygiene
59
3. Hygienic macro expansion
Simple substitution semantics:
let syntax x = m in e end → e[m/x]
if BV(e) ∩ FV(m) =
(m . s) → transcribe(e, match(p, s))
if m = (macro p : => e) and BV(s) ∩ FV(e) = and BV(e) # s
60
3. Expansion up to x.let syntax m = macro (e) : (expr) => (e x) in x.(m x) end
61
3. Expansion up to x.let syntax m = macro (e) : (expr) => (e x) in x′.(m x′) end
→
x.x′.((macro (e) : (expr) => (e x)) x′)
→
x.x′.(x′ x)
62
3. Confluence
(with ([x 1]) (or #f x))
((x.(with ([t #f]) (if t t x))) 1)
((x.(or #f x)) 1)(with ([x 1]) (with ([t #f]) (if t t x)))
63
3. Confluence
Theorem: Let s be a term such that , ⊢ s : . Ifs → s1 and s → s2 then there exists an s′ such that s1 →
* s′ and s2 →*
s′.
* *
s
s1 s2
s′
64
A Theory of Hygienic Macros
m, a model of hygienic macros
1. Macro binding specifications (types)2. Theory of -equivalence3. High-level expansion semantics4. Definition and proof of hygiene
AKA, paydirt
65
4. Hygiene
The expansion relation → is hygienic if for any well-typed expressions e0 and e′0 such that e0 = e′0 and fully-expanded expressions e and e′ such thate0 →
* e and e′0 →
* e′, then e = e′.
e0
e′0
e
e′
*
*
Hygienic = respects !
66
4. Remember confluence?
* *
s up to-equivalence
s1 s2
s′
67
4. Hygiene
Theorem: the relation → is hygienic.
e0
e′0
e′
e
68
Research Plan
69
Additional properties and proofs
Joint confluence with evaluation:
(with ([x ((y.y) 1)]) x)
(with ([x 1]) x)((x.x) ((y.y) 1))
((x.x) 1)
70
Evaluation of classic algorithms
KFFD ’86 Macros that Work (Clinger and Rees ’91) Proof of correctness, if possible
71
Elements of syntax-rules
Core features: Recursive macros Case dispatch
Advanced features: Macro-defining macros
Conveniences: Variable-length sequences Ellipsis patterns (Kohlbecker and Wand ’87)
72
Research plan
Formal model (Completed.) Correctness proofs (Completed.) Elements of syntax-rules
(Expected May 2008.) Evaluation of classic algorithms
(Expected June 2008.) Additional properties and proofs
(Expected August 2008.) Dissertation and defense
(Expected November 2008.)
73
Related Work
74
Related work
Hygienic macro systems Kohlbecker et al ’86: Hygienic macro expansion Kohlbecker and Wand ’87: Macro-by-example Bawden and Rees ’88: Syntactic closures Clinger and Rees ’91: Macros that work Dybvig et al ’93: Syntactic abstraction in Scheme Krishnamurthi ’01: Linguistic reuse
Models of macros Bove and Arbilla ’92: Confluent calculus of expansion Gasbichler ’06: Modules with hygienic macros
75
Related workMacro types Culpepper and Felleisen ’03: Well-shaped macros
Variables and binding Higher-order abstract syntax, de Bruijn, “locally nameless” Gabbay, Pitts: Nominal logic
Staged programming Davies and Pfenning: Modal analysis of staged computation Taha et al: Multi-stage programming, MetaML, MacroML Nanevski: Meta-programming with names and necessity. Kim et al ’06: Polymorphic modal type system for Lisp-like
multi-staged languages
76
Hygienic = respects !
Thank you.
77
Thank you.