![Page 1: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/1.jpg)
A Theory ofHygienic Macros
David Herman, Mitchell WandNortheastern University
![Page 2: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/2.jpg)
2
A Theory of Hygienic Macros
Introduction to macros and hygiene
The challenge of formalizing hygiene
m and its properties
![Page 3: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/3.jpg)
3
In the beginning MACDEF SUM A,B,C LOAD A ADD B STORE CENDDEF
![Page 4: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/4.jpg)
4
The essence of macros
Derived (user-defined) syntactic constructs Defined by rewriting to existing constructs Translated at compile-time (“macro expansion”)
repeat s until e
begin s ; while (not e) s end end
![Page 5: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/5.jpg)
5
The essence of macros
Derived (user-defined) syntactic constructs Defined by rewriting to existing constructs Translated at compile-time (“macro expansion”)
(repeat s until e)
(begin s (while (not e) s))
![Page 6: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/6.jpg)
6
Example: short-circuit or
(or (foo "bar") (baz 42))
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
evaluate only if (foo "bar") produces #f
evaluate first; evaluate only once
pattern; matches subexpressions
e1 and e2
template; rewrites expression using
e1 and e2
![Page 7: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/7.jpg)
7
The power of macros
Abstractions for control, scope, program structure Powerful meta-programming idioms Simplifies language core (“extensible languages”)
functions, recursion, iteration, primitive datatypes
local bindings, custom loop forms, test harnesses, data
conditionals,constructors,
regularexpressions,
languagedesigner
library writer
little lang-uages, …
short-circuit operators,
![Page 8: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/8.jpg)
8
A brief history of hygiene
![Page 9: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/9.jpg)
9
Naïve macro expansion
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t #f) (if t t t)))
![Page 10: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/10.jpg)
10
Naïve macros are leaky abstractions(define-macro (or e1 e2) (with (t e1) (if t t e2)))
A leaky abstraction is a brittle abstraction.
Please don’t refer to any variablesnamed t in expression e2!
Oh, and don’t use or in a contextthat shadows the names with or if!
![Page 11: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/11.jpg)
11
For reasons of hygiene it will always be assumed that the bound variables that occur in a certain expression are different from the free ones. This can be fulfilled by renaming bound variables.
—H. P. Barendregt
Kohlbecker: This calls for hygiene
![Page 12: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/12.jpg)
12
Hygienic macro expansion
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t′ #f) (if t′ t′ t)))
![Page 13: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/13.jpg)
13
Hygiene and syntactic abstraction
functions, recursion, iteration, primitive datatypes
libraries
functions, recursion, iteration, primitive datatypes
local bindings, custom loop forms, test harnesses, data
conditionals,constructors,
regularexpressions,little lang-uages, …
short-circuit operators,
language
![Page 14: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/14.jpg)
14
Hygiene and syntactic abstraction
functions, recursion, iteration, primitive datatypes
custom loop forms, derived datatypes, first-class module systems, object-orientation, test harnesses, data
constructors, laziness, short-circuit boolean operators, iteration protocols,
static analyses, parser generators, partial evaluation, debuggers,
coroutines, exceptions, threads, type checkers,
introspection, etc…
local bindings, software contracts,
language
libraries
![Page 15: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/15.jpg)
15
What do we know about hygiene?
![Page 16: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/16.jpg)
16
Hygiene by example
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t #f) (if t t t)))
![Page 17: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/17.jpg)
17
Hygiene by informal rules
Kohlbecker et al, ’86:
Generated identifiers that become binding instances in the completely expanded program must only bind variables that are generated at the same transcription step.
Clinger and Rees, ’91:
1. It is impossible to write a macro that inserts a binding that can capture references other than those inserted by the macro.
2. It is impossible to write a macro that inserts a reference that can be captured by bindings other than those inserted by the macro.
SRFI 72:
A binding for an identifier can only capture a reference to another if both were present in the source or introduced during a single evaluation of a syntax or quasisyntax form, with the understanding that the evaluation of any nested, unquoted syntax or quasisyntax forms counts as part of the evaluation of an enclosing quasisyntax.
![Page 18: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/18.jpg)
18
Hygiene by implementation
![Page 19: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/19.jpg)
19
But which implementation?
?
Kohlbecker et al, ’86 Clinger & Rees ’91 Dybvig et al, ’93 Van Tonder, ’05
![Page 20: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/20.jpg)
20
Insight #1: Lexical scope
(define-macro (or e1 e2) (with (t e1) (if t t e2)))
(with (x 1) (or #f x))(with (x 1) (with (t #f) (if t t x)))
(with (t 1) (or #f t))(with (t 1) (with (t′ #f) (if t′ t′ t)))
=
=
![Page 21: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/21.jpg)
21
Hygienic macro expansionpreserves -equivalence.
The essence of hygienic macros
![Page 22: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/22.jpg)
22
What variables are bound by with?(define-macro (with (a e1) e2) )
(with (x 1) (or #f x))
(with (t 1) (or #f t))
=
…
![Page 23: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/23.jpg)
23
What is the binding structure?
(define-macro (murky a e) (begin (set! a e) (lambda (a) e)))
(lambda (foo) (murky foo (+ foo 1)))
![Page 24: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/24.jpg)
24
What is the binding structure?
(define-macro (turing-machine e)
)
![Page 25: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/25.jpg)
25
What is the binding structure?
(define-macro (indecisive a e) … (set! a e) … (lambda (a) e))
(lambda (foo) (indecisive foo (+ foo 1)))
![Page 26: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/26.jpg)
26
What variables are bound by with?(define-macro (with (a e1) e2) …) )
Standard answer: look at results of expansion
(with (x 1) (or #f x))((lambda (x) ((lambda (t) (if t t x)) #f) 1)
(with (t 1) (or #f t))((lambda (t) ((lambda (t′) (if t′ t′ t)) #f) 1)
…
![Page 27: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/27.jpg)
27
Using expansion to define scope
To specify hygienic expansion, we need to know the binding structure (scope) of programs.
To specify the binding structure of programs, we need to know the results ofhygienic expansion.
Oops.
![Page 28: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/28.jpg)
28
Inside every large language is a small language struggling to get out…
—Igarashi, Pierce, and Wadler
![Page 29: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/29.jpg)
29
Insight #2: Specifying lexical scope
![Page 30: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/30.jpg)
30
What variables are bound by with?(define-macro (with (a e1) e2) …) )
In practice, programmers write binding specifications:
with takes the form (with (a e1) e2) and produces an expression, where:
a is an identifier e1 and e2 are expressions a is bound in e2
…
![Page 31: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/31.jpg)
31
A shape type for with
(with (<a> expr) expra) → expr
binder
expression in same
environment
expression in scope of a
![Page 32: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/32.jpg)
32
Explicitly type-annotated macros
let val x = 1 in … x …end
(with (x 1) … x …)
(define-macro (with (a e1) e2) : expr ((a : var) (e1 : expr) (e2 : expr a)) )
…
…
![Page 33: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/33.jpg)
33
Shape-directed -conversion
(with (<a> expr) expra) → expr
(with (x 1) (or #f x))
(with (t 1) (or #f t))
=
![Page 34: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/34.jpg)
34
Shape-directed -conversion
(with (<a> expr) expra) → expr
(with (z 1) (or #f z))
(with (t 1) (or #f t))
=
![Page 35: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/35.jpg)
35
Shape-directed -conversion
(with (<a> expr) expra) → expr
(with (z 1) (or #f z))
(with (z 1) (or #f z))
=
![Page 36: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/36.jpg)
36
Quick recap
Macro shape types binding structure Binding structure -equivalence -equivalence correctness of hygienic
expansion
So, what of hygienic expansion?
![Page 37: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/37.jpg)
37
m: a model of hygienic macros
(with (x 1) x)
(x.x) 1
S-expressions
compiler’s core language
![Page 38: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/38.jpg)
38
m: a model of hygienic macros
e ::= var | var.e | e e| let syntax var = m in e end| (op s …)
core language
source
![Page 39: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/39.jpg)
39
Hygienic macro expansion
let syntax x = m in e end → e[m/x]
if BV(e) ∩ FV(m) =
((macro p => e) s …) → (e)
if (p) = s for some substitution and BV(s) ∩ FV(e) = and BV(e) # s
![Page 40: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/40.jpg)
40
Properties of m
![Page 41: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/41.jpg)
41
Confluence
(with (x 1) (or #f x))
(x.(with (t #f) (if t t x))) 1
(x.(or #f x)) 1(with (x 1) (with (t #f) (if t t x)))
![Page 42: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/42.jpg)
42
Hygiene
For (well-typed) e0 = e′0 and fully-expanded e and e′, if e0 →
* e and e′0 →
* e′, then e = e′.
e0
e′0
e
e′
*
*
![Page 43: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/43.jpg)
43
Hygiene…from confluence
* *
s
s1 s′1
s′
![Page 44: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/44.jpg)
44
Hygiene…from confluence
* *
s0
s1 s′1
s2
s′0
s′2
![Page 45: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/45.jpg)
45
Contributions
Semantics for hygienic expansion, accommodating: Lexically scoped macros Binding forms of arbitrary shape
Formal definitions and proofs -equivalence in presence of macros Shape-directed -conversion Confluence and hygiene
![Page 46: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/46.jpg)
46
What we’ve gained
Formalized programmers’ mental model of the binding structure of macros.
Showed the model has good theoretical properties.
Validated intuition: hygienic macro expansion frees the programmer to -convert with impunity.
But—more features yet to model…
![Page 47: A Theory of Hygienic Macros David Herman, Mitchell Wand Northeastern University](https://reader038.vdocuments.us/reader038/viewer/2022110211/56649ee55503460f94bf57fe/html5/thumbnails/47.jpg)
47
Thank you.
Hygienic macro expansionpreserves -equivalence.
Moral: