Metaprogramming

CS251ProgrammingLanguagesSpring2017,LynTurbakDepartmentofComputerScienceWellesleyCollege

TheseslidesborrowheavilyfromBenWood’sFall‘15slides.

Howtoimplementaprogramminglanguage

InterpretaAonAninterpreterwriDenintheimplementaAonlanguagereadsaprogramwriDeninthesourcelanguageandevaluatesit.

TranslaAon(a.k.a.compilaAon)Antranslator(a.k.a.compiler)wriDenintheimplementaAonlanguagereadsaprogramwriDeninthesourcelanguageandtranslatesittoanequivalentprograminthetargetlanguage.

ButnowweneedimplementaAonsof:implementaAonlanguagetargetlanguage

Metaprogramming 2

Metaprogramming:InterpretaAon

InterpreterforlanguageLonmachineM

MachineMPrograminlanguageL

Metaprogramming 3

Interpreters

Data

Output

Source Program

Interpreter =

virtual machine

Metaprogramming 4

Metaprogramming:TranslaAon

InterpreterforlanguageBonmachineM

MachineM

PrograminlanguageA AtoBtranslator

PrograminlanguageB

Metaprogramming 5

if (x == 0) { x = x + 1; } ...

cmp (1000), $0 bne L add (1000), $1 L: ...

C Source Program C Compiler

x86 Target Program

Compiler

x86 Target Program

x86 computer

Data

Output

Metaprogramming 6

ThankstoBenWoodfortheseandfollowingpictures

JavaCompiler

if (x == 0) { x = x + 1; } ...

load 0 ifne L load 0 inc store 0 L: ...

Source Program Java Compiler

Target Program

(compare compiled C to compiled Java) Metaprogramming 7

InterpretersvsCompilers

InterpretersNoworkaheadofLmeIncrementalmaybeinefficient

CompilersAllworkaheadofLmeSeewholeprogram(ormoreofprogram)TimeandresourcesforanalysisandopLmizaLon

Metaprogramming 8

Compilers...whoseoutputisinterpreted

Target Program Java

Virtual Machine

Data

Output

Source Program Java Compiler

Target Program

Doesn’tthislookfamiliar? Metaprogramming 9

Interpreters...thatusecompilers.

Target Program

Virtual Machine

Data

Output

Source Program Compiler

Metaprogramming 10

TypicalCompiler

Source Program

Lexical Analyzer

Syntax Analyzer

Semantic Analyzer

Intermediate Code Generator

Code Optimizer

Code Generator

Target Program

Analysis

Synthesis

Metaprogramming 11

JITCompilersandOpAmizaAon

Target Program

Virtual

Machine Data

Output

Just In Time Compiler

PerformanceMonitor

Source Program

Compiler

•  HotSpotJVM•  JikesRVM•  SpiderMonkey•  v8•  Transmeta•  ...

Metaprogramming 12

VirtualMachineModel

High-LevelLanguageProgram

VirtualMachineLanguage

Bytecodecompiler

Virtualmachine(interpreter)

JITcompiler

runAme

compileAme

Ahead-of-Lmecompiler

NaLveMachineLanguage

Metaprogramming 13

Howtoimplementaprogramminglanguage

InterpreterRuleP-in-Lprogram Linterpretermachine

PmachineTranslatorRule

P-in-Sprogram S-to-TtranslatormachineP-in-Tprogram

Candescribebyderivinga“proof”oftheimplementaLonusingtheseinferencerules:

Metaprogramming 14

ImplementaAonDerivaAonExample

Provehowtoimplementa"251webpagemachine"using:

•  251-web-page-in-HTMLprogram(awebpagewriDeninHTML)•  HTML-interpreter-in-Cprogram(awebbrowserwriDeninC)•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)

Nopeakingahead!

Metaprogramming 15

ImplementaAonDerivaAonExampleSoluAon

Wecanomitsomeoccurrencesof“program”and“machine”:

Metaprogramming 16

ImplementaAonDerivaAonAreTreesAndsowecanrepresentthemasnestedstructures,likenestedbulletedlists:

251webpagemachine(I)q  251-web-page-in-HTMLprogramq  HTMLinterpretermachine(I)

²  HTML-interpreter-in-x86program(T)o  HTML-interpreter-in-Cprogramo  C-to-x86compilermachine(I)

•  C-to-x86compiler-in-x86program•  X86computer

²  x86computer

q  251-web-page-in-HTMLprogramo  HTML-interpreter-in-Cprogram

•  C-to-x86compiler-in-x86program•  X86computer

o  C-to-x86compilermachine(I)²  HTML-interpreter-in-x86program(T)²  x86computer

q  HTMLinterpretermachine(I)251webpagemachine(I)

Versionthatshowsconclusionsbelowbullets.MoresimilartoderivaLonswithhorizontallines,buthardertocreate,andread

Preferred“top-down”versionthatshowsconclusionsabovebullets.

Metaprogramming 17

Metaprogramming:BootstrappingPuzzles

HowcanwewriteaJava-to-x86compilerinJava?

HowcanwewriteSchemeinterpreterinScheme?

Metaprogramming 18

MetacircularityandBootstrappingManyexamples:•  LispinLisp/SchemeinScheme/RacketinRacket•  PythoninPython:PyPy•  JavainJava:JikesRVM,MaxineVM•  …•  C-to-x86compilerinC•  evalconstructinlanguageslikeLisp,JavaScript

Howcanthisbepossible?

Keyinsightstobootstrapping:•  ThefirstimplementaLonofalanguagecannotbein

itself,butmustbeinsomeotherlanguage.•  OnceyouhaveoneimplementaLonofalanguage,you

canimplementitinitself.Metaprogramming 19

Supposeyouaregiven:•  Scheme-interpreter-in-Pythonprogram•  Pythonmachine•  Scheme-interpreter-in-SchemeprogramHowdoyoucreateaSchemeinterpretermachineusingtheScheme-interpreter-in-Schemeprogram?

MetacircularityExample1:Problem

Metaprogramming 20

Supposeyouaregiven:•  Scheme-interpreter-in-Pythonprogram•  Pythonmachine•  Scheme-interpreter-in-SchemeprogramHowdoyoucreateaSchemeinterpretermachineusingtheScheme-interpreter-in-Schemeprogram?

Schemeinterpretermachine#2(I)q Scheme-interpreter-in-Schemeprogramq Scheme-interpretermachine#1(I)

² Scheme-interpreter-in-Pythonprogram² Pythonmachine

ButwhycreateSchemeinterpretermachine#2whenyoualreadyhaveScheme-interpretermachine#1?

MetacircularityExample1:SoluAon

Metaprogramming 21

Supposeyouaregiven:•  Scheme-subset-interpreter-in-Pythonprogram(implements

onlycoreSchemefeatures;nodesugaringorotherfrills)•  Pythonmachine•  Full-Scheme-interpreter-in-SchemeprogramHowdoyoucreateaFull-SchemeinterpretermachineusingtheFull-Scheme-interpreter-in-Schemeprogram?

Full-Schemeinterpretermachine(I)q Scheme-interpreter-in-Schemeprogramq Scheme-subsetinterpretermachine#1(I)

² Scheme-subset-interpreter-in-Pythonprogram² Pythonmachine

MetacircularityExample1:MoreRealisAc

Metaprogramming 22

Supposeyouaregiven:•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)•  C-to-x86-translator-in-C-subsetprogram

HowdoyoucompiletheC-to-x86-translator-in-C?

MetacircularityExample2:Problem

Metaprogramming 23

Supposeyouaregiven:•  C-to-x86-translator-in-x86program(aCcompilerwriDeninx86)•  x86interpretermachine(anx86computer)•  C-to-x86-translator-in-Cprogram

HowdoyoucompiletheC-to-x86-translator-in-C?

MetacircularityExample2:SoluAon

C-to-x86-translatormachine#2(I)q C-to-x86-translator-in-x86program#2(T)

² C-to-x86-translator-in-C² C-to-x86-translatormachine#1(I)

o  C-to-x86-translator-in-x86program#1o  x86computer

q x86computer

ButwhycreateC-to-x86-translator-in-x86program#2(T)whenyoualreadyhaveC-to-x86-translator-in-x86program#1?

Metaprogramming 24

Supposeyouaregiven:•  C-subset-to-x86-translator-in-x86program

(acompilerforasubsetofCwriDeninx86)•  x86interpretermachine(anx86computer)•  Full-C-to-x86-translator-in-C-subsetprogram

(acompilerforthefullClanguagewriDeninasubsetofC)

HowdoyoucreateaFull-C-to-x86-translatormachine?

MetacircularityExample2:MoreRealisAc

Full-C-to-x86-translatormachine(I)q Full-C-to-x86-translator-in-x86program(T)

² Full-C-to-x86-translator-in-C-subset² C-subset-to-x86-translatormachine(I)

o  C-subset-to-x86-translator-in-x86programo  x86computer

q x86computerMetaprogramming 25

AlonglineofCcompilersC-version_n-to-target_n-translatormachine(I)q  C-version_n-to-target_n-translatorprogramintarget_n-1(T)

²  C-version_n-to-target_n-translatorprograminC-version_n-1²  C-version_n-1-to-target_n-1translatormachine(I)

o  C-version_n-1-to-target_n_1-translatorprogramintarget_n-2(T)

Ø  C-version_2-to-target_2-translator-programintarget_1(T)

§  C-version_2-to-target_2-translatorprograminC-version_1§  C-version_1-to-target_1translatormachine(I)

•  C-version_1-to-target_1-translatorprograminassembly_0•  assembly_0computer

Ø  target_1computer

o  target_n-2computerq  target_n-1computer

…

…

o  TheversionsofCandtargetlanguagescanchangeateachstage.o  Trojanhorsesfromearliersourcefilescanremainintranslatormachinesevenif

they’renotinlatersourcefile!SeeKenThompson’sReflec4ononTrus4ngTrustMetaprogramming 26

MoreMetaprogramminginSML

•  We’vealreadyseenPostFixandIntexSML•  AsequencesofexpressionlanguagesimplementedinSMLthatlookcloserandclosertoRacket:•  Bindex:addnaming•  Valex:addmorevaluetypes,dynamictypechecking,desugaring

•  HOFL:firstclassfuncLonvalues,closurediagrams

Metaprogramming 27

Remember:language!=implementaAon•  Easytoconfuse"thewaythislanguageisusually

implemented"or"theimplementaLonIuse"with"thelanguageitself.”

•  JavaandRacketcanbecompiledtox86

•  CcanbeinterpretedinRacket

•  x86canbecompiledtoJavaScript

•  CanwecompileC/C++toJavascript?hDp://kripken.github.io/emscripten-site/

Metaprogramming 28