Download - LLVM Dev Meeting 2016 - ORC
![Page 1: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/1.jpg)
ORCLLVM’s Next Generation of JIT API
![Page 2: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/2.jpg)
Contents
• LLVM JIT APIs Past, Present and Future
• I will praise MCJIT
• Then I will critique MCJIT
• Then I’ll introduce ORC
• Code examples available in the Building A JIT tutorial on llvm.org
![Page 3: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/3.jpg)
Use CasesKaleidoscope LLDB
High Performance JITs Interpreters and REPLs
Simple and safe Cross-target compilation
Ability to configureoptimizations and codegen
Lazy compilation
Equivalence with static compile
![Page 4: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/4.jpg)
Requirements
, configurable for advanced users
, in-process for application scripting
• , non-lazy for high-performance cases
• Simple for beginners
• Cross-target for LLDB
• Lazy for interpreters
We can support all of these requirements
But not behind a single interface…
![Page 5: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/5.jpg)
ExecutionEngine
void addModule(Module*);
void *getPointerToFunction(Function*);
void addGlobalMapping(Function*, void*);
// Many terrible things that, trust me, you // really don’t want to know about.
![Page 6: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/6.jpg)
• MCJIT (LLVM 2.9 — present)
• Implements ExecutionEngine
• Cross-target, no lazy compilation
JIT Implementations
• Legacy JIT (LLVM 1.0 — 3.5)
• Introduced ExecutionEngine
• Lazy compilation, in-process only
• ORC (LLVM 3.7 — present)
• Forward looking API
• Does NOT implement ExecutionEngine
![Page 7: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/7.jpg)
MCJIT Design
• Static Pipeline with JIT Linker
• Efficient code and tool re-use
• Supports cross-target JITing
• Does not support lazy compilation
MCJITCodeGen
MC
RuntimeDyld
Object
LLVM IR
Raw Bits
Address
![Page 8: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/8.jpg)
MCJIT Implementation
• Only accessible via ExecutionEngine
• Caused ExecutionEngine to bloat
• Can not support all of ExecutionEngine
![Page 9: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/9.jpg)
ExecutionEngine
void *getPointerToFunction(Function*) uint64_t getFunctionAddress(const std::string&) void *getPointerToNamedFunction(StringRef) void *getPointerToFunctionOrStub(Function*) uint64_t getAddressToGlobalIfAvailable(StringRef) void *getPointerToGlobalIfAvailable(StringRef) void *getPointerToGlobal(const GlobalValue*) uint64_t getGlobalValueAddress(const std::string&) void *getOrEmitGlobalVariable(const GlobalVariable*)
Symbol Query Horrors…
![Page 10: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/10.jpg)
MCJIT Implementation
• Only accessible via ExecutionEngine
• Caused ExecutionEngine to bloat
• Can not support all of ExecutionEngine
• Limited visibility into internal actions
• No automatic memory management
![Page 11: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/11.jpg)
ORC — On Request CompilationA Modular MCJIT
![Page 12: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/12.jpg)
Modularizing MCJIT
MCJITCodeGen
MC
RuntimeDyld
Object
Raw Bits
![Page 13: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/13.jpg)
Modularizing MCJIT
Link Layer
Compile LayerCodeGen
MC
RuntimeDyld
Object
Raw Bits
Compile Layerforwards symbol
queries to thelink layer
![Page 14: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/14.jpg)
Initial Benefits
• Layers can be tested in isolation
Link Layer
Compile LayerCodeGen
MC
RuntimeDyld
Object
Raw Bits
![Page 15: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/15.jpg)
Initial Benefits
• Layers can be tested in isolation
Link LayerRuntimeDyld
Object
Raw Bits
• E.g. Unit test the link layer
![Page 16: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/16.jpg)
Initial Benefits
• Layers can be tested in isolation
Link Layer
Compile LayerCodeGen
MC
RuntimeDyld
Object
Raw Bits
• Observe events without callbacks
• E.g. Add a notification layer
• E.g. Unit test the link layer
![Page 17: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/17.jpg)
The Layer Interface
• Handle addModule(Module*, MemMgr*, Resolver*)
• Memory manager owns executable bits
• Resolver provides symbol resolution
• JITSymbol findSymbol(StringRef, bool)
• void removeModule(Handle)
![Page 18: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/18.jpg)
Example: Basic Composition
…ObjectLinkingLayer LinkLayer;SimpleCompiler Compiler(TargetMachine());IRCompileLayer<…> CompileLayer(LinkLayer, Compiler);…
![Page 19: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/19.jpg)
Example: Basic Composition
…ObjectLinkingLayer LinkLayer;SimpleCompiler Compiler(TargetMachine());IRCompileLayer<…> CompileLayer(LinkLayer, Compiler);…
class MyJIT {};
![Page 20: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/20.jpg)
Example: Basic Composition
…ObjectLinkingLayer LinkLayer;SimpleCompiler Compiler(TargetMachine());IRCompileLayer<…> CompileLayer(LinkLayer, Compiler);…
![Page 21: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/21.jpg)
Example: Basic Composition
…ObjectLinkingLayer LinkLayer;SimpleCompiler Compiler(TargetMachine());IRCompileLayer<…> CompileLayer(LinkLayer, Compiler);CompileLayer.addModule(Mod, MemMgr, SymResolver);auto FooSym = CompileLayer.findSymbol(“foo”, true);auto Foo = reinterpret_cast<int(*)()>(FooSym.getAddress());int Result = Foo(); // <—— Call into JIT’d code.…
![Page 22: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/22.jpg)
Memory Managers
• Own executable code, free it on destruction
• Inherit from RuntimeDyld::MemoryManager
• Custom memory managers supported
• SectionMemoryManager provides a good default
![Page 23: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/23.jpg)
Symbol Resolversauto Resolver = createLambdaResolver( [&](StringRef Name) { return CompileLayer.findSymbol(Name, false); }, [&](StringRef Name) { return getSymbolAddressInProcess(Name); });
• First lambda implements in-image lookup
• Second implements external lookup
![Page 24: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/24.jpg)
The Story So Far
• Layers wrap up JIT functionality to make it composable
• Build custom JITs by composing layers
• Memory managers handle memory ownership
• Symbol resolvers handle symbol resolution
![Page 25: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/25.jpg)
Adding New Features
• New layers provide new features
• Compile On Demand Layer
• addModule builds function stubsthat trigger lazy compilation
• Symbol queries resolve to stubs
Compile On Demand
Compile
Link
![Page 26: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/26.jpg)
Without Laziness
ObjectLinkingLayer LinkLayer; SimpleCompiler Compiler(TargetMachine()); IRCompileLayer<…> CompileLayer(LinkLayer, Compiler);
CompileLayer.addModule(Mod, MemMgr, SymResolver); CompileLayer.findSymbol(“foo”, true);auto FooSym = auto Foo = reinterpret_cast<int(*)()>(FooSym.getAddress()); int Result = Foo(); // <—— Call foo.
![Page 27: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/27.jpg)
With Laziness
ObjectLinkingLayer LinkLayer; SimpleCompiler Compiler(TargetMachine()); IRCompileLayer<…> CompileLayer(LinkLayer, Compiler);CompileOnDemandLayer<…> CODLayer(CompileLayer, …);
CODLayer.addModule(Mod, MemMgr, SymResolver); CODLayer.findSymbol(“foo”, true);auto FooSym = auto Foo = reinterpret_cast<int(*)()>(FooSym.getAddress()); int Result = Foo(); // <—— Call foo’s stub.
![Page 28: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/28.jpg)
COD Layer Requirements
• Indirect Stubs Manager
• Create named indirect stubs (indirect jumps via pointers)
• Modify stub pointers
• Compile Callback Manager
• Create compile callbacks (re-entry points in the compiler)
![Page 29: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/29.jpg)
Compile Callbacksfoo: jmp *foo$ptr
foo$cc: push foo_key jmp Resolver
Resolver
foo$impl: … ret
ORC/LLVM
bar: … call foo …
.ptr foo$ptr
![Page 30: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/30.jpg)
Callbacks and Stubsauto StubsMgr = … ; auto CCMgr = … ;
auto CC = CCMgr.getCompileCallback(); StubsMgr.createStub(“foo”, CC.getAddress(), Exported);
CC.setCompileAction([&]() -> TargetAddress { });
auto FooSym = StubsMgr.findStub(“foo”, true); auto Foo = static_cast<int(*)()>(FooSym.getAddress()); int Result = Foo();
printf(“Hello world”); return 0;
Prints “Hello world”, then jumps to 0
![Page 31: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/31.jpg)
Callbacks and Stubsauto StubsMgr = … ; auto CCMgr = … ;
auto CC = CCMgr.getCompileCallback(); StubsMgr.createStub(“foo”, CC.getAddress(), Exported);
CC.setCompileAction([&]() -> TargetAddress { });
auto FooSym = StubsMgr.findStub(“foo”, true); auto Foo = static_cast<int(*)()>(FooSym.getAddress()); int Result = Foo();
CompileLayer.addModule( return CompileLayer.findSymbol(“foo”, true).getAddress();
Lazily compiles “foo” from existing IR
FooModule, MemMgr, Resolver);
![Page 32: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/32.jpg)
Callbacks and Stubsauto StubsMgr = … ; auto CCMgr = … ;
auto CC = CCMgr.getCompileCallback(); StubsMgr.createStub(“foo”, CC.getAddress(), Exported);
CC.setCompileAction([&]() -> TargetAddress { });
auto FooSym = StubsMgr.findStub(“foo”, true); auto Foo = static_cast<int(*)()>(FooSym.getAddress()); int Result = Foo();
CompileLayer.addModule( return CompileLayer.findSymbol(“foo”, true).getAddress();
Lazily compiles “foo” from AST
IRGen(FooAST), MemMgr, Resolver);
![Page 33: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/33.jpg)
Laziness Recap
• Callbacks and Stubs
• Provide direct access to lazy compilation
• Push laziness earlier in the compiler pipeline
• CompileOnDemand provides off-the-shelf laziness for IR
• ORC supports arbitrary laziness with a clean API
![Page 34: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/34.jpg)
Adding New Layers
• Transform Layer
• addModule runs a user-suppliedtransform function on the module
• Symbol queries are forwarded
• Above C.O.D.: Eager optimizations
• Below C.O.D.: Lazy optimizations
Transform
Compile
Link
Transform
Compile On Demand
![Page 35: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/35.jpg)
Layers and Modularity
Pick features “off the shelf”
Mix and match components:experiment with new designs
Create, modify and share new featureswithout breaking existing clients
![Page 36: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/36.jpg)
Remote JIT Support
![Page 37: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/37.jpg)
Remote JIT Support• Execute code on a different process / machine / architecture
• Enables JIT code to be sandboxed
• MCJIT supported remote compilation, but required a lot of manual work
• OrcRemoteTarget client/server provides high level API
• Remote mapped memory, stub and callback managers
• Symbol queries
• Execute remote functions
![Page 38: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/38.jpg)
Local Lazinessauto StubsMgr = auto CCMgr =
auto CC = CCMgr.getCompileCallback(); StubsMgr.createStub(“foo”, CC.getAddress(), Exported);
CC.setCompileAction([&]() -> TargetAddress { CompileLayer.addModule(IRGen(FooAST), MemMgr, Resolver); return CompileLayer.findSymbol(“foo”, true).getAddress(); });
auto FooSym = StubsMgr.findStub(“foo”, true);auto Foo = static_cast<int(*)()>(FooSym.getAddress()); int Result = Foo();
auto StubsMgr = auto CCMgr =
auto CC = CCMgr.getCompileCallback(); StubsMgr.createStub(“foo”, CC.getAddress(), Exported);
CC.setCompileAction([&]() -> TargetAddress { CompileLayer.addModule
… ;… ;
![Page 39: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/39.jpg)
auto StubsMgr = auto CCMgr =
auto CC = CCMgr.getCompileCallback(); StubsMgr.createStub(“foo”, CC.getAddress(), Exported);
CC.setCompileAction([&]() -> TargetAddress { CompileLayer.addModule(
RT.createStubsMgr();
Remote Lazinessauto RT = … ;
IRGen(FooAST), RT.createMemMgr(), Resolver);
return CompileLayer.findSymbol(“foo”, true).getAddress(); });
auto FooSym = StubsMgr.findStub(“foo”, true);int Result = RT.callIntVoid(FooSym.getAddress());
RT.createCallbackMgr();
![Page 40: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/40.jpg)
Demo
![Page 41: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/41.jpg)
Remote JIT Support
• Remote JITing with ORC is easy
• Remoteness is orthogonal to other features, including laziness
• Security implications are serious
• Sandbox the server, authenticate the client, encrypt the channel
• Treat like mains electricity: very useful, but safety first!
![Page 42: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/42.jpg)
Future Opportunities
• New development modes: edit/test vs edit/compile/test
• Remote interpreters for development on embedded devices
• Distributing work for clusters
• Compute
• Database queries
![Page 43: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/43.jpg)
ORC vs MCJIT• Same underlying architecture: static compiler + JIT linker
• ORC
• Offers a strict superset of features
• A more flexible API
• Supports remoteness and laziness
• Has better memory management
• OrcMCJITReplacement provides a transition path
![Page 44: LLVM Dev Meeting 2016 - ORC](https://reader031.vdocuments.us/reader031/viewer/2022020910/620085f78d5c0a4e49270c82/html5/thumbnails/44.jpg)
Future Goals• Kill off ExecutionEngine, design a new in-tree JIT (for LLI and C-API)
• New layers and components (e.g. hot function recompilation)
• API cleanup: Core abstractions are in place but need polish
• More architectural and relocation support (Fix RuntimeDyldELF!)
• Check out the Building A JIT tutorial
• Get involved: http://llvm.org/bugs, OrcJIT component