build an ide - eclipse 2009
TRANSCRIPT
© 2007 by «Author»; made available under the EPL v1.0 | Date | Other Information, if necessary© 2009 by IBM; made available under the EPL v1.0 | Mar 23, 2009
PIMP Your Eclipse: Building an IDE using IMP
Robert M. Fuhrer, Philippe Charles,Stanley M. Sutton Jr.(IBM T. J. Watson Research Center)Jurgen Vinju(Centrum Wiskunde en Informatica)
2 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.02
Outline
Introduction IDE Development Process Overview (20 mins) The Target Language (5 mins) Interactive Exercises: Developing an IDE (2 hours) Advanced Topics (30 mins) IMP Architectural Overview (10 mins) Where to go from here
3 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.03
IMP Goals
Facilitate the development of robust, high-functionality, language-specific IDEs in Eclipse
Enable language developers and users to get on with the interestingwork
Ultimately: make it easier to approximate the functionality of theEclipse JDT for other languages
4 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.04
IMP Target: Desired IDE Functionality
New Project/Type/… creation wizards refactoring launch & debug: launch configs,
breakpoints, backtraces, values,evaluation
Eclipse JDT, IntelliJ set a very high bar!
syntax highlighting, compiler annotations,hover help, source folding, formatting…
structural views
navigation (hyperlinks, “Open Type”, …)
content assist, quick fixes
compiler w/ incremental build,automatic dependency tracking
5 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.05
Nicely-formatted rich text hovers for documentation of given entity
Quick fixes: “Remove catch clause” when no one throws the corresponding exception
“Change method signature” – add a formal parameter of the appropriate type
“Add package to Export-Package directive in bundle manifest”
Refactorings: “Introduce Parameter” – make selected expression within a method a parameter at
the method’s call sites
“Infer Type Arguments” – migrate application to generics by determining theappropriate type parameters for each instance of an already generic type
“Introduce Type Parameter” – help make existing classes generic
Raise the level of abstraction at which programmer interacts with the code
void foo() { try { // doesn’t throw Foo } catch (Foo f) { // … }}
void foo() { // M not exported M m = new M();}
The “High Bar” of Language-specific Functionality
void foo() { //…}void bar() { foo(3);}
void foo() { // doesn’t throw Foo}
void foo(int i) { //…}void bar() { foo(3);}
<Manifest.mf>// …Export-Packages: org.foo.MPackage
void foo() { bar(3);}void bar(int N) { for(int i=0; i < N; i++){ //… }}
void foo() { bar();}void bar() { for(int i=0; i < 3; i++){ //… }}
void foo() { List<String> l= new ArrayList<String>(); l.add(“hi”); String s; s= /*(String)*/l.get(0);}
void foo() { List l= new ArrayList(); l.add(“hi”); String s; s= (String) l.get(0);}
class Cell<T> { T data; T getData() { return data; } void setData(T o) { data= o; }}
class Cell { Object data; Object getData() { return data; } void setData(Object o) { data= o; }}
6 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.06
IMP IDE for LPG
Source text isLPG grammarspecification
Navigableoutline ofgrammarsections
Hoverannotations
Syntaxcoloring
Hyperlinking(not shown)
Parsing anderror reporting
Sourcefolding
And there’s more
7 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.07
IMP Approach Keep IDE developer focused on language concepts
data structures (e.g. ASTs) and processing, not Eclipse UI componentry Leverage common themes, structures, concepts, semantics
Encapsulate common views/behavior within IMP runtime framework Define responsibilities for language-specific behavior as “IDE services” or
“language services” with corresponding extension points & APIs Encapsulate language and language processing concepts within IMP APIs
Remain technology-agnostic: parsing, analysis, … Design for customization Use meta-tooling to reify the IDE development flow
Templates/generators/wizards for IDE services Language-definition support (grammars, parsers)
Ease the IDE development process Generate language ASTs and parsers from grammar (e.g., LPG) Direct developer focus to relevant APIs and customization sites Provide DSLs to ease service implementations (preferences, presentation, etc.) Cheat sheets, documentation, default implementations, examples
Enable IDE developers to get on with the interesting work!
8 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.08
IMP & the Eclipse Plug-in Architecture Plug-in – set of contributions
Your IDE: one or more of these Smallest unit of Eclipse functionality Big example: HTML editor Small example: action that creates zip files
Extension point – named entity for collectingcontributions
Example: extension point for workbenchpreference UI
Generally, one per IMP language/IDE service
Extension – a contribution (an IMP service impl) Example: IMP outline tree model builder
IMP Runtimeplug-in
IDE plug-in A
extension (svc impl)extension point
IDE plug-in B
9 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.09
Installing IMP
As of 3/23/09, we recommend either 3.3.2 or 3.4.2
Many issues w/ 3.4.2 and p2 installs/updates, so please (for now): Use fresh 3.4.2 install
Install only IMP and IMP-related features in one shot
Vote for bugzilla bug on 3.4.2 p2:
https://bugs.eclipse.org/bugs/show_bug.cgi?id=268619
10 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.010
Installing IMP - Eclipse 3.3
11 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.011
Installing IMP - Eclipse 3.3 (cont.)
12 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.012
Installing IMP - Eclipse 3.3 (cont.)
13 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.013
Installing IMP - Eclipse 3.3 (cont.)
14 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.014
Installing IMP - Eclipse 3.3 (cont.) - Select Features
Except: IMP Program AnalysisUI depends on Zest (a graphlayout engine) - which is notpart of 3.3 Classic, so disablethis unless you installed itpreviously…
Select everything on theIMP Update Site!
15 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.015
Installing IMP - Eclipse 3.3 (cont.) - Licenses
16 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.016
Installing IMP - Eclipse 3.3 (cont.) - Go for it!
17 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.017
Installing IMP - Eclipse 3.4
18 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.018
Installing IMP - Eclipse 3.4
19 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.019
Installing IMP - Eclipse 3.4 (cont.) - Add Site
20 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.020
Installing IMP - Eclipse 3.4 (cont.) - Select Features
21 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.021
Installing IMP - Eclipse 3.4 (cont.) - Confirm
22 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.022
Installing IMP - Eclipse 3.4 (cont.) - Licenses
23 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.023
Outline
Introduction IDE Development Process Overview The Target Language Interactive Exercises: Developing an IDE Advanced Topics IMP Architectural Overview Where to go from here
24 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.024
IMP Development Process Overview Define “language descriptor”
Identify base language (if any), file name extensions, unique ID to locatecorresponding service implementations …
Define (or reuse) lexical and grammar specifications Used as basis for many language/IDE services Use any parser generator you like! (LPG, SDF, Antlr, …) IMP includes LPG tooling: creates a grammar skeleton; you complete it; ASTs
auto-generated Scannerless parsing is an option (for tricky languages like JavaScript, AspectJ) or: be macho and do it all yourself (with a little help from IMP)
Define language service implementations Wizard/compiler creates plugin metadata to hook into framework Wizard inserts example code for a simple language (not derived from AST, yet) Complete service implementation as desired In any order, though a few constraints (e.g., ref resolver before content assist)
IDE is usable at each step!
25 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.025
IMP Development Process Overview (cont.)
Lather, rinse, repeat,…
Some IDE services are “factored” ⇒ multiple levels of customization more investment ⇒ more language-specific behavior
IDE developer chooses a point in the spectrum from generic to fully custom
E.g. for hover help, if the IDE developer: does nothing
user gets trivial syntactic feedback on the kind of construct (feh) supplies a reference resolver
user gets a presentation of the referenced entity (can be good enough) supplies a documentation provider
user gets the documentation on the referenced entity (better) etc.
26 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.026
IMP Development Process Overview: Cheat Sheets
“Click toPerform”
Descriptionof stepProcess
overview
Initiateeach step
27 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.027
IMP Development Process Overview: User-Visible “IDEServices” Source editor Content assistance Hover help Outline view Quick outline Token coloring Indexed search (via PDB)° Compiler/tool annotations and
markers Hyperlinking Auto-editing Source folding
Annotation hover Formatting View filtering° Refactoring contributions Editor action contributions Preference pages Call graph° Type hierarchy°
° Under development
28 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.028
* no coding required ° under development
IMP Development Process: IDE DeveloperResponsibilities Language description* Parser; message handling Content Proposer Token Colorer Folding Updater Model Listener Index Contributor ° Auto-edit Strategy Editor actions Formatter
Structured source differ° Reference resolvers Incremental builder Compiler Nature enabler* Label Provider Image Decorator Outline content provider Refactoring contributions Documentation provider Preference service and pages
(via simple DSL)
All services are optional!
29 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.029
Outline
Introduction IDE Development Process OverviewThe Target Language Interactive Exercises: Developing an IDE Advanced Topics IMP Architectural Overview Where to go from here
30 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.030
The Target Language - “LEG”
The “Little Expression Grammar” Imperative, procedural, block-structured language Minimal control structures: if-then-else, while Type variable decls, assignments Only simple scalar types: int, double, boolean Functions with formal parameters, return statements Entry point: main()
int factorial(int n) { if (n <= 1) return 1; return n * factorial(n-1);}int main() { return factorial(10);}
31 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.031
Outline
Introduction IDE Development Process Overview The Target Language Interactive Exercises: Developing an IDE Advanced Topics IMP Architectural Overview Where to go from here
32 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.032
Tutorial Exercises Exercise 1: Prep’ing the Chassis
a) Registering your language w/ IMPb) Grammar & lexerc) Launch time!d) Grammar mods
Exercise 2: Wheels and Chromea) Syntax coloringb) Defining the outlinec) Making source foldable
Exercise 3: Engine Modsa) Resolving referencesb) Providing content assistancec) Providing an incremental builderd) Supporting ‘Mark Occurrences’
Exercise 4: Fuzzy Dicea) Enhancing hover helpb) Preference pages
Advanced Topics (time allowing): Source code formatting
Presentation specifications
Error recovery
Providing a custom editor name &icon
Using other kinds ofparser/compiler front ends
Simple breakpoint support
Deployment the Easy Way
33 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.033
Exercise 1: Prep’ing the Chassis
34 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.034
Exercise 0: Create a New Plugin Project
35 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.035
Exercise 0:Create a Plugin Project(cont.)
36 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.036
Exercise 0:Create a Plugin Project(cont.)
37 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.037
Exercise 0: Switch to the IMP IDE Perspective
38 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.038
Exercise 1a: Registering Your Language with IMP
New ->Programming
languagedescriptor
Languageproject and
IDFilename
extensions Parentlanguage
(leave blank)
39 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.039
Exercise 1a: Registering Your Language with IMP (cont.)
As a side-effect, this will also generate a new “bundle activator” class Overwrites the one generated by the “New Plugin Project” wizard
Adds functionality such as access to language ID, plugin log, etc.
Nothing interesting in the previously-generated one, so just accept the“nag box”
40 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.040
New extension oforg.eclipse.imp.runtime.languageDescription
Result of Creating New Language Descriptor
41 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.041
Exercise 1b: Parsing/Lexing: The Grammar and Lexer Specs
ParseController
class
Languageoptions
Auto-generate
AST classes
42 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.042
Exercise 1b: Anatomy of an LPG Grammar
%Options - package decl, enable AST gen, etc. - typically keep as is
%Globals - placed at beginning of generated Parser source file, e.g.additional imports
%Terminals - declare aliases for terminals that invalid Java™ symbols
%Start - identify the top-level grammar construct - usually somethinglike “sourceFile”
%Rules - the grammar rules - the good stuff
%Headers - additional Parser class members, nested classes
-- this is a comment in an LPG grammar
Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc. in the United States, other countries, or both.
43 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.043
Exercise 1b: Anatomy of an LPG Grammar (cont.) declaration ::= primitiveType identifier
compilationUnit$$functionDeclaration ::= %empty | compilationUnit functionDeclaration
identifier ::= IDENTIFIER /. IAst decl; public void setDeclaration(IAst decl) { this.decl = decl; } public IAst getDeclaration() { return decl; } ./
standard left-recursive rulefor a possibly-empty list
grammar annotation torequest ArrayList-likeAPI for generated AST
additional members forgenerated Java rule class
1 interface per non-terminal
1 class per rule
44 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.044
Exercise 1b: Anatomy of an LPG Lexer
%Options - package decl, etc. - typically keep as is
%Globals - additional imports
%Export - identify the terminal symbols to be exported to the parser
%Terminals - aliases for characters (optional)
%Start - ‘Token’ - typically keep as is
%Rules - the token syntax - same formalism/power as for grammars!(more powerful than regexps used by scanner generators like lex/flex)
-- this is a comment in an LPG grammar
45 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.045
Exercise 1b: Anatomy of an LPG Lexer (cont.)
Token ::= identifier /. checkForKeyWord(); ./ | number /. makeToken($_NUMBER); ./ | DoubleLiteral /. makeToken($_DoubleLiteral); ./ | white /. skipToken(); ./ | slc /. makeComment($_SINGLE_LINE_COMMENT); ./
number ::= digit | number digit
identifier ::= letter | identifier letter | identifier digit
%EXPORT Foo ➜ makeToken($_Foo)
syntax of a NUMBER token
valid token types
create “adjunct” token
46 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.046
Exercise 1b: Potential Issues Running LPG Builder
How to know if LPG builder ran correctly: LEGParser.java, LEGLexer.java, etc. generated in same package as grammar Package created containing generated AST classes
1. LPG Builder may not work when Eclipse is installed in a path that containsembedded spaces.• Remedy: Move Eclipse install folder to a path without embedded spaces
2. LPG executable doesn’t run properly• Cause: On Mac OS and Linux(?), the LPG executable file is not always marked as
executable on the file system.• Remedy: Run
chmod a+x /path/to/lpg-macosx_x86
on the file identified in the Error Log View
47 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.047
Result of Creating New Grammar & Parser
Sample grammartemplate instantiated
and opened —contains example
specificationsParser/AST classes
auto-generatedby LPG Builder
Developer pointedto starting point
for customization
48 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.048
Exercise 1c: Running Your IDE
49 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.049
Exercise 1c (cont.): Running Your IDE
50 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.050
Exercise 1c (cont.): Running Your IDE
Makes sure your plugindependencies are correctbefore launching (avoidsClassNotFoundExceptions)
51 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.051
Exercise 1c (cont.): Running Your IDE
Platform-specific LPG pluginfragments (whichever onesaren’t intended for theplatform you’re running)
Annotation-processingplugins (only valid forJava 6.0)
Trim your launch configto avoid irrelevantvalidation complaints
52 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.052
Exercise 1c (cont.): Running Your IDE
53 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.053
Exercise 1c (cont.): Running Your IDE
54 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.054
Running LEG Editor with Live Parsing
Project withsource
Source textwith
parse errorannotations
It’s alive!
55 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.055
Exercise 1d: Enhancing the Grammar
Add support for the usual comparison operators “<=“ and “>=“ to thelexer and grammar, which our definition of factorial() uses
56 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.056
Exercise 1d (cont.): Enhancing the Grammar (soln.)
Add to LegParser.g:
%Terminals LESS_EQUAL ::= '<=’ GREATER_EQUAL ::= ’>=’%End
-- …
%Rules | expression ’<=' term | expression ’>=' term%End
Add to LegLexer.gi:%Export -- … GREATER_EQUAL LESS_EQUAL%End
%Rules | ’<' '=’ /.makeToken($_LESS_EQUAL); ./ | ’>' '=’ /.makeToken($_GREATER_EQUAL); ./%End
57 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.057
Exercise 2: Wheels and Chrome
58 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.058
Exercise 2a: Token Coloring
59 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.059
Exercise 2a: Token Coloring (cont.)public class LEGColorer extends TokenColorerBase implements ITokenColorer, LEGParsersym {
protected final TextAttribute identifierAttr, numberAttr, keywordAttr; // …
public LEGColorer() { super(); Display d = Display.getDefault(); numberAttr = new TextAttribute(d.getSystemColor(SWT.COLOR_DARK_BLUE), null, SWT.BOLD); keywordAttr = new TextAttribute(d.getSystemColor(SWT.COLOR_DARK_RED), null, SWT.ITALIC);}
public TextAttribute getColoring(IParseController controller, Object o) { IToken token= (IToken) o;
switch (token.getKind()) {
case TK_IDENTIFIER: return identifierAttr; case TK_NUMBER: return numberAttr; // … default: if (((SimpleLPGParseController) controller).isKeyword(token.getKind())) return keywordAttr; return super.getColoring(controller, token); }}
public IRegion calculateDamageExtent(IRegion seed) { return seed;}
}
N.B.: Colors are “managed system resources” - very limited in number.IMPORTANT: Use either system Colors or Colors from a fixed palette.If too many Colors are instantiated, Eclipse may crash!
returns null for the default font
or, return an expanded region to re-color
or whatever things your getTokenIterator() returns
token kind symbols defined by lexer spec
good resource: http://colorbrewer.org
60 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.060
Exercise 2a (cont.): Enhancing the Token Colorer
Add appropriate code to LegTokenColorer to handle single-linecomments and doubles
Hints: Add a TextAttribute field decl for the comment attribute and initialize it
Use numberAttribute for doubles
Add cases to the switch statement in getColoring()
Use JDT hyperlinking (or “Find Declaration”) to locate TK_IDENTIFIER inorder to find the appropriate token symbols to use
61 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.061
Exercise 2a (cont.): Enhancing the Token Colorer (soln.)public class LEGTokenColorer implements ITokenColorer { TextAttribute commentAttribute; // … public TokenColorer() { // … commentAttribute = new TextAttribute( display.getSystemColor(SWT.COLOR_GREEN), null, SWT.ITALIC); // … } public TextAttribute getColoring(IParseController ctlr, Object o) { // … switch (token.getKind()) { // … case TK_DoubleLiteral: return numberAttribute; case TK_SINGLE_LINE_COMMENT: return commentAttribute; // … } }
}
62 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.062
Result of New Token Colorer
Implementationtemplate
instantiatedand opened
Contains exampleimplementation —returns appropriateTextAttribute based
on token type
Developer pointedto starting point
for customization
63 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.063
Exercise 2b:Defining the Outline
64 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.064
Exercise 2b: Defining an Outline (Model Builder)public class LEGModelBuilder extends TreeModelBuilderBase {
public void visitTree(Object root) { if (root == null) return; ASTNode rootNode= (ASTNode) root; LEGModelVisitor visitor= new LEGModelVisitor();
rootNode.accept(visitor); } private class LEGModelVisitor extends AbstractVisitor { // … public boolean visit(block n) { pushSubItem(n); return true; }
public void endVisit(block n) { popSubItem(); }
public boolean visit(declarationStmt0 n) { createSubItem(n); return true; } }
}
create item w/ children
“close” item w/ children
create item w/ no children
generated, but if you useanother kind of visitor…
LPG-generated visitor class
continue visiting sub-structure
65 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.065
Exercise 2b: Enhancing the Model Builder
Add functionDeclaration nodes to the model
Hints: Add visit() and endVisit() overrides to the visitor class for
functionDeclaration, and call pushSubItem() andpopSubItem()
66 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.066
Exercise 2b: Defining an Outline (Label Provider)
public class LEGLabelProvider implements ILabelProvider {
private static ImageRegistry sImageRegistry = Activator.getInstance().getImageRegistry();
private static Image DEFAULT_IMAGE = sImageRegistry.get(ImageResources.LEG_DEFAULT_IMAGE);
private static Image FILE_IMAGE = sImageRegistry.get(ImageResources.LEG_FILE); private static Image FILE_WITH_WARNING_IMAGE = sImageRegistry.get(ImageResources.LEG_FILE_WARNING); private static Image FILE_WITH_ERROR_IMAGE = sImageRegistry.get(ImageResources.LEG_FILE_ERROR);
// …
A: image resource management
allocate image resources with care -use an ImageRegistry (as provided byIMP-generated bundle activator class)
67 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.067
Exercise 2b: Defining an Outline (Label Provider) public Image getImage(Object element) {
if (element instanceof ISourceEntity) { ISourceEntity entity= (ISourceEntity) element; return getImageFor(entity.getResource()); } if (element instanceof IResource) { return getImageFor((IResource) element); } ASTNode n= (element instanceof ModelTreeNode) ? (ASTNode) ((ModelTreeNode) element).getASTNode() : (ASTNode) element; return getImageFor(n); }
public Image getImageFor(IResource res) { if (res instanceof IFile) { int sev = MarkerUtils.getMaxProblemMarkerSeverity((Ifile) res, IResource.DEPTH_ONE); switch(sev) { case IMarker.SEVERITY_ERROR: return FILE_WITH_ERROR_IMAGE; case IMarker.SEVERITY_WARNING: return FILE_WITH_WARNING_IMAGE; default: return FILE_IMAGE; } } return null; // use default image }
B: providing images
IMP Project Explorer
Package Explorer, Navigator
IMP outline page
decorate files w/ marker indicators
68 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.068
Exercise 2b: Defining an Outline (Label Provider)
public static Image getImageFor(ASTNode n) { return DEFAULT_IMAGE; }
B: providing images
69 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.069
Exercise 2b: Defining an Outline (Label Provider) public String getText(Object element) { ASTNode n= (element instanceof ModelTreeNode) ? (ASTNode) ((ModelTreeNode) element).getASTNode() : (ASTNode) element;
if (n instanceof IcompilationUnit) return "Compilation unit"; if (n instanceof block) return "Block"; // … if (n instanceof functionDeclaration) { functionHeader hdr = (functionHeader) ((functionDeclaration) n).getfunctionHeader(); return hdr.getidentifier().toString(); } return "<???>"; }
C: providing text labels
70 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.070
Exercise 2b: Enhancing the Label Provider
Add formal argument types to the label produced forfunctionDeclarations
Add custom icons for functionDeclarations and for the main()function.
Hints: Copy & modify icons from the “icons” folder of the LEG project to create
2 new icons (e.g. “funcDecl.gif” and “mainDecl.gif”)
Copy the image resource maintenance code in the bundle Activator classand ILEGResources interface for these 2 new icons.
Add case to LEGLabelProvider.getImageFor(ASTNode n) forfunctionDeclaration
71 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.071
Exercise 2b: Enhancing the Label Provider (soln.) public String getText(Object element) { ASTNode n= (element instanceof ModelTreeNode) ? (ASTNode) ((ModelTreeNode) element).getASTNode() : (ASTNode) element; // … if (n instanceof functionDeclaration) { functionHeader hdr = (functionHeader) ((functionDeclaration) n).getfunctionHeader(); StringBuilder sb= new StringBuilder(); sb.append(hdr.getType()); sb.append(" "); sb.append(hdr.getidentifier().toString()); sb.append("("); declarationList parms = hdr.getparameters(); for(int i=0; i < parms.size(); i++) { if (i > 0) sb.append(", "); sb.append(parms.getdeclarationAt(i).getprimitiveType()); } sb.append(")"); return sb.toString(); } // … }
72 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.072
Exercise 2b: Enhancing the Label Provider (soln.)public interface ILEGResources { // … public static final String FUNC_DECL= "funcDecl"; public static final String MAIN_DECL= "mainDecl"; // …
}
public class Activator extends PluginBase { protected void initializeImageRegistry(ImageRegistry reg) { // … path= ICONS_PATH.append(”funcDecl.gif"); imageDescriptor= createImageDescriptor(bundle, path); reg.put(ILEGResources.FUNC_DECL, imageDescriptor);
path= ICONS_PATH.append(”mainDecl.gif"); imageDescriptor= createImageDescriptor(bundle, path); reg.put(ILEGResources.MAIN_DECL, imageDescriptor); }
}
73 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.073
Exercise 2b: Enhancing the Label Provider (soln.)public class LEGLabelProvider implements ILabelProvider { // … private static Image FUNC_DECL_IMAGE = sImageRegistry.get(ILEGResources.FUNC_DECL); private static Image MAIN_DECL_IMAGE = sImageRegistry.get(ILEGResources.MAIN_DECL); // … public static Image getImageFor(ASTNode n) { if (n instanceof functionDeclaration) { functionDeclaration fd= (functionDeclaration) n; if(fd.getfunctionHeader().getidentifier().toString().equals("main")) return MAIN_DECL_IMAGE; else return FUNC_DECL_IMAGE; } return DEFAULT_IMAGE; }
}
74 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.074
Exercise 2c: Making Source Foldable
75 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.075
Exercise 2c: Making Source Foldablepublic class LEGFoldingUpdater extends LPGFolderBase {
private void makeFoldable(ASTNode n) { makeAnnotation(n.getLeftIToken(), n.getRightIToken()); }
private class FoldingVisitor extends AbstractVisitor { public boolean visit(block n) { makeFoldable(n); return true; } };
public void sendVisitorToAST(HashMap newAnnotations, List annotations, Object astRoot) { ASTNode theAST= (ASTNode) astRoot; prsStream = theAST.getLeftIToken().getPrsStream(); AbstractVisitor abstractVisitor = new FoldingVisitor(); theAST.accept(abstractVisitor); makeAdjunctAnnotations(); }
}
N.B.: Folding is line-based, so if your regiondoesn’t include an EOL, it won’t be foldable
override for each node to make foldablemake it so!
optional: makes comments (aka “adjuncts”) foldable
continue visiting sub-structure
76 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.076
Exercise 2c: Enhancing the Source Folder
Add functionDeclarations to the set of foldable entities Hint: add an override for functionDeclaration to the visitor class,
and call makeFoldable() Be sure to return true, since you want the visitor to visit the
functionDeclaration‘s body
77 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.077
Exercise 2c: Enhancing the Source Folder (soln.)
private class FoldingVisitor extends AbstractVisitor { // … public boolean visit(functionDeclaration n) { makeFoldable(n); return true; }
}
78 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.078
Exercise 3: Engine Mods
79 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.079
Prelude to Exercise 3a: Language ServicesUnderlying Hover Help and Hyperlinking Hover help:
Use ISourcePositionLocator to identify AST node under text cursor If there’s an IReferenceResolver:
If this AST node is a reference, use the IReferenceResolver to determine thecorresponding declaring AST node
If there’s an IDocumentationProvider: Pass resulting entity to IDocumentationProvider
Else: Obtain source text for target
Hyperlinking: Use ISourcePositionLocator to identify AST node under text cursor If this AST node is a reference, use the IReferenceResolver to determine the
corresponding declaring AST node If the resolver produces something other than the original node:
Use the ISourcePositionLocator to locate the declaration’s text (possibly inanother file)
80 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.080
Exercise 3a: Resolving References
81 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.081
Exercise 3a: Resolving Referencespublic class LEGReferenceResolver implements IReferenceResolver {
public LEGReferenceResolver () { }
public String getLinkText(Object node) { return node.toString(); }
public Object getLinkTarget(Object node, IParseController ctlr) { if (node instanceof Iidentifier && ctlr.getCurrentAst() != null) { identifier id = (identifier) node; LEGParser parser = (LEGParser) ((SimpleLPGParseController)controller).getParser(); SymbolTable<IAst> symtab = parser.getEnclosingSymbolTable(id);
return symtab.findDeclaration(id.toString()); }
return null; }
}
Get the text to be shown in the popup post-itview before hyperlinking to that node
If the given AST node is a reference to something,return the target of the reference (usually a decl).
82 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.082
Exercise 3a: Resolving References(Digression on LEG’s Use of Symbol Tables)
functionDeclaration ::= functionHeader block /. SymbolTable<IAst> symbolTable;
public void setSymbolTable(SymbolTable<IAst>symbolTable) { this.symbolTable = symbolTable; }
public SymbolTable<IAst> getSymbolTable() { return symbolTable; } ./
N.B.: In LEG, each function decl defines a local scope
add a local SymbolTable field
setter for parser to set up the scope
getter for name lookup to find the scope
How to look up references in LEG?
Other possible sources of binding info: index (using PDB), compiler front-end
<LegParser.g excerpt>
83 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.083
Exercise 3a: Resolving References(Digression on LEG’s Use of Symbol Tables)
private final class SymbolTableVisitor extends AbstractVisitor { public boolean visit(block n) { n.setSymbolTable(symbolTableStack.push( new SymbolTable<IAst>(symbolTableStack.peek()))); return true; } public void endVisit(block n) { symbolTableStack.pop(); } public boolean visit(declaration n) { IToken id = n.getidentifier().getIToken(); SymbolTable<IAst> symbol_table = symbolTableStack.peek();
if (symbol_table.get(id.toString()) == null) symbol_table.put(id.toString(), n); else emitError(id, "Illegal redeclaration of " + id.toString()); return true; } public boolean visit(identifier n) { IToken id = n.getIDENTIFIER(); IAst decl = symbolTableStack.peek().findDeclaration(id.toString());
if (decl == null) emitError(id, "Undeclared variable " + id.toString()); else n.setDeclaration(decl); return true; }
}
push symtab for new scope
pop symtab when leaving scope
install var decl in symtab
find decl in symtab, put direct ref in ident node
<LegParser.g excerpt>
84 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.084
Exercise 3a Digression: ISourcePositionLocator
public interface ISourcePositionLocator { Object findNode(Object astRoot, int offset);
Object findNode(Object astRoot, int startOffset, int endOffset);
int getStartOffset(Object entity);
int getEndOffset(Object entity);
int getLength(Object entity);
IPath getPath(Object entity);}
textual offsets are in chars
the innermost AST node whose textual extent contains the given text offset
the innermost AST node that spans the given text extent
“entity”: AST node, token, or type system object
the char offset of the beginning of the textual extent of the given entity
the char offset of the end of the textual extent of the given entity
the char length of the textual extent of the given AST node
the workspace-relative or file-system absolute path to the compilationunit (source or compiled, if no source) containing the given AST nodeor ISourceEntity. The path is in Eclipse "portable" format, using '/' for thepath separator. See org.eclipse.core.runtime.IPath#toPortableString().
How do we locate source entities?You can obtain an ISourcePositionLocator fromIParseController.getNodeLocator()
85 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.085
Exercise 3b: Providing Content Assistance
86 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.086
Exercise 3b: Providing Content Assistance
public class LEGContentProposer implements IContentProposer {
public ICompletionProposal[] getContentProposals(
IParseController ctlr,
int offset,
ITextViewer viewer) { // …}
}
offset at which contentassist was requestedviewer for the editorrequesting assistance
gives access to AST, etc.
87 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.087
Exercise 3b: Content Assistance: SourceProposals
public class SourceProposal implements ICompletionProposal { String proposal;
String newText;
String prefix;
int offset;
int cursorLoc;
String addlInfo;}
Shown to the user in the popup view
The text to be added to the source buffer
The prefix being completed (if any;typically for completing an identifier)
The location of the text being completed(buffer-relative)
Where to place the cursor once the textis inserted, if the proposal is accepted(buffer-relative)
Info to show in a popup pane alongside theproposal list when this proposal is selected
88 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.088
Exercise 3b: Content Assistance: SourceProposals
proposal
newText = “wait()”
addlInfo
offset prefix = “wa”
cursorLoc = just after “wait()”
89 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.089
Exercise 3b: Providing Content Assistancepublic class LEGContentProposer implements IContentProposer {
public ICompletionProposal[] getContentProposals(IParseController ctlr, int offset, ITextViewer viewer) { List<ICompletionProposal> result = new ArrayList();
if (ctlr.getCurrentAst() != null) { IToken token = getToken(ctlr, offset); String prefix = getPrefix(token, offset); LEGParser parser = (LEGParser) ((SimpleLPGParseController) ctlr).getParser(); ISourcePositionLocator locator = ctlr.getNodeLocator(); ASTNode node = (ASTNode) locator.findNode(ctlr.getCurrentAst(), token.getStartOffset(), token.getEndOffset()); if (node != null) { result= computeProposals(token, prefix, node, parser); } } else { result.add(new ErrorProposal("No proposals available - syntaxerrors", offset)); }
return result.toArray(new ICompletionProposal[result.size()]); } // …
90 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.090
Exercise 3b: Providing Content Assistance (cont.)private List<ICompletionProposal> computeProposals(String prefix,
ASTNode node, int offset, LEGParser parser) { List<ICompletionProposal> result = new ArrayList(); if (node.getParent() instanceof Iexpression || node.getParent() instanceof assignmentStmt || node.getParent() instanceof BadAssignment || node.getParent() instanceof returnStmt) { Map<String, IAst> decls = collectVisibleDecls( parser.getEnclosingSymbolTable(node)); List<IAst> matchingDecls = filterSymbols(decls, prefix);
for (IAst decl: matchingDecls) { result.add(createProposalForDecl(decl, prefix, offset)); } } else { result.add(new ErrorProposal("No completion exists for prefix: " + prefix, offset)); } return result;
}
91 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.091
Exercise 3b: Providing Content Assistance (cont.)
private Map<String,IAst> collectVisibleDecls( SymbolTable<IAst> innerScope) {
Map<String,IAst> result = new HashMap<String,IAst>();
// Move outward from innermost enclosing scope for (SymbolTable<IAst> s = innerScope; s != null; s = s.getParent()) { for (String key: s.keySet()) { if (! result.containsKey(key)) // omit shadowed declarations result.put(key, s.get(key)); } } } return result;
}
92 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.092
Exercise 3b: Providing Content Assistance (cont.)private SourceProposal createProposalForDecl(IAst decl, String prefix, int offset) { String propDescrip = "", newText = "";
if (decl instanceof declaration) { newText = ((declaration) decl).getidentifier().toString(); propDescrip = ((declaration) decl).getprimitiveType() + " " + newText; } else if (decl instanceof functionDeclaration) { functionDeclaration fdecl = (functionDeclaration) decl; functionHeader fhdr = fdecl.getfunctionHeader(); declarationList parms = fhdr.getparameters(); newText = fhdr.getidentifier() + "("; for (int i = 0; i < parms.size(); i++) { Ideclaration parm = parms.getdeclarationAt(i); newText += ((declaration0) parm).getprimitiveType() + (i < parms.size() - 1 ? ", " : ""); } newText += ")"; propDescrip = fhdr.getType() + " " + newText; } return new SourceProposal(propDescrip, newText, prefix, offset);
}
Here’s the Beef!
handle vardecls
handle funcdecls
93 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.093
Exercise 3c:Providing anIncremental Builder
…SMAP support is only useful ifyour compiler generates Java code…
Enabling this will cause a “line mappingtable” to be generated in each class file
94 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.094
Exercise 3c: Providing an Incremental Builder IDE developer has these responsibilities:
a) Pass options from preferences page to compile
b) Identify suitable source files
c) Route compiler messages to markers
d) Inform IMP of inter-compilation-unit dependencies
e) Provide “Nature Enabler” to configure builder on existing projects
f) [optional] Provide “New Project Wizard” that configures nature (JDT doesthis)
g) Oh, and build :-) Two basic options for the compiler:
Call out to an existing compiler
Write your own, starting from AST (IMP is not a compiler framework – useany one of the really good ones already out there! Polyglot, JAstAdd, …)
95 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.095
Exercise 3c: Providing an Incremental BuilderDigression on Natures & Builders Builders
A builder instance is associated with a given project (accessible fromyour builder class via getProject())
A builder scans the project for relevant input (source) files to process A builder creates markers on resources to indicate build problems
Natures A project has one or more natures Natures are often associated with languages (but could also be
associated with tools) A nature is typically used to enable various bits of functionality E.g., the JDT’s “Java nature” enables the JDT’s Java compiler The nature typically adds the associated builder(s) to the project’s list of
“build commands”
96 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.096
Exercise 3c: Providing an Incremental Buildera) Pass options from preferences to compiler
public class BuilderBase extends IncrementalProjectBuilder { protected IPreferencesService fPrefService = new PreferencesService(null, getPlugin().getLanguageID());
} protected IProject[] build(int kind, Map args, IProgressMonitor mon) { if (fPrefService.getProject() == null) { fPrefService.setProject(getProject()); // Use project-specific prefs } }
}
public class LegBuilder extends BuilderBase { protected void compile(IFile file, IProgressMonitor mon) { // … fEmitDiagnostics = fPrefService.getBooleanPreference( LegConstants.P_EMITDIAGNOSTICS); // … }
public String getIncludePath() { return fPrefService.getStringPreference(LegConstants.P_INCLUDEPATH); }
}
97 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.097
Exercise 3c: Providing an Incremental Builderb) Identifying suitable source files
public class MyBuilder extends BuilderBase { // … protected boolean isSourceFile(IFile file) { return !file.isDerived() && file.getFileExtension().equals(“g”);
}
protected boolean isNonRootSourceFile(IFile file) { return !file.isDerived() && file.getFileExtension().equals(“gi”); }
}“non-root” files are scanned fordependencies, but not passedto compile()
“root” files are both passed tocompile() and scanned fordependencies
98 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.098
Exercise 3c: Providing an Incremental Builderc) Route compiler messages to markersBuilderBase.createMarker(IResource errorResource, int startLine, int startChar, int endChar, String description, int severity)
where errorResource is typically a source file
where startChar, endChar are file- (not line-) relative offsets
where severity is one of: IMarker.SEVERITY_INFO IMarker.SEVERITY_WARNING IMarker.SEVERITY_ERROR
99 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.099
Exercise 3c: Providing an Incremental Builderd) Inform IMP of inter-compilation-unit dependenciespublic class BuilderBase { protected DependencyInfo fDependencyInfo;
protected abstract void collectDependencies(IFilefile);
}
Typically: parse file, traverse AST and look for references to entities inother files
Register dependencies via calls to DependencyInfo method
public void addDependency(String fromPath, String uponPath);
100 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0100
Exercise 3c: Providing an Incremental Buildere) Providing a “Nature Enabler”public class EnableNature implements IWorkbenchWindowActionDelegate { private IProject fProject;
public EnableBuilder() { } public void dispose() { } public void init(IWorkbenchWindow window) { }
public void run(IAction action) { new MyNature().addToProject(fProject); }
public void selectionChanged(IAction action, ISelection selection) { if (selection instanceof IStructuredSelection) { IStructuredSelection ss= (IStructuredSelection) selection; Object first= ss.getFirstElement();
if (first instanceof IProject) { fProject= (IProject) first; } else if (first instanceof IJavaProject) { fProject= ((IJavaProject) first).getProject(); } } }}
all generated code;no editing needed
Use the “New NatureEnabler” Wizard
track selection so action isready to run when invoked
action class - shows upin context menu
101 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0101
Exercise 3c: Providing an Incremental BuilderUsing a Console for Diagnosticspublic class MyBuilder extends BuilderBase { private static final String BUILDER_CONSOLE = ”My Builder";
protected MessageConsoleStream getConsoleStream() { findConsole(BUILDER_CONSOLE).newMessageStream(); }
protected void compile(IFile file, IProgressMonitor m) { MessageConsoleStream msgStream = getConsoleStream(); // … msgStream.println(“…”); }
}
102 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0102
Exercise 3c: Providing an Incremental BuilderThe Poor-Man’s Compilerpublic class LEGBuilder extends BuilderBase { // … protected void compile(final IFile file, IProgressMonitor m) { try { // … LEGCompiler compiler= new LEGCompiler(PROBLEM_MARKER_ID); compiler.compile(file, monitor); // … // runParserForCompiler(file, monitor);
doRefresh(file.getParent()); // N.B.: Assumes generated // files go into parent folder } catch (Exception e) { // catch all Exceptions: any exception could break the // builder infra-structure. getPlugin().logException(e.getMessage(), e); } }
}
103 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0103
Exercise 3c: Enhancing the Poor-Man’s Compiler
Add support to the compiler for the “<=“ and “>=“ operators you addedto the language earlier
Hints: Add endVisit() overrides to the TranslatorVisitor for AST node
types expression8 and expression9 and generate the appropriate Javacode
See how the compiler already handles “== “
104 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0104
Exercise 3c: Enhancing the Poor-Man’s Compiler (soln.)
public class LEGCompiler { // … private final class TranslatorVisitor extends AbstractVisitor { // … public void endVisit(expression8 n) { String right= (String) fTranslationStack.pop(); String left= (String) fTranslationStack.pop(); fTranslationStack.push(left + " <= " + right); } public void endVisit(expression9 n) { String right= (String) fTranslationStack.pop(); String left= (String) fTranslationStack.pop(); fTranslationStack.push(left + " >= " + right); } }
}
105 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0105
A Cautionary Tale: Adapting an Existing Compiler Front-endfor Use in an IDE
An existing compiler doesn’t necessarily make a good IDE back-end: “Synthetic” AST nodes (e.g. Java default constructor) not explicitly marked Syntactic desugaring done too early – difficult/impossible to recover original Source position information is often fake, missing, or inaccurate
E.g., parent extent doesn’t contain that of its children; bogus synthetic nodeextent
Can’t correlate source selections to AST nodes Can’t reproduce source text from AST (e.g., after rewriting)
In short: ASTs of even well-written compilers make transformation hard Analysis results (e.g., type inference, control- and data-flow) usually not exposed
in a reusable manner Error recovery – may be optimized for parsing speed vs. good diagnostics Error messages – may get “error cascade” Engineering trade-offs between speed of batch compilation vs. incremental
compilation
or: “Hey, your front-end is my back-end!”
106 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0106
Exercise 3d: Supporting ‘Mark Occurrences’
107 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0107
Exercise 3d: Supporting ‘Mark Occurrences’
• In IMP, there can be many kinds of “occurrences,” even per language:• definition/reference (“classic” JDT behavior)
• use/def (data flow)
• executes-in-parallel
• whatever definition suits your language
public interface IOccurrenceMarker extends ILanguageService {
public String getKindName();
public List getOccurrencesOf(IParseController ctlr, Object astNode);
}
used to identify this particularkind of occurrence in the UI
produces the list of AST nodes (from the samefile) that represent occurrences of astNode
108 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0108
Exercise 4: Fuzzy Dice
109 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0109
Exercise 4a: Enhancing Hover Help with aDocumentation Provider
110 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0110
Exercise 4a: Enhancing Hover Help with aDocumentation Provider
public class LEGDocumentationProvider implements IDocumentationProvider { public String getDocumentation(Object entity, IParseController ctlr) { if (entity == null) return null;
if (entity instanceof ASTNode) { // Possibilities: source text, associated docs (e.g. JavaDoc).
// Produce HTML for nicer hovers!
if (entity instanceof declaration) { declaration decl = (declaration) entity;
return decl.getprimitiveType().toString() + " " + decl.getidentifier().toString(); } }
return null; }
}
111 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0111
Exercise 4a: Enhancing the Documentation Provider
Enhance the IDocumentationProvider:
If the entity passed in is a functionDeclaration, find theimmediately-preceding comment (aka “adjunct” in LPG) that starts withthe “LEG Doc” prefix (“//**”), if any, and return that as thedocumentation.
Hints Use ASTNode.getPrecedingAdjuncts() to access the relevant
comments
112 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0112
Exercise 4a: Enhancing the Documentation Provider(soln.)public class LEGDocumentationProvider implements IDocumentationProvider{ public String getDocumentation(Object entity, IParseController ctlr){ // … if (entity instanceof functionDeclaration) { functionDeclaration fdecl= (functionDeclaration) entity; IToken[] adjuncts= fdecl.getPrecedingAdjuncts();
for(int i=0; i < adjuncts.length; i++) { String adjStr= adjuncts[i].toString(); if (adjStr.startsWith("//**")) { return adjStr.substring(4); } } } // … }
}
113 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0113
Exercise 4b: Adding a Preferences Page
114 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0114
package org.imp.leg.preferences;
page Leg {fields { boolean UseDefaultIncludePath { defvalue true; } dirlist IncludePathToUse { tooltip "A semicolon-separated folder list to search for include files"; defvalue ".;..;${pluginResource:org.eclipse.imp.leg/include}"; } unless UseDefaultIncludePath string SourceFileExtensions { tooltip "A comma-separated list of file name extensions identifying thesource files to process"; defvalue "leg"; } boolean GenerateLog { tooltip ”Place diagnostics from the build process in a log file"; defvalue true; } int MaxLogEntries { label "Maximum # of log entries"; range 0 .. 1000; defvalue 100; } if GenerateLog
}
Exercise 4b: Anatomy of a Preferences Specification
“Preference Page”UI components
generated directlyfrom this!
page name in UI - can be hierarchical,e.g., “Leg.Compiler”
package where generated code goes
115 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0115
Exercise 4b: Anatomy of a Preferences Specification
page Leg { fields { boolean UseDefaultIncludePath }}
page Leg.Compiler { fields { boolean GenerateLog // …
}}
hierarchical page name
116 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0116
Exercise 4b: Anatomy of a Preferences SpecificationField specifications:
field-type field-key { label “Label on pref page”; tooltip “…”; defvalue …;}
field-types:boolean, color, combo,dirList, double, file,font, int, radio, string
If field-key == CamelCaseLikeThis, and no explicit label,label => “Camel Case Like This”
default if no defvalue:boolean => false,color => black,combo/radio => first listed value,file/dirList/string => “”,int/double => range min (0 if none),font => N/A
field-key => constant in LEGConstants.java, used incalls to IPreferencesService.getXXXPreference(String field-key)
int MaxLogEntries { label "Maximum # of Log Entries"; tooltip “Log is truncated to this size”; defvalue 100;}
117 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0117
Exercise 4b: Anatomy of a Preferences Specification
radio/combo fields:combo field-key { values { valueKey1 “valueLabel1”, … }; defvalue valueKeyi;}
conditional fields:field-type field-key { // …} if boolean-field-key
constrained int/double fields:int TabSize { range 1 .. 20; }
substitution variables in defvalue(also allowed in user-specified values): ${os}, ${arch}, ${ws}, ${nl} ${workspaceLoc} ${projectLoc:projectName} ${pluginLoc:bundleID} ${pluginVersion:bundleID} ${pluginResource:bundleID/path}
118 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0118
Outline
Introduction IDE Development Process Overview The Target Language Interactive Exercises: Developing an IDEAdvanced Topics IMP Architectural Overview Where to go from here
119 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0119
Advanced Topics 1: Source Formatting
Generic rule-based rewrite engine Rules expressed using combination of
Pattern-matching (over concrete syntax) "Box" language layout operators (H - horizontal, V - vertical, A - array, I -
indent, etc.) with parameters (hs - horizontal sep, is = indentation) [CWI] Example: V [ H hs=0 [ “if” “(“ “<Expression>” “)” “{“ ]
I is=$blockIndent [ “<StatementList>” ] “}”]
Pattern example (extracted from above rule): if (<Expression>) { <StatementList> }
Meta-variables act as placeholders for syntactic constructs (e.g. “<Expression>”,“<StatementList>”)
User-settable formatting parameters (e.g. “$blockIndent”) automatically surfaced inIDE preferences (at least some day :-) )
120 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0120
Advanced Topics 1: Source Formatting (cont.)
1. Create service extension using “New Formatting Spec” wizard2. Create an extended parser/scanner for your language for the pattern
matcher Add tokens for “meta-variables” (e.g. <Expression> or
<StatementList> in prev. example)
Meta-variables serve as placeholders for syntactic constructs thatappear in the rules
Specify multiple “start symbols” Formatting rules are concrete syntax, but aren’t complete programs
in the target language Each rule is parsed independently (of course)
3. Create/edit formatting rules using IMP-supplied rule editor
4. Safe (no code loss): set of rules need not cover the entire language
121 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0121
text { font normal = "courier"; IDENTIFIER = { color = "BLUE"; font = normal; style = { bold }; } NUMBER = { color = "RED"; style = { regular }; } DoubleLiteral = { color = "YELLOW"; style = { regular }; } SINGLE_LINE_COMMENT = { color = "GREEN"; font = normal; style = { italic }; } } outline { node functionDeclaration; node block; node declarationStmt0; node declarationStmt1; } foldable { node functionDeclaration; }
}
package org.eclipse.imp.foo;import foo.imp.parser.Ast.*;
language foo { presentation { icon funcIcon = "func.gif"; icon blockIcon = "block.gif"; icon declIcon = "decl.gif";
functionDeclaration f = {
label = {:f.getfunctionHeader().getidentifier(). toString() :}; icon = funcIcon; } block b = { label = "Block"; icon = blockIcon; } declarationStmt0 d = { label = {:d.getdeclaration().getprimitiveType() + " " +d.getdeclaration().getidentifier().toString():}; icon = declIcon; } }
Advanced Topics 2: Presentation Specifications
AST node types
token kinds
{: Java code :}
icon file paths
New!
122 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0122
Advanced Topic 3: Error Handling Errors are the norm! ⇒ must not cripple the IDE! All analyses must produce something reasonable wherever possible
LPG: systematic, semi-automatic errorrecovery for parsing & creating “prosthetic”AST nodes (work in progress)
void A() { int x= 5; foo blah; for(int i=0; i < a.length; i++) { int y= a[i] * a[j]; x += y; }}
dangling ref
mangled statementA()
body
int x= 5; BadStmt for
int i=0;
body
i < a.length i++
header
declStmt
… a[j] …
123 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0123
Advanced Topic 4: Adding a custom editor name & icon
Add an extension like the following to your plugin.xml:
<extension point="org.eclipse.ui.editors"> <editor class="org.eclipse.imp.editor.UniversalEditor" extensions="g,gi" icon="icons/grammarfile.gif" id="lpg.editor" name="LPG Grammar Editor"> </editor> </extension>
Coming soon: Will use existing attributes (language name, icon) in thelanguage descriptor to do the above automatically
124 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0124
Advanced Topic 5: Using Other Kinds of Parserspublic interface IParseController extends ILanguageService { Language getLanguage();
void initialize(IPath filePath, ISourceProject project, IMessageHandler handler);
ISourceProject getProject();
IPath getPath();
Object parse(String input, boolean _, IProgressMonitor monitor);
Object getCurrentAst();
Iterator getTokenIterator(IRegion region);
ISourcePositionLocator getNodeLocator();
ILanguageSyntaxProperties getSyntaxProperties();
IAnnotationTypeInfo getAnnotationTypeInfo();}
125 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0125
Advanced Topic 5: Using Other Kinds of Parsers (cont.)class MyParseController extends ParseControllerBase { public Object parse(String input, boolean _, IProgressMonitor m) { // Parse the given input (editor buffer’s contents), // and return the resulting AST. See ParseControllerBase // for the path, the containing project, and where to store // the resulting AST. }
public Iterator getTokenIterator(IRegion region) { // Return an Iterator over tokens, or anything else you // want to syntax color. Your ITokenColorer must understand // what to do with whatever the Iterator produces. }
public ISourcePositionLocator getNodeLocator() { // Return the ISourcePositionLocator for this language. }
public ILanguageSyntaxProperties getSyntaxProperties() { // … (see next slide) }
}
your responsibilities
126 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0126
Advanced Topic 5: Using Other Kinds of Parsers (cont.)
interface ILanguageSyntaxProperties { public String getSingleLineCommentPrefix();
public String getBlockCommentStart();
public String getBlockCommentContinuation();
public String getBlockCommentEnd();
public String[][] getFences();
public String getIdentifierConstituentChars();
public int[] getIdentifierComponents(String ident);}
e.g. { { "[", "]" }, { "(", ")" }, { "{", "}" }, { "/.", "./" ]
Enables various editor commands(e.g. “Go to Matching Fence”)
127 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0127
Advanced Topic 6: Simple Breakpoint Support [Applies only if your compiler generates Java™ source]
Add line breakpoint support by adding SMAP (JSR-44) attributes to generatedJava class files
Arrange for your compiler to insert “//#line” comments in generated Javasource to indicate location of corresponding original source
• Have your nature’s configure() method add the SMAPINature to new projects,and identify Java as a down-stream builder:
public class MyNature extends ProjectNatureBase { // … public void addToProject(IProject project) { super.addToProject(project); new SmapiProjectNature("g").addToProject(project); } // … protected String getDownstreamBuilderID() { return "org.eclipse.jdt.core.javabuilder"; }}
file-name extension ofyour source files
128 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0128
Advanced Topic 6: Simple Breakpoint Support
IMP/SMAPI classfile post-processor: looks for your language’s source files corresponding to Java source files scans for “//#line” comments
builds a line mapping table
inserts the appropriate SMAP attributes into the classfile
129 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0129
Advanced Topic 7: Deployment the Easy Way
N.B.: deployment != ~employment
130 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0130
Outline
Introduction IDE Development Process Overview The Target Language Interactive Exercises: Developing an IDE Advanced Topics IMP Architectural Overview Where to go from here
131 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0131
IMP Components
Runtime framework for core IDE component behavior Views: editor, outline page, markers, annotations, etc. Incremental build support Analysis …
Meta-tooling for language-specific IDEs Language-definition support for syntax, well-behaved ASTs, analyses DSLs to more easily implement language services: preferences,
presentation (beta), source formatting (beta) Quick fix/content assist, refactoring support – transformation & analysis
languages (coming) …
132 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0132
IMP Language Service Dispatching
IMP Source Editor& Eclipse Platform
AnalysisScheduler
Compiler Front-End
Analyzers
Source File“foo.leg”
…{ “*.leg” ⇒ “leg” language }…
Language Registry
ServiceControllers
Fact Basebindings, call graph,
type hierarchy,…
Tokens ASTetc.
Decorated ASTfor “foo.leg”
Bindings Types
Ref Resolver
Doc Provider
Indexer
Syntax Colorer
Outliner
Formatter
IDE Services for “leg”
Refactoring
src Δ’s
on demand
etc.
src Δ’s
language-specificlanguage-independent
133 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0133
IMP Language Service Scheduling
more expensive analysis
Quick Fix, Refactoring
Outline, Pkg ExplorerUpdates
Syntax color updates
Index Creation
Indexed Search
Lexical Analysis
Syntactic Analysis
Specialized Analyses
Model Update Broadcaster
Explicit User Actions
Editing Actions
Background Jobs
Presentation RepairerDamage region
idle time
src text
134 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0134
IMP Support for Language Service & Analysis Implementation
Lexical Analysis
SyntaxColoring Syntactic Analysis (Parsing)
Binding Analysis
ContentAssist
EagerParsing
Flow Analyses
Quick FixRefactoring
Refactoring-SpecificAnalyses
Search
IndexCreation
LexicalSpecification
GrammarSpecification,
Auto-generated AST’sGrammar Annotations:Scoping & Definitions
DeclarativePresentationSpecifications
Index EntrySpecifications
AST PatternMatching
WALA Analyses &IMP AnalysisFramework
Declarative ASTRewrite
SpecificationsError Recovery
Grammar Specs,Declarative Assist
Specifications
more expensive analyses
Outlining/Folding
Type Analysis
LanguageInheritance
135 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0135
IMP Analysis Architecture
Analysis & TransformationSpecifications
Specification Compiler(s)
Analysis-specificData Schemas
Analysis-specificAlgebraic Data Types
ConstraintSolver
StateSolver
SourceFact Extractors
Fact Browser& ViewersRefactorings
RelationalSolver
Fact Base{type,context,value}*
Pluggable AnalysisManager
RelationalEngine
ConstraintEngine
StateEngine
Core Data StructureImplementations
(relations, maps, sets,…)
GenericLayer
Analysis-SpecificLayer
SpecificationLanguage
Layer
AnalysisClients
analyzer registration
Search
136 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0136
IMP Analysis Languages
Base Language(Sets, Relations, Maps,Algebraic Data Types,
Operators, Fact access)
Constraint-basedAlgorithm Language
(constraint rep’n,update ops)
State Machine-basedAlgorithm Language
(FSM rep’n,state ops)
Relation-basedAlgorithm Language
(fixed-point ops)
core data structures,primitives, operators,strong type system
domain-specificrep’ns,
operators,semantics
LPG Family FactExtraction Language
Polyglot Family FactExtraction Language
SDF Family FactExtraction Language
JAstAdd Family FactExtraction Language
language extension
Analyzers
Extractors concrete syntax,pattern matching
137 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0137
IMP Static Analysis Support Using WALA
WALA extensible static analysis framework General framework encompassing many classic analyses
pointer analysis, escape analysis, effects analysis, call graph, type analysis multiple precisions (CHA, RTA, 0-CFA, 1-CFA, etc.)
General iterative solver framework for expressing new analyses
Robust, highly scalable (capable of analyzing millions of LOC)
Handles static and dynamic languages
Currently supports Java, JavaScript, PHP, X10
Open-source (http://wala.sourceforge.net) Adding support to WALA for a new language:
Implement translator from your source AST’s into WALA AST’s
Define new instruction types for WALA IR as needed (~12 for X10)
Implement constraint handlers for new IR instructions to enable existing analyses(e.g. pointer analysis, effects analysis, escape analysis)
138 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0138
WALA IR Generation
IBM JS AST
Polyglot Java AST
Polyglot X10 AST
JS CAst AST
Java CAst -> IRtranslator
IBM JS -> CAsttranslator
Rhino JS AST
Polyglot Java -> CAsttranslator
Generic OO -> CAsttranslator services
DOMO IR
X10 -> CAsttranslator
OO CAst AST
Java CAst AST
X10 CAst AST
Eclipse Java AST
Generic Java -> CAsttranslator
Eclipse Java -> CAsttranslator
Rhino JS -> CAsttranslator
Generic JS -> CAsttranslator
Generic OOCAst -> IRtranslator
X10 CAst -> IRtranslator
JS CAst -> IRtranslator
inheritancedata flow
139 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0139
Related Work
GUIDE (C. Laffra): Inspiration for early IMP prototype LDT Dynamic Languages Toolkit (Eclipse Technology Project)
DLTK focuses on dynamic languages; IMP doesn’t
DLTK uses generic language model for program representation; IMPdoesn’t rely on a single model
DLTK not based on meta-tooling; IMP is
DLTK aims for language interoperability, IMP for IDE and languageextensibility
140 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0140
Outline
Introduction IDE Development Process Overview The Target Language Interactive Exercises: Developing an IDE Advanced Topics IMP Architectural Overview Where to go from here
141 Presentation Title | Presentation Subtitle | © 2007 by «Author»; made available under the EPL v1.0 PIMP Your Eclipse: Building an IDE using IMP | © 2009 by IBM; made available under the EPL v1.0141
Where to Go from Here
IMP http://www.eclipse.org/imp
LPG (formerly JikesPG) http://sourceforge.net/project/lpg
X10 http://x10.sourceforge.net/
WALA http://wala.sourceforge.net/
Polyglot http://www.cs.cornell.edu/projects/polyglot/
© 2007 by «Author»; made available under the EPL v1.0 | Date | Other Information, if necessary© 2009 by IBM; made available under the EPL v1.0 | Mar 23, 2009
The End
Questions?