Practical Domain-Specific Languages with Groovy
Guillaume Laforge
Groovy Project Manager
SpringSource
jeudi 2 juillet 2009
• Groovy Project Manager
• JSR-241 Spec Lead
• Head of Groovy Developmentat SpringSource
• Initiator of the Grails framework
• Co-author of Groovy in Action
• Speaker: JavaOne, QCon, JavaZone, Sun TechDays, Devoxx, The Spring Experience, JAX, Dynamic Language World, IJTC, GR8Conf, DSL DevCon and more...
Guillaume Laforge
2
jeudi 2 juillet 2009
A few words about Groovy
• Groovy is a dynamic language for the JVM– with a Meta Object Protocol
– compiles directly to bytecode, seamless Java interop
• Open Source ASL 2 project hosted at Codehaus
• Relaxed grammar derived from Java 5
– + borrowed good ideas from Ruby, Python, Smalltalk
• Fast... for a dynlang on the JVM
• Closures, properties, optional typing, BigDecimal by default, nice wrapper APIs, and more...
3
jeudi 2 juillet 2009
• The context and the usual issues we face
• Some real-life examples of Domain-Specific Languages
• Groovy’s DSL capabilities
• Integrating a DSL in your application
• Considerations to remember when designing your own DSL
Agenda
4
jeudi 2 juillet 2009
The contextjeudi 2 juillet 2009
Subject Matter Experts,Business analysts...
jeudi 2 juillet 2009
HAI
CAN HAS STDIO?
I HAS A VAR
IM IN YR LOOP
UP VAR!!1
VISIBLE VAR
IZ VAR BIGGER THAN 10?
KTHXBYE
IM OUTTA YR LOOP
KTHXBYE
Developer producing LOLCODE
jeudi 2 juillet 2009
Lots of languages...
jeudi 2 juillet 2009
And in the end......nobody understands each other
jeudi 2 juillet 2009
10
Expressing requirements...
jeudi 2 juillet 2009
DSL: a potential solution?
•Use a more expressive language than a general purpose one
•Share a common metaphore of understanding between developers and subject matter experts
•Have domain experts help with the design of the business logic of an application
•Avoid cluttering business code with too much boilerplate technical code
•Cleanly separate business logic from application code
•Let business rules have their own lifecycle
11
jeudi 2 juillet 2009
Towards more readability (1)
12
jeudi 2 juillet 2009
Towards more readability (1)
12
jeudi 2 juillet 2009
Towards more readability (1)
20%
12
jeudi 2 juillet 2009
Towards more readability (2)
13
jeudi 2 juillet 2009
Towards more readability (2)
13
jeudi 2 juillet 2009
Towards more readability (2)
80%
13
jeudi 2 juillet 2009
• The context and the usual issues we face
• Some real-life examples of Domain-Specific Languages
• Groovy’s DSL capabilities
• Integrating a DSL in your application
• Considerations to remember when designing your own DSL
Agenda
14
jeudi 2 juillet 2009
• In our everyday life, we’re surrounded by DSLs
– Technical dialects
– Notations
– Business languages
15
A collection of DSLs
jeudi 2 juillet 2009
16
Technical dialects
jeudi 2 juillet 2009
SQL
jeudi 2 juillet 2009
18
^[\w-\.]+@([\w-]){2,4}$
jeudi 2 juillet 2009
19
Notations
jeudi 2 juillet 2009
1. e4 e52. Nf3 Nc63. Bb5 a6
jeudi 2 juillet 2009
L2 U F-1 B L2 F B -1 U L2 jeudi 2 juillet 2009
Visual!jeudi 2 juillet 2009
23
Business languages
jeudi 2 juillet 2009
Real-life Groovy examples
• Anti-malaria drug resistance simulation
• Human Resources employee skills representation
• Insurance policies risk calculation engine
• Loan acceptance rules engine for a financial platform
• Mathematica-like lingua for nuclear safety simulations
• Market data feeds evolution scenarios
• and more...
24
jeudi 2 juillet 2009
• The context and the usual issues we face
• Some real-life examples of Domain-Specific Languages
• Groovy’s DSL capabilities
• Integrating a DSL in your application
• Considerations to remember when designing your own DSL
Agenda
25
jeudi 2 juillet 2009
A flexible & malleable syntax
• No need to write full-blown classes, use scripts
• Optional typing (def)– in scripts, you can even omit the def keyword
• Native syntax constructs
• Parentheses & semi-colons are optional
• Named arguments
• BigDecimal by default for decimal numbers
• Closures for custom control structures
• Operator overloading
26
jeudi 2 juillet 2009
Scripts vs classes
• Hide all the boilerplate technical code– an end-user doesn’t need to know about classes
–public class Rule { public static void main(String[] args) { System.out.println(“Hello”); }}
–println “Hello”
27
jeudi 2 juillet 2009
Optional typing
• No need to bother with types or even generics– unless you want to!
• Imagine an interest rate lookup table method returning some generified type:
–Rate<LoanType, Duration, BigDecimal>[] lookupTable() { ... }def table = lookupTable()
• No need to repeat the horrible generics type info!
28
jeudi 2 juillet 2009
Native syntax constructs
• Lists– [Monday, Tuesday, Wednesday]
• Maps– [CA: ‘California’, TX: ‘Texas’]
• Ranges– def bizDays = Monday..Friday
– def allowedAge = 18..65
– You can create your own custom ranges
29
jeudi 2 juillet 2009
Optional parens & semis
• Make statements and expressions look more like natural languages
– move(left);
– move left
30
jeudi 2 juillet 2009
Named arguments
• In Groovy you can mix named and unnamed arguments for method parameters– named params are actually put in a map parameter
– plus optional parens & semis
• take 1.pill, of: Chloroquinine, after: 6.hours
• Corresponds to a method signature like:
–def take(Map m, MedicineQuantity mq)
31
jeudi 2 juillet 2009
BigDecimal by default
• Main reason why financial institutions often decide to use Groovy for their business rules!– Although these days rounding issues are overrated!
• Java vs Groovy for a simple interpolation equation
• BigDecimal uMinusv = c.subtract(a); BigDecimal vMinusl = b.subtract(c); BigDecimal uMinusl = a.subtract(b); return e.multiply(uMinusv) .add(d.multiply(vMinusl)) .divide(uMinusl, 10, BigDecimal.ROUND_HALF_UP);
• (d * (b - c) + e * (c - a)) / (a - b)
32
jeudi 2 juillet 2009
• When closures are last, they can be put “out” of the parentheses surrounding parameters
• unless (account.balance < 100.euros, { account.debit 100.euros })
• unless (account.balance < 100.euros) { account.debit 100.euros}
• Signature def unless(boolean b, Closure c)
33
Custom control structuresThanks to closures
jeudi 2 juillet 2009
Operator overloading
• Currency amounts–15.euros + 10.dollars
• Distance handling–10.kilometers - 10.meters
• Workflow, concurrency–taskA | taskB & taskC
• Credit an account–account << 10.dollarsaccount += 10.dollarsaccount.credit 10.dollars
a + b a.plus(b)
a - b a.minus(b)
a * b a.multiply(b)
a / b a.divide(b)
a % b a.modulo(b)
a ** b a.power(b)
a | b a.or(b)
a & b a.and(b)
a ^ b a.xor(b)
a[b] a.getAt(b)
a << b a.leftShift(b)
a >> b a.rightShift(b)
+a a.positive()
-a a.negative()
~a a.bitwiseNegate()
34
jeudi 2 juillet 2009
Groovy’s dynamic heart:
The MOP!MetaObject Protocol
jeudi 2 juillet 2009
Groovy’s MOP
• All the accesses to methods, properties, constructors, operators, etc. can be intercepted thanks to the MOP
• While Java’s behavior is hard-wired at compile-time in the class
• Groovy’s runtime behavior is adaptable at runtime through the metaclass.
• Different hooks for changing the runtime behavior– GroovyObject, custom MetaClass implementation, categories,
ExpandoMetaClass
36
jeudi 2 juillet 2009
Adding properties to numbers
• Three possible approaches
– create a Category
• a category is a kind of decorator for default MCs
– create a custom MetaClass
• a full-blown MC class to implement and to set on the POGO instance
– use ExpandoMetaClass
• friendlier DSL approach but with a catch
37
jeudi 2 juillet 2009
Adding properties to numbers with an ExpandoMetaClass
• Number.metaClass.getMeters = {-> new Distance(delegate, Unit.METERS) }
100.meters
38
jeudi 2 juillet 2009
The Builder pattern
jeudi 2 juillet 2009
The Groovy MarkupBuilder
•def mkp = new MarkupBuilder()mkp.html { head { title “Groovy in Action” } body { div(width: ‘100’) { p(class: ‘para) { span “Best book ever!” } } }}
40
jeudi 2 juillet 2009
A builder for HR
• softskills { ideas { capture 2 formulate 3 } ...}knowhow { languages { java 4 groovy 5 } ...}
41
jeudi 2 juillet 2009
A builder for HR
• softskills { ideas { capture 2 formulate 3 } ...}knowhow { languages { java 4 groovy 5 } ...}
41
jeudi 2 juillet 2009
Builders
• Builders are... – a mechanism for creating any tree-structered graph
– the realization of the GoF builder pattern at the syntax level in Groovy
– simply a clever use of chained method invocation, closures, parentheses omission, and use of the GroovyObject methods
• Existing builders– XML, Object graph, Swing, Ant, JMX, and more...
42
jeudi 2 juillet 2009
Compile-time metaprogramming
• Groovy 1.6 introduced AST Transformations
• Compile-time == No runtime performance penalty!
Transformation
43
jeudi 2 juillet 2009
AST Transformations
• Two kinds of transformations
– Global transformations
• applicable to all compilation units
– Local transformations
• applicable to marked program elements
• using specific marker annotations
44
jeudi 2 juillet 2009
Example #1: @Singleton
• Let’s revisit this evil (anti-)pattern! public class Evil { public static final Evil instance = new Evil (); private Evil () {} Evil getInstance() { return instance; } }
• In Groovy! @Singleton class Evil {}
• Also a “lazy” version! @Singleton(lazy = true) class Evil {}
45
jeudi 2 juillet 2009
• You can delegate to fields of your classes
–class Employee { def doTheWork() { “done” }}class Manager { @Delegate Employee slave = new Employee()}def god = new Manager()assert god.doTheWork() == “done”
• Damn manager who will get all the praise...
46
Example #2: @DelegateNot just for managers!
jeudi 2 juillet 2009
Global transformations
• Implement ASTTransformation
• Annotate the transfo specifying a compilation phase
• @GroovyASTTransformation(phase=CompilePhase.CONVERSION)public class MyTransformation implements ASTTransformation { public void visit(ASTNode[] nodes, SourceUnit unit) { ... }}
• For discovery, create the file META-INF/services/org.codehaus.groovy.transform.ASTTransformation
• Add the fully qualified name of the class in that file
47
jeudi 2 juillet 2009
Local transformations
• Same approach as Global transformations
• But you don’t need the META-INF file
• Instead create an annotation to specify on which element the transformation should apply
• @Retention(RetentionPolicy.SOURCE)@Target([ElementType.METHOD])@GroovyASTTransformationClass( ["fqn.MyTransformation"])public @interface WithLogging {...}
48
jeudi 2 juillet 2009
• The context and the usual issues we face
• Some real-life examples of Domain-Specific Languages
• Groovy’s DSL capabilities
• Integrating a DSL in your application
• Considerations to remember when designing your own DSL
Agenda
49
jeudi 2 juillet 2009
• Java 6’s javax.script.* APIs (aka JSR-223)
• Spring’s language namespace
• Groovy’s own mechanisms
• But a key idea is to externalize those DSL programs– DSL programs can have their own lifecycle
– no need to redeploy an application because of a rule change
– business people won’t see the technical code
50
Various integration mechanisms
jeudi 2 juillet 2009
Java 6’s javax.script.* API
• Groovy 1.6 provides its own implementation of the javax.script.* API
• ScriptEngineManager mgr = new ScriptEngineManager();ScriptEngine engine = mgr.getEngineByName(“Groovy”);
String result = (String)engine.eval(“2+3”);
51
jeudi 2 juillet 2009
Spring’s lang namespace
• POGOs (Plain Old Groovy Objects) can be pre-compiled as any POJO and used interchangeably with POJOs in a Spring application
• But Groovy scripts & classes can be loaded at runtime through the <lang:groovy/> namespace and tag
• Reloadable on change
• Customizable through a custom MetaClass
• <lang:groovy id="events" script-source="classpath:dsl/eventsChart.groovy" customizer-ref="eventsMetaClass" />
52
jeudi 2 juillet 2009
Groovy’s own mechanisms
• Eval
– for evaluating simple expressions
• GroovyShell
– for more complex scripts and DSLs
• GroovyClassLoader
– the most powerful mechanism
53
jeudi 2 juillet 2009
Eval
• Simple mechanism to evaluate math-like formulas
• Eval.me ( ‘3*4’)Eval.x (1, ‘3*x + 4’)Eval.xy (1, 2, ‘x + y’)Eval.xyz(1, 2, 3, ‘x * y - z’)
54
jeudi 2 juillet 2009
• A Binding provides a context of execution– can implement lazy evaluation if needed
• A base script class can be specified
• def binding = new Binding()binding.mass = 22.3binding.velocity = 10.6def shell = new GroovyShell(binding)shell.evaluate(“mass * velocity ** 2 / 2”)
55
GroovyShell
jeudi 2 juillet 2009
GroovyClassLoader
• Most powerful mechanism– could also visit or change the AST
– scripts & classes can be loaded from elsewhere
– more control on compilation
• GroovyClassLoader gcl = new GroovyClassLoader();Class clazz = gcl.parseClass( new File(“f.groovy”));GroovyObject instance = (GroovyObject)clazz.newInstance();instance.setMetaClass(customMC);
56
jeudi 2 juillet 2009
Externalize business rules
• Although Groovy DSLs can be embedded in normal Groovy classes, you should externalize them
• Store them elsewhere– in a database, an XML file, etc.
• Benefits– Business rules are not entangled
in technical application code
– Business rules can have their own lifecycle, without requiring application redeployments
57
jeudi 2 juillet 2009
• The context and the usual issues we face
• Some real-life examples of Domain-Specific Languages
• Groovy’s DSL capabilities
• Integrating a DSL in your application
• Considerations to remember when designing your own DSL
Agenda
58
jeudi 2 juillet 2009
Start small, with key concepts
Beware overengineering!
jeudi 2 juillet 2009
Grow your language progressively
jeudi 2 juillet 2009
Get your hands dirty
Play with the end-usersjeudi 2 juillet 2009
Let your DSL fly, it’s not yours, it’s theirs!
jeudi 2 juillet 2009
Tight feedback loop
Iterative process
jeudi 2 juillet 2009
Stay humble.
You can’t get it right the first time.
Don’t design alone at your deskInvolve the end users from the start
jeudi 2 juillet 2009
Playing it safein a sandbox
jeudi 2 juillet 2009
Various levels of sandboxing
• Groovy supports the usual Java Security Managers
• Use metaprogramming tricks to prevent calling / instantiating certain classes
• Create a special GroovyClassLoader AST code visitor to filter only the nodes of the AST you want to keep– ArithmeticShell in Groovy’s samples
66
jeudi 2 juillet 2009
Test, test, test!
• Don’t just test for nominal cases– Explicitly test for errors!
• Ensure end-users get meaningful error messages
67
jeudi 2 juillet 2009
• Summary
• Questions & Answers
Agenda
68
jeudi 2 juillet 2009
Summary
• Groovy’s a great fit for Domain-Specific Languages– Malleable & flexible syntax
– Full object-orientation
• Metaprogramming capabilities– Runtime metaprogramming
– Compile-time metaprogramming
• Groovy’s very often used for mission-critical DSLs
69
jeudi 2 juillet 2009
I kan haz my cheezburgr naw?Or do ya reely haz keshtionz?
?jeudi 2 juillet 2009
Appendix
71
jeudi 2 juillet 2009
• http://www.flickr.com/photos/wheatfields/420088151/sizes/l/
• http://www.flickr.com/photos/therefromhere/518053737/sizes/l/
• http://www.flickr.com/photos/romainguy/230416692/sizes/l/
• http://www.flickr.com/photos/addictive_picasso/2874279971/sizes/l/
• http://www.flickr.com/photos/huangjiahui/3127634297/sizes/l/
• http://www.flickr.com/photos/25831000@N08/3064515804/sizes/o/
• http://www.flickr.com/photos/lanier67/3147696168/sizes/l/
• http://www.flickr.com/photos/ktb/4916063/sizes/o/
• http://www.flickr.com/photos/nathonline/918128338/sizes/l/
• http://www.flickr.com/photos/kevinsteele/39300193/sizes/l/
• http://commons.wikimedia.org/wiki/File:Brueghel-tower-of-babel.jpg
• http://commons.wikimedia.org/wiki/File:Platypus.jpg
• http://www.flickr.com/photos/joaomoura/2317171808/sizes/l/
• http://www.flickr.com/photos/wiccked/132687067/
• http://www.flickr.com/photos/xcbiker/386876546/sizes/l/
72
jeudi 2 juillet 2009
• http://www.flickr.com/photos/pietel/152403711/sizes/o/
• http://www.flickr.com/photos/forezt/192554677/sizes/o/
• http://keremkosaner.files.wordpress.com/2008/04/softwaredevelopment.gif
• http://www.jouy.inra.fr
• http://www.flickr.com/photos/ejpphoto/408101818/sizes/o/
• http://www.flickr.com/photos/solaro/2127576608/sizes/l/
• http://www.flickr.com/photos/biggreymare/2846899405/sizes/l/
• http://www.flickr.com/photos/timsamoff/252370986/sizes/l/
• http://www.flickr.com/photos/29738009@N08/2975466425/sizes/l/
• http://www.flickr.com/photos/howie_berlin/180121635/sizes/o/
• http://www.flickr.com/photos/yogi/1281980605/sizes/l/
• http://www.flickr.com/photos/dorseygraphics/1336468896/sizes/l/
73
jeudi 2 juillet 2009