Leveraging Scala Macros for Better

ValidationTomer Gabel, Wix

JavaOne 2014

I Have a Dream

• Definition:

case class Person( firstName: String, lastName: String )

implicit val personValidator = validator[Person] { p ⇒ p.firstName is notEmpty p.lastName is notEmpty }

I Have a Dream

• Usage:

validate(Person("Wernher", "von Braun”)) == Success

validate(Person("", "No First Name”)) == Failure(Set(RuleViolation( value = "", constraint = "must not be empty", description = "firstName" )))

ENTER: ACCORD.

Basic Architecture

API

Combinator Library

DSL

Macro Transformation

The Accord API

• Validation can succeed or fail• A failure comprises one or more

violations

sealed trait Resultcase object Success extends Resultcase class Failure(violations: Set[Violation]) extends Result

• The validator typeclass:

trait Validator[-T] extends (T ⇒ Result)

Why Macros?

• Quick refresher:

implicit val personValidator = validator[Person] { p ⇒ p.firstName is notEmpty p.lastName is notEmpty }

Implicit “and”

Automatic descriptiongeneration

Full Disclosure

Macros are experimental

Macros are hard

I will gloss over a lot of details

… and simplify a lot of things

Abstract Syntax Trees

• An intermediate representation of

code

– Structure (semantics)

–Metadata (e.g. types) – optional!

• Provided by the reflection API

• Alas, mutable

– Until Dotty comes along

Abstract Syntax Trees

def method(param: String) = param.toUpperCase

Abstract Syntax Trees

def method(param: String) = param.toUpperCase

Apply( Select( Ident(newTermName("param")), newTermName("toUpperCase") ), List())

Abstract Syntax Trees

def method(param: String) = param.toUpperCase

ValDef( Modifiers(PARAM), newTermName("param"), Select( Ident(scala.Predef), newTypeName("String") ), EmptyTree // Value)

Abstract Syntax Trees

def method(param: String) = param.toUpperCase

DefDef( Modifiers(), newTermName("method"), List(), // Type parameters List( // Parameter lists List(parameter) ), TypeTree(), // Return type implementation)

Def Macro 101

• Looks and acts like a normal functiondef radix(s: String, base: Int): Longval result = radix("2710", 16)// result == 10000L

• Two fundamental differences:– Invoked at compile time instead of

runtime– Operates on ASTs instead of values

Def Macro 101

• Needs a signature & implementation

def radix(s: String, base: Int): Long = macro radixImpl

def radixImpl (c: Context) (s: c.Expr[String], base: c.Expr[Int]): c.Expr[Long]

Values

ASTs

Def Macro 101

• What’s in a

context?

– Enclosures

(position)

– Error handling

– Logging

– Infrastructure

Basic Architecture

API

Combinator Library

DSL

Macro Transformation

Overview

implicit val personValidator = validator[Person] { p ⇒ p.firstName is notEmpty p.lastName is notEmpty }

• The validator macro:– Rewrites each rule by addition a

description– Aggregates rules with an and combinator

Macro Application

Validation Rules

Signature

def validator[T](v: T ⇒ Unit): Validator[T] = macro ValidationTransform.apply[T]

def apply[T : c.WeakTypeTag] (c: Context) (v: c.Expr[T ⇒ Unit]): c.Expr[Validator[T]]

Brace yourselves

Here be dragons

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Search for Rule

• A rule is an expression of type

Validator[_]

• We search by:

– Recursively pattern matching over an

AST

– On match, apply a function on the

subtree

– Encoded as a partial function from Tree

to R

Search for Rule

def collectFromPattern[R] (tree: Tree) (pattern: PartialFunction[Tree, R]): List[R] = { var found: Vector[R] = Vector.empty new Traverser { override def traverse(subtree: Tree) { if (pattern isDefinedAt subtree) found = found :+ pattern(subtree) else super.traverse(subtree) } }.traverse(tree) found.toList}

Search for Rule

• Putting it together:

case class Rule(ouv: Tree, validation: Tree)

def processRule(subtree: Tree): Rule = ???

def findRules(body: Tree): Seq[Rule] = { val validatorType = typeOf[Validator[_]]

collectFromPattern(body) { case subtree if subtree.tpe <:< validatorType ⇒ processRule(subtree) }}

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Process Rule

• The user writes:p.firstName is notEmpty

• The compiler emits:Contextualizer(p.firstName).is(notEmpty)

Object Under Validation (OUV)

Validation

Type: Validator[_]

Process Rule

Contextualizer(p.firstName).is(notEmpty)

• This is effectively an Apply AST node

• The left-hand side is the OUV

• The right-hand side is the validation

– But we can use the entire expression!

• Contextualizer is our entry point

Process Rule

Contextualizer(p.firstName).is(notEmpty)

Apply

Select

Apply

TypeApply

Contextualizer

String

SelectIdent(“p”)

firstNameis

notEmpty

Process Rule

Contextualizer(p.firstName).is(notEmpty)

Apply

Select

Apply

TypeApply

Contextualizer

String

SelectIdent(“p”)

firstNameis

notEmpty

Process Rule

Apply

TypeApply

Contextualizer

String

Select

Ident(“p”)

firstName

Process Rule

Apply

TypeApply

Contextualizer

Φ

Select

Ident(“p”)

firstName

Process Rule

Apply

TypeApply

Contextualizer

Φ

Select

Ident(“p”)

firstName

case Apply(TypeApply(Select(_, `term`), _), ouv :: Nil) ⇒

Process Rule

Apply

TypeApply

Contextualizer

Φ

OUV

Φ

Φ

Process Rule

• Putting it together:val term = newTermName("Contextualizer")

def processRule(subtree: Tree): Rule = extractFromPattern(subtree) { case Apply(TypeApply(Select(_, `term`), _), ouv :: Nil) ⇒ Rule(ouv, subtree) } getOrElse abort(subtree.pos, "Not a valid rule")

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Generate Description

Contextualizer(p.firstName).is(notEmpty)

• Consider the object under validation• In this example, it is a field accessor• The function prototype is the entry

pointSelect

Ident(“p”)

firstName

validator[Person] { p ⇒ ...}

Generate Description

• How to get at the prototype?• The macro signature includes the rule block:

def apply[T : c.WeakTypeTag] (c: Context) (v: c.Expr[T ⇒ Unit]): c.Expr[Validator[T]]

• To extract the prototype:

val Function(prototype :: Nil, body) = v.tree // prototype: ValDef

Generate Description

• Putting it all together:

def describeRule(rule: ValidationRule) = { val para = prototype.name val Select(Ident(`para`), description) = rule.ouv description.toString}

Walkthrough

Search for rule

Process rule

Generate description

Rewrite rule

Rewrite Rule

• We’re constructing a Validator[Person]

• A rule is itself a Validator[T]. For example:Contextualizer(p.firstName).is(notEmpty)

• We need to:– Lift the rule to validate the enclosing

type– Apply the description to the result

Quasiquotes

• Provide an easy way to construct ASTs:

Apply( Select( Ident(newTermName"x"), newTermName("$plus") ), List( Ident(newTermName("y")) ))

q"x + y"

Quasiquotes

• Quasiquotes also let you splice trees:

def greeting(whom: c.Expr[String]) = q"Hello \"$whom\"!"

• And can be used in pattern matching:

val q"$x + $y" = tree

Rewrite Rule

Contextualizer(p.firstName).is(notEmpty)

new Validator[Person] { def apply(p: Person) = { val validation = Contextualizer(p.firstName).is(notEmpty) validation(p.firstName) withDescription "firstName" }}

Rewrite Rule

• Putting it all together:def rewriteRule(rule: ValidationRule) = { val desc = describeRule(rule) val tree = Literal(Constant(desc)) q""" new com.wix.accord.Validator[${weakTypeOf[T]}] { def apply($prototype) = { val validation = ${rule.validation} validation(${rule.ouv}) withDescription $tree } } """}

The Last Mile

Epilogue

• The finishing touch: and combinator

def apply[T : c.WeakTypeTag] (c: Context) (v: c.Expr[T ⇒ Unit]): c.Expr[Validator[T]] = {

val Function(prototype :: Nil, body) = v.tree // ... all the stuff we just discussed

val rules = findRules(body) map rewriteRule val result = q"new com.wix.accord.combinators.And(..$rules)" c.Expr[Validator[T]](result)}

WE’RE DONE HERE!Thank you for listening

tomer@tomergabel.com

@tomerg

http://il.linkedin.com/in/tomergabel

Check out Accord at:http://github.com/wix/accord