functional programming ii

Functional Programming II

ExperTalks 2015-09-25Prashant Kalkar, Equal Experts

Christian Hujer, Nelkinda Software Craft

Little more on side effects

Consider this example

val hitMonster = monster.hitByShooting(gunShot)

// reduces the power of gunShot

val hitPlayer = player.hitByShooting(gunShot)

Changes to the gunshot are happening on the slide, hence they are called as Side Effects.

Not referentially transparent.

Functional way

val (hitMonster, usedGunShot) =

monster.hitByShooting(gunShot)

val hitPlayer = player.hitByShooting(usedGunShot)

Side effect is now visible in the return type.

No more side effect, just effect of the function.

New function is referentially transparent.

Recursion

Recursion

recursion/riˈkərZHən/nounSee recursion.

Recursion

recursion/riˈkərZHən/nounSee recursion. See also tail recursion.

Recursion

recursion/riˈkərZHən/nounSee recursion. See also tail recursion.

tail recursion/tāl riˈkərZHən/nounIf you aren’t sick of it already, see tail recursion.

Recursion

Recursion

Recursion

“If you already know what recursion is, just remember the answer.Otherwise, find someone who is standing closer to Douglas Hofstadter than you are;then ask him or her what recursion is.”

‒ Andrew Plotkin

A recursive law:Hofstadter’s Law

“It always takes longer than you expect, even when you take into account Hofstadter’s Law.”

‒ Douglas Hofstadter,Gödel, Escher, Bach:

An Eternal Golden Braid

Recursive Acronyms

GNUGNU's Not Unix

WINEWINE Is Not an Emulator

PHPPHP Hypertext Preprocessor

Recursion in Nature

Recursion in Arts

Recursion in Arts

Recursion Use Cases

● Being Awesome● Traversing Data Structures

○ Lists○ Trees

● Formulas● Algorithms

● Often less error-prone than iteration!

Recursion Example: Factorial

unsigned long factorial(unsigned long n) {

return

n == 0 ? 1

: n * factorial(n - 1);

}Base Case

Generic Case

Reduction

Recursion Example: Echo

Case 1: Empty Listecho([]) :- /* Base Case */

nl.

Case 2: Single Wordecho([N]) :- /* Special 1-Element Case */

write(N), echo([]).

Case 3: Multiple Wordsecho([H|T]) :- /* Generic Case */

write(H), write(' '), echo([T]).

Recursion Example: Echo

Case 1: Empty Listecho([]) :- /* Base Case */

nl.

Case 2: Single Wordecho([N]) :- /* Special 1-Element Case */

write(N), echo([]).

Case 3: Multiple Wordsecho([H|T]) :- /* Generic Case */

write(H), write(' '), echo([T]).

Recursion Example: Lsimport java.io.*;

public class Ls { public static void main(final String... args) { if (args.length == 0) ls(new File(".")); for (final String pathname : args)

ls (new File(pathname)); } public static void ls(final File path) { System.out.println(path); if (path.isDirectory()) for (final File child : path.listFiles()) ls(child); }}

Recursive Max Function in Haskell

maximum' :: (Ord a) => [a] -> a

maximum' [] = error "maximum of empty list"

maximum' [x] = x

maximum' (x:xs)

| x > maxTail = x

| otherwise = maxTail

where maxTail = maximum' xs

Recursive Max Function in Haskell

maximum' :: (Ord a) => [a] -> a

maximum' [] = error "maximum of empty list"

maximum' [x] = x

maximum' (x:xs) = max x (maximum' xs)

Quicksort in Haskell

qsort :: (Ord a) => [a] -> [a]

qsort [] = []

qsort (x:xs) =

qsort [a | a <- xs, a <= x]

++ [x] ++

qsort [a | a <- xs, a > x]

Disclaimer: This is not a true Quicksort (it’s not in-place), and not the fastest possible.

But behold its beauty!

How to DevelopLinear Recursive Algorithms

1. Define Base caseaka Case 0 or Case 1

2. Define special cases, if any3. Define generic case

aka Case n → n-1 or Case n+1 → n

Recursion in C

unsigned long factorial(unsigned long n) {

return

n == 0 ? 1

: n * factorial(n - 1);

}

Disassembly (gcc -O1)factorial:.LFB0:

.cfi_startprocmovl $1, %eaxtestq %rdi, %rdije .L6pushq %rbx.cfi_def_cfa_offset 16.cfi_offset 3, -16movq %rdi, %rbxleaq -1(%rdi), %rdicall factorialimulq %rbx, %raxpopq %rbx.cfi_restore 3.cfi_def_cfa_offset 8

.L6:rep ret.cfi_endproc

Disassembly (clang -O1)factorial: # @factorial

.cfi_startproc# BB#0:

movl $1, %eaxtestq %rdi, %rdije .LBB0_2.align 16, 0x90

.LBB0_1: # %tailrecurse # =>This Inner Loop Header: Depth=1

imulq %rdi, %raxdecq %rdijne .LBB0_1

.LBB0_2: # %tailrecurse._crit_edgeretq.cfi_endproc

Disassembly (gcc -O2)factorial:.LFB0:

.cfi_startproctestq %rdi, %rdimovl $1, %eaxje .L4.p2align 4,,10.p2align 3

.L3:imulq %rdi, %raxsubq $1, %rdijne .L3rep ret

.L4:rep ret.cfi_endproc

Conclusion

There are cases in which modern compilers● convert recursion into tail recursion● compile a tail recursion into a loop

Note: javac is stupid!Oracle, are you listening?!

Fibonacci

unsigned long fibo(unsigned long n){ return

n < 2 ? 1 : fibo(n - 1) + fibo(n - 2);}

⇒ Too tough for most compilers!

ConvertingRecursion → Tail Recursion

typedef unsigned long u64;

u64 fiboT(u64 a, u64 b, u64 n){ return n < 2 ? a : fiboT(b, a + b, n - 1);}

u64 fibo(u64 n){ return fiboT(1, 2, n);}

Why are Functional Languagesbetter at Recursion?

● Pure functions● Memoization● (Why do impure functions not allow

memoization?)● Lazy Evaluation

Algebraic Data Types

Algebraic data type

Composite Type

Defined by one more more data constructors.

data List a = Nil | Cons x (List xs)

● ADT algebra○ Sum of data constructors - Disjoint unions○ Each constructor is Product of its arguments

ADT Example

List definition in Scala

sealed trait List[+A]

case object Nil extends List[Nothing]

case class Cons[+A](head: A, tail: List[A])

extends List[A]

In OO terms these constructors can be mapped to subtypes.

ADT Example

ADTs have finite number of constructors or subtypes.

For list we have two constructors or subtypesEmpty list Non empty list with head and tail.

These constructions are exhaustive and no new subtype will ever be added.

More ADT Examples

OptionSome | None

EitherLeft | Right

TrySuccess | Failure

FutureSuccess | Failure

Switch cases good or bad

OO programmers consider Switch cases bad. Conditional code. Hard to maintainHunt down existing cases to add new cases. Program will misbehave if you miss to update one.

FP enhanced switch cases to pattern matching and promote them.

What’s the Truth??

Types and Operations

● Type against OperationsTy

pe

Operations

Infinite Type, Finite Operations

Finite Type, Infinite Operations

Infinite Type, Finite OperationsImplemented as Polymorphism (class hierarchy).

Easy to add new Type by Subtyping.

Difficult to add new Operations.

Finite Type, Infinite OperationsRepresented by Algebraic data types (ADTs)

Easy to add new operations using Pattern matching.

Difficult to add new Types.

Pattern Matching

Check for given pattern in the input.

Constant Pattern

def describe(x: Any) = x match {

case 5 => "five"

case "hello" => "hi!"

case () => "Unit" // => everything is a value

}

Switch cases.. Enhanced

Pattern Types

● Constructor Pattern

sealed class Expr

case class Number(num: Double) extends Expr

case class UnOp(operator: String, arg: Expr)

extends Expr

expr match {

case UnOp("-", UnOp("-", Number(a))) =>

println("Double Negation for" + a)

case _ => // return type unit ().

}

Sequence Pattern

expr match {

case List(0, _, _) =>

println("3 element, starting with 0")

case List(0, _*) => println("starting with 0")

case List(_, _, *) => println("at least two")

case e => println("something else" + e)

// variable pattern

}

Variable pattern is catch all pattern.

Type Pattern

def generalSize(x: Any) = x match {

case s: String => s.length

// typecast to string

case m: Map[_, _] => m.size

case _ => -1

}

Type pattern

def isIntIntMap(x: Any) = x match {

case m: Map[Int, Int] => true

case _ => false

}

<console>:11: warning: non-variable type argument

Int in type pattern Map[Int,Int] is unchecked since

it is eliminated by erasure

case m: Map[Int, Int] => true

^

Type erasers

Generic types in java and scala are only available at compile time.

Compiler use this information for type checking and then erase it.

Java and Scala arrays are exception to this rule.

def isStringArray(x: Any) = x match {

case a: Array[String] => "yes" // this works

case _ => "no"

}

Pattern matching with ADTs

def printIfExists(a: Option[Int]) = {

a match {

case Some(x) =>

println("value provided " + x)

}

}

<console>:10: warning: match may not be exhaustive.It would fail on the following input: None

This check is possible because of ADTs

Pattern match as Higher order function

def doubleTheOdds(numbers: List[Int]) = {

numbers.map {

case x if x%2 != 0 => x * 2

case y => y

}

}

The match is happening on the input of the map function.

If condition in pattern matching is called as Guard Condition.

Partial functions with Pattern Matching

def printIfExists:

PartialFunction[Option[Int], Unit] = {

case Some(x) =>

println("value provided " + x)

}

This will compile without any warnings.

Also no match required (similar to HOF pattern matching)

Pattern matching in variable declaration

Tuple values

val fullName = ("Jack", "Sparrow")

val (firstName, lastName) = fullName

Case class objects

val exp = BinOp("*", Number(5), Number(1))

val BinOp(op, left, right) = exp

Pattern matching as value

val number = 10

val numberDescription = number match {

case 5 => "five"

case 10 => "Ten"

case _ => "something else"

}

Everything is value in function programming.

Also said as everything is an expression (not statement).

Pattern Matching final Example

val numbers = List(1, 2, 3, 4, 5, 6)numbers.collect { case x if x%2 == 0 => x * 2 }output => List(4, 8, 12)

This is equivalent to

val partial: PartialFunction[Int, Int] = { case x if x % 2 == 0 => x * 2

}

numbers.filter(partial.isDefinedAt).map(partial)

I’ve seen the future.

● It’s logical.

● It has pattern matching……on the right side, which is left!

Pattern matching beyond FP?

main(Argv) :- echo(Argv).

echo([]) :-

nl.

echo([Last]) :-

write(Last), echo([]).

echo([H|T]) :-

write(H), write(' '), echo(T).

Echo in Prolog

Expression Problem

Problem

What can be done when neither Types nor Operations are limited.

Is there a way to Add a new type easilyAdd a new operation easily

This problem is well known as Expression Problem.

Philip Wadler 1998

American Computer Scientist

He coined the term Expression Problem

Expression Problem

Definition

Expression Problem:The goal is to define a datatype by cases, where one can add new cases to the datatype and new functions over the datatype, without recompiling existing code, and while retaining static type safety (e.g., no casts)

Used in discussing strengths and weaknesses (expressive power) of various programming paradigms and programming languages.

The problem manifest itself while working with language expressions.

interface Exp { void print(); }

class Lit implements Exp {

protected int value;

Lit(int value) { this.value = value; }

public void print() { System.out.print(value); }

}

class Add implements Exp {

protected Exp left, right;

Add(Exp l, Exp r) { left = l; right = r; }

public void print() {

return left.print(); Syso.print(“+”); right.print();

}

}

interface EvalExp extends Exp {

int eval();

}

class EvalAdd extends Add implements EvalExp {

public int eval() {

return left.eval() + right.eval();

}

}

// Ouch! This will not compile.

Generics to the Rescue

We can solve the compilation problem by somehow limiting the left and right to EvalExp

We can do that by introducing a type parameter for EvalAdd class.

class EvalAdd<A extends EvalExp> extends Add implements EvalExp

But this does not change the type of the left or right.

We will have to parameterize Add too.

Simple Solution with Generics

class Add<A extends Exp> implements Exp { protected A left, right;

Add(A l, A r) { left = l; right = r; } public void print() { … }

}

class EvalAdd<A extends EvalExp> extends Add<A> implements EvalExp {

public int eval() {

return left.eval() + right.eval();

// Now this will work.

}

}

Analysing the solution

The solution is possible because of Generics which came from FP languages into Java.

But we do not like this solution.

The base interface Exp does not enforce type parameter on derived classes.

This version was simplified version of actual proposed solution.

Type classes

Expression problem can be solved by using type classes

Haskell was the first language to introduce type classes.

They are supported by Scala with the help of implicits.

Example

sealed trait Exp

case class Lit(value: Int) extends Exp

case class Add(left: Exp, right: Exp) extends Exp

object ExpEvaluator {

def eval(exp : Exp): Int = exp match {

case Lit(value) => value

case Add(left, right) =>

eval(left) + eval(right)

}

}

We would like to convert Expression to Json.

object JsonWriter {

def write(value: JsonValue): String = // ...

def write(value: Exp) = // ...

}

But, we should be able to use JsonWriter for any object not just Expressions.

How about this

trait JsonConvertible {

def toJson: JsValue

}

sealed trait Exp extends JsonConvertible

object JsonWriter {

def write(value: JsonValue): String = // ...

def write(value: JsonConvertible) =

write(value.toJson)

}

This works.

JsonWriter.write(Add(Lit(2), Lit(3)))

But wait! .. Why the Expression class needs to know anything about Json.

We would like to remove the Expression dependency on JsonCovertible.

This implementation is also called as Subtype Polymorphism

We can solve the coupling problem by introducing a Type class as follows

// type class

trait JsonConverter[A] {

def convertToJson(value: A): JsonValue

}

object JsonWriter {

def write(value: JsonValue): String = // ...

def write[A](value: A, conv: JsonConverter[A]) =

write(conv.convertToJson(value))

}

We need to implement a convertor.

def expJsonConverter = new JsonConverter[Exp] {

def convertToJson(exp: Exp): JsonValue = { ... }

}

JsonWriter.write(Add(Lit(2), Lit(3)),

expJsonConverter)

This removed the coupling between Exp and Json.

But we have to pass in a parameter, which can quickly become complex for Composite type.

Scala implicits can help here.

implicit def expJsonConverter = // ...

def write[A](value: A)(implicit conv: JsonConverter[A]) =

write(conv.convertToJson(value))

JsonWriter.write(Add(Lit(2), Lit(3)))

Implicits parameters are passed in by the Compiler.

The call is now exactly like when JsonCovertible was used.

More about Type classes

Type classes provide something called as Ad-hoc polymorphism.

Type classes allow you to add new functionality to the existing Types, without changing them.Note, they are different than extension methods.

Like extension method, they do not attach themself to the type they are working on.

More about Type classes

Type classes provide polymorphism on their type parameters.

Interfaces in OO, provide polymorphism over the calling object.

Type classes can accept more than one type parameters and hence can provide polymorphism on two different type parameters. This is not possible for interfaces.

More about Type classes

Compiler picks-up the right type classes on the type parameter.

So unlike interfaces, implementation is chosen at compile time

Type classes depend on their type parameters and not with the object so they are more like Generic Static Method.

Solving Expression Problem with Type classes

trait Exp

case class Lit(value: Int) extends Exp

case class Add[A <: Exp, B <: Exp](left: A, right:

B) extends Exp

[A <: Exp] is same as <A extends Exp> in Java

Define type class and implicit implementations

// type classtrait Eval[E] {

def eval(e: E): Int

}

implicit def litEval = new Eval[Lit] {

def eval(l: Lit) = l.value

}

Implementation for Add is little complex.

implicit def addEval[A <: Exp, B <: Exp]

(implicit e1: Eval[A], e2: Eval[B]) = {

new Eval[Add[A, B]] {

def eval(a: Add[A, B]) =

e1.eval(a.left) + e2.eval(a.right)

}

}

The method takes two implicit arguments to evaluate left and right exp.

In, Add[A, B], A and B are type of Exp which can be Add or Lit

Evaluating the Expressions

def expressionEvaluator[A <: Exp]

(exp: A)(implicit e : Eval[A]) = {

e.eval(exp)

}

Lets use it to evaluate something// (1 + 2) + 3val exp1 = Add(Add(Lit(1), Lit(2)), Lit(3))

println("" + expressionEvaluator(exp1))

// output => 6

Let add a new type

case class Mult[A <: Exp, B <: Exp]

(left: A, right: B) extends Exp

Define an Implicit evaluator for Mult (same as Add)

implicit def mulEval[A <: Exp, B <: Exp]

(implicit e1: Eval[A], e2: Eval[B]) = {

new Eval[Mult[A, B]] {

def eval(m : Mult[A, B]) =

e1.eval(m.left) * e2.eval(m.right)

}

}

This will allow us to evaluate Expression with Mult

// (3 + 4) * 7val exp2 = Mult(Add(Lit(3), Lit(4)), Lit(7))

println("" + expressionEvaluator(exp2))

// output => 49

Lets try to add new operation like Print.

Type class for Print operation

// type classtrait Print[P] {

def print(p: P): Unit

}

Implicit implementation for Lit type

implicit def litPrint = new Print[Lit] {

def print(l: Lit) = Console.print(l.value)

}

Similarly implicit implementation for Add and Mult

implicit def addPrint[A <: Exp, B <: Exp]

(implicit p1: Print[A], p2 : Print[B]) = {

new Print[Add[A, B]] {

def print(a : Add[A, B]) = {

p1.print(a.left);

Console.print(" + ");

p2.print(a.right);

}

}

Implementation for Mult is exactly same.

Define a method to print the expression

def printExpressions[A <: Exp]

(exp : A)(implicit p : Print[A]) = {

p.print(exp)

}

Use it to print the expressions.// (3 + 4) * 7val exp2 = Mult(Add(Lit(3), Lit(4)), Lit(7))

printExpressions(exp2)

// output => 3 + 4 * 7

TestingFunctional Programming

● In General: Easier than OO (without FP)○ Pure Functions are almost trivial to test.○ No Side-effects ⇒ No additional states○ Higher Order Functions can be mocked easily.

● But what about TDD? How do you approach TDD for Functional Programming?

Testing Functional Programming

Transformation Priority Premise

● What is a Transformation?

● Let’s talk about Refactoring.

“Uncle Bob”, My Hero Robert C. Martin →

Refactoring

“Refactoring is changing the structure of code without significantly changing its behavior.”

‒ Robert C. Martin

Transformation

“Transformation is changing the behavior of code without significantly changing its structure.”

‒ Robert C. Martin

Transformation Priority Premise({} → nil) no code at all -> code that employs nil(nil->constant)

(constant → constant+) a simple constant to a more complex constant(constant → scalar) replacing a constant with a variable or an argument(statement → statements) adding more unconditional statements.(unconditional → if) splitting the execution path(scalar → array)

(array → container)

(statement → recursion)

(if → while)

(expression → function)replacing an expression with a function or algorithm

(variable → assignment) replacing the value of a variable

Beyond FP

● FP and OO are NOT mutually exclusive● FOOP - Functional Object Oriented

Programming● Logic Programming

Thank you!Questions?

Backup Slides

TODO Testing

● Testing Pure First-Order Functions● Testing Pure Higher-Order Functions

○ Mocking in FP● Transformation Priority Premise for FP

○ Example how to develop from Base Case to N-Case

List Implementation - optional

package mylist

sealed trait List[A] {

def ::(e: A): List[A] = mylist.::(e, this)

def :::(l: List[A]): List[A] = ???

}

case object Nil extends List[Nothing]

case class ::[A](h: A, tail: List[A]) extends List[A]

Topic List[P] Side Effects Continued[C] Recursion[P] Algebraic Datatypes[P] Pattern Matching[P] Expression Problem[C] Streams in Java (creation, consumption, filtering, map, reduce etc.)[P] Streams in Scala (creation, consumption, filtering, map, reduce etc.)[C] Testing in Functional Programming (TDD for FP, BDD, TPP for FP)MonadsMonoidsFunctorsApplicativesF-Bounded Types

Recursion TODO● Replacing Assignments with Calls● Replacing Loops with Recursion● Developing Recursive Algorithms

○ Base Case / Most Degenerate Case○ N-Case○ List-Variants: Empty, 1 Element, Multiple○ Head-Tail-Recursion vs Partition Recursion

● Tail-Recursion○ Why○ How to convert normal recursion into tail recursion (example: factorial) by introducing a

result parameter○ Examples in C with Disassemblies by GCC, clang, armcc, c251○ Example in Prolog○ Example in Scala○ Example in Clojure○ Limitations in Java