open xke - poc d'une architecture distribuée de calculs financiers par xavier bucchiotty
TRANSCRIPT
![Page 1: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/1.jpg)
Système distribué de calculs financiers
Par Xavier Bucchiotty
![Page 2: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/2.jpg)
ME
@xbucchiotty
https://github.com/xbucchiotty
http://blog.xebia.fr/author/xbucchiotty
![Page 3: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/3.jpg)
Build a testable,
composable and scalable
cash-flow system
![Page 4: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/4.jpg)
Stream API Iteratees Akka actor Akka cluster
Step 4Step 1 Step 2 Step 3
![Page 5: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/5.jpg)
Use caseFinancial debt management
![Page 6: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/6.jpg)
CAUTION
![Page 7: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/7.jpg)
![Page 8: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/8.jpg)
initial = 1000 €duration = 5 yearsfixed interets rate = 5%
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
1000 €
![Page 9: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/9.jpg)
initial = 1000 €duration = 5 yearsfixed interets rate = 5%
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
1000 €
date = last date + (1 year)
![Page 10: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/10.jpg)
initial = 1000 €duration = 5 yearsfixed interets rate = 5%
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
1000 €
amort = initial / duration
![Page 11: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/11.jpg)
initial = 1000 €duration = 5 yearsfixed interets rate = 5%
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
1000 €
outstanding = last oustanding - amort
![Page 12: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/12.jpg)
initial = 1000 €duration = 5 yearsfixed interets rate = 5%
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
1000 €
interests = last outstanding * rate
![Page 13: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/13.jpg)
val f = (last: Row) => new Row {
def date = last.date + (1 year)
def amortization = last amortization
def outstanding = last.outstanding - amortization
def interests = last.outstanding * fixedRate
}
![Page 14: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/14.jpg)
Step 1Stream API
![Page 15: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/15.jpg)
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
![Page 16: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/16.jpg)
Date Amort Interests Outstanding
2013-01-01 200 € 50 € 800 €
2014-01-01 200 € 40 € 600 €
2015-01-01 200 € 30 € 400 €
2016-01-01 200 € 20 € 200 €
2017-01-01 200 € 10 € 0 €
first
f(first)
f(f(first))
![Page 17: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/17.jpg)
case class Loan( ... ) {
def first: Row
def f:(Row => Row)
def rows = Stream.iterate(first)(f) .take(duration)
}
![Page 18: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/18.jpg)
case class Portfolio(loans: Seq[Loan]) {
def rows =
loans.stream.flatMap(_.rows)
}
![Page 19: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/19.jpg)
3450 €Total
Date Amort Interests Total paid
2013-01-01 200 € 50 € 250 €2014-01-01 200 € 40 € 240 €2015-01-01 200 € 30 € 230 €2016-01-01 200 € 20 € 220 €2017-01-01 200 € 10 € 210 €2013-01-01 200 € 50 € 250 €2014-01-01 200 € 40 € 240 €2015-01-01 200 € 30 € 230 €2016-01-01 200 € 20 € 220 €2017-01-01 200 € 10 € 210 €2013-01-01 200 € 50 € 250 €2014-01-01 200 € 40 € 240 €2015-01-01 200 € 30 € 230 €2016-01-01 200 € 20 € 220 €2017-01-01 200 € 10 € 210 €
Loan 1
Loan 2
Loan 3
![Page 20: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/20.jpg)
// Produce rowsval totalPaid = portfolio.rows
// Transform rows to amount.map(row => row.interests + row.amortization)
//Consume amount.foldLeft(0 EUR)(_ + _)
![Page 21: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/21.jpg)
// Produce rowsval totalPaid = portfolio.rows
// Transform rows to amount.map(row => row.interests + row.amortization)
//Consume amount.foldLeft(0 EUR)(_ + _)
type RowProducer = Iterable[Row]
type RowTransformer[T] = (Row=>T)
type AmountConsumer[T] = (Iterable[Amount]=>T)
![Page 22: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/22.jpg)
//LoanStream.iterate(first)(f) take duration
//Porfolioloans => loans flatMap (loan => loan.rows)
RowProducer(Iterable[Row])
+ on demand computation- sequential computation
![Page 23: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/23.jpg)
object RowTransformer {
val totalPaid = (row: Row) =>
row.interests + row.amortization
}
+ function composition- type limited to «map»
RowTransformer(Row => T)
![Page 24: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/24.jpg)
object AmountConsumer {
def sum = (rows: Iterable[Amount]) => rows.foldLeft(Amount(0, EUR))(_ + _)
}
AmountConsumer(Iterable[Amount] => T)
+ function composition- synchronism
![Page 25: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/25.jpg)
Stream API
Step 1
5000 loans50 rows
~ 560 ms
![Page 26: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/26.jpg)
On demand computation
Function composition
Sequential computation
Synchronism
Transformation limited to «map»
Pros Cons
![Page 27: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/27.jpg)
Step 2Iteratees
![Page 28: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/28.jpg)
Integrating Play iterateeslibraryDependencies ++= Seq( "com.typesafe.play" %% "play-iteratees" % "2.2.0-RC2")
![Page 29: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/29.jpg)
Enumerator
Iteratee
Producer
Input Status
Consumer
![Page 30: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/30.jpg)
Enumerator
Iteratee
Input StatusIteratees are immutable
Asynchronous by design
Type safe
![Page 31: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/31.jpg)
Enumerator
enumerate and interleave
![Page 32: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/32.jpg)
case class Loan(initial: Amount, duration: Int, rowIt: RowIt) {
def rows(implicit ctx: ExecutionContext) =
Stream.iterate(first)(f).take(duration)
}
Data producer
Enumerator.enumerate(
)
![Page 33: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/33.jpg)
case class Portfolio(loans: Seq[Loansan]) {
def rows(implicit ctx: ExecutionContext) =
}
producers can be combined
Enumerator.interleave(loans.map(_.rows))
![Page 34: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/34.jpg)
2013-01-01 200 € 50 € 250 €2014-01-01 200 € 40 € 240 €2015-01-01 200 € 30 € 230 €2016-01-01 200 € 20 € 220 €2017-01-01 200 € 10 € 210 €
2013-01-01 200 € 50 € 250 €2014-01-01 200 € 40 € 240 €2015-01-01 200 € 30 € 230 €2016-01-01 200 € 20 € 220 €2017-01-01 200 € 10 € 210 €
Date Amort Interests Total paid
2013-01-01 200 € 50 € 250 €2014-01-01 200 € 40 € 240 €2015-01-01 200 € 30 € 230 €2016-01-01 200 € 20 € 220 €2017-01-01 200 € 10 € 210 €
3450 €Total
![Page 35: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/35.jpg)
Iteratee
Consumer as a state machine
![Page 36: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/36.jpg)
Iteratees consume Input
![Page 37: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/37.jpg)
object Input {
case class El[+E](e: E)
case object Empty
case object EOF
}
![Page 38: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/38.jpg)
and propagates a state
![Page 39: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/39.jpg)
object Step {
case class Done[+A, E](a: A, remaining: Input[E])
case class Cont[E, +A](k: Input[E] => Iteratee[E, A])
case class Error[E](msg: String, input: Input[E])
}
![Page 40: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/40.jpg)
Enumerator
Iterateedef step = ...val count = 0
Input
El(...)
Status
Continue
Iterateedef step = ...val count = 1
computes
![Page 41: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/41.jpg)
Iterateedef step = ...val count = 1
Iterateedef step = ...val count = 1
Enumerator
Input
EOF
Status
Done
computes
![Page 42: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/42.jpg)
Iterateedef step = ...val count = 1
Enumerator
Input
El(...)
Status
Error
Iterateedef step = ...val error = "Runtime Error"
computes
![Page 43: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/43.jpg)
val last: RowConsumer[Option[Row]] = {
def step(last: Option[Row]): K[Row,Option[Row]]= {
case Input.Empty => Cont(step(last))
case Input.EOF => Done(last, Input.EOF)
case Input.El(e) => Cont(step(Some(e)))
}
Cont(step(Option.empty[Row]))
}
![Page 44: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/44.jpg)
object AmountConsumer {
val sum: AmountConsumer[Amount] =
}
(rows: Iterable[Amount]) => rows.foldLeft(Amount(0, EUR))(_ + _)
![Page 45: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/45.jpg)
object AmountConsumer {
val sum: AmountConsumer[Amount] =
}
Iteratee.fold[Amount, Amount](Amount(0, EUR))(_ + _)
![Page 46: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/46.jpg)
import RowTransformer.totalPaidimport AmountConsumer.sum
val totalPaidComputation: Future[Amount] = portfolio.rows.run(sum)
![Page 47: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/47.jpg)
import RowTransformer.totalPaidimport AmountConsumer.sum
val totalPaidComputation: Future[Amount] = portfolio.rows |>>> sum
![Page 48: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/48.jpg)
Enumeratee
map and filter
![Page 49: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/49.jpg)
Enumerator
Iteratee
Producer
Input Status
Consumer
![Page 50: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/50.jpg)
Enumerator
Iteratee
Producer
Input[A]
Status
Consumer
EnumerateeTransformation
Input[B]
![Page 51: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/51.jpg)
Data transformation
object RowTransformer {
val totalPaid =
Enumeratee.map[Row](row =>
row.interests + row.amortization
)
}
![Page 52: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/52.jpg)
def until(date: DateMidnight) = Enumeratee.filter[Row](
row => !row.date.isAfter(date)
)
Data filtering
![Page 53: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/53.jpg)
type RowProducer = Iterable[Row]
type RowProducer = Enumerator[Row]
type AmountConsumer[T] = (Iterable[Amount]=>T)
type RowTransformer[T] = (Row=>T)
type RowTransformer[T] = Enumeratee[Row, T]
type AmountConsumer[T] = Iteratee[Amount, T]
![Page 54: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/54.jpg)
Futures are composable
map, flatMap, filteronComplete, onSuccess, onError, recover
![Page 55: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/55.jpg)
// Produce rowsval totalPaidComputation: Future[Amount] = portfolio.rows &> totalPaid |>>> sum
// Blocking the thread to wait for the resultval totalPaid =
Await.result(
totalPaidComputation,
atMost = defaultTimeout)
totalPaid should equal(3480 EUR)
![Page 56: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/56.jpg)
We still have function compositionand prepares the code for asynchronism
![Page 57: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/57.jpg)
RowProducer//LoanEnumerator.enumerate( Stream.iterate(first)(f).take(duration))
//PorfolioEnumerator.interleave(loans.map(_.rows))
+ on demand computation+ parallel computation
![Page 58: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/58.jpg)
RowTransformer
+ Function composition+ map, filter, ...
val totalPaid = Enumeratee.map[Row](row =>
row.interests + row.amortization
)
![Page 59: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/59.jpg)
AmountConsumer
+ Function composition+ Asynchronism
def sum = Iteratee.fold[Amount, Amount]
(Amount(0, EUR))(_ + _)
![Page 60: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/60.jpg)
Stream API
Step 1
5000 loans50 rows
~ 560 ms
Iteratees
Step 2
5000 loans50 rows
~ 3500 ms?
![Page 61: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/61.jpg)
simple test
complex test Thread.sleep((Math.random() * 1000) % 2) toLong)
![Page 62: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/62.jpg)
Stream API
Step 1
5000 loans50 rows
~ 560 ms
with pause~ 144900 ms
Iteratees
Step 2
5000 loans50 rows
~ 3500 ms
with pause~ 157285 ms
?
![Page 63: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/63.jpg)
Cost of using this implementation of iteratees
is greater than gain of interleaving for such small
operations
![Page 64: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/64.jpg)
Bulk interleaving
![Page 65: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/65.jpg)
//Portfolioval split =loans.map(_.stream).grouped(loans.size / 4)
![Page 66: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/66.jpg)
Stream API
Step 1
5000 loans50 rows
~ 560 ms
with pause~ 144900 ms
Iteratees
Step 2
5000 loans50 rows
~ 4571 ms
with pause~ 39042 ms
![Page 67: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/67.jpg)
On demand computation
Function composition
Sequential computation
Synchronism
Transformation limited to «map»
Pros Cons
![Page 68: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/68.jpg)
On demand computation
Function composition
Sequential computation
Synchronism
Pros Cons
![Page 69: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/69.jpg)
On demand computation
Pros Cons
Function composition
Parallel computation
Asynchronism
No error management
No elasticity
No resilience
![Page 70: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/70.jpg)
Step 3Akka actor
![Page 71: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/71.jpg)
Integrating AkkalibraryDependencies ++= Seq( "com.typesafe.akka" %% "akka-actor" % "2.2.0")
![Page 72: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/72.jpg)
Actors are objects
They communicate with each other by messages
asynchronously
![Page 73: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/73.jpg)
class Backend extends Actor {
def receive = {
case Compute(loan) => sender.tell( msg = loan.stream.toList, sender = self)
}}
case class Compute(loan: Loan)
![Page 74: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/74.jpg)
case class Loan
def rows(implicit calculator: ActorRef, ctx: ExecutionContext) = {
val responseFuture = ask(calculator,Compute(this))
val rowsFuture = responseFuture .mapTo[List[Row]]
rowsFuture.map(Enumerator.enumerate(_)) ) }}
![Page 75: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/75.jpg)
val system = ActorSystem.create("ScalaIOSystem")
val calculator = system.actorOf(Props[Backend].withRouter(
RoundRobinRouter(nrOfInstances = 10)),"calculator")
}
![Page 76: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/76.jpg)
Supervisionval simpleStrategy = OneForOneStrategy() { case _: AskTimeoutException => Resume case _: RuntimeException => Escalate}
system.actorOf(Props[Backend]....withSupervisorStrategy(simpleStrategy)), "calculator")
![Page 77: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/77.jpg)
Router
Routee 3
Routee 2
Routee 1
ComputeCompute
![Page 78: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/78.jpg)
Router
Routee 3
Routee 2
Routee 1
AskTimeoutException
Resume
![Page 79: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/79.jpg)
Router
Routee 3
Routee 2
Routee 1
Actor System
![Page 80: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/80.jpg)
RowProducer//Loanask(calculator,Compute(this))
.mapTo[List[Row]]
.map(Enumerator.enumerate(_))
//PorfolioEnumerator.interleave(loans.map(_.rows))
+ parallel computation- on demand computation
![Page 81: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/81.jpg)
RowTransformer
+ Nothing changed
val totalPaid = Enumeratee.map[Row](row =>
row.interests + row.amortization
)
![Page 82: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/82.jpg)
AmountConsumerdef sum = Iteratee.fold[Amount, Amount]
(Amount(0, EUR))(_ + _)
+ Nothing changed
![Page 83: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/83.jpg)
5000 loans50 rows
~ 4571 ms
with pause~ 39042 ms
Stream API
Step 1
5000 loans50 rows
~ 560 ms
with pause~ 144900 ms
Iteratees
Step 2
Akka actor
Step 3
5000 loans50 rows
~ 4271 ms
with pause~ 40882 ms
![Page 84: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/84.jpg)
On demand computation
Function composition
Parallel computation
Asynchronism
Pros Cons
No error management
No elasticity
No resilience
![Page 85: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/85.jpg)
On demand computation
Function composition
Parallel computation
Asynchronism
Pros Cons
No error management
No elasticity
No resilience
![Page 86: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/86.jpg)
No on demand computation
Function composition
Parallel computation
Asynchronism
Error management
Pros Cons
No elasticity
No resilience
![Page 87: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/87.jpg)
Step 4Akka cluster
![Page 88: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/88.jpg)
Integrating Akka ClusterlibraryDependencies ++= Seq( "com.typesafe.akka" %% "akka-cluster" % "2.2.0")
![Page 89: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/89.jpg)
Cluster RouterClusterRouterConfig
Can create actors on different nodes of the cluster Role Local actors or not Control number of actors per node per system
![Page 90: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/90.jpg)
Cluster RouterAdaptiveLoadBalancingRouter
Collect metrics (CPU, HEAP, LOAD) via JMX or Hyperic Sigar and make load balancing
![Page 91: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/91.jpg)
val calculator = system.actorOf(Props[Backend].withRouter(
RoundRobinRouter(nrOfInstances = 10)),"calculator")
}
val calculator = system.actorOf(Props[Backend] .withRouter(ClusterRouterConfig(
local = localRouter, settings = clusterSettings))
, "calculator")}
![Page 92: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/92.jpg)
Router
Routee 3
Routee 1
Actor System
Routee 4
Routee 3
Actor System
Routee 6
Routee 5
Actor System
Elasticity
![Page 93: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/93.jpg)
application.conf
cluster {
seed-nodes = ["akka.tcp://[email protected]:2551","akka.tcp://[email protected]:2552"] auto-down = on
}
![Page 94: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/94.jpg)
Router
Routee 3
Routee 1
Actor System
Routee 4
Routee 3
Actor System
Routee 6
Routee 5
Actor System
Resilience
![Page 95: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/95.jpg)
//Loanask(calculator,Compute(this))
.mapTo[List[Row]]
.map(Enumerator.enumerate(_))
//PorfolioEnumerator.interleave(loans.map(_.rows))
RowProducer
+ Nothing changed
![Page 96: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/96.jpg)
RowTransformer
+ Nothing changed
val totalPaid = Enumeratee.map[Row](row =>
row.interests + row.amortization
)
![Page 97: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/97.jpg)
AmountConsumerdef sum = Iteratee.fold[Amount, Amount]
(Amount(0, EUR))(_ + _)
+ Nothing changed
![Page 98: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/98.jpg)
Function composition
Parallel computation
Asynchronism
Error management
Pros Cons
No on demand computation
No elasticity
No resilience
![Page 99: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/99.jpg)
Function composition
Parallel computation
Asynchronism
Error management
Pros Cons
No on demand computation
No elasticity
No resilience
![Page 100: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/100.jpg)
Function composition
Parallel computation
Asynchronism
Error management
Elasticity
Resilience
Network serialization
Pros Cons
No on demand computation
![Page 101: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/101.jpg)
Stream API
Step 1
5000 loans50 rows
~ 560 ms
with pause~ 144900 ms
Iteratees
Step 2
5000 loans50 rows
~ 4571 ms
with pause~ 39042 ms
Akka actor
Step 3
5000 loans50 rows
~ 4271 ms
with pause~ 40882 ms
Akka cluster
Step 4
5000 loans50 rows
~ 6213 ms
with pause~ 77957 ms
1 node / 2 actors
![Page 102: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/102.jpg)
Stream API
Step 1
5000 loans50 rows
~ 560 ms
with pause~ 144900 ms
Iteratees
Step 2
5000 loans50 rows
~ 4571 ms
with pause~ 39042 ms
Akka actor
Step 3
5000 loans50 rows
~ 4271 ms
with pause~ 40882 ms
Akka cluster
Step 4
5000 loans50 rows
~ 5547 ms
with pause~ 39695 ms
2 nodes / 4 actors
![Page 103: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/103.jpg)
Conclusion
![Page 104: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/104.jpg)
Stream API
Step 1
powerful library
low memory
performance when single
threaded
Iteratees
Step 2
Akka actor
Step 3
error management
control on parallel execution via configuration
Akka cluster
Step 4
elasticity
resilience
monitoring
elegant API
enable asynchronism
and parallelism
![Page 105: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/105.jpg)
It’s all about trade-off
![Page 106: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/106.jpg)
But do you really need distribution?
![Page 107: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/107.jpg)
Hot subject
Recet blog post from «Mandubian» for Scalaz stream machines and iteratees [1]
Recent presentation from «Heather Miller» for spores (distribuables closures) [2]
Recent release of Scala 2.10.3 and performance optimization of Promise
Release candidate of play-iteratee module with performance optimization
Lots of stuff in the roadmap of Akka cluster 2.3.0
![Page 108: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/108.jpg)
Hot subject
[1] : http://mandubian.com/2013/08/21/playztream/
[2] : https://speakerdeck.com/heathermiller/on-pickles-and-spores-improving-support-for-distributed-programming-in-scala
![Page 109: Open XKE - POC d'une architecture distribuée de calculs financiers par Xavier Bucchiotty](https://reader035.vdocuments.us/reader035/viewer/2022070315/554fb6dcb4c9057b298b544e/html5/thumbnails/109.jpg)
YOUFOR watching
THANK
Merci!