Apache Flink

Akka – Actors for the win

Till Rohrmann

trohrmann@apache.org

Problems of the old RPC

service• Proprietary server/client architecture to

wrap RPCs

• Shortcomings:

– Blocking calls without timeouts

– Poor exception handling

– Error-prone programming abstraction

– Limited scalability

flink.apache.org 1

Akka – Actor-based

concurrency• Framework to write distributed and highly

reactive applications

• Actor-based concurrency inspired by Erlang’sactor model

• Open source, Apache License 2.0

2flink.apache.org

Actor model

flink.apache.org 3

How can Akka help us?

• Nice programming abstraction

• Asynchronous messages with callbacks

• Location transparency of actors

• Fault-tolerant and self-healing

• High throughput and scalability

• Smaller code base to maintain

flink.apache.org 5

Akka’s integration with Flink

• Job/TaskManager and JobClient are actors

• Old RPC are replaced by messages

• Easy to extend functionality by defining new messages

flink.apache.org 6

JobClientJobManag

er

TaskManag

er

TaskManag

er

Job

Possible improvements

• Make more components an actor to

increase parallelism

– Scheduler/InstanceManager

– ExecutionGraph/Execution

• Make the system more reactive by

asynchronous calls

• Use Akka persistence to recover state of

failed actors

flink.apache.org 7

flink.apache.org

github.com/apache/flink

meetup.com/Apache-Flink-

Meetup