the+haxl+project+at+facebook

7/27/2019 The+Haxl+Project+at+Facebook

1/86

The Haxl Project at

FacebookSimon MarlowJon Coens

Louis BrandyJon Purdy& others


2/86

Business

Logicservice API

Databases

Other

back-end

services


3/86

Use case: fighting spam

Business

Logic

Databases

Other

back-end

services

www (PHP)

Is this thing

spam?

YES/NO


4/86

Use case: fighting spam

Business

Logic

Databases

Other

back-end

services

www (PHP)

Site-integrity engineers

push new rules

hundreds of

times per day


5/86

Data dependencies in a computation database

thrift

memcache


6/86

Code wants to be structured hierarchically

abstraction

modularity

database

thrift

memcache


7/86


abstraction

modularity

database

thrift

memcache


8/86


abstraction

modularity

database

thrift

memcache


9/86


abstraction

modularity

database

thrift

memcache


10/86

Execution wants to be structured horizontally

Overlap multiple requests

Batch requests to the same data source Cache multiple requests for the same data

database

thrift

memcache


11/86

Furthermore, each data source has differentcharacteristics

Batch request API?

Sync or async API?

Set up a new connection for each request, or keep apool of connections around?

Want to abstract away from allof this in thebusiness logic layer


12/86

But we know how to do this!


13/86


Concurrency.

Threads let us keep our abstractions & modularity whileexecuting things at the same time.

Caching/batching can be implemented as a service inthe process

as we do with the IO manager in GHC


14/86


Concurrency.




But concurrency (the programing model) isnt whatwe want here.


15/86


Concurrency.




But concurrency (the programing model) isnt whatwe want here.

Example...


16/86

x and y are Facebook users

suppose we want to compute the number of

friends that x and y have in common simplest way to write this:

length (intersect (friendsOf x) (friendsOf y))


17/86

Brief detour: TAO

TAO implements Facebooks data model most important data source we need to deal with

Data is a graph Nodes are objects, identified by 64-bit ID Edges are assocs (directed; a pair of 64-bit IDs)

Objects and assocs have a type

object fields determined by the type

Basic operations: Get the object with a given ID Get the assocs of a given type from a given ID

User A

User B

User C

User D

FRIENDS


18/86

Back to our example


(friendsOf x) makes a request to TAO to get all theIDs for which there is an assoc of type FRIEND (x,_).

TAO has a multi-get API; very important that wesubmit (friendsOf x) and (friendsOf y) as a singleoperation.


19/86

Using concurrency

This:



20/86

Using concurrency

This:

Becomes this:


dom1 = putMVar m1)forkIO (friendsOf y >>= putMVar m2)fx


21/86


22/86

Using the async package:

doax


23/86


24/86

Using Control.Concurrent.Async.concurrently:

do(fx,fy)


25/86


26/86

Why not concurrency?

friendsOf x and friendsOf y are obviously independent

obviously both needed

pure


27/86




pure

Caching is not just an optimisation: iffriendsOf x is requested twice, we mustget the same

answer both times

caching is a requirement


28/86




pure

Caching is not just an optimisation: iffriendsOf x is requested twice, we mustget the same

answer both times

caching is a requirement

we dont want the programmer to have to ask forconcurrency here


29/86

Could we use unsafePerformIO?

we could do caching this way, but not concurrency.Execution will stop at the first data fetch.


friendsOf = unsafePerformIO ( .. )


30/86

Central problem

Reorder execution of an expression to perform datafetching optimally.

The programming model has no side effects (other

than reading)


31/86

What we would like to do:

explore the expression along all branches to get a set of data

fetches


32/86


submit the data fetches


33/86


wait for the responses


34/86


now the computation is unblocked along multiple paths

... explore again collect the next batch of data fetches

and so on

Round 0

Round 1

Round 2


35/86

Facebooks existing solution to this problem: FXL

Lets you write

And optimises the data fetching correctly.

But its an interpreter, and works with an explicitrepresentation of the computation graph.

Length(Intersect(FriendsOf(X),FriendsOf(Y)))


36/86

We want to run compiled code for efficiency

And take advantage of Haskell high quality implementation

great libraries for writing business logic etc.

So, how can we implement the right data fetchingbehaviour in a Haskell DSL?


37/86

Start with a concurrency monad

newtype Haxl a = Haxl { unHaxl :: Result a }

data Result a = Done a

| Blocked (Haxl a)

instance Monad Haxl wherereturn a = Haxl (Done a)m >>= k = Haxl $case unHaxl m of

Done a -> unHaxl (k a)Blocked r -> Blocked (r >>= k)


38/86

Old idea:

Scholz 1995A Concurrency Monad Based on Constructor

Primitives ClaessensA Poor Mans Concurrency Monad(JFP 1999)

Called the Resumption Monad in Harrisons Cheap(But Functional) Threads (JFP 2004)

Used to good effect in Li & Zdancewics PLDI07 paper

Combining events and threads for scalable networkservices implementation and evaluation of monadic,application-level concurrency primitives

Nowadays its a Free Monad


39/86

The concurrency monad lets us run a computationuntil it blocks, do something, then resume it

But we need to know what it blocked on... Could add some info to the Blocked constructor


40/86

newtype Haxl a = Haxl { unHaxl :: IO (Result a) }

data Result a = Done a| Blocked (Haxl a)

instance Monad Haxl wherereturn a = Haxl (return (Done a))m >>= k = Haxl $ doa unHaxl (k a)Blocked r -> return (Blocked (r >>= k))

dataFetch :: Request a -> Haxl adataFetch r = doaddRequest rHaxl (return (Blocked (Haxl (getResult r))))

We choose to put it in IO instead:


41/86

Ok so far, but we still get blocked at the first datafetch.

numCommonFriends x y = dofx


42/86

To explore multiple branches, we need to useApplicative

instance Applicative Haxl wherepure = returnHaxl f Haxl a = Haxl $ dor return (Done (f' a'))Blocked a' -> return (Blocked (f' a'))

Blocked f' -> do

ra return (Blocked (f' a'))


43/86

This is precisely the advantage ofApplicative overMonad:

Applicative allows exploration of the structure of thecomputation

Our example is now written:

Or:

numCommonFriends x y =length (intersect friendsOf x friendsOf y)

numCommonFriends x y =length common (friendsOf x) (friendsOf y)where common = liftA2 intersect


44/86

Note that we still have the Monad!

The Monad allows us to make decisions based on

values when we need to.

Batching will not explore the then/else branches exactly what we want.

dofs


45/86

But it does mean the programmer should useApplicative composition to get batching.

This is suboptimal:

So our plan is to guide programmers away from thisby using a combination of:

APIs that discourage it (e.g. providing lifted operations)

Code analysis and automatic feedback

Profiling

dofx


46/86

We really want bulk operations to benefit frombatching.

But this doesnt work: mapM uses Monad ratherthan Applicative composition.

This is why traverse exists:

So in our library, we make mapM = traverse

Also: sequence = sequenceA

traverse :: (Traversable t, Applicative f)=> (a -> f b) -> t a -> f (t b)

friendsOfFriends id =concat (mapM friendsOf =


47/86


48/86

---- Return the names of all the friends of the user that are-- members of the Functional Programming group.--

fpFriends :: Id -> Haxl [Text]fpFriends id = do

fs


49/86

But we need a different filterM. The standard one:

filterM :: (Monad m) => (a -> m Bool) -> [a] -> m [a]

filterM _ [] = return []filterM p (x:xs) = do

flg


50/86

But we need a different filterM. The standard one:

and the one that batches the data fetches correctly:

filterM :: (Applicative f, Monad f) => (a -> f Bool) -> [a] -> f [a]filterM pred xs = do

bools m Bool) -> [a] -> m [a]

filterM _ [] = return []filterM p (x:xs) = do

flg


51/86

Lets write the same example in a slightly differentway:

nameOfFPFriend :: Id -> Haxl (Maybe Text)nameOfFPFriend id = dob Haxl [Text]fpFriends2 id = dofs


52/86

friendsOf

memberOfGroup

getName

Before there were obviously 3 rounds.


53/86

friendsOf

memberOfGroup

getName


But in the new version, we packaged up two fetches

in nameOfFPFriend

nameOfFPFriend


54/86

friendsOf

memberOfGroup

getName


But in the new version, we packaged up two fetches

in nameOfFPFriend

The monad still executes everything in 3 rounds.

nameOfFPFriend

b l k ll h /f l ff


55/86

But maybe we dont like all this mapM/filterM stuff

b d l k ll h /f l ff


56/86


A query-style API might be nicer.

B b d lik ll hi M/fil M ff


57/86



newtype HaxlQuery a = HaxlQuery { query :: Haxl [a] }-- basically ListT Haxl, but ListT doesnt work!-- instances of Monad, MonadPlus

selectFrom :: Haxl [a] -> HaxlQuery aselectFrom = HaxlQuery

liftH :: Haxl a -> HaxlQuery aliftH m = HaxlQuery (return m)

suchThat :: Haxl Bool -> HaxlQuery ()suchThat m = liftH m >>= guard

B b d lik ll hi M/fil M ff


58/86



fpFriends3 :: Id -> Haxl [Text]fpFriends3 id =query [ name | f HaxlQuery ()suchThat m = liftH m >>= guard


59/86

DSL tricks

instance Num a => Num (Haxl a) where(+) = liftA2 (+)(-) = liftA2 (-)-- etc.

simonFriends id =filterM (isCalled Simon) =


60/86

Implementation

DataSource abstraction

Replaying requests

Scaling

Hot-code swapping

Experience

Status etc.


61/86

Data Source Abstraction

We want to structure the system like this:

Core code includes the monad, caching support etc.

Core is generic: no data sources built-in

Core

TAO

Memcache

other

service...

Data sources


62/86

How do we arrange this?

Three ways that a data source interacts with core:

issuing a data fetch request

persistent state

fetching the data

Package this up in a type class

Lets look at requests first...

class DataSource req where

...

parameterised

over the type of

requests


63/86

Example Request type

Core has a single way to issue a request

Note how the result type matches up.

data ExampleReq a whereCountAardvarks :: String -> ExampleReq IntListWombats :: Id -> ExampleReq [Id]deriving Typeable

its a GADT, where the typeparameter is the type of

the result of this request

dataFetch :: DataSource req => req a -> Haxl a


64/86

Fetching requests

It is Cores job to keep track of requests submittedvia dataFetch

When the computation is blocked, we have to fetch

the batch of requests from the data source

class DataSource req wherefetch :: [BlockedFetch req] -> IO ()

data BlockedFetch req= forall a . BlockedFetch (req a) (MVar a)


65/86

How do we store the requests?

Core doesnt know which data sources are in use

Need to use dynamic typing (Typeable) toimplement the store of pending requests and the

cache.class (Typeable1 req, Hashable1 req, Eq1 req)

=> DataSource req where ...

class Eq1 req where

eq1 :: req a -> req a -> Bool

unfortunate that we

need this


66/86

Data Source State

Core keeps track of the state for each data source

Passes it to fetch

class (Typeable1 req, Hashable1 req, Eq1 req)=> DataSource req where

data DataState req

fetch :: DataState req-> [BlockedFetch req]-> IO ()

The state for a data source

is an associated datatype


67/86

Clean data source abstraction

Means that we can plug in any set of data sourcesat runtime

e.g. mock data sources for testing

core code can be built & tested independently


68/86

Replayability

The Haxl monad and the type system give us:

Guarantee of no side effects, except via dataFetch

Guarantee that everything is cached

The ability to replay requests...


69/86

user code


70/86

user code Haxl Core


71/86

user code Haxl Core data

sources


72/86


sources

Cache


73/86


sources

Cache


74/86


sources

Cache


75/86


sources

Cache


76/86


sources

Cache

The data sources change over time

But if wepersist the cache, we can re-run the user code and

get the same results Great for

testing fault diagnosis profiling


77/86

Scaling

Each server has lots of cores, pounded by requestsfrom other boxes constantly.


78/86

Hot code swapping

1-2K machines, new code pushed many times perday

Use GHCs built-in linker

Had to modify it to unload code GC detects when it is safe to release old code

We can swap in new code while requests are still

running on the old code


79/86

Haskell Extensions?


80/86

Haskell Extensions?

Other people on the team are Haskell beginners


81/86

Haskell Extensions?


Goal: keep the code simple, use few extensions


82/86

Haskell Extensions?



{-# LANGUAGE ExistentialQuantification,

TypeFamilies,GADTs,RankNTypes,ScopedTypeVariables,DeriveDataTypeable,StandaloneDeriving,

MultiParamTypeClasses,FlexibleInstances

#-}


83/86

Haskell Extensions?



But were doing ok.

{-# LANGUAGE ExistentialQuantification,

TypeFamilies,GADTs,RankNTypes,ScopedTypeVariables,DeriveDataTypeable,StandaloneDeriving,

MultiParamTypeClasses,FlexibleInstances

#-}


84/86

Status

Prototyped most features (including hot codeswapping & scaling)

Core is written

We have a few data sources, more in the works

Busy hooking it up to the infrastructure

Can play around with the system in GHCi, including

data sources


85/86

Key points

Haxl Monad:

Implicit concurrency with Applicative

Automatic batching of data fetches

Haskells type system guarantees the user cantbreak the rules

... and gives us guaranteed replayability

Clean separation of core from datasources

Scaling & hot-code swapping


86/86

Questions?

the+haxl+project+at+facebook

Documents