scala + akka + ning/async-http-client - vancouver scala meetup february 2015
TRANSCRIPT
![Page 1: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/1.jpg)
![Page 2: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/2.jpg)
Scala + Akka
+
ning/async-http-client
Or: How I stopped worrying about network
latency and drank the reactive cool-aid
![Page 3: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/3.jpg)
Intro
$ whoami
![Page 4: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/4.jpg)
RSS is dead, long live RSS!
● In constant decline over the years as
consumer-facing tech
● Still very popular between “content services”
● RSS/Atom feeds @ HSo Follow feeds on behalf of users
o Post to social networks based on content
o ~250K active feeds, ~400K messages daily
![Page 5: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/5.jpg)
Scaling webfeeds using PHP
![Page 6: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/6.jpg)
But latency isn’t the problem; blocking is.
![Page 7: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/7.jpg)
Enter Scala + Akka + async HTTP client
![Page 8: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/8.jpg)
Akka: a short detour for the uninitiated
“Toolkit and runtime for building highly
concurrent, distributed, and fault tolerant
applications on the JVM”
● Implementation of the Actor model
● Scala, Java
● Adheres strictly to the Reactive Manifesto:o Responsive, Resilient, Elastic, Message-driven
![Page 9: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/9.jpg)
Akka (continued)
Source: http://www.scottlogic.com/blog/rdoyle/assets/ActorModel.png
![Page 10: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/10.jpg)
Ning’s async-http-client
![Page 11: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/11.jpg)
… in an actor
![Page 12: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/12.jpg)
Actors in Scala webfeed service
![Page 13: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/13.jpg)
So, what did this buy us?
![Page 14: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/14.jpg)
Now
BeforeBefore ● 120 processes
● 6 X m3.xlarge
● 2 processes
● 2 X m3.xlarge
![Page 15: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/15.jpg)
90 webfeed/sec 300 webfeed/sec
![Page 16: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/16.jpg)
CPU utilization & load average - barely breaking a sweat!
![Page 17: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/17.jpg)
Some lessons learned
● You can't ignore back pressureo Pull-based approach
o Akka Streams to the rescue?
● scala.xml.pull.XMLEventReader is not
production ready
● Be weary of the resulting load on the rest of
your infrastructure!
![Page 18: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/18.jpg)
Some lessons learned (continued)
● Actors should only ever do one thing (and do
it well)o Easy to modify the actor topology (and it will
happen!)
o Easier to describe supervisor’s responsibilities
● Let it crash!
![Page 19: Scala + Akka + ning/async-http-client - Vancouver Scala meetup February 2015](https://reader034.vdocuments.us/reader034/viewer/2022050907/55a644311a28ab9d518b48fc/html5/thumbnails/19.jpg)
Thanks!
http://akka.io/
https://github.com/AsyncHttpClient/async-http-client
We’re hiring!
https://hootsuite.com/about/careers
@yanik_berube