StarGazer: !

Map Analytics in StarCraft II

Guang Yang

(Demo)

Pipeline

10

IngestionStreaming

Batch

Data-store

API/Front-end

Ingestion

11

API

AWS EC2 Node

AWS EC2 Node

AWS EC2 Node

AWS EC2 Node

Serialize via Avro

Kafka Topics: !- Stream !- Batch

Example API Request: http://api.ggtracker.com/api/v1/matches/3529593.json

Serialization via Avro

- reinforces schema !

- splittable on HDFS !

- backward compatible !- saves space (binary)

Batch

13

Avro Files (on HDFS) SchemaRDD (in Spark SQL) Tables in Cassandra

- in-memory !- expressive commands

!- full benefits of RDD

- AP !- tunable consistency

Streaming

14

Guang Yang

- B.A. in Computational and Applied Mathematics (Rice University)

- M.S. in Industrial Engineering & Operations Research (UC Berkeley)

- Got into Diamond League as Terran without making any Tanks

- Email: gy8@berkeley.edu

- GitHub: github.com/gy8

15

Shetland, Scotland, UK