map analytics in starcraft ii (2/3/2015)
Post on 15-Jul-2015
33 Views
Preview:
TRANSCRIPT
StarGazer: !
Map Analytics in StarCraft II
Guang Yang
(Demo)
Pipeline
10
IngestionStreaming
Batch
Data-store
API/Front-end
Ingestion
11
API
AWS EC2 Node
AWS EC2 Node
AWS EC2 Node
AWS EC2 Node
Serialize via Avro
Kafka Topics: !- Stream !- Batch
Example API Request: http://api.ggtracker.com/api/v1/matches/3529593.json
Serialization via Avro
- reinforces schema !
- splittable on HDFS !
- backward compatible !- saves space (binary)
Batch
13
Avro Files (on HDFS) SchemaRDD (in Spark SQL) Tables in Cassandra
- in-memory !- expressive commands
!- full benefits of RDD
- AP !- tunable consistency
Streaming
14
Guang Yang
- B.A. in Computational and Applied Mathematics (Rice University)
- M.S. in Industrial Engineering & Operations Research (UC Berkeley)
- Got into Diamond League as Terran without making any Tanks
- Email: gy8@berkeley.edu
- GitHub: github.com/gy8
15
Shetland, Scotland, UK
top related