(gam403) from 0 to 60 million player hours in 400b star systems
TRANSCRIPT
© 2015, Amazon Web Services, Inc. or its Affiliates. All rights reserved.
David Walsh, COO
Dav Stott – Senior Server Developer
October 2015
GAM403
From 0 to 60 Million Player Hours
Frontier Developments plc
in 400 Billion Star Systems
Boss level challenge 1
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
Multiplayer architecture
Shared Galaxy
AWS Elastic Beanstalk PHP
MongoDB
Redis
Amazon
DynamoDBAmazon
S3
STUN / TURN servers
•NAT traversal and network relay services
nslookup
turn.server
Amazon Route 53
Latency routing
eu-west-1
EC2
eu-west-1a
EC2
eu-west-1b
TURNTURN
•Uses worldwide AWS data centers to reduce P2P latency
ap-southeast-2
EC2
ap-southeast-
2a
EC2
ap-southeast-
2b
TURN TURN
eu-west-1a
edServer and Auto Scaling
edServer.exe
Amazon CloudWatch
Players
Frames/second
metrics.py
EU #19
Amazon EC2 edServer builds
hosted in Amazon S3
watchDog.py
EC2 management
API
edServer and Auto Scaling
eu-west-1
EU1 EU2 EU3
EU4 EU5 EU6
EU7 EU8 EU9
EU10 EU11 EU12
EU13 EU14 EU15
EU16 EU17 EU18
??
EU20 EU21 EU22
EU2
3
EU24 EU25
EU26 EU27 EU28
edServer and Matchmaking
Giving players the best possible online experience
GET /server
API
EU/GB
Travelled to Barnards Star
In Wing0 5 10 15 20 25
Server OC 1
Server EU 2
Server EU 3
Alpha Centauri
Sol
Barnards Star
Wings
edServer and load balancing
Quiet servers are easier to terminate
Stack players up to server load capacities
Central web API, autoscaling, and authentication
eu-west-1
Availability Zone A
DynamoDB
RDS
Amazon ElastiCache
Auto Scaling group
PHP
S3
PHP
Availability Zone B
A few learning points
aws autoscaling suspend-processes --scaling-processes AZRebalance --region eu-west-1 --auto-scaling-group-name <ASGNAME>
boto.ec2.autoscale.connect_to_region().terminate_instance(instanceId, true)
A few learning points
EC2 API rate limit
PHP’s Composer is vulnerable at deployment time
Orchestration and configuration in source control
Commanders and ships
Relational Document
MySQL MongoDB 3
id 103911108
credits 9119226
name A.N.Other
alive 1
currentShip 0
currentShipType FerDeLance
{"_id": {"commander_id": 103911108, "slot": 0},"name": "FerDeLance","modules": {
"HugeHardpoint1": "Hpt_BeamLaser_Huge","MediumHardpoint1": "Hpt_PulseLaserBurst_Small","MediumHardpoint2": "Hpt_PulseLaserBurst_Small","Armour":"FerDeLance_Armour_Grade1","FrameShiftDrive": "Int_Hyperdrive_Size4_Class5"
},"health": {"hull": 1000000, "shieldup": true},"fuel": {"capacity": 8, "lvl": 6.554478},"reserve": {"lvl": 0.9162},
}
MongoDB 3 on EC2
eu-west-1
t1.
micro
Amazon
Route 53
mongo-a 10.1.2.3
mongo-b 10.2.2.3
mongo-c 10.3.2.3
mongo-
backup
10.1.2.4
mongo-
arbiter
10.2.2.4
Boss level challenge 2
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
• Set in the real 1:1 sized Milky Way galaxy
• Accurately simulate 400 B star systems
Name: LHS 6309 A 1.
Class: TerrestrialClassification_EarthLike.
Mass: 1.8877(Gas:0.000, Volatiles:0.685, Rock:0.212, Metal:0.102)
Radius: 1.475 Earths.
Surface temp: 144.821 K.
Black body temp: 144.821 K.
Approx magma temp: 3740.786 K.
Semi-major axis: 0.025 AU.
Eccentricity: 0.076.
Inclination: -0.000 radians?]
Surface pressure: 4961.263 Atm.
Volcanism type: WaterGeysers.
Atmosphere composition: eAtmosphereClass_EarthLike(Nitrogen(1.000),
Oxygen(0.000), Neon(0.000)).
Description: Tidally locked Earthlike body with thick atmosphere and major water
geysers volcanism.
Data storage
EC2 SSD instance storage
Elasticache Redis
RDS MySQL and Mongo
S3
Sophisticated simulation
Sp
ee
d
Co
nsis
ten
cy
Only store “interesting” data
Boss level challenge 3
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
• Set in the real 1:1 sized Milky Way galaxy
• Accurately simulate 400 B star systems
• Exploration, dynamic trading, outfitting
Exploration
Ship navigation computers (DynamoDB)
First discovery credits (S3)
{
"bodies": {"M": {"36028797039542902": { "N": "7" }}},
"cmdr_id": {"N": "119161"},
"ss_addr": {"N": "20578934"}
}
{
"system": {"name": "Sagittarius A*"},
"bodies": {
"36028797039542902": {
"name": "Sagittarius A* A",
"cmdrname": "Zulu Romeo",
"discovered": "2014-12-01 13:48:22“
}
}
Game event logs
eu-west-1
Availability Zone A
Auto Scaling group
PHP PHP
Availability Zone B
Amazon SQS
Queue 1
Queue 2
Trading and outfitting
Overlays:
Default stock of 1000
Player buys 200
Current stock of 800
Gets stored as 0.8
Gradually restocks over time
Until it returns to 1000 when it gets deleted
Players
Politics
Supply and demand
Core simulation
Supply and demand
eu-west-1
Availability Zone A Availability Zone B
Auto Scaling group
PHP PHP
{“buy”:{“fish”:200, “mkt”: “Lave”}}
{“buy”:{“fish”:75, “mkt”: “Lave”}}
{“sell”:{“fish”:275, “mkt”: “Sol”}}
{“buy”:{“tea”:20, “mkt”: “Lave”}}
{“sell”:{“cobalt”:200, “mkt”: “Lave”}}
{“buy”:{“tea”:500, “mkt”: “Lave”}}
{“sell”:{“cobalt”:40, “mkt”: “Lave”}}
{“buy”:{“fish”:200, “mkt”: “Lave”}}
{“sell”:{“tea”:500, “mkt”: “Sol”}}
{“lave”: {“fish”: 0.8,
“cobalt”: 1.1,
“tea”: 0.5}
{“sol”: {“fish”: 1.2,
“tea”: 1.5}
Game analytics
Shared
API
SQS
Queue
dispatchers
ElasticsearchKibana 4
Kibana 3
Custom
reports On premise servers
Boss level challenge 4
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
• Set in the real 1:1 sized Milky Way galaxy
• Accurately simulate 400 B star systems
• Exploration and dynamic trading
• Player-driven story
Player-driven story
eu-west-1
Availability Zone A Availability Zone B
Auto Scaling group
PHP PHP
{
“faction: "9469194806713“,
"boom": 148,
"bust": 7,
"famine": 10,
"influence": 49
}
Sobek Boys
vs
Federal Bridge
{boom: +1} {inf: -4}
Community goals and leaderboards
eu-west-1
Availability Zone A Availability Zone B
Auto Scaling group
PHP PHP
ZADD "goal:1" 42 "Cmdr Dav"ZADD "goal:1" 89 "Cmdr Jameson"ZADD "goal:1" 100 "Cmdr Thrust"
ZREVRANGE "goal:1" 0 -11) "Cmdr Thrust"2) "Cmdr Jameson"3) "Cmdr Dav"
ZREVRANK "goal:1" "Cmdr Dav"(integer) 2
Boss level challenge 5
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
• Set in the real 1:1 sized Milky Way galaxy
• Accurately simulate 400 B star systems
• Exploration and dynamic trading
• Player-driven story
• Sell and distribute the game ourselves
Store and distribution
https://frontierstore.net
RDS MySQL
Login servers
S3
Game builds
Amazon
CloudFront
Boss level challenge 6
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
• Set in the real 1:1 sized Milky Way galaxy
• Accurately simulate 400 B star systems
• Exploration and dynamic trading
• Player-driven story
• Sell and distribute the game ourselves
• Add platforms, features, seasons
Close quarters combat championships
eu-west-1
Availability Zone A Availability Zone B
Auto Scaling group
PHP PHP
• Instant action PvP
• Join lobbies
• Background matchmaker
Boss level challenge 6
• Have similar impact as Elite did in 1984 using
• Modern CPU and GPU power
• Network connectivity
• Massively multiplayer
• Set in the real 1:1 sized Milky Way galaxy
• Accurately simulate 400 B star systems
• Exploration and dynamic trading
• Player-driven story
• Sell and distribute the game ourselves
• Add platforms, features, seasons