Data Center AwarenessScott Hernandez, 10gen

Agenda

● Infrastructure● Players and moving pieces● Goals and challenges● MongoDB solutions

○ Now (Replication, Sharding, Tagging)○ Future (+=, Shard affinity, more Tagging)

Multiple Online Data Centers

● Europe○ App○ Data

● Asia

○ App

● USA○ App○ Data

Infrastructure

● Geographically disparate clients● Geo-DNS/Load Balancer (LB)● Accelerators/Reverse-Proxies● Server/Application

Players Introduction

● Users ● Data Centers

○ Application Servers○ Databases

● Application Awareness

○ Configuration○ Why?

Goals

● Reduce network○ latency○ Inter-dc traffic

● Localize resource use○ Reduce failure cases○ Increase availability○ Isolate dependencies

● Provide multiple active sites● Partition geo/regional data

Challenges

● Data concistency○ User experience○ Backup and operational needs

● Scaling● Partitioning/Sharding

Multiple Online Data Centers

● Europe○ App○ Data

● Asia

○ App

● USA○ App○ Data

Non-default behaviors

Default:● Primary read/write● No stale reads

Multi-DC needs:● Read locally● Support some stale

reads

Replication

● Replica Sets○ Possible to read from non-primary replicas○ Copy of data○ Write Concern

■ Tagging■ Verifiable writes

● Provides

○ Isolation (possible stale reads)○ Availability○ Distribution of read, possibly stale (WriteConcern)

Replica Writes

Client USA-NY

USA-CA

EUROPE

Write Replica_Safe

Ack

ClientQuery

Tagged Writes

Uses● Multiple Racks/DCs● Backups● Disaster Recovery

Sharding + Replication

● Range-based sharding (chunks)○ Not tag aware○ Random distribution (balancer)

● Shards made up of Replica Sets○ All advantages

● Writes to (primary) shard per chunk● Reads

○ From Primary by default○ Optional non-primary reads

New Features

● ReadPreference○ Primary (only)○ Secondary (preferred)○ PrimaryFirst?○ SecondayOnly?○ Any (closest)

● Replica Sets

○ ReadPreference● Sharding

○ Tagged balancing○ ReadPreference

Sharding - Reads

● Local when non-primary● Tagged

○ Custom Tagging○ By region/dc/rack?

Sharding - Balancing/Distribution

● Tag chunks/ranges● Possible super-chunks (grouping)