Technical Director, 10gen

@jonnyeight alvin@10gen.com alvinonmongodb.com

Alvin Richards

#MongoDBdays

Capacity Planning: Choosing What to Deploy and When

http://hci.stanford.edu/jheer/files/zoo/ex/maps/napoleon.html

But first a story…

What are the consequences of not planning?

Capacity Planning: Why, What, When

Why?

What are the consequences of not planning?

Capacity Planning: Why, What, When

Why?

•  Availability •  Throughput •  Latency

What?

Capacity Planning: Why, What, When

Before it's too late!

When?

Capacity Planning: Why, What, When

Start Launch Version 2

Before it's too late!

When?

Capacity Planning: Why, What, When

Start Launch Version 2

Problems

•  Capacity –  Under –  Over –  Just right?

•  Prediction Models –  User/Load –  System(s) Behavior

•  Change Velocity (reaction time) –  Data/Resource-Allocation/Provisioning

Resource Usage

•  Storage –  IOPS –  Size –  Data & Loading Patterns

•  Memory –  Working Set

•  CPU –  Speed –  Cores

•  Network –  Latency –  Throughput

7,200 rpm SATA ~ 75-100 IOPS

15,000 rpm SAS ~ 175-210 IOPS

Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS

Amazon SSD 9,000 – 120,000 IOPS

Storage Capability Example IOPs

7,200 rpm SATA ~ 75-100 IOPS

15,000 rpm SAS ~ 175-210 IOPS

Amazon EBS/Provisioned ~ 100 IOPS "up to" 2,000 IOPS

Amazon SSD 9,000 – 120,000 IOPS

Storage Capability Example IOPs

Intel X25-E (SLC) ~ 5,000 IOPS

Fusion IO ~ 135,000 IOPS

Violin Memory 6000 ~ 1,000,000 IOPS

Storage Measuring and Monitoring

Storage Measuring and Monitoring

Storage Measuring and Monitoring

Storage

•  Active

•  Archival

•  Loading Patterns

•  Integration (BI/DW)

db.blogs.stats() {

"ns" : "test.blogs", "count" : 1338330, "size" : 46915928, "avgObjSize" : 35.05557523181876, "storageSize" : 86092032, "numExtents" : 12, "nindexes" : 2, "lastExtentSize" : 20872960, "paddingFactor" : 1, "flags" : 0, "totalIndexSize" : 99860480, "indexSizes" : { "_id_" : 55877632, "name_1" : 43982848 }, "ok" : 1

}

stats()

Size of data

Average document size

Size on Disk

Size of all indexes

Size of each index

•  Sorting

•  Aggregation

•  Connections

•  Working Set –  Active Data in Memory –  Measured Over Periods

•  Ratio to Storage

Memory

•  New in 2.4 – db.serverStatus( { workingSet: 1 } )

Memory Measuring and Monitoring

MOPs

PFs

Memory and Storage MOPS: MongoDB Ops/Sec

% Disk Util

MOPS

Memory and Storage

Estimating Sample Use Case •  User log on

•  Read "user" collection based on supplied credentials •  Update "last seen" and "ip" on "user" collection •  Insert into "events" collection •  Update "activity" collection for time series metrics

Estimating Sample Use Case – Worst Case

Index Read(s)

Index Write(s)

Journal Write

Data Read(s)

Data Write(s)

OpLog Write(s)

Read "user" collection based on supplied credentials

4k 4k

Update "last seen" and "ip" on "user" collection

Maybe* Maybe** sizeof(bson) 4k 4k 4k

Insert into "events" collection

4k per Index

sizeof(bson) 4k 4k

Upsert "activity" collection for time series metrics

4k 4k per index

sizeof(bson)

4k 4k 4k

* Only if page is faulted out by the O/S before this operation executes ** if indexes exists on attributes updated, index update will also occur

Estimating Sample Use Case •  User log on

•  Read "user" collection based on supplied credentials •  8k Read

•  Update "last seen" and "ip" on "user" collection •  4k Read, 4k Write (single index), 4k Write OpLog

•  Insert into "events" collection •  8k Write (single index), 4k Write OpLog

•  Update "activity" collection for time series metrics •  8k Read, 8k Write (single index), 4k Write OpLog

•  System load •  20k Read, 36k Write

CPU

•  Non-indexed Data

•  Sorting

•  Aggregation –  Map/Reduce –  Framework

•  Data –  Fields –  Nesting –  Arrays/Embedded-Docs

MOPs

CPU

MOPs

CPU %

CPU

•  Non-indexed Data

•  Sorting

•  Aggregation –  Map/Reduce –  Framework

•  Data –  Fields –  Nesting –  Arrays/Embedded-Docs

CPU

•  Latency –  WriteConcern –  ReadPreference –  Batching –  Documents (and Collections)

•  Throughput –  Update / Write Patterns –  Reads / Queries –  SAN / NAS –  Virtualization

Network

All of these use the same resources:

•  Single Instance

•  Multiple Instances (Replica Set)

•  Cluster (Sharding)

•  Data Centers

Deployment Types

•  Storage

•  Memory

•  CPU

•  Network

•  Application Metrics

Monitoring

•  MMS (MongoDB Monitoring Service)

•  MongoDB: mongotop, mongostat

•  Linux: iostat, vmstat, sar, etc

•  Windows: Perfmon

•  Load testing

Tools

•  Load/Users –  Response Time/TTFB

•  System Performance –  Peak Usage –  Min Usage

Models

•  Limitations –  Data Movement –  Allocation/Provisioning (servers/mem/disk)

•  Improvement –  Limit Size of Change (if you can) –  Increase Frequency –  Practice

Velocity of Change

•  Repeat Testing

•  Repeat Evaluations

•  Repeat Deployment

Repeat (continuously)

•  What is the working set? –  How does that equate to memory –  How much disk access will that require

•  How efficient are the queries?

•  What is the rate of data change?

•  How big are the highs and lows?

Starter Questions

Technical Director, 10gen

@jonnyeight alvin@10gen.com alvinonmongodb.com

Alvin Richards

#MongoDBdays

Thank You