![Page 1: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/1.jpg)
MONGIOPS Your Favorite Datastore, Only Faster…
![Page 2: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/2.jpg)
A MONGO SERVER
This is a typical server…
December 5, 2012 Fusion-io Confidential 2
Dell R720 CPU Cores: 32 w/HT Memory: 128 GB
…is your working set > 128GB?
![Page 3: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/3.jpg)
DO
LL
AR
S
COST OF DRAM MODULES
0
200
400
600
800
1000
1200
1400
1600
4GB 8GB 16GB 32GB
Fusion-io Confidential December 5, 2012 3
$ $$ $$$
$$$$$$
![Page 4: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/4.jpg)
A MONGO CLUSTER
With NoSQL Databases, we tend to scale out for DRAM
December 5, 2012 Fusion-io Confidential 4
Combined Resources CPU Cores: 96! Memory: 384 GB
Do we need that many cores to serve reads and writes?
![Page 5: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/5.jpg)
RE
LA
TIV
E P
ER
FO
RM
AN
CE
(L
OG
AR
ITH
MIC
SC
AL
E)
STORAGE IS SLOW
December 5, 2012 5 Fusion-io Confidential Source: http://www.micronblogs.com/category/history-of-digital-storage/
1
100,000
100
1,000
10,000
10
1988 2008
HDD Performance
![Page 6: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/6.jpg)
TECHNOLOGY ENABLERS
© Fusion-io
FLASH MEMORY
SOFTWARE-ENABLED REPROGRAMMABLE
CONTROLLERS
PCIe Ecosystem
![Page 7: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/7.jpg)
SSD TREATS MEMORY LIKE DISK
© Fusion-io
![Page 8: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/8.jpg)
MONGO ON FUSION-IO
August, 2012 Fusion-io Confidential 8
Step 1: Mount ioDrive at /var/lib/mongo Step 2:
![Page 9: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/9.jpg)
IODRIVE2 SPECIFICATIONS
ioDrive2 Capacity 400GB 600GB 365GB 785GB 1.2TB
NAND Type SLC (Single Level Cell)
SLC (Single Level Cell)
MLC (Multi Level Cell)
MLC (Multi Level Cell)
MLC (Multi Level Cell)
Read Bandwidth (1 MB) 1.4 GB/s 1.5 GB/s 910 MB/s 1.5 GB/s 1.5 GB/s
Write Bandwidth (1 MB) 1.3 GB/s 1.3 GB/s 590 MB/s 1.1 GB/s 1.3 GB/s
Read IOPS (512 Byte) 190,000 193,000 137,000 141,000 143,000
Write IOPS (512Byte) 535,000 535,000 535,000 535,000 535,000
Read Access Latency 47 µs 47 µs 68 µs 68 µs 68 µs
Write Access Latency 15 µs 15 µs 15 µs 15 µs 15 µs
Bus Interface PCI-Express 2.0 x4
Weight Less than 11 ounces
Operating Systems 64-Bit Microsoft XP/Vista/Win7/Server 2003/Server 2008, RHEL 4/5/6, SLES 10/11, OEL v5/6,
VMware ESX 4.0/4.1/ESXi 4.1/5.0*, Solaris 10 U8/U9, OSX* 10.6.7/10.7, HP-UX* 11i *ioDrive2 and ioDrive2 Duo
August, 2012 Fusion-io Confidential 9
![Page 10: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/10.jpg)
IODRIVE2 DUO SPECIFICATIONS
ioDrive2 Capacity 1.2TB 2.4TB
NAND Type SLC (Single Level Cell) MLC (Multi Level Cell)
Read Bandwidth (1 MB) 3.0 GB/s 3.0 GB/s
Write Bandwidth (1 MB) 2.5 GB/s 2.5 GB/s
Read IOPS (512 Byte) 385,000 285,000
Write IOPS (512Byte) 725,000 725,000
Read Access Latency 47 µs 68 µs
Write Access Latency 15 µs 15 µs
Bus Interface PCI-Express 2.0 x8
Weight Less than 16 ounces
Operating Systems 64-Bit Microsoft XP/Vista/Win7/Server 2003/Server 2008, RHEL 4/5/6, SLES 10/11, OEL v5/6,
VMware ESX 4.0/4.1/ESXi 4.1/5.0*, Solaris 10 U8/U9, OSX* 10.6.7/10.7, HP-UX* 11i *ioDrive2 and ioDrive2 Duo
August, 2012 Fusion-io Confidential 10
![Page 11: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/11.jpg)
GO TO OUR WEBSITE
December 5, 2012 Fusion-io Confidential 11
![Page 12: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/12.jpg)
ONE IODRIVE VS A 10-DISK RAID-0
December 5, 2012 Fusion-io Confidential 12
vs
ioDrive 2 – 1.2TB 10 x 7200 RPM SAS disks in raid-0 mdarray
![Page 13: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/13.jpg)
YCSB DETAILS
▸ 2 x 8-core 2.9GHz Xeon/64GB/2x10GbE ▸ XFS
Test ▸ Workload A: 50/50% read/write mix ▸ Workload B: 95/5% read/write mix ▸ Workload C: 100% read ▸ Workload F: 50/50% read/read+modify+write mix
December 5, 2012 Fusion-io Confidential 13
![Page 14: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/14.jpg)
Workload A - 50/50% read/write mix
December 5, 2012 Fusion-io Confidential 14
![Page 15: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/15.jpg)
Workload B - 95/5% read/write mix
December 5, 2012 Fusion-io Confidential 15
![Page 16: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/16.jpg)
Workload C - 100% read
December 5, 2012 Fusion-io Confidential 16
![Page 17: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/17.jpg)
Workload F - 50/50% read/read+modify+write mix
December 5, 2012 Fusion-io Confidential 17
![Page 18: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/18.jpg)
AV
G R
EA
D L
AT
EN
CY
IN
MIL
ISE
CO
ND
S
YCSB: LOWEST READ LATENCY WINS
December 5, 2012 18
0
20
40
60
80
100
120
140
160
180
W O R K L O A D A W O R K L O A D B W O R K L O A D F : U P D A T E W O R K L O A D F : R M W
Fusion-io Disk
Fusion-io Confidential
30x
Fast
er
27x
Fast
er
20x
Fast
er
40x
Fast
er
![Page 19: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/19.jpg)
AV
G U
PD
AT
E L
AT
EN
CY
IN
MIL
ISE
CO
ND
S
YCSB: LOWEST UPDATE LATENCY WINS
December 5, 2012 19
0
50
100
150
200
250
300
350
400
450
W O R K L O A D A W O R K L O A D B W O R K L O A D F : U P D A T E W O R K L O A D F : R M W
Fusion-io Disk
Fusion-io Confidential
18x
Fast
er
4x F
aste
r
14x
Fast
er
16x
Fast
er
![Page 20: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/20.jpg)
ARCHITECTURAL IMPACT OF FUSION-IO
▸ Avoid scaling out for DRAM • Nodes can handle higher transaction load • Terabytes of low latency persistent storage on single nodes • Potentially avoid sharding
▸ Use less DRAM per node • Lower cost servers • DRAM available for applications
December 5, 2012 20 Fusion-io Confidential
![Page 21: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/21.jpg)
Talksum Data Stream Real Time for Big Data
![Page 22: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/22.jpg)
Talksum Data Stream Appliance
• Real Time Data Management & Analy1cs
• Filters, Monitors, Counts, Transforms & Reduces
• OS Level Processing for High Performance
Talksum Data Stream
System Logs
App Data
Device Stats
OtherData
Accepts Data fromany source
Filter &Route
Real-TimeMonitors & Analytics
Real Time Reduced
Data
RemoteSites
Outputs to Storage,Applications or other Systems
BI Tools
Analytic Apps
StoredProcess
Third Party
API
Device
Storage
![Page 23: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/23.jpg)
WHY DO WE USE MONGO?
▸ First Class JSON Support • Perfect match for CEE enhanced RFC3164 • “Schemaless” = great for log data with minor variations
▸ Capped Collections • Control over space utilization important on an appliance
▸ Atomic Operations • Great for working with metrics
▸ Aggregation Framework • Convenience of SQL “group by”
▸ Map Reduce • Familiar paradigm
December 5, 2012 Fusion-io Confidential 23
![Page 24: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/24.jpg)
ANALYZING NETFLOW DATA
Analyzing a day’s worth of Netflow Data to generate sent octets metrics for each source ip, destination ip, and source->destination ip combination using atomic increment operations
{ _id : src_key }, { $inc : { sntOcts : snt_oct } } { _id : dst_key }, { $inc : { sntOcts : snt_oct } } { _id : src_dst_key }, { $inc : { sntOcts : snt_oct }}
To test IO, messages sent as fast as possible.
No Queue! Messages dropped when IO capacity overflows.
December 5, 2012 Fusion-io Confidential 24
![Page 25: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/25.jpg)
NETFLOW MESSAGES PROCESSED A SECOND
Atomic Updates: each message is 3 Mongo operations.
December 5, 2012 25 Fusion-io Confidential
![Page 26: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/26.jpg)
RESULTS OVER 1 HOUR (3600 SECONDS)
▸ SATA Drive • Total messages processed: 2,602,550 • Messages Per Second: 3,614 • Total Mongo Operations: 7,807,650 • Mongo Operations Per Second: 10,481
▸ Fusion-io Drive • Total messages processed: 7,811,851 • Messages Per Second: 10,849 • Total Mongo Operations: 23,435,553 • Mongo Operations Per Second: 30,549
December 5, 2012 26 Fusion-io Confidential
![Page 27: MongIOPS: Your Favorite Datastore, Only Faster](https://reader034.vdocuments.us/reader034/viewer/2022052619/555823fed8b42a25588b4cd6/html5/thumbnails/27.jpg)
T H A N K Y O U