mongodb days silicon valley: best practices for upgrading to mongodb
TRANSCRIPT
Best Practices for Upgrading to MongoDB 3.2
Achille Brighton
This talk
• What's cool in MongoDB version 3.2• Upgrading MongoDB In Outline• The Fine Details• Upgrading Revisited• One weird trick
What’s cool in 3.2
More use cases. Pluggable storage engines enables you to use MongoDB in
more projects with the same core database.
Mission-critical apps. MongoDB delivers major advances in the critical areas of
governance, high availability, and disaster recovery.
New tools for new users. Now MongoDB is an integral part of the tooling and
workflows of Data Analysts, DBAs, and Operations teams.
3.2 Overview
Encrypted Storage
What you get • Encryption of sensitive data for regulated industries• ~ 15% overhead
– Better than many 3rd party encryption
Note• Based on the WiredTiger Storage Engine • Available in MongoDB Enterprise Advanced
Document Validation
What you get • Implement data governance • Enforce data quality across• Use familiar MongoDB queries to specify rules
More power to the DBAs • The DBA can specify which documents in a collection
should be validated • Failed validations can be configured
– Hard error– Just a warning
Partial Indexes
$lookup Join
What you get • Equivalent to left join• Use with aggregation
$lookup: { localField : "stock", foreignField: "stockQuote", from : "other_collection", as : "joined_stock" }
CSRS – Config Server Replica Set
CSRS
Overview• Config server replica sets can span more than 3 data
centers with up to 50 replica set members supported
What you get • Simplified sharded deployments
– Config servers are deployed as replica sets• Improved metadata consistency • Easily scale to many data centers
Fast Failover
• Enhanced algorithm detects failure and isolation of primary.• Clusters more resilient to overloaded or unreliable networks
What you get • 2 seconds failover
– Exact time is dependent on system configuration – Tune the timing with the electionTimeoutMillis
Storage Engines
In-Memory Storage
• Predictable throughput and latency• In-memory computing without trading away guarantees
of disk-based databases like – Rich query flexibility – Real-time analytics– Scalable capacity – Durability
• Durability via replica set secondaries (using mmap/wt)
WiredTiger is the Default
What you get • Best general-purpose storage engine• 7-10x better throughput• Up to 80% compression
“Out of the box” • No special configuration needed
7x-10x Performance, 50%-80% Less StorageHow: WiredTiger Storage Engine• Same data model, same query
language, same ops• Write performance gains driven
by document-level concurrency control
• Storage savings driven by native compression
• 100% backwards compatible• Non-disruptive upgrade
MongoDB 3.0MongoDB 2.6
Performance
50%-80% Less Storage via Compression
• Better storage utilization• Higher I/O scalability• Multiple compression options
– Snappy– zlib– None
• Data and journal compressed on disk• Indexes compressed on disk and in memory
Compression in action
New Tools for New Users
Data Analysts• BI Connector• $lookup
• Aggregation Enhancements: Faster & simpler real-time analytics
DBAs
MongoDB Compass
• Visualize schema• View Documents• Graphically build queries• Authenticated access
Operations
Integration with standard operational workflow reduces overhead• Start from global view of infrastructure: APM tools
integration, e.g. New Relic, AppDynamics.• Then, drill down: Profiler visualization in Ops Manager • Then, deploy: Automated index builds• Next, refine: Partial Indexes improve resource utilization
Ops Manager & MMS
Single-click provisioning, scaling & upgrades, admin tasks
Monitoring, with charts, dashboards and alerts on 100+ metrics
Backup and restore, with point-in-time recovery, support for sharded clusters
MongoDB Ops Manager / MMS
The Best Way to Manage MongoDB In Your Data CenterUp to 95% Reduction in Operational Overhead
How Ops Manager / MMS Helps You
Scale EasilyMeet SLAs
Best Practices, Automated
Cut Management Overhead
Upgrading MongoDB in Outline
• Manual upgrades – lots of steps, but doable• Automatic – Ops Manager / MMS
Manual Upgrade
Upgrading a cluster manually
• Follow the steps in the fine documentation:– http://docs.mongodb.org/manual/release-notes/3.0-upgrade/– http://docs.mongodb.org/manual/release-notes/3.2-upgrade/
• (coming soon)– First result if Googling for “MongoDB Upgrade”
• It's critical to note that upgrade order matters:– For replica sets, first upgrade the secondaries, then
the primaries.– For a sharded cluster, follow the 7 steps in the
documentation.
Automatic
Upgrading a cluster with Automation
• Ops Manager / MMS is the easiest, fastest, and best way to upgrade a MongoDB deployment– One-click upgrades.– Rolling upgrades, so zero downtime.
Upgrades
• Select Desired Version from Dropdown
The fine details
• Backward compatibility & your application• Backward compatibility & your operations• Database behavior & performance• Enabling 3.2 Features
Backward Compatibility
Backward Compatibility: App
• Comprehensive documentation about incompatible changes– https://docs.mongodb.org/manual/release-notes/3.2-compatibility
• Highlights– You may need to upgrade your applications' drivers
before upgrading MongoDB!– Some commands/methods deprecated, removed,
changed.
• Audit your codebase for uses of deprecated functionality.
Backward Compatibility: Ops
• Comprehensive documentation about incompatible changes– https://docs.mongodb.org/manual/release-notes/3.2-compatibility
• Highlights– Some settings deprecated, others mmapv1-only.– Stricter replica set configuration validation– mongo* tools' options changed– Legacy (pre-2.6) user account model removed
• Audit scripts & other infrastructure you've got around.
Database Behavior & Performance
Database Behavior & Performance
• Always test your application against new version before upgrading production– Apps sometimes rely on unspecified behaviors (e.g.
stability of results from unsorted queries)– Query optimizer improvements may affect index
selection (check important queries with explain)
Good practice: Test against Staging
• Run all your performance correctness tests against a staging environment for some time before upgrading production.
• Best practice: run a real application workload against a 3.0 staging environment, too.– Replicate the workload manually– Use Facebook’s Flashback tool
Upgrading to use 3.2 Features
Protocol Version Upgrade
• Protocol Version 0 (Current)• Protocol Version 1 (New)
– Conceptually:• cfg=rs.conf()• cfg.protocolVersion=1• rs.reconfig(cfg)
CSRS – Config Replicaset
• Allows up to 50 config servers• New clusters only, or brief downtime
Encryption at Rest
• Rolling upgrade
Document Validation
• Validation Level– Off– Strict– Moderate (allows existing invalid documents)
• Validation Action– Warn– Error
Recap
Upgrading Revisited
• Plan• Practice• Productionize
Plan Your Upgrade
Plan Your Upgrade
• Review the compatibility notes– Audit apps, ops for deprecated details
• Upgrade drivers, scripts, if necessary– Might require some recoding, in edge cases
• Make sure you've got good tests– App behavior, performance, etc.
• Write a checklist of upgrade steps for your environments
Practice Your Upgrade
Practice Your Upgrade
• Upgrade apps & database in staging environment• Test, test, test.
– If no problems appear within (say) a week of continuous testing, probably a decent candidate.
Productionize Your Upgrade
We Can Help!
• MongoDB's Professional Services group offers consulting engagements.– Ensuring upward compatibility for your apps, tools– Assisting with performance & behavior testing around
database upgrades– Preparing & executing upgrade plans with your team
in your environments.
– Let us know: [email protected]