effectively deploying mongodb on aem
TRANSCRIPT
![Page 1: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/1.jpg)
Effectively Scale and Operate AEM with MongoDB
![Page 2: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/2.jpg)
Agenda
MongoDB Introduction MongoMK Sizing Deployment Operational Best Practices
![Page 3: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/3.jpg)
Introduction
Norberto Leite Technical Evangelist Madrid, Spain http://www.mongodb.com/norberto @nleite [email protected]
![Page 4: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/4.jpg)
introduction
![Page 5: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/5.jpg)
MongoDB
GENERAL PURPOSE DOCUMENT DATABASE OPEN-SOURCE
![Page 6: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/6.jpg)
MongoDB is Fully Featured
![Page 7: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/7.jpg)
Query Operators
Condi&onal Operators $all, $exists, $mod, $ne, $in, $nin, $nor, $or, $size, $type $lt, $lte, $gt, $gte
// find customers with any claims > db.customers.find( {claims: {$exists: true }} ) // find customers matching a regular expression > db.customers.find( {last: /^rog*/i } ) // count customers by city > db.customers.find( {city: ‘Philadelphia’} ).count()
![Page 8: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/8.jpg)
Indexes
// Index nested documents > db.customers.ensureIndex( “policies.agent”:1 ) > db.customers.find({‘policies.agent’:’Fred’})
// geospatial index > db.customers.ensureIndex( “property.location”: “2d” ) > db.customers.find( “property.location” : { $near : [22,42] } )
// text index > db.customers.ensureIndex( “policies.notes”: “text” )
![Page 9: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/9.jpg)
Documents are Rich Data Structures { ! first_name: ‘Paul’,! surname: ‘Miller’,! cell: 447557505611, ! city: ‘London’,! location: [45.123,47.232],! Profession: [banking, finance, trader],! cars: [ ! { model: ‘Bentley’,! year: 1973,! value: 100000, … },! { model: ‘Rolls Royce’,! year: 1965,! value: 330000, … }! ]!}!
Fields can contain an array of sub-documents
Fields
Fields can contain arrays
Typed field values String
Number
![Page 10: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/10.jpg)
Over 10,000,000 downloads
300,000 Students for MongoDB
University
35,000 attendees to
MongoDB events annually
Over 1,000 Partners
Over 2,000!Paying Customers
![Page 11: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/11.jpg)
![Page 12: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/12.jpg)
MongoMK
![Page 13: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/13.jpg)
MongoDB MicroKernel
• AEM 6 Scalable Storage Solution • Flexible • Multi-Datacenter Deployments • Geo Distributed Content • Auto-Failover
![Page 14: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/14.jpg)
MongoMK
Nodes Blobs
Se4ngs
Changes
ClusterNodes
Metadata collec&on Content node structure Internal AEM indexes MulL version structures
Blobs collec&on Binary file chunks Enforces data de-‐duplicaLon
Se4ngs, Changes, ClusterNodes Internal AEM collecLons ConfiguraLon and seRngs data
![Page 15: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/15.jpg)
MongoMK - Content
Metadata Binary / Blobs
![Page 16: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/16.jpg)
Binary / Blobs
MongoMK – Metadata
Metadata
![Page 17: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/17.jpg)
MongoMK – Data Model
![Page 18: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/18.jpg)
MongoMK – Version Control
Revisions of content are maintained as separate trees
![Page 19: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/19.jpg)
MongoMK – Data Model
![Page 20: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/20.jpg)
MongoMK – Versioning & Concurrency
System provides version and concurrency control of content revisions.
![Page 21: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/21.jpg)
MongoMK – Versioning & Concurrency
![Page 22: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/22.jpg)
Binary / Blobs
MongoMK – Binary Data
Metadata
![Page 23: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/23.jpg)
AEM Blob Storage
TAR MONGODB S3
![Page 24: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/24.jpg)
Blobs on MongoMK
Posi&ve Penalty
Unified Storage Layer More Storage Requirements
Reduced OperaLonal Overhead More I/O Intensive
Binary De-‐duplicaLon Higher Bandwidth Requirements
![Page 25: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/25.jpg)
MongoMK - Blobs
![Page 26: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/26.jpg)
MongoMK - Blobs
![Page 27: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/27.jpg)
Sizing
![Page 28: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/28.jpg)
MONGODB SIZING
Availability
Volume
Expected Latency Working Data Set
![Page 29: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/29.jpg)
Availability
![Page 30: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/30.jpg)
Sizing - Availability
AEM
Edi&ng
Cura&ng
Valida&ng
Primary
Secondary
Secondary
![Page 31: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/31.jpg)
Sizing - Availability
AEM
Edi&ng
Cura&ng
Valida&ng
Primary
Secondary
Primary
![Page 32: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/32.jpg)
Sizing - Availability
AEM -‐ author AEM -‐ author AEM -‐ author
Primary
Secondary Secondary
Datacenter West Datacenter East Datacenter Center
AEM -‐ author AEM -‐ author
![Page 33: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/33.jpg)
Sizing - Availability
AEM -‐ author AEM -‐ author AEM -‐ author
Primary
Secondary Secondary
Backups (hidden) Hot Backups (delayed)
![Page 34: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/34.jpg)
Volume
![Page 35: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/35.jpg)
Sizing - Volume
Indexes ProperLes MulL Version
Nodes
Blobs
Full Text Search Indexes
Binary Chunks
![Page 36: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/36.jpg)
Sizing - Volume
• Space required by – Data – Indexes
• Read / Write Ratio • Computational Unit Capacity
– RAM – Disk
• Types of Disks! – CPU
![Page 37: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/37.jpg)
Working Set
![Page 38: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/38.jpg)
Sizing – Working Set
Working Set
Rest of your Database
![Page 39: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/39.jpg)
Sizing – Working Set
• Percentage of data that is constantly request by the application – Indexes – Recent Used Data
• Read / Write – Impacts the calculation of the RAM requirement
• Working Set Should Fit In RAM
![Page 40: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/40.jpg)
Working Set Not in RAM
![Page 41: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/41.jpg)
Sizing – Working Set
• AEM Calculation of Working Set – Internal MongoDB Indexes – Constantly accessed Assets
• % of data access – AEM Indexes
• Multi-version Indexes • Lucene Indexes
![Page 42: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/42.jpg)
Working Set Can Be Distributed Across Shards
![Page 43: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/43.jpg)
Sizing - Latency
Primary
Secondary
Secondary
AEM
AEM
AEM
ONLY SECONDARY READS!
![Page 44: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/44.jpg)
Deploy
![Page 45: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/45.jpg)
Rules for a Good Deployment
Prototype
Test
Monitor Scale
Automate
![Page 46: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/46.jpg)
AEM + Ops Manager
Scale Easily Meet SLAs
Best PracLces, Automated
Cut Management Overhead
![Page 47: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/47.jpg)
Demo
![Page 48: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/48.jpg)
Operational Best Practices
![Page 49: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/49.jpg)
Node Documents { "_id": "1:/norberto", "_deleted": { "r14e211fe903-0-1": "false" }, "jcr:created": { "r14e211fe903-0-1": "\"dat:2015-06-23T17:53:36.257+02:00\"" }, "jcr:createdBy": { "r14e211fe903-0-1": "\"admin\"" }, "_commitRoot": { "r14e211fe903-0-1": "0" }, "jcr:primaryType": { "r14e211fe903-0-1": "\"nam:nt:folder\"" }, "_modified": NumberLong("1435074815"), "_modCount": NumberLong("1") }
{ "_id": "1:/norberto", … "_commitRoot": { "r14e211fe903-0-1": "0", "r14e2122a4bb-0-1": "0" }, … "_modified": NumberLong("1435074995"), "_modCount": NumberLong("3"), "_children": true, "_lastRev": { "r0-0-1": "r14e2122a4bb-0-1" } }
![Page 50: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/50.jpg)
Document Moves 2015-05-18T17:07:10.264+0200 [conn1298] command aem.$cmd command: findAndModify { findandmodify: "nodes", query: { _id: "10:/content/www/...", _modCount: 3 }, fields: { _id: 1 }, update: { $set: { ... }, $max: { _modified: 1431961630 }, $inc: { _modCount: 1 } } } update: { $set: { ... }, $max: { _modified: 1431961630 }, $inc: { _modCount: 1 } } nscanned:1 nscannedObjects:1 nmoved:1 nMatched:1 nModified:1 keyUpdates:0 numYields:0 locks(micros) w:116270 reslen:203 116ms!
nscanned:1464 nscannedObjects:1044 nmoved:288 nMatched:1044 nModified:1044 keyUpdates:1 numYields:11 locks(micros) w:242369 169ms!
nscanned:10829 nscannedObjects:10000 nmoved:1316 nMatched:10000 nModified:10000 keyUpdates:1 numYields:89 l!ocks(micros) w:2372208 1234ms!
![Page 51: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/51.jpg)
Fragmentation
AEM will be constantly growing the documents generating document moves and subsequently
producing fragmentation
db.adminCommand( { repairDatabase: 1 } )!
h_p://docs.mongodb.org/manual/reference/command/repairDatabase/
![Page 52: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/52.jpg)
Monitoring Performance Indicators
Memory, Background Flushing Avg and Opcounters: make sure you understand these
metrics!
h_ps://docs.mms.mongodb.com/
![Page 53: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/53.jpg)
Working Set Measurement
h_ps://docs.mms.mongodb.com/
Page Faults and Memory aligned with Db Storage will give you and good indication of how well provision is your machine and if you need to
scale out
![Page 54: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/54.jpg)
To Sum It Up
![Page 55: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/55.jpg)
Takeway
AEM + MongoDB is a Scalable Solution Provisioning is Key Monitor, monitor, monitor Wish List is Being Attended We Want to Help You!
![Page 56: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/56.jpg)
![Page 57: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/57.jpg)
![Page 58: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/58.jpg)
MongoMK Deployment for AEM h_ps://info-‐mongodb-‐com.s3.amazonaws.com/MongoDB_AEM6_ConsulLng_Datasheet.pdf
![Page 59: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/59.jpg)
Upcoming Webinar
https://www.mongodb.com/webinar/aem-and-mongodb
![Page 61: Effectively Deploying MongoDB on AEM](https://reader030.vdocuments.us/reader030/viewer/2022032503/55beaa5cbb61eb95198b4592/html5/thumbnails/61.jpg)