couchbase server 2.0 and cross data center replication (xdcr)
TRANSCRIPT
![Page 1: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/1.jpg)
1
Couchbase Server 2.0 and Cross Data Center replication
Dipti BorkarMarty Schoch
![Page 2: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/2.jpg)
2
Couchbase Server 2.0 - Webinar Series
Couchbase Server 2.0 and Indexing/Querying
Couchbase Server 2.0 and Incremental Map Reduce for Real-Time Analytics
Couchbase Server 2.0 and Cross Data Center Replication
Couchbase Server 2.0 and Full-Text Search Integration
Couchbase Server 2.0 Use Cases Overview
Introducing Couchbase Server 2.0
http://www.couchbase.com/webinars
![Page 3: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/3.jpg)
3
New in Two
JSON support
Indexing and Querying
Cross data center replication
Incremental Map Reduce
![Page 4: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/4.jpg)
4
XDCR: Cross Data Center Replication
US DATA CENTER
EUROPE DATA CENTER
ASIA DATA CENTER
http://blog.groosy.com/wp-content/uploads/2011/10/internet-map.jpg
![Page 5: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/5.jpg)
5
XDCR: Cross Data Center Replication
• Replicate your Couchbase data across clusters• Clusters may be spread across geos• Configured on a per-bucket basis• Supports unidirectional and bidirectional operation• Application can read and write from both clusters
(active – active replication)• Scales out linearly• Different from intra-cluster replication
![Page 6: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/6.jpg)
6
Intra-cluster Replication
![Page 7: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/7.jpg)
7
Cross Datacenter Replication (XDCR)
![Page 8: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/8.jpg)
8
33 2
Single node - Couchbase Write Operation with XDCR2
Managed Cache
Dis
k Q
ueue
Disk
Replication Queue
App Server
Couchbase Server Node
Doc 1Doc 1
Doc 1
To other node
XDCR Queue
Doc 1
To other cluster
![Page 9: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/9.jpg)
9
Internal Data Flow
1. Document written to managed cache
2. Document added to intra-cluster replication queue
3. Document added to disk queue
4. XDCR push replicates to other clusters
![Page 10: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/10.jpg)
10
XDCR in action
COUCHBASE SERVER CLUSTERNYC DATA CENTERACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2 SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
DocRAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
COUCHBASE SERVER CLUSTERSF DATA CENTER
ACTIVE
Doc
Doc 2
SERVER 1
Doc 9
SERVER 2 SERVER 3
RAM
Doc Doc Doc
ACTIVE
Doc
Doc
Doc RAM
ACTIVE
Doc
Doc
DocRAM
DISK
Doc Doc Doc
DISK
Doc Doc Doc
DISK
![Page 11: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/11.jpg)
11
XDCR ARCHITECTURE
![Page 12: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/12.jpg)
12
Bucket A
Bucket B
Bucket C
Cluster 1
Bucket A
Bucket B
Bucket C
Cluster 2
Bucket-level XDCR
![Page 13: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/13.jpg)
13
Continuous Reliable Replication
• All data mutations replicated to destination cluster• Multiple streams round-robin across vBuckets in
parallel (32 default)• Automatic resume after network disruption
![Page 14: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/14.jpg)
14
Cluster Topology Aware
• Automatically handles node addition and removal in source and destination clusters
![Page 15: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/15.jpg)
15
Efficient
• Couchbase Server de-duplicates writes to disk– With multiple updates to
the same document only the last version is written to disk
– Only this last change written to disk is passed to XDCR
• Document revisions are compared between clusters prior to transfer
![Page 16: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/16.jpg)
16
Active-Active Conflict Resolution
• Couchbase Server provides strong consistency at the document level within a cluster
• XDCR provides eventual consistency across clusters• If a document is mutated on both clusters, both
clusters will pick the same “winner”• In case of conflict, document with the most updates
will be considered the “winner”
3 33
5
![Page 17: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/17.jpg)
17
CONFIGURATION AND
MONITORING
![Page 18: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/18.jpg)
18
STEP 1: Define Remote Cluster
![Page 19: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/19.jpg)
19
STEP 2: Start Replication
![Page 20: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/20.jpg)
20
Monitor Ongoing Replications
![Page 21: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/21.jpg)
21
Detailed Replication Progress
• Source Cluster
• Destination Cluster
![Page 22: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/22.jpg)
22
XDCR TOPOLOGIES
![Page 23: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/23.jpg)
23
Unidirectional
• Hot spare• Read slave• Development/Testing copies
![Page 24: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/24.jpg)
24
Bidirectional
• Multiple Active Masters• Disaster Recovery• Datacenter Locality
![Page 25: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/25.jpg)
25
Chain
![Page 26: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/26.jpg)
26
Propagation
![Page 27: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/27.jpg)
27
XDCR in the Cloud
• Server Naming– Optimal configuration using DNS name that resolves to
internal address for intra-cluster communication and public address for inter-cluster communication
• Security– XDCR traffic is not encrypted, plan topology accordingly– Consider 3rd party Amazon VPN solutions
![Page 28: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/28.jpg)
28
USE CASES
![Page 29: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/29.jpg)
29
Data Locality
• Data closer to your users is faster for your users
![Page 30: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/30.jpg)
30
Disaster Recovery
• Ensure 24x7x365 data availability even if an entire data center goes down
![Page 31: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/31.jpg)
31
Development and Testing
• Test code changes with actual production data without interrupting your production cluster
• Give developers local databases with real data, easy to dispose and recreate
Test and Dev Staging Production
![Page 32: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/32.jpg)
32
Impact of XDCR on the cluster
Your clusters need to be sized for XDCR• XDCR is CPU intensive – Configure the number of parallel streams based on your CPU
capacity
• You are doubling your I/O usage– I/O capacity needs to be sized correctly
• You will need more memory particularly for bidirectional XDCR – Memory capacity needs to be sized correctly
![Page 33: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/33.jpg)
33
DEMO
![Page 34: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/34.jpg)
34
Additional Resources
• Couchbase Server Manual - http://www.couchbase.com/docs/couchbase-manual-2.0/couchbase-admin-tasks-xdcr.html
• Getting Started with XDCR blog - http://blog.couchbase.com/cross-data-center-replication-step-step-guide-amazon-aws
![Page 35: Couchbase Server 2.0 and Cross Data Center Replication (XDCR)](https://reader038.vdocuments.us/reader038/viewer/2022110121/558c8d5dd8b42a54678b4757/html5/thumbnails/35.jpg)
35
Q & A